*/
The judiciary is familiar with addressing bias but GenAI presents new challenges. Sophie Mitchell examines how, why and when gender bias might present itself in judicial use of AI tools and calls for urgent debate
Microsoft’s ‘CoPilot Chat’ is now being rolled out on judicial office holders’ devices. The tool is to assist judges, who remain personally responsible for material which is produced in their name.
Concerns over the potential unfairness, discrimination and opacity of AI are not new. But is enough awareness being raised about how technology-facilitated misogyny and abuse against women and girls may have an impact upon the use and performance of these technologies in the justice system?
The AI Guidance for Judicial Office Holders (31 October 2025) warns users to ‘be aware of bias’. The judiciary is generally already very familiar with addressing bias. But emerging technologies are presenting new challenges. This article focuses on gender bias but clearly, it can take many forms (and many of the issues raised also apply to AI use in the justice system more generally).
Generative Artificial Intelligence (GenAI) is the type of Al that creates content (e.g., CoPilot, ChatGPT and other models that respond to written prompts). They can do so because they have consumed huge amounts of data during their training; they then use that input to generate their own content. However, GenAI models risk regurgitating, or worse amplifying, the harms and inequalities inherent within the material they have consumed.
An example of this is Microsoft’s experimental chatbot called ‘Tay’. Microsoft’s aim was to ‘experiment with and conduct research on conversational understanding’, with Tay able to learn from the conversations and get progressively ‘smarter’. Within hours, the bot started spouting foul racist and sexist abuse (‘I f*cking hate feminists and they should all die and burn in hell’… ‘Hitler was right I hate the jews’). Microsoft promptly took it offline.
Within 24 hours, the experiment unveiled the risks of Al absorbing and regurgitating human prejudice, which in turn risks normalising it online.
In 2024, UNESCO carried out a study of popular generative Al platforms, including OpenAI’s GPT-3.5 and GPT-2, and Meta’s Llama 2. It found ‘unequivocal evidence of bias against women in content generated’, as well as homophobia and racial stereotyping.
There is a need for increasing awareness about this because generative AI uses machine learning techniques to continue to create new data that has similar characteristics to the data that goes into it. And online harms are getting worse, with a disproportionate impact on women, girls and minority groups.
In August 2024, the Home Office announced that extreme misogyny will be treated as a form of extremism.
As part of ongoing attempts to tackle online harms, the Parliamentary Research Briefing, ‘Violence against women and girls in schools and among children and young people’ dated 7 August 2025, summarised recent studies about inequality online as follows:
Journalist Laura Bates, in her book, The New Age of Sexism: How the AI Revolution is Reinventing Misogyny (Simon & Schuster UK: 2025) cites Baroness Helena Kennedy: ‘the law has always trailed behind social change’ and ‘women have always had to struggle to get the law to deliver for them’. Baroness Kennedy gave her the example of the decades-long efforts to make the courts recognise coercive control as a form of domestic abuse, which she compares to the emotional trauma of experiencing sexual violence in an environment like the metaverse, suggesting that it may take a long time for such harms to be recognised in law.
Machines assisting judges by summarising evidence, or creating a first draft judgment, in cases involving controlling or coercive behaviour, online harms or discrimination, would require close scrutiny of the result. Those types of cases may also be more vulnerable to fabricated caselaw or misunderstanding of the law by the machine, particularly where the law is complex or developing.
In any instance where the machine is invited to be creative with text, it may not use inclusive language.
There is a clear need to balance the speed, efficiency and social benefits which AI has to offer, with caution and the need to spread awareness about concerns such as the potential for bias. For now, at least, AI tools assist the judiciary, but they are not replacing direct judicial engagement with evidence. Any user of AI must keep the risks of inherent bias in mind and be ready to correct it, particularly when cases involve data that is vulnerable to bias.
There is growing global interest in how AI can improve access to justice, including how it can increase court capacity. The government’s policy paper, ‘AI action plan for justice’ dated 31 July 2025, promises to ‘embed AI across justice services… while still preserving the right to have any final determination made by a human judge’.
With reference to rapidly developing AI technology, on 15 October 2025, Sir Geoffrey Vos gave a speech calling for a ‘serious debate now, before it is too late’, to consider: (a) what human rights people should have in the light of ever more capable AI, and (b) what decisions should remain in human hands.
There are many members of the legal profession now discussing AI, but few opportunities for them to do so in a manner that enables cross-collaboration with the technology experts who may be able to fill gaps in their knowledge (for instance, about the likely effectiveness of de-biassing techniques). Lack of transparency in AI technology makes an informed discussion difficult for all of us. Lack of technical expertise in the subject matter makes it even harder. Confidentiality rings could be used as appropriate, where there are sensitive data issues.
According to the Alan Turing Institute, only 22% of AI/data professionals are women, presenting the obvious risk that the lack of diversity in the AI ecosystem results in harmful feedback loops of social bias being built into machine learning systems. Further, developers are unlikely to have expertise in law and human rights. Any tools developed by private companies are more likely to be driven by efficiency of market solutions which can generate profits, rather than equality and the rule of law. So it’s all the more important that everyone is advocated for by the legal profession.
In May 2025, the Commons Home Affairs Committee opened an inquiry: ‘Combatting new forms of extremism’. In October 2025, the Metropolitan Police Service gave evidence that one of the factors contributing to the rise in misogyny extremism is the growing divide between the extremely internet literate younger generation and an older generation who are largely unaware of the risks and/or are unable or unwilling to regulate use of the internet.
It is this ‘growing divide’ between the generations that must be bridged before it risks leaving vulnerable women and girls exposed, and boys radicalised, as those more likely to help them are often unaware of the extent of online harms, and how they may influence harmful algorithms, AI stereotyping and generally contaminate data input. Views from the youngest members of the legal profession could be invaluable.
AI, algorithmic and automated tools are increasingly being used across the public sector to make decisions affecting individuals. The Public Law Project is tracking examples of automated decision making (ADM) in government on its ‘TAG register’.
The Public Law Project states that, as of 17 November 2025, there are 55 automated tools tracked on the register; 83.6% of these tools were only uncovered or more fully understood through the submission of FOIA requests.
On 5 November 2025, Lord Sales explored some of the ways in which administrative law (specifically, judicial review methodology) is and is not yet adequately suited to scrutinising the use of AI and ADM in the administrative sphere (‘AI and Public Law: Automated Decision-Making in Government’). He observed that, as opacity of an ADM system could make it very difficult to detect bias, at least until the ADM system has produced a substantial number of decisions in which patterns can be identified, courts and litigants could become more proactive in identifying cases which raise systemic issues and marshalling them together in a composite procedure, by using pilot cases or group litigation techniques.
There are already several well-known instances of the use of opaque algorithms in administrative decision-making in the UK: the Home Office has used a ‘visa streaming’ tool to grade entry visa applications, assigning risk ratings that significantly impact application outcomes; the Department for Work and Pensions is using an opaque algorithm to determine which claims for universal credit it would investigate; local authorities in the UK have been applying algorithms that are difficult to scrutinise to support decisions on transportation, houses in multiple occupation, and children’s social care.
Ongoing analysis of the automated decision making starting to be rolled out across the justice system, and the adequacy of their transparency levels, will be crucial to help inform the debate about the extended use of AI within the justice system.
Drawing the threads together, there need to be more mechanisms through which the legal profession can bring their experience, queries and concerns to the table, enabling the MOJ to consider them. Experience from the legal profession ought to help inform the design, development and intended use of AI in the justice system, to ensure that a modern, inclusive application of the rule of law and human rights are built in. Once bias is embedded in a machine, it is much harder to diagnose and resolve the issue, and it may be even harder to challenge it.
In February, Sophie is speaking at the Westminster Legal Policy Forum, where stakeholders and policymakers will consider the ‘Next steps for AI use in the justice system and courts modernisation in England and Wales’. She welcomes readers’ input to help further the debate.
Sophie Mitchell is a media law specialist and longstanding member of the Attorny General’s Counsel Panel, routinely acting for the Ministry of Justice and HM Courts and Tribunals Service in cases involving online harms, data and information law.
Microsoft’s ‘CoPilot Chat’ is now being rolled out on judicial office holders’ devices. The tool is to assist judges, who remain personally responsible for material which is produced in their name.
Concerns over the potential unfairness, discrimination and opacity of AI are not new. But is enough awareness being raised about how technology-facilitated misogyny and abuse against women and girls may have an impact upon the use and performance of these technologies in the justice system?
The AI Guidance for Judicial Office Holders (31 October 2025) warns users to ‘be aware of bias’. The judiciary is generally already very familiar with addressing bias. But emerging technologies are presenting new challenges. This article focuses on gender bias but clearly, it can take many forms (and many of the issues raised also apply to AI use in the justice system more generally).
Generative Artificial Intelligence (GenAI) is the type of Al that creates content (e.g., CoPilot, ChatGPT and other models that respond to written prompts). They can do so because they have consumed huge amounts of data during their training; they then use that input to generate their own content. However, GenAI models risk regurgitating, or worse amplifying, the harms and inequalities inherent within the material they have consumed.
An example of this is Microsoft’s experimental chatbot called ‘Tay’. Microsoft’s aim was to ‘experiment with and conduct research on conversational understanding’, with Tay able to learn from the conversations and get progressively ‘smarter’. Within hours, the bot started spouting foul racist and sexist abuse (‘I f*cking hate feminists and they should all die and burn in hell’… ‘Hitler was right I hate the jews’). Microsoft promptly took it offline.
Within 24 hours, the experiment unveiled the risks of Al absorbing and regurgitating human prejudice, which in turn risks normalising it online.
In 2024, UNESCO carried out a study of popular generative Al platforms, including OpenAI’s GPT-3.5 and GPT-2, and Meta’s Llama 2. It found ‘unequivocal evidence of bias against women in content generated’, as well as homophobia and racial stereotyping.
There is a need for increasing awareness about this because generative AI uses machine learning techniques to continue to create new data that has similar characteristics to the data that goes into it. And online harms are getting worse, with a disproportionate impact on women, girls and minority groups.
In August 2024, the Home Office announced that extreme misogyny will be treated as a form of extremism.
As part of ongoing attempts to tackle online harms, the Parliamentary Research Briefing, ‘Violence against women and girls in schools and among children and young people’ dated 7 August 2025, summarised recent studies about inequality online as follows:
Journalist Laura Bates, in her book, The New Age of Sexism: How the AI Revolution is Reinventing Misogyny (Simon & Schuster UK: 2025) cites Baroness Helena Kennedy: ‘the law has always trailed behind social change’ and ‘women have always had to struggle to get the law to deliver for them’. Baroness Kennedy gave her the example of the decades-long efforts to make the courts recognise coercive control as a form of domestic abuse, which she compares to the emotional trauma of experiencing sexual violence in an environment like the metaverse, suggesting that it may take a long time for such harms to be recognised in law.
Machines assisting judges by summarising evidence, or creating a first draft judgment, in cases involving controlling or coercive behaviour, online harms or discrimination, would require close scrutiny of the result. Those types of cases may also be more vulnerable to fabricated caselaw or misunderstanding of the law by the machine, particularly where the law is complex or developing.
In any instance where the machine is invited to be creative with text, it may not use inclusive language.
There is a clear need to balance the speed, efficiency and social benefits which AI has to offer, with caution and the need to spread awareness about concerns such as the potential for bias. For now, at least, AI tools assist the judiciary, but they are not replacing direct judicial engagement with evidence. Any user of AI must keep the risks of inherent bias in mind and be ready to correct it, particularly when cases involve data that is vulnerable to bias.
There is growing global interest in how AI can improve access to justice, including how it can increase court capacity. The government’s policy paper, ‘AI action plan for justice’ dated 31 July 2025, promises to ‘embed AI across justice services… while still preserving the right to have any final determination made by a human judge’.
With reference to rapidly developing AI technology, on 15 October 2025, Sir Geoffrey Vos gave a speech calling for a ‘serious debate now, before it is too late’, to consider: (a) what human rights people should have in the light of ever more capable AI, and (b) what decisions should remain in human hands.
There are many members of the legal profession now discussing AI, but few opportunities for them to do so in a manner that enables cross-collaboration with the technology experts who may be able to fill gaps in their knowledge (for instance, about the likely effectiveness of de-biassing techniques). Lack of transparency in AI technology makes an informed discussion difficult for all of us. Lack of technical expertise in the subject matter makes it even harder. Confidentiality rings could be used as appropriate, where there are sensitive data issues.
According to the Alan Turing Institute, only 22% of AI/data professionals are women, presenting the obvious risk that the lack of diversity in the AI ecosystem results in harmful feedback loops of social bias being built into machine learning systems. Further, developers are unlikely to have expertise in law and human rights. Any tools developed by private companies are more likely to be driven by efficiency of market solutions which can generate profits, rather than equality and the rule of law. So it’s all the more important that everyone is advocated for by the legal profession.
In May 2025, the Commons Home Affairs Committee opened an inquiry: ‘Combatting new forms of extremism’. In October 2025, the Metropolitan Police Service gave evidence that one of the factors contributing to the rise in misogyny extremism is the growing divide between the extremely internet literate younger generation and an older generation who are largely unaware of the risks and/or are unable or unwilling to regulate use of the internet.
It is this ‘growing divide’ between the generations that must be bridged before it risks leaving vulnerable women and girls exposed, and boys radicalised, as those more likely to help them are often unaware of the extent of online harms, and how they may influence harmful algorithms, AI stereotyping and generally contaminate data input. Views from the youngest members of the legal profession could be invaluable.
AI, algorithmic and automated tools are increasingly being used across the public sector to make decisions affecting individuals. The Public Law Project is tracking examples of automated decision making (ADM) in government on its ‘TAG register’.
The Public Law Project states that, as of 17 November 2025, there are 55 automated tools tracked on the register; 83.6% of these tools were only uncovered or more fully understood through the submission of FOIA requests.
On 5 November 2025, Lord Sales explored some of the ways in which administrative law (specifically, judicial review methodology) is and is not yet adequately suited to scrutinising the use of AI and ADM in the administrative sphere (‘AI and Public Law: Automated Decision-Making in Government’). He observed that, as opacity of an ADM system could make it very difficult to detect bias, at least until the ADM system has produced a substantial number of decisions in which patterns can be identified, courts and litigants could become more proactive in identifying cases which raise systemic issues and marshalling them together in a composite procedure, by using pilot cases or group litigation techniques.
There are already several well-known instances of the use of opaque algorithms in administrative decision-making in the UK: the Home Office has used a ‘visa streaming’ tool to grade entry visa applications, assigning risk ratings that significantly impact application outcomes; the Department for Work and Pensions is using an opaque algorithm to determine which claims for universal credit it would investigate; local authorities in the UK have been applying algorithms that are difficult to scrutinise to support decisions on transportation, houses in multiple occupation, and children’s social care.
Ongoing analysis of the automated decision making starting to be rolled out across the justice system, and the adequacy of their transparency levels, will be crucial to help inform the debate about the extended use of AI within the justice system.
Drawing the threads together, there need to be more mechanisms through which the legal profession can bring their experience, queries and concerns to the table, enabling the MOJ to consider them. Experience from the legal profession ought to help inform the design, development and intended use of AI in the justice system, to ensure that a modern, inclusive application of the rule of law and human rights are built in. Once bias is embedded in a machine, it is much harder to diagnose and resolve the issue, and it may be even harder to challenge it.
In February, Sophie is speaking at the Westminster Legal Policy Forum, where stakeholders and policymakers will consider the ‘Next steps for AI use in the justice system and courts modernisation in England and Wales’. She welcomes readers’ input to help further the debate.
The judiciary is familiar with addressing bias but GenAI presents new challenges. Sophie Mitchell examines how, why and when gender bias might present itself in judicial use of AI tools and calls for urgent debate
Kirsty Brimelow KC, Chair of the Bar, sets our course for 2026
What meaningful steps can you take in 2026 to advance your legal career? asks Thomas Cowan of St Pauls Chambers
Marie Law, Director of Toxicology at AlphaBiolabs, explains why drugs may appear in test results, despite the donor denying use of them
Asks Louise Crush of Westgate Wealth Management
AlphaBiolabs has donated £500 to The Christie Charity through its Giving Back initiative, helping to support cancer care, treatment and research across Greater Manchester, Cheshire and further afield
Q&A with criminal barrister Nick Murphy, who moved to New Park Court Chambers on the North Eastern Circuit in search of a better work-life balance
Jury-less trial proposals threaten fairness, legitimacy and democracy without ending the backlog, writes Professor Cheryl Thomas KC (Hon), the UK’s leading expert on juries, judges and courts
Human rights cases don’t come bigger than this. Tim Otty KC, lead counsel for the Government of Ukraine in its case before the European Court against Russia, talks about the significance of this landmark ruling and other pro bono highlights from his career at the Bar. Interview by Anthony Inglese CB
Are you ready for the new way to do tax returns? David Southern KC explains the biggest change since HMRC launched self-assessment more than 30 years ago... and its impact on the Bar
Professor Dominic Regan and Seán Jones KC present their best buys for this holiday season
Marking one year since a Bar disciplinary tribunal dismissed all charges against her, Dr Charlotte Proudman discusses the experience, her formative years and next steps. Interview by Anthony Inglese CB
