*/
Beyond the intrusiveness of automated facial recognition is concern about ‘false positives’ and potential for racial bias in the technology
By Sailesh Mehta and Shahid Khan
The Automated Facial Recognition genie has long been out of the bottle. The question is whether it can be controlled. Police forces and secret services have been using the technology for some time. Its backers hail it as an unbiased tool for catching criminals, finding missing persons and protecting the public, particularly against terrorists. Civil liberty groups warn of the chilling effect such technology has already had on the right to privacy, the freedom of movement and speech – and its potential for abuse. It can be used to track every movement of an individual in real time, and to keep tabs on their network of connections. The integrated use of the technology to connect it with cars driven, phone calls made, and items purchased, are reminiscent of mass surveillance in the totalitarian state depicted in Orwell’s 1984.
An aspect of automated facial recognition (AFR) was tested in the Court of Appeal recently in R (on the application of Edward Bridges) v Chief Constable of South Wales Police [2020] EWCA Civ 1058. The case concerned the use of facial recognition technology by the South Wales Police in a pilot project. Cameras would capture images of individuals attending concerts or football matches from afar, biometric data extracted from the photos and an algorithmic comparison made with data of faces on a watchlist database. Fifty faces per second can be scanned by the system. If there is no match, then the facial biometric data of the individual is immediately destroyed. If there is a positive match, then the system requires a police officer and another person to make a decision to stop the individual which may lead to a possible arrest. Edward Bridges challenged the lawfulness of the use of the technology. Beyond the intrusiveness of such surveillance, is concern about ‘false positives’ – incorrect identifications by the algorithms used in the automated process.
The case was heard in the context of growing international evidence of the potential for misuse of the technology and its potential for bias. Late last year, the National Institute of Standards and Technology (NIST), an American organisation, carried out an evaluation of 189 software algorithms from 99 developers – a majority of the industry. The tests showed a worrying range of inaccuracy – with higher rates of false positives for Asian and African-American faces relative to images of Caucasians. The differentials often ranged from a factor of 10 to 100 times. There were also worryingly high levels of differentials for African American women’s faces. The authors of the study understood that false positives, when one face is being compared to a large database, could lead to a person wrongly being put on a watch list or even arrested. Further, that the higher rates of error for Black and Asian faces were a function of the ‘training’ algorithm used by the commercial programmes which educates the technology about facial recognition. But it was difficult, if not impossible, to delve into the ‘training’ algorithms because such matters were closely guarded commercial secrets. The author of the study stated that they had ‘found empirical evidence for the existence of demographic differentials in the majority of the face recognition algorithms studied’. They had found an inherent racial bias in the algorithms. The study found that the best way to reduce this bias was to use more diverse data to train the algorithms.
In Bridges, the Court of Appeal found that there was a breach of Article 8 of the European Convention on Human Rights – the right to respect for private and family life. Any interference with this right must be in accordance with the law. It said interference with the Article 8 right was not in accordance with the law in that (i) it is not clear who can be placed on the watchlist – the discretion was too wide – and (ii) there do not appear to be any criteria for determining where the cameras could be deployed.
The court also found a breach of the Public Sector Equality Duty (PSED) as set out in s 149(1) of the Equality Act 2010. The police had never sought to satisfy themselves that the software program did not have an unacceptable bias on grounds of race or sex. Expert evidence before the court suggested that algorithms could have such a bias. As a result, the police had not done all that they reasonably could to fulfil the duty to ensure equality.
Thus, the court has temporarily stopped the use of this technology. It will resume as soon as the police set up a series of checks and balances which satisfy the PSED and the discretion as to where to place the cameras and who should be on the watchlist is better regulated. The genie is only taking a momentary rest.
The Automated Facial Recognition genie has long been out of the bottle. The question is whether it can be controlled. Police forces and secret services have been using the technology for some time. Its backers hail it as an unbiased tool for catching criminals, finding missing persons and protecting the public, particularly against terrorists. Civil liberty groups warn of the chilling effect such technology has already had on the right to privacy, the freedom of movement and speech – and its potential for abuse. It can be used to track every movement of an individual in real time, and to keep tabs on their network of connections. The integrated use of the technology to connect it with cars driven, phone calls made, and items purchased, are reminiscent of mass surveillance in the totalitarian state depicted in Orwell’s 1984.
An aspect of automated facial recognition (AFR) was tested in the Court of Appeal recently in R (on the application of Edward Bridges) v Chief Constable of South Wales Police [2020] EWCA Civ 1058. The case concerned the use of facial recognition technology by the South Wales Police in a pilot project. Cameras would capture images of individuals attending concerts or football matches from afar, biometric data extracted from the photos and an algorithmic comparison made with data of faces on a watchlist database. Fifty faces per second can be scanned by the system. If there is no match, then the facial biometric data of the individual is immediately destroyed. If there is a positive match, then the system requires a police officer and another person to make a decision to stop the individual which may lead to a possible arrest. Edward Bridges challenged the lawfulness of the use of the technology. Beyond the intrusiveness of such surveillance, is concern about ‘false positives’ – incorrect identifications by the algorithms used in the automated process.
The case was heard in the context of growing international evidence of the potential for misuse of the technology and its potential for bias. Late last year, the National Institute of Standards and Technology (NIST), an American organisation, carried out an evaluation of 189 software algorithms from 99 developers – a majority of the industry. The tests showed a worrying range of inaccuracy – with higher rates of false positives for Asian and African-American faces relative to images of Caucasians. The differentials often ranged from a factor of 10 to 100 times. There were also worryingly high levels of differentials for African American women’s faces. The authors of the study understood that false positives, when one face is being compared to a large database, could lead to a person wrongly being put on a watch list or even arrested. Further, that the higher rates of error for Black and Asian faces were a function of the ‘training’ algorithm used by the commercial programmes which educates the technology about facial recognition. But it was difficult, if not impossible, to delve into the ‘training’ algorithms because such matters were closely guarded commercial secrets. The author of the study stated that they had ‘found empirical evidence for the existence of demographic differentials in the majority of the face recognition algorithms studied’. They had found an inherent racial bias in the algorithms. The study found that the best way to reduce this bias was to use more diverse data to train the algorithms.
In Bridges, the Court of Appeal found that there was a breach of Article 8 of the European Convention on Human Rights – the right to respect for private and family life. Any interference with this right must be in accordance with the law. It said interference with the Article 8 right was not in accordance with the law in that (i) it is not clear who can be placed on the watchlist – the discretion was too wide – and (ii) there do not appear to be any criteria for determining where the cameras could be deployed.
The court also found a breach of the Public Sector Equality Duty (PSED) as set out in s 149(1) of the Equality Act 2010. The police had never sought to satisfy themselves that the software program did not have an unacceptable bias on grounds of race or sex. Expert evidence before the court suggested that algorithms could have such a bias. As a result, the police had not done all that they reasonably could to fulfil the duty to ensure equality.
Thus, the court has temporarily stopped the use of this technology. It will resume as soon as the police set up a series of checks and balances which satisfy the PSED and the discretion as to where to place the cameras and who should be on the watchlist is better regulated. The genie is only taking a momentary rest.
Beyond the intrusiveness of automated facial recognition is concern about ‘false positives’ and potential for racial bias in the technology
By Sailesh Mehta and Shahid Khan
Sam Townend KC explains the Bar Council’s efforts towards ensuring a bright future for the profession
Giovanni D’Avola explores the issue of over-citation of unreported cases and the ‘added value’ elements of a law report
Louise Crush explores the key points and opportunities for tax efficiency
Westgate Wealth Management Ltd is a Partner Practice of FTSE 100 company St. James’s Place – one of the top UK Wealth Management firms. We offer a holistic service of distinct quality, integrity, and excellence with the aim to build a professional and valuable relationship with our clients, helping to provide them with security now, prosperity in the future and the highest standard of service in all of our dealings.
Is now the time to review your financial position, having reached a career milestone? asks Louise Crush
If you were to host a dinner party with 10 guests, and you asked them to explain what financial planning is and how it differs to financial advice, you’d receive 10 different answers. The variety of answers highlights the ongoing need to clarify and promote the value of financial planning.
On the 50th anniversary of the pub bombings, even now it is still unresolved. Chris Mullin, the journalist and former MP who led the campaign leading to the release of the Birmingham Six, looks back at events
Adele Akers’ reflections on health and wellbeing support at the very junior end of the Bar
Not one to say, ‘I told you so,’ Sam Thomas continues his cyber series with the key learnings from the major supply chain attack affecting 80 law firms and at least one set of chambers at the end of 2023
One year on and the Court of Appeal fails to quash convictions after receiving evidence of racism in the jury room, and there are still no revisions to the Equal Treatment Bench Book , says Keir Monteith KC
Most of us like to think we would risk our career in order to meet our ethical obligations, so why have so many lawyers failed to hold the line? asks Flora Page
