Most U.S. executive departments are planning to expand their use of facial recognition technology, according to a recently released Government Accountability Office (GAO) report. The report—requested by members of the House Judiciary Committee and the House Oversight and Reform Committee amid increasing concerns about the misuse of facial recognition technology—found that 18 of 24 federal agencies surveyed, including 11 of the 15 executive departments, employed at least one facial recognition system in 2020. Ten executive departments plan to expand their use of facial recognition systems through 2023.

The report discussed federal use of facial recognition technology, ranging from the relatively mundane (employees unlocking their smartphones), to the more sophisticated and contentious, such as running criminal suspect and victim photos through commercial facial recognition systems like a controversial database compiled by the legally embattled Clearview AI. 

Digital access, domestic law enforcement, and physical security are the most prevalent federal uses of facial recognition technology, with the departments of Homeland Security, Defense and Justice owning two-thirds of federal facial recognition systems. The government is building databases internally, utilizing commercial software, and partnering with state and local authorities to conduct facial recognition searches.

The expanding federal use of facial recognition systems has been met with widespread skepticism by privacy and civil liberties advocates, who question the technology’s accuracy and worry about its misuse. In media coverage of the report, commentators expressed concerns that federal facial recognition use may lead to improper surveillance and false arrests, exacerbated by racial and gender bias in the technology. 

Scope of the Report

The report defined facial recognition as a biometric technology that “can verify or identify individuals by their faces.” It described facial recognition systems as those that detect individuals by images of their faces obtained from photos or videos, either to verify a known person or to identify an unknown person. While recognizing that facial detection (determining whether an image contains a face) and facial analysis (identifying what a face’s characteristics and activity reveal about a person) are “related to, but distinct from, facial recognition,” the report included both facial detection and analysis technologies within the umbrella of “facial recognition technology” (FRT). 

The report examined two questions: how agencies used FRT in fiscal year 2020 and how agencies plan to expand their use of FRT through fiscal year 2023. The report additionally surveyed agencies’ FRT research and development activities, their use of nonfederal FRT systems, and their regulation of nonfederal use of FRT. 

To compile the report, the GAO analyzed the current and planned use of facial recognition technologies by 24 federal agencies, including all 15 executive departments. The findings are the result of a 16-month audit that included surveying agencies, interviewing subject matter experts and reviewing documents to determine FRT usage across the federal government. Its findings build on a sequence of GAO reports on FRT usage, including a 2016 report on FBI use of FRT, a 2020 report on Department of Homeland Security (DHS) use of FRT at ports of entry, and a June 2021 report on federal law enforcement use of FRT.

While the report provides an insightful overview of FRT use across the federal government, it is not all-encompassing. The report discusses only unclassified FRT systems. Further, the GAO surveyed only the 24 agencies subject to the Chief Financial Officers Act of 1990. The report thus did not review potential FRT usage by independent intelligence agencies like the CIA and the Office of the Director of National Intelligence. Likewise, it did not review FRT use by independent regulatory agencies that have investigative and enforcement responsibilities, such as the Federal Trade Commission and the Securities and Exchange Commission. 

The Report’s Findings: Uses of FRT

The GAO determined that 18 of the 24 agencies it surveyed are using FRT, with 10 of those agencies planning to expand their use of FRT through 2023. Fourteen agencies used FRT simply to unlock agency-issued smartphones, presumably by employing built-in facial recognition systems such as Face ID for iPhone. Twelve agencies employed FRT for other reasons: six used FRT to support domestic law enforcement investigations, five for facility security and access control, four for national security and defense, two for border and transportation security, and five for other purposes. 

Domestic law enforcement. Six federal agencies use FRT for domestic law enforcement lead generation. Federal law enforcement officers who use FRT for lead generation appear to search photos of unidentified criminal suspects or victims against databases of known individuals. These databases typically include government-collected information, such as driver’s license photos or mugshots, but may also include publicly available information, such as photos posted on social media, particularly if they are assembled by a commercial vendor. For example, the FBI’s Next Generation Identification Interstate Photo System compares photos of unknown criminal suspects to a repository of known individuals identified in mugshots. It is used not only by the Department of Justice but also by Department of Defense and DHS criminal investigators. 

Five agencies—DHS, the Justice Department, the Department of Health and Human Services (HHS), the Department of the Interior, and the Defense Department—used a controversial FRT system from Clearview AI, a New York City-based facial recognition startup that is facing multiple lawsuits for compiling a dataset of more than 3 billion images by scraping personal images from social media profiles. DHS Immigration and Customs Enforcement (ICE) and the Secret Service used Clearview AI to identify perpetrators and victims in child exploitation investigations, suggesting they may have uploaded photos of minors into Clearview’s application. A few agencies appear to be ramping up their use of Clearview AI. HHS’s Office of Inspector General and the Defense Department’s Air Force Office of Special Investigations engaged in operational pilots of Clearview AI in 2020, while two additional agencies plan to use Clearview AI for the first time. The report also noted that federal agents have accessed Clearview AI through state-run fusion centers, including the New York State Intelligence Center.

Physical security. Five agencies are using FRT for monitoring and surveilling facilities, including with real-time detection systems. For example, HHS’s Centers for Disease Control and Prevention uses the commercial FRT AnyVision to alert for the presence of individuals at CDC facilities who are watchlisted or otherwise suspected of criminal activity. The departments of Commerce and Energy used facial recognition as part of their access control systems for secure facilities, such as in continuity of operations at data centers or nuclear facilities.

National security and defense. Four agencies had unclassified FRT systems for national security and defense: DHS, the Defense Department, the Justice Department and the State Department. These FRT systems typically are used to identify known and suspected terrorists or to obtain derogatory information about a counterterrorism or counterintelligence suspect. For example, State’s Integrated Biometric System enables consulates and passport agencies to run visa and passport applicants’ photos against photos in terrorist watchlist databases.

Border and transportation security. Only two agencies identified having FRT systems for border and transportation security: DHS and State. However, DHS’s border and transportation security systems appear to be some of the more large-scale FRT systems employed by the federal government. For example, Customs and Border Protection’s (CBP’s) Automated Targeting System matches photos of international travelers, applicants for traveler programs, and other subjects of interest against a database of people associated with derogatory information. CBP’s Traveler Verification Service, meanwhile, compares live photos of travelers arriving at or departing a port of entry against DHS databases. 

Other use cases. The Defense Department, the Justice Department, the National Aeronautics and Space Administration, the General Services Administration, and the Social Security Administration all used FRT for additional purposes. The Defense Department is developing the Real-Time Automated Personnel Identification System to issue identification cards with consistent formatting and to enable employees to verify their identity. The Justice Department deployed FRT in educational settings to demonstrate how the technology works. NASA’s Johnson Space Center piloted FRT verification software for employees who had forgotten their badge but canceled the program due to cost. The General Services Administration and the Social Security Administration conducted employee pilots to assess using FRT for accessing government websites. 

The Report’s Findings: Investing in the Technology

Ten agencies engaged in research and development of FRT systems, and four agencies entered into FRT-related agreements with nonfederal partners. 

Research and development. Ten federal agencies worked on FRT-related research and development in fiscal year 2020. These efforts ranged from internal projects to develop proprietary government technology to grant-making for funding external research. DHS has hosted Biometric Technology Rallies—industry challenges to develop facial recognition and other biometric solutions—with a focus in 2020 on facial identification of people wearing masks. The Justice Department researched the relationship between skin tone and false-match rates, the detection of deepfakes and other synthetic faces, and the teaming of human forensic examiners with FRT systems. State, meanwhile, researched the impact of aging on the accuracy of facial recognition, with a focus on how FRT could work with children’s passport photos. The Commerce Department, through the National Institute for Standards and Technology, conducted broader research on standards, performance management methods, image quality, testing and evaluation technologies, and interoperability. While some agencies provided cost estimates on particular FRT projects—for example, DHS obligated more than $2 million for independent testing of an automated Transportation Security Administration airport checkpoint processing system—the report’s funding data is too inconsistent to provide a holistic picture of federal research and development spending on FRT systems. 

Other federal agencies explored use cases for automated eye tracking, an emerging use case for facial analysis. The Department of Transportation used eye tracking to evaluate the performance of commercial motor vehicle drivers, train operators and air traffic controllers. NASA used eye tracking in human factors research, while the Department of Veterans Affairs purchased eye tracking equipment to help speech-impaired veterans communicate. 

Ownership of FRT. Agencies varied in the extent to which they owned FRT systems internally or chose to rely on external systems. The Defense Department, the Justice Department and DHS owned the most FRT systems internally, whereas some agencies, such as the Department of the Treasury, did not own an FRT system but, instead, used federal and commercially owned systems. Agencies used memoranda of understanding—agreements that require no external approval—to share access to their FRT systems with interagency and nonfederal partners. Additionally, some federal FRT systems pulled directly from other federal and nonfederal FRT databases as part of their facial identification searches. For example, the FBI’s Facial Analysis, Comparison, and Evaluation (FACE) Services Unit had access in 2020 to the FRT systems of two federal entities and 21 states.

There is significant FRT cooperation between federal agencies and state and local partners. In addition to multistate cooperation through the FBI’s FACE Services Unit, the DHS Multi-State Facial Recognition Community of Interest has facial recognition data from at least 15 states and four localities. Through the community, authorized users such as CBP and ICE agents may request facial recognition searches through state and local entities, including fusion centers. 

Federal agencies also maintain regional partnerships to apply FRT to local law enforcement and security priorities. For example, DHS awarded a contract to the Lehigh County district attorney’s office in Pennsylvania to enhance ICE use of the Gang Intelligence Application, a database of transnational gang members. The Department of the Interior Park Police, meanwhile, indirectly searched the National Capital Region Facial Recognition Investigative Leads System (NCRFRILS) through a Maryland law enforcement partner in July 2020, in what appears to have been a controversial attempt to identify a June 2020 Lafayette Square protester. The Park Police planned to seek direct access to NCRFRILS in April 2021, but the regional FRT program was halted in May after Virginia passed a law that restricted the use of FRT by state and local law enforcement. 

Two federal agencies—DHS and State—have also entered into FRT agreements with foreign governments. DHS worked with Australia and the United Kingdom to assess FRT capabilities, while State donated FRT equipment to Mexico and Guatemala and trained Mexican officials on how to use it. 

Only one agency, DHS, regulated the use of FRT by other entities in 2020. TSA amended its aircraft operator security program to permit the use of FRT systems to identify passengers checking baggage for flights. 

The Future of FRT Use

Critics of FRT have long raised concerns about its accuracy, privacy implications and risk of abuse. Nonetheless, the GAO report revealed that 10 agencies plan to expand their use of FRT through fiscal year 2023. Nine agencies will begin using new FRT systems. For example, State plans to pilot a new system for screening suspicious individuals, such as known or suspected terrorists, attempting to travel through partner countries. While the Justice Department’s research into the effects of deepfakes and skin tone on FRT accuracy demonstrates its awareness of FRT’s imperfections, it too plans to deploy two new FRT systems: one for transportation safety and one for physical security. Three agencies also plan to continue evaluating existing FRT systems, including DHS, which will expand its pilot of TSA automated identification in airport check-in verification to the Detroit Metropolitan Wayne County Airport. DHS also plans to upgrade an existing biometric storage system. 

Agency plans to expand FRT use prompted many commentators to raise concerns. Popular Science called greater government use of facial recognition “bad,” quoting an Electronic Frontier Foundation attorney’s observation that government use of facial recognition “can lead to an increase in false arrests and citizen surveillance.” Engadget stated that facial recognition systems are “still far from being perfectly accurate, especially when identifying women and [people of color].” Other reporting also emphasized the “demographic differentials” in many facial recognition systems—biases that have contributed to the technology’s facing “growing opposition.” 

The report’s release coincides with a period of reevaluation of FRT, particularly as it pertains to government use. Even commercial vendors of facial recognition systems have expressed wariness. Amazon banned police use of its facial recognition software Rekognition in June 2020 pending congressional implementation of “appropriate rules” and extended the prohibition indefinitely in May 2021. Alphabet and Google CEO Sundar Pichai has also emphasized the need for a regulatory framework to govern the use of facial recognition. 

Multiple efforts to curtail government use of facial recognition are underway. Virginia, Massachusetts and Maine and more than a dozen cities—including Boston, Portland and San Francisco—have banned or restricted the use of FRT by public officials. While federal use of FRT remains unregulated today, Sens. Markey, Sanders, Warren and others have co-sponsored legislation to stop government use of facial recognition and other biometric technology.

As the GAO report demonstrates, federal departments are poised to expand their use of facial recognition systems across a wide range of use cases in the absence of federal regulation.