Benefits and Challenges of Smartphone Applications to Augment Contact Tracing (GAO-21-104622)
Summary
A GAO technology assessment report to congressional addressees, GAO-21-104622, dated September 2021, on exposure notification. It examines smartphone applications developed with the spread of COVID-19 to supplement manual contact tracing, a public health measure. GAO explains that exposure notification apps determine the proximity of users and notify people who have been in close contact with another user who was likely infectious. It states that the apps are expected to enhance the speed and reach of contact tracing, and the report examines their benefits and challenges.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
Full text
United States Government Accountability Office
Report to Congressional Addressees
September 2021 TECHNOLOGY ASSESSMENT
Exposure Notification
Benefits and Challenges of Smartphone
Applications to Augment Contact Tracing
GAO-21-104622
The cover image displays a stylized depiction of the use of an exposure notification app by various individuals and an
example of an exposure notification message.
Cover source: GAO. | GAO-21-104622
TECHNOLOGY ASSESSMENT
Highlights of GAO-21-104622, a report to
congressional addressees Exposure Notification
September 2021 Benefits and Challenges of Smartphone
Applications to Augment Contact Tracing
Why GAO did this study What GAO found
With the emergence and rapid global Exposure notification applications (apps)—which determine the proximity of users
spread of COVID-19, smartphone and notify people who have been in close contact with another user who was likely
apps have been developed to
supplement manual contact tracing, infectious—are expected to enhance the speed and reach of contact tracing and
which is a public health measure used help slow the spread of infectious diseases such as COVID-19. As of June 2021,
to slow the spread of infectious almost half (26/56) of U.S. states, territories, and the District of Columbia had
disease. deployed an app for COVID-19, all using a system developed jointly by Google and
GAO was asked to conduct a Apple (see figure). In the absence of a national app, states independently launched
technology assessment of exposure apps, resulting in a staggered rollout over 10 months beginning in August 2020.
notification apps. This report
Map of deployment of exposure notification apps by U.S. states and territories, as of
discusses (1) the benefits of exposure
notification apps; (2) the current level June 2021
of deployment in the U.S.; (3)
challenges affecting their use; and (4)
policy options that may help address
these challenges for future use.
To address these objectives, GAO
reviewed agency documentation, met
with officials from several federal
agencies, and conducted a review of
technical and policy literature. GAO
also interviewed representatives
from companies involved in the
development of exposure notification
apps, public health organizations,
federally funded research and
development centers, and academic Reported app development costs for selected states varied, ranging from no cost
researchers. In addition, GAO (provided by a nonprofit organization) to $700,000. Marketing costs for selected
analyzed information from a selection states ranged from $380,000 to $3.2 million. Reported app download levels in the
of states. GAO is identifying policy selected states ranged from 200,000 to more than 2 million, as of June 2021.
options in this report.
GAO identified several challenges limiting app use and the ability of states and
GAO received technical comments on others to determine whether the apps were effective:
a draft of this report from five federal
agencies and five organizations Accuracy of Technical limitations to measuring distance and exposure can result in
included in the review, which it measurements inaccurate exposure notifications.
incorporated as appropriate. Privacy and The public may lack confidence that its privacy is being protected, in
security concerns part, due to a lack of independent privacy and security assessments and
a lack of federal legal protections.
Adoption States have faced challenges attracting public interest in downloading
and using an exposure notification app.
Verification code States faced challenges in promptly providing people who tested
delays positive for COVID-19 with a verification code necessary to notify other
close contacts of potential exposure using the app.
Evidence of Limited data are available to evaluate the effectiveness of the apps.
effectiveness
View GAO-21-104622. For more information, Source: GAO. | GAO-21-104622
contact Karen L. Howard at (202) 512-6888
or, howardk@gao.gov or Vijay A. D’Souza, at
(202) 512-6240, dsouzav@gao.gov.
United States Government Accountability Office
GAO developed the following four policy options that could help address challenges related to exposure notification apps. The policy
options identify possible actions by policymakers, which may include Congress, other elected officials, federal agencies, state and
local governments, and industry. See below for details of the policy options and relevant opportunities and considerations.
Policy Options to Help Address Challenges of Exposure Notification Apps for Future Use
Opportunities Considerations
Research and Development • Research on technological limitations could help • The research needed may be costly.
(report page 41) increase accuracy, encouraging users to download • Improvements may not be cost-effective,
Policymakers could promote and use the apps. since existing apps may already be
research and development to • Research on technologies and architectures other sufficiently accurate.
address technological than those used by U.S. states could lead to • Research may result in apps that are not
limitations. improvements. functional for the next pandemic, since the
• Partnerships with technology companies could spur current apps were developed for COVID-19.
innovation and help with integrating improvements.
Privacy and Security • Uniform standards and best practices could help • Policymakers would need to balance the
Standards and Practices address real and perceived risks to the public’s data, need for privacy and security with the costs
(report page 42) potentially increasing adoption. of implementing standards and practices.
Policymakers could promote • Standards developed by a broad coalition of • Implementation of privacy requirements
uniform privacy and security stakeholders could increase the likelihood of may need to be flexible, since jurisdictions
standards and practices for stakeholder agreement and buy-in. could use different approaches.
exposure notification apps. • Standards and practices could be
challenging to oversee and enforce.
Best Practices (report page • Best practices could help authorities better promote • Best practices could require consensus from
43) app adoption. many public- and private-sector
Policymakers could promote • Best practices could help state public health stakeholders, which can be time- and
best practices for authorities by providing information on procedures resource-intensive.
approaches to increasing and potential approaches for distributing • Current best practices may have limited
adoption and to measure verification codes in a timely manner. relevance to a future pandemic.
the effectiveness of • Best practices could help public health authorities • In some cases, stakeholders may lack
exposure notification apps. establish a more rigorous way to measure the sufficient information or the experience to
extent of app use and any resulting improvements develop best practices.
in notifying exposed people.
National Strategy (report • Enhanced national coordination that builds on • A coordinated national approach would
page 44) the underlying infrastructure and lessons likely have associated costs and require
Policymakers could learned from COVID-19 could prompt faster sustained funding during the pandemic.
collaborate to enhance deployment of apps in the future. • Coordination of groups with divergent
the pandemic national • A future national marketing campaign with perspectives and interests may pose
strategy and promote a cohesive and coherent messaging could result challenges to defining outcomes, measuring
coordinated approach in wider adoption. performance, and establishing a leadership
to the development • Policymakers could recommend a national app that approach.
and deployment of public health authorities could decide to use based • It is unclear whether potential users
exposure notification on their individual needs. A national app could add would be more or less likely to trust a
apps. more functions by integrating exposure notification national exposure notification app than
capabilities with test scheduling and vaccine one developed by a state government.
delivery coordination.
Source: GAO. | GAO-21-104622
This is a work of the U.S. government and is not subject to copyright protection in the United States. The published
product may be reproduced and distributed in its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you
wish to reproduce this material separately.
Table of Contents
Introduction ........................................................................................................................ 1
1 Background ...................................................................................................................... 4
1.1 COVID-19 ........................................................................................................................... 4
1.2 Manual contact tracing for COVID-19 ............................................................................... 4
1.3 Roles of states, federal agencies, and other stakeholders ................................................ 6
2 Benefits and Design of Exposure Notification Apps ........................................................ 9
2.1 Exposure notification apps are expected to provide enhanced speed and reach ............ 9
2.2 How exposure notification apps work ............................................................................. 10
2.3 Apps can use either a centralized, decentralized, or hybrid system to manage data.....12
2.4 States are widely using the Google and Apple Exposure Notifications system ..............14
3 Deployment and Adoption of Exposure Notification Apps ........................................... 18
3.1 About half of the states have deployed an exposure notification app ........................... 18
3.2 About half of the states use a customized app ............................................................... 21
3.3 States reported that app development time and cost varied based on
several factors .............................................................................................................. 25
3.4 Officials reported varying download levels and use........................................................ 26
4 Challenges Associated with Exposure Notification Apps ............................................... 27
4.1 Accuracy of measurements ............................................................................................. 27
4.2 Privacy and security concerns ......................................................................................... 28
4.3 Adoption and use of apps ................................................................................................ 31
4.4 Verification code delays .................................................................................................. 32
4.5 Limited evidence of effectiveness ................................................................................... 33
5 Policy Options That Could Help Address Challenges for Future Use ............................. 41
5.1 Policy option: Research and development ...................................................................... 41
5.2 Policy option: Privacy and security standards and best practices ................................... 42
5.3 Policy option: Best practices to measure effectiveness .................................................. 43
5.4 Policy option: Enhance the national strategy .................................................................. 44
Exposure Notification GAO-21-104622 i
6 Agency and Expert Comments ....................................................................................... 46
Appendix I: Objectives, Scope, and Methodology ............................................................ 47
Appendix II: Exposure Notification App Adoption Rates for Selected U.S. States ........... 51
Appendix III: GAO Contacts and Staff Acknowledgments ................................................ 54
Exposure Notification GAO-21-104622 ii
Abbreviations
APHL Association of Public Health Laboratories
BLE Bluetooth Low Energy
CDC Centers for Disease Control and Prevention
COVID-19 Coronavirus Disease 2019
DHS Department of Homeland Security
FCC Federal Communications Commission
FTC Federal Trade Commission
GPS Global Positioning System
HHS Health and Human Services
NIH National Institutes of Health
NIST National Institute of Standards and Technology
QR quick response
WHO World Health Organization
Exposure Notification GAO-21-104622 iii
441 G St. N.W.
Washington, DC 20548
Introduction
September 9, 2021
Congressional Addressees
For more than a century, public health authorities have used contact tracing to track and limit
the spread of infectious diseases. Manual contact tracing involves interviewing infected people
to identify others they have been in contact with, notifying those contacts that may have been
exposed, and advising the infected individual and contacts to take appropriate measures.
Manual contact tracing can be effective, but it has limitations. Specifically, it is a resource-
intensive process, and it is most effective during the early stage of an outbreak, when case
numbers tend to be lower, or during phases with fewer cases. 1 In addition, its effectiveness
relies, in part, on prompt and complete identification of individuals and notification of contacts,
which could be difficult with a rapidly spreading disease. Other limitations of manual contact
tracing include the reliance on human recall of contacts and movements (which can be prone to
error), and the inability to identify strangers. Further, people may not be forthcoming in sharing
information about their contacts.
With the emergence and rapid spread of the highly infectious Coronavirus Disease 2019 (COVID-
19), digital contact tracing technologies have been developed to supplement manual contact
tracing and help address its limitations. One such technology is a type of application (app)
developed for use on a smartphone. 2 Referred to as an exposure notification app, 3 it is intended
to be used to notify a smartphone user who has been in close contact with another user who
later tested positive for COVID-19. 4 This type of app is intended to reduce transmission by
notifying potentially exposed people faster than manual contact tracing, including contacts the
infected person may not have known.
1
Center for Infectious Disease Research and Policy, COVID-19: the CIDRAP Viewpoint, (Minneapolis, Minn.: University of Minnesota,
June 2, 2020).
2
An estimated 85 percent of adults in the U.S. own a smartphone, according to a survey conducted by the Pew Research Center.
However, this decreases to an estimated 61 percent for people 65 and older. In addition, the rate varies based on other factors,
including income level and whether a person lives in an urban or rural area. This estimate was based on a survey of U.S. adults,
conducted between Jan. 25 and Feb. 8, 2021. See Pew Research Center, Mobile Fact Sheet, (Washington, D.C.: Apr. 7, 2021),
accessed July 1, 2021, https://www.pewresearch.org/internet/fact-sheet/mobile.
3
An exposure notification app can include both software that a user downloads to use on a smartphone, or a function built into the
phone’s operating system that can be activated by users.
4
While contact tracers have previously used smartphone apps, including for data entry, we are unaware of any exposure notification
apps in public use prior to the COVID-19 pandemic.
Exposure Notification GAO-21-104622 1
In July 2020, we issued a Science & Tech Spotlight overview of exposure notification app
technology. 5 Since then, there has been an increase in the development and use of these apps.
You asked us to conduct an assessment of these technologies. This report discusses (1) the
benefits and design of exposure notification apps, (2) the current level of deployment in the
U.S., (3) challenges affecting their use, and (4) policy options that could help address key
challenges for future use.
To address all of these objectives, we reviewed documentation and met with officials from
selected federal agencies involved in providing guidance, funding research, and directing other
efforts related to exposure notification apps. In addition, we interviewed representatives from
entities involved in the development of exposure notification apps; public health organizations;
federally funded research and development centers; academic researchers; and
nongovernmental organizations. We also conducted a review of literature discussing exposure
notification apps, including their benefits, design, and challenges, as well as relevant policy
options.
To identify the current level of deployment, we developed an inventory of exposure notification
apps that had been deployed by U.S. states, territories, and the District of Columbia (hereafter
referred to as states) as of June 2021. States that had an app in a pilot phase at the time of our
review were included the category of “states that had not deployed an app as of June 2021.”
To obtain additional information associated with the development and use of these apps, we
interviewed state public health officials from a non-generalizable sample of nine states that had
deployed an exposure notification app as of January 1, 2021: Alabama, Colorado, Connecticut,
Minnesota, Nevada, North Carolina, Pennsylvania, Virginia, and Washington. We also reached
out to two other states (Louisiana and Utah) that deployed apps in the later stages of our
evidence collection and received written feedback to structured questions about the status of
their efforts to deploy an app, their rates of adoption, and other topics. We selected the sample
of nine states to obtain a range of views, based on factors such as deployment date,
geographical distribution, number of COVID-19 cases, and app developer. Because the selection
was based on a non-generalizable sample, the results were not used to make inferences about
all states that had deployed an app. In addition to our interviews with officials from the selected
states, we conducted a review of each of the selected states apps, including the key functions,
features, and privacy use policies for those apps.
To obtain perspectives from states that had not deployed an app, we collected information from
a non-generalizable selection of seven states that had not deployed an app at the time of our
review (Montana, Nebraska, Oregon, Rhode Island, South Carolina, Texas, and West Virginia).
5
GAO, Science & Tech Spotlight: Contact Tracing Apps, GAO-20-666SP (Washington, D.C.: July 28, 2020). While the terms exposure
notification apps and contact tracing apps have been used to describe these technologies, we will use the term exposure notification
apps in this review.
Exposure Notification GAO-21-104622 2
For these states, we conducted an interview with officials from one state and obtained written
responses to a semi-structured set of questions for the other six.
We identified policy options that may address the identified challenges based on our literature
review and interviews. We assessed each policy option by identifying potential benefits and
considerations of implementing them, as identified over the course of our review. See appendix
I for a detailed description of our objectives, scope, and methodology.
We conducted our work from November 2020 to September 2021 in accordance with all
sections of GAO’s Quality Assurance Framework that are relevant to technology assessments.
The framework requires that we plan and perform the engagement to obtain sufficient and
appropriate evidence to meet our stated objectives and to discuss any limitations to our work.
We believe that the information and data obtained, and the analysis conducted, provide a
reasonable basis for any findings and conclusions in this product.
Exposure Notification GAO-21-104622 3
1 Background
1.1 COVID-19 nation. 7 In the U.S., there have been more
than 596,000 reported deaths 8 and 32
The outbreak of COVID-19 was first million reported confirmed and probable
reported on December 31, 2019, in Wuhan, cases as of July 2021. 9 In addition, despite
China. 6 In the weeks that followed, the virus strides made in getting people vaccinated,
quickly spread around the globe. On the threat of variants is growing, including
January 31, 2020, the Secretary of Health evidence of increased transmissibility. As a
and Human Services declared a public result, uncertainty about the future of the
health emergency for the U.S., retroactive COVID-19 pandemic remains.
to January 27, which followed a World
Health Organization (WHO) declaration on 1.2 Manual contact tracing for
January 30 that the outbreak constituted a
public health emergency of international
COVID-19
concern. On March 11, 2020, WHO
characterized the COVID-19 outbreak as a Contact tracing is a key component in
global pandemic due to its levels of spread controlling the transmission and spread of
and its severity. COVID-19 is highly infectious diseases, according to the
contagious and may be spread by people Centers for Disease Control and Prevention
who are not showing symptoms (i.e., (CDC). 10 Contact tracing is intended to
“asymptomatic”) or before symptoms separate the people who have (or may
appear (“pre-symptomatic”). have) an infectious disease from those who
do not and provide information on other
More than a year later, as we have measures the potentially exposed contacts
should take, such as being tested for the
previously reported, the pandemic has
disease or self-isolating. Together, the test,
resulted in catastrophic loss of life and
trace, and isolate strategy is part of the
substantial damage to the global economy,
and to the stability and security of our
6 9
This disease is caused by SARS-CoV-2 (Severe Acute Data on COVID-19 cases in the U.S. are based on aggregate
Respiratory Syndrome, Coronavirus 2). case reporting to the Centers for Disease Control and
7 Prevention, COVID Data Tracker, (Atlanta, Ga.), accessed July
GAO, COVID-19: Key Insights from the GAO’s Oversight of 7, 2021,
the Federal Public Health Response, GAO-21-396T https://covid.cdc.gov/covid-data-tracker/#datatracker-home
(Washington, D.C.: Feb. 24, 2021). , and include probable and confirmed cases as reported by
8 states and jurisdictions. Centers for Disease Control and
CDC’s National Center for Health Statistics COVID-19 death
counts in the U.S. are based on provisional counts from Prevention (CDC) COVID-19 counts are subject to change
death certificate data, which do not distinguish between due to delays or updates in reported data from states and
laboratory-confirmed and probable COVID-19 deaths. territories. According to CDC, the actual number of COVID-
Provisional death counts are incomplete due to an average 19 cases is unknown for a variety of reasons, including that
delay of 2 weeks (a range of 1–8 weeks or longer) for death people who have been infected may not have been tested or
certificate processing. Data include deaths occurring from may not have sought medical care.
January 2020 through the week ending on July 3, 2021. 10
Centers for Disease Control and Prevention, Operational
Centers for Disease Control and Prevention, National Center Consideration for Adapting a Contact Tracing Program to
for Health Statistics, (Atlanta, Ga.), accessed July 7, 2021, Respond to the COVID-19 Pandemic in non-US Settings,
https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm. (Atlanta, Ga.: June 23, 2021).
Exposure Notification GAO-21-104622 4
broader effort to limit the transmission of agencies typically maintain an existing
infectious diseases such as COVID-19. capacity to conduct contact tracing for
infectious diseases, this capacity is generally
Contact tracers are the people who sufficient only to respond to relatively small
manually trace the contacts of each person or isolated outbreaks.
who has tested positive for COVID-19.
Contact tracers begin the process by Contact tracing is resource intensive,
interviewing the person with the positive because, as cases increase, the contact
test result in order to identify others whom tracer will need more time to contact all
that person might have contacted. Next, potentially exposed persons. Hence, more
the tracer advises the person and the and more tracers will be needed to ensure
contacts to take containment measures comprehensive contact tracing of all
(e.g., a 14-day quarantine for COVID-19), diagnosed cases and potentially exposed
and coordinates or provides information on persons. The particular features of the
any needed care, testing recommendations, COVID-19 pandemic—asymptomatic
and resources. 11 persons and the ability to spread rapidly—
require a significantly large workforce of
For COVID-19, CDC defines a close contact contact tracers. According to the National
as anyone who has been within 6 feet of an Association of County and City Health
infected person for a total of 15 minutes or Officials, the benchmark rate is 30 contact
more over a 24-hour period (for example, tracers per 100,000 people. This equates to
three individual 5-minute exposures for a about 98,460 contact tracers needed to
total of 15 minutes). 12 According to CDC, cover the entire U.S. population. 13
infected persons can spread COVID-19 According to CDC, state health departments
starting from 48 hours (or 2 days) before reported a total of 51,855 employed
they have symptoms or test positive for contact tracers for the month of December
COVID-19. 2020, which was about one month before
the peak of reported new cases in the U.S. 14
In a public health emergency such as the
COVID-19 pandemic, it is critical that each To supplement the capabilities of manual
state has a sufficient workforce of contact contact tracing, several states have used
tracers in order to contain the disease. smartphone apps. These apps include those
Although state and local public health that help people monitor their COVID-19
11 13
Consistent with CDC guidance, except in certain National Association of County and City Health Officials,
circumstances, people who have been in close contact with Position Statement: Building COVID-19 Contact Tracing
someone who has COVID-19 should quarantine. However, Capacity in Health Departments to Support Reopening
people who have been fully vaccinated and people who American Society Safely (Washington, D.C.: Apr. 16, 2020).
were previously diagnosed with COVID-19 within the last 3 14
months may not need to quarantine. This number represents a best estimate for a 1-month
snapshot and may not include contact tracers employed at
12
See Centers for Disease Control and Prevention, the local health department or community level, according
“Appendices,” COVID-19, (Atlanta, Ga.: Updated July 2, to CDC documentation. Data are reported monthly, and
2021), accessed July 2, 2021, estimates will continue, and be updated regularly, according
https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t to CDC. We used the reported estimates from December
racing/contact-tracing-plan/appendix.html#contact. 2020 to illustrate capacity just before the peak cases
observed in January 2021.
Exposure Notification GAO-21-104622 5
symptoms, assist people in recalling the Federal agencies
places they had visited when providing that
information to a contact tracer, and Federal agencies—including CDC, the
exposure notification apps. 15 Department of Homeland Security (DHS),
the National Institutes of Health (NIH), and
1.3 Roles of states, federal agencies, the National Institute of Standards and
and other stakeholders Technology (NIST)—have taken various
steps to assist states in the development
Various entities have a role in the and use of exposure notification apps,
deployment and use of exposure including issuing guidance, distributing
notification apps within the U.S. These funds to states, and funding research.
entities include states, federal agencies, and
other stakeholders, such as national public Specifically, in May 2020 and December
health organizations and organizations 2020, CDC issued two guidance documents
involved in research and development of on digital contact tracing tools, which
the apps. included discussion of exposure notification
apps. 16 The guidance is intended to provide
States health departments with minimum and
preferred characteristics of the apps,
including those for contact notification and
In the U.S., public health authorities at the
data security. 17 In addition, CDC distributes
state, territorial, and local levels plan and
funds to states—through established
coordinate pandemic response actions
mechanisms such as its Epidemiology and
within their jurisdictions. In addition, these
Laboratory Capacity for Prevention and
authorities generally lead contact tracing
Control of Emerging Infectious Diseases
efforts, including the implementation of
cooperative agreement, which currently
related technologies, such as exposure
provides funds to 64 jurisdictions to detect,
notification apps.
prevent, and respond to the growing
threats posed by infectious diseases,
including for the development and use of
exposure notification apps, according to
15 16
For example, in April 2020, Utah deployed an app which Centers for Disease Control and Prevention, Preliminary
allowed residents to check their symptoms, and privately Criteria for the Evaluation of Digital Contact Tracing Tools
share a subset of their location information with public for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and
health officials to aid in the contact tracing process. In Guidelines for the Implementation and Use of Digital Tools to
summer 2020, Utah disabled the location–based services in Augment Traditional Contact Tracing, version 1.0 (Atlanta,
this app. Ga: Dec. 15, 2020).
17
These characteristics included that the apps should enable
health departments to define different exposure risk levels
used to identify contacts based on how close and how long
their exposure was and to require user consent before their
data are shared with a health department.
Exposure Notification GAO-21-104622 6
CDC documentation. 18 (See ch. 3 for years. NIH has also funded various projects
additional information on uses of this related to contact tracing tools. 20
funding.)
In early 2020, NIST began work on a project,
CDC has also funded research on exposure which is currently ongoing, to study and
notification apps, including research develop exposure notification systems with
performed by the Massachusetts Institute strong privacy and cybersecurity
of Technology’s Lincoln Laboratory, to protocols. 21 As a part of this project, NIST
examine barriers to adoption and the held an event in June 2020 to help facilitate
efficacy of the underlying technologies used research aimed at improving the
by various apps. Further, CDC officials performance of these kinds of apps.
stated that the agency has provided Further, in January 2021, NIST held a
ongoing support and consultation to states workshop on challenges associated with
interested in implementing exposure exposure notification apps.
notification apps. For example, since early
August 2020, CDC has coordinated with Other stakeholders
Lincoln Laboratory to host meetings with
state public health authorities where they National public health organizations have
can discuss issues related to app issued guidance and provided other support
development and deployment. to state public health authorities to assist in
the development and deployment of
In addition, DHS’s Science and Technology exposure notification apps. These
Directorate provided funding to two organizations include the Association of
projects through its Silicon Valley Public Health Laboratories (APHL),
Innovation Program. 19 These projects are Association of State and Territorial Health
intended to develop criteria the apps can be Officials, Council of State and Territorial
tested against and to enable the capability Epidemiologists, National Association of
to test the apps using the criteria. According County and City Health Officials, Linux
to DHS officials, they expect that these Foundation Public Health, the Public Health
projects will be completed in the next 2 Informatics Institute, and others. Other key
stakeholders include entities involved in the
research and development of exposure
18 19
The 64 jurisdictions receiving awards under the DHS intends for this program to find new technologies
Epidemiology and Laboratory Capacity for Prevention and that strengthen national security.
Control of Emerging Infectious Diseases cooperative 20
agreements include all 50 states, several large metro areas, These projects included tools to identify businesses and
and U.S. territories and affiliates. A full list of recipients is hot spots visited by people with COVID-19 and development
provided on the Centers for Disease Control and of a digital health pass to enable businesses to verify health
Prevention’s website. See Centers for Disease Control and credentials.
Prevention, National Center for Emerging and Zoonotic 21
A system (or protocol) provides a framework that
Infectious Diseases, Division of Preparedness and Emerging determines the function of a particular software application,
Infections, Recipients, Project Officers, and Jurisdictional like an app on a smartphone.
Assignment Listing, (Atlanta, Ga.: last reviewed June 16,
2021), accessed June 30, 2021,
https://www.cdc.gov/ncezid/dpei/elc/advisor-list.html.
Exposure Notification GAO-21-104622 7
notification apps or analysis of their they developed this system to help
performance. Specifically, researchers, governments and the global community
organizations, and technology companies slow the spread of the COVID-19 pandemic.
have played a key role in the design of the In addition, Google and Apple collaborated
systems used by exposure notification with Microsoft and APHL to establish and
apps. 22 host servers to facilitate the system.
Further, Google and Apple collaborated
In May 2020, Google and Apple—the two with the MITRE Corporation to deploy the
primary developers of operating systems Exposure Notification Private Analytics
for smartphones—collaborated on the portal, which provides public health
development of an exposure notification authorities with data on the performance of
system used by the states discussed later in the states’ apps. This effort involves several
the report. According to Google and Apple, other partners, including the Internet
Security Research Group and NIH.
22
These include, for example, the TCN Coalition and
Massachusetts Institute of Technology Pact developed
systems—referred to as the Temporary Contact Numbers
Protocol, or TCN Protocol; and the Private Automated
Contact Tracing (PACT) protocol, respectively.
Exposure Notification GAO-21-104622 8
2 Benefits and Design of Exposure Notification Apps
Exposure notification apps are intended to documentation, and publications that we
automate and augment the manual contact reviewed.
tracing process, with enhanced speed and
reach being among the expected benefits, Speed. Apps are expected to allow for
according to scientific literature, state faster identification and notification of
officials, federal agency documents, and contacts. After a positive test result is
representatives of stakeholder received, apps automate the process of
organizations we interviewed. They work by identifying and notifying contacts. This
using proximity detection to determine automation can lead to faster notification,
when two smartphone users are in close which in turn can lead to faster changes in
contact, then notifying all contacts of a user individual behavior aimed at helping slow
who later reports a positive test result for disease transmission, namely testing and
quarantine, according to selected studies. 23
COVID-19. The apps can use a centralized,
decentralized, or hybrid system to collect,
store, and manage data. Many states within Reach. Apps are also expected to provide
the U.S. are using apps based on a more complete and faster identification of
decentralized system that was developed contacts because, unlike manual contact
jointly by Google and Apple. tracing, they do not rely on a person’s
memory to identify the people they came
into contact with, according to CDC
2.1 Exposure notification apps are
documents. In addition, according to a Pew
expected to provide enhanced Research Center report, 41 percent of
speed and reach Americans asked about their views on
speaking with a public health official
Exposure notification apps are expected to reported that they are unlikely to talk with
provide two key benefits—speed and reach. contact tracers. The report also noted that
Specifically, they are expected to allow for younger adults, those with lower incomes,
more timely identification and notification and those with less formal education are
of contacts and greater (more complete) especially unlikely to engage with manual
coverage of contacts, according to the contact tracers. 24 Apps may provide a way
majority of the selected states, CDC
to increase coverage of these populations.
In addition, apps can reach people even
23 Institute of Technology, Oct. 29, 2020), and J.A. Moreno
John Hopkins University and Association of State and
Territorial Health Officials, A National Plan to Enable Lopez et al., “Anatomy of Digital Contact Tracing: Role of
Comprehensive COVID-19 Case Finding and Contact Tracing Age, Transmission, Setting, Adoption, And Case Detection,”
in the US (Baltimore, Md.: Johns Hopkins University, Apr. 10, Science Advances, vol. 7, no. 15 (2020): eabd8750.
2020), Massachusetts Institute of Technology Lincoln 24
Pew Research Center, The Challenges of Contact Tracing
Laboratory, Realizing the Promise of Automated Contact as U.S. Battles COVID-19, (last updated Oct. 30, 2020),
Tracing Technology to Control the Spread of COVID-19: accessed July 2, 2021,
Recommendations for Smartphone App Deployment, Use, https://www.pewresearch.org/fact-tank/2020/10/30/key-fi
and Iterative Assessment (Cambridge, Mass.: Massachusetts ndings-about-americans-views-on-covid-19-contact-tracing/.
Exposure Notification GAO-21-104622 9
when manual contact tracing resources are signal can receive and store these
limited or overwhelmed. encounter messages. The distance
between two phones can be estimated
2.2 How exposure notification apps by comparing the strength of the BLE
signal when it was sent with its strength
work when it is received. 27
• If one or more of the messages later
Exposure notification apps use proximity
turns out to have been from a contact
detection to determine whether two app
who tested positive for COVID-19, a
users are in close contact. The app then
central server or a user’s smartphone
notifies a person if they had been in close
analyzes the encounter message to
contact with another user who was likely
determine whether the user’s risk of
infectious at the time, and who voluntarily
exposure exceeds a predetermined
confirmed their diagnosis in the app.
threshold. The risk analysis includes
Proximity detection involves a series of
factors such as the time spent at
automated actions that determine the
various distances and when the contact
proximity of two persons, notify them of
occurred in relation to when the
potential exposure, and provide guidance in contact was most infectious. 28
the case of exposure.
The formula used to calculate the level of
The proximity detection steps are described
risk, including the specific risk factors, can
more fully here.
be set by the public health agency. The
assessment generally involves determining
• An exposure notification app whether the encounter meets the CDC’s
periodically broadcasts messages
definition of a close contact (i.e., at least 15
(referred to as encounter messages)
minutes within 6 feet within 24 hours). The
using a wireless radio transmission
apps do not consider other factors that
technology—Bluetooth Low Energy
affect the risk of infection. For example,
(BLE) 25—that contain, among other
they do not consider whether the users
things, a random identifier and the
were wearing masks, or whether the
strength of the signal sent (i.e.,
transmitted power). 26 Any other phone encounter occurred in a well ventilated
location (e.g., indoors or outdoors).
that has the same or similar app
installed and is in range of the user’s
25 27
BLE is a wireless radio transmission technology with a The BLE signal will weaken as the distance between two
range of around 30 feet. BLE started to be included in smartphones increases. The strength of the signal when it is
smartphones in 2011, and is now included on most received is referred to as a received signal strength
smartphones to enable communication between devices, indication measurement.
such as smart watches and wireless headphones. 28
This can be determined based on when the person first
26
To help preserve user privacy and to limit the ability to had symptoms or was tested.
track the movements of other users, the random identifiers
are changed on a periodic basis (e.g., every 10 minutes).
Further, the identifiers do not reveal any personal
information about other users.
Exposure Notification GAO-21-104622 10
If a user’s risk of exposure exceeds the risk of measurements and other potential
threshold, the user receives an exposure technologies.
notification from the app. 29 The exposure
notification can also include other Location data can also be used instead of,
information, such as when the exposure or in addition to, the data gathered using
occurred and the next steps the person the BLE messages. Location data are not
should take, such as getting tested, currently used by U.S. states. However,
monitoring symptoms, and self- other nations (e.g., Israel) have apps that
quarantining. See figure 1 for an example of use Global Positioning System (GPS) data to
an exposure notification message. track and record a person’s location,
including the date and time. 30 Further, apps
The distance measured between two can track user locations by having the user
phones using BLE is only an estimate, and scan a quick response (QR) code at a
its accuracy can be affected by various specific location (e.g., venue, restaurant). 31
factors. See section 4.1 for additional The app then records the location, date,
discussion of factors affecting the accuracy
29 31
In certain systems, a public health provider could provide A QR code is a barcode with the ability to encode different
the notification in lieu of an app notification. types of information. Each location needs to have a unique
30 QR code and it must be accessible (e.g., posted at the
Other methods to determine a smartphone’s location entrance to a building).
include assisted-GPS, the triangulation of cell towers, and
Wi-Fi access point identification.
Exposure Notification GAO-21-104622 11
and time. 32 See figure 2 for an example of a 2.3 Apps can use either a
QR code. centralized, decentralized, or hybrid
system to manage data
Exposure notification apps can use a
centralized, decentralized, or hybrid
architecture for collecting, storing, and
analyzing data. 35 The main difference
between these types of architecture is the
extent to which the information used to
determine exposure is stored and analyzed
on a central server or on an individual
smartphone. These differences affect the
privacy protections built into the system. A
decentralized architecture may help
preserve a user’s privacy more than a
centralized architecture. These types of
architecture are described more fully here.
• In a centralized architecture, most of
A user’s recent locations can be compared the data are stored on a central server,
with a list of locations of people who have which also analyzes the data to
tested positive for COVID-19 to determine determine which users may have been
the risk of exposure. 33 However, GPS exposed. For example, the central
location estimates are only accurate within server collects personal information as
about a 16-foot radius outdoors. In a part of users’ registration and
addition, the accuracy decreases near generates the random identifiers used
buildings, bridges, and trees, and indoors or for the encounter messages. A public
underground. 34 Thus, the location estimates health authority can access this
may not always be reliable in determining information (including information on
whether two people were in close contact. which users were in contact) and
aggregate it to perform further analyses
of the data to identify additional
potential exposures and to identify
potential surges in cases to help inform
32 35
The app uses the phone’s camera to scan the QR code. Data architecture is a framework that comprises of
33 models, policies, rules, and standards that govern the
The list of locations from infected persons can also include collection, storage, arrangement, integration, and use of
locations obtained through manual contact tracing. data in organizations.
34
National Coordination Office for Space-Based Positioning,
Navigation, and Timing, GPS Accuracy, (Washington, D.C.:
last update Apr. 22, 2020), accessed May 17, 2020,
https://www.gps.gov/systems/gps/performance/accuracy.
Exposure Notification GAO-21-104622 12
broader mitigation and response contact tracing efforts, and identifying
efforts. The central server can also where infections may be occurring. For
incorporate data from other sources, example, this architecture does not
such as manual contact tracing (e.g., allow authorities to know who received
locations an infected person visited). an exposure notification.
However, storing data on a centralized
• A hybrid architecture incorporates
server can also reveal potentially
aspects of both architectures.
sensitive information to governmental
Specifically, the random identifier
organizations, or others who gain
generation for encounter messages
access to the server. An example of a
remains decentralized (i.e., handled by
centralized app is one used by the
user smartphones) to help preserve
nation of Singapore, which it deployed
privacy, while the risk analysis and
in March 2020.
notifications are handled by the central
• In a decentralized architecture, most of server. Hybrid systems have been
the data are located on users’ developed by researchers, but we are
smartphones, with only limited data on not aware of their use at a national or
a central server. Each user’s device state level.
analyzes the data to determine whether
an exposure has occurred. This Table 1 provides a comparison of the
approach may help preserve personal different architectures, including how data
privacy; however, it also limits the data are managed and the key advantages and
that are available to public health disadvantages to each approach.
authorities for determining the
effectiveness of the apps, informing
Exposure Notification GAO-21-104622 13
Table 1: Advantages and disadvantages of centralized, decentralized, and hybrid data architectures
used in exposure notification apps
Centralized Decentralized Hybrid
Where most information is central server smartphone device smartphone device
stored
Where random identifiers central server smartphone device smartphone device
are generated
Where exposure data are central server smartphone device central server
analyzed
Level of data access by higher lower moderate
public health authorities
Key advantages Public health authorities Seeks to preserve Seeks to preserve
can access data to individual privacy by individual privacy and
perform analysis, limiting data accessible to provides health
identify additional entities (e.g., public health authorities with useful
exposure and potential authorities). data.
surges, and inform
response efforts.a
Key disadvantages Data could reveal Limits the data that are Data could reveal
potentially sensitive available to public health potentially sensitive
information to public authorities for determining information to public
health authorities or how well the app works health authorities or
other entities that gain and to inform response other entities that gain
access to the server. efforts. access to the server.
Source: Based on GAO review of technical and industry documentation. I GAO-21-104622
aThe central server can also incorporate data from other sources, such as manual contact tracing (which can provide
the location of an infected person among other things).
2.4 States are widely using the Google customizing their own exposure notification
and Apple Exposure Notifications apps.
system
In September 2020, Google and Apple
provided public health authorities with an
U.S. states with apps are using the Google and
additional option (referred to as the Express
Apple Exposure Notifications system. 36 (See
option), which was intended to make it easier
ch. 3 for additional information on for authorities to use the Google Apple
deployment of apps by state public health system by removing the need for the
authorities.) The Google Apple system was authorities to build their own custom apps. In
released in May 2020 as an application- this option, Google developed an app for
programming interface to be used by public Android-based phones, and Apple deployed
health authorities in developing and app-less functionality, such that a person can
36
For the purposes of this report we refer to the Google and
Apple Exposure Notifications system as the Google Apple
system.
Exposure Notification GAO-21-104622 14
enable the system for their area (if available) encounter messages. If there is a match, the
through the settings on an iPhone. 37 app analyzes the risk of exposure based on
the method and parameters established by
The Google Apple system uses BLE and a the public health authority. If the risk exceeds
decentralized architecture. In this system, a predetermined threshold, the app displays
each user’s app creates a temporary key an exposure notification to the app user. The
(changes every 24 hours) that the app uses to notification can include guidance and
generate random identifiers, and to encrypt instructions. Figure 3 provides an overview of
information provided in the encounter this process.
messages. The app then exchanges a random
identifier with other users’ apps, and In August 2020, Microsoft partnered with
maintains a list of the encounter messages APHL to establish a key server that could be
that the user has received. To help preserve used by all U.S. states—the National Key
the privacy of the users, the encounter Server. With its launch, apps from different
messages do not include personal information U.S. states could be interoperable, so that app
or location data. users can find out if they have been exposed
without needing to download and use apps
If the user tests positive for COVID-19, a from multiple states. 38 This feature is
public health authority uses a verification particularly important in regions where
server to generate a verification code, and commuters regularly cross jurisdictional
then sends the user the code to verify the boundaries (e.g., in neighboring areas of
positive test result. A user can then Washington D.C., Maryland, and Virginia).
voluntarily input this code in the app to Further, according to APHL, the server also
submit their recent temporary keys (e.g., reduces the burden of each state’s public
prior 14 days) to a key server. If a user health agency needing to build and host its
chooses not to input the code in the app, the own key server.
user will not enable the notification of other
recent close contacts who were also using the
In addition to the National Key Server, APHL
app of potential exposure.
manages a central verification server,
referred to as the Multi-tenant Verification
Each exposure notification app periodically Server, which was launched in September
downloads the temporary keys from people 2020. APHL made the verification server
who had recently tested positive from the key available to reduce the effort needed by
server and then compares it with its list of public health agencies to bring an exposure
37 38
The Express option works differently on iPhones and Android App interoperability means that a person using an app from
phones. For iPhones, the Express option is built into the one state could receive an exposure notification based on an
operating system and can be activated by users in the iPhone encounter with any other person who had an app that also
settings. To receive exposure notifications, no app is required used this server, such as a person from another state.
and therefore it is said to have “app-less” functionality. For
Android smartphones, states develop the configuration
settings (e.g., risk parameters), and Google then develops the
app, which states can then use. For Apple devices, the Google
Apple system works on iPhones running at least iOS 12.5. For
Android, the system works for any smartphone capable of
running Android version 6.
Exposure Notification GAO-21-104622 15
notification app to their jurisdiction. As a part
of the Google Apple system, APHL noted that
a verification server is necessary to ensure a
user has received a positive test result before
uploading their temporary keys to the
National Key Server. APHL also noted that
rather than each public health agency
standing up its own verification server and
deciding on a verification approach, providing
one verification server reduces the time and
cost to deploy the Google Apple system. For
the states and territories with apps, nearly all
were using the National Key Server, while
over two-thirds were using the Multi-tenant
Verification Server as of August 2021,
according to APHL. 39
39
Although a verification server is necessary, it does not have
to be the Multi-tenant Verification Server, so some states
elected to use their own verification servers.
Exposure Notification GAO-21-104622 16
Exposure Notification GAO-21-104622 17
3 Deployment and Adoption of Exposure Notification Apps
Almost half of U.S. states have deployed an 3.1 About half of the states have
exposure notification app. In the absence of a deployed an exposure notification
national exposure notification app, states app
have independently launched their own apps
at different times, resulting in a staggered
As of June 2021, 26 of 56 U.S. states
rollout. States have developed and deployed (including territories and the District of
apps using the Google Apple system and Columbia) have deployed an app (see fig. 4).
about half have customized their apps, which Unlike other countries, the U.S. does not have
provides the apps with more flexibility and a national exposure notification app; instead,
functionality. According to officials from states have independently deployed
selected states, development time, costs, and individual apps.
levels of adoption have varied.
Exposure Notification GAO-21-104622 18
The patchwork of app deployment shown in The 26 states deployed apps over a span of 10
figure 4 arises from the fact that public health months, in a staggered rollout beginning in
authorities at the state and territorial levels August 2020. Figure 5 provides a timeline of
decide whether and when to deploy an app. app deployment and related events. Virginia
Furthermore, there was no existing option for was the first state to deploy an exposure
a national app that states could use, notification app, in August 2020, and
according to CDC documentation. As of June Massachusetts was the most recent, in June
2021, 26 out of 56 states had deployed apps. 2021. Seventeen of the states deployed an
Officials from seven selected states that had app between October 2020 and June 2021,
not deployed an app cited several reasons for which was after the Express option was made
that decision, including limited cell phone available (see fig. 5).
coverage in rural areas or other challenges
related to the deployment and use of an app
(see ch. 4 for additional detail). They also
cited competing priorities, such as natural
disaster response or vaccine distribution
efforts, and were concerned that exposure
notification app development would divert
limited resources away from other priorities.
Exposure Notification GAO-21-104622 19
Note: The timeline indicates the Google Apple exposure notification option that was initially deployed by the state
(i.e., custom or Express). Seven states deployed the Express option after initially deploying a custom app (Minnesota,
Nevada, Hawaii, New York, Virginia, Louisiana, and New Jersey) between January and April 2021. In addition to
exposure notification apps based on the Google Apple system, a few states developed smartphone apps to help
people monitor their COVID-19 symptoms or assist in recalling the places they had visited when providing that
information to a contact tracer. For example, in April 2020, Utah deployed an app which allowed residents to check
their symptoms, and privately share a subset of their location information with public health officials to aid in the
contact tracing process. In summer 2020, Utah disabled the location-based services in this app.
Exposure Notification GAO-21-104622 20
3.2 About half of the states use a
customized app
Of the 26 U.S. states that had deployed
apps as of June 2021, all are using a version
of the Google Apple system. 40 With this
system, public health authorities can
choose to develop customized apps, use the
Express option, or use both in tandem (see
table 2). States could use, for example, a
customized app for Android and the Express
option for iOS smartphones. 41
40 41
In addition to the U.S., most countries with an app use BLE One of the selected states uses a customized app for both
(primarily using the Google Apple system); approximately Android and iOS and also enabled the Express option for iOS.
one-third use GPS. Some countries use both BLE and QR
codes; for example, the United Kingdom’s National Health
Service’s app uses both the Google Apple system and QR
codes to check-in to locations.
Exposure Notification GAO-21-104622 21
Table 2: U.S. states deployment of apps using the Google Apple Exposure Notifications system
States Customized app Express option
Alabama ● ○
Arizona ● ○
California ○ ●
Colorado ○ ●
Connecticut ○ ●
Delaware ● ○
District of Columbia ○ ●
Guam ● ○
Hawaii ● ●
Louisiana ● ●
Maryland ○ ●
Massachusetts ○ ●
Michigan ● ○
Minnesota ● ●
Nevada ● ●
New Jersey ● ●
New Mexico ○ ●
New York ● ●
North Carolina ● ○
North Dakota ● ○
Pennsylvania ● ○
Utah ○ ●
Virginia ● ●
Washington ○ ●
Wisconsin ○ ●
Wyoming ● ○
Total 16 17
Legend: ● = State or territory that is using a version of the Google and Apple Exposure Notifications system; ○ =
State or territory that is not using the identified version of the Google Apple system.
Source: GAO compilation of data from selected states, related documents, interviews, and other sources. I GAO-21-104622
Note: State public health authorities can deploy customized exposure notification apps, which may offer unique
functions and features. State public health authorities can also elect to use the Express option of the Google Apple
system. The Express option provides convenience and efficiency but potentially less flexibility to tailor exposure
notification functionality. The total in the table (33) does not equal the number of states with apps (26) because some
states use both a customized app and the Express option.
To build a customized exposure notification university partners). For the Express option,
app, public health authorities can seek public health authorities can provide Google
external technical support (e.g., third-party and Apple an electronic configuration file that
developers, nonprofit organizations, or includes instructions and content, including
Exposure Notification GAO-21-104622 22
the risk parameters for enabling an exposure For example, a customized app may help
notification and messaging for app users. users identify testing facilities and access
While states can use their own internal state-level statistics about COVID-19
technical team or seek outside help to infections and death rates. State officials
develop their app, officials from all nine of the noted that they included these features to
selected states we interviewed said they had provide information to the public outside of
limited technical expertise and resources and their agencies’ websites, which they hoped
received varying levels of external support to would encourage people to download and
deploy their apps, regardless of whether they use their app. Based on our observations of
used customized apps or the Express option. exposure notification apps for the selected
states, a common customized function was
According to Google and Apple the ability to share the app with others.
representatives, the Express option was Figure 6 shows screenshots for two
developed to help states quickly and easily customized apps. These images illustrate how
deploy their app. However, the Express an app can be tailored to offer unique
option does not offer states the same functions in the user interface. For example,
flexibility to tailor the functions and features one screenshot illustrates a unique function,
of their app as do customized exposure “Healthcheck,” which allows app users to
notification apps. report any COVID-19 related symptoms,
exposure history, and testing history to their
public health authority.
Exposure Notification GAO-21-104622 23
Note: The number in the image on the left (490238) is an illustration of a verification code that would be provided by
a public health authority to an app user to verify the positive results of a COVID-19 test. A user can then voluntarily
input this code in the app to submit their recent temporary keys.
Exposure Notification GAO-21-104622 24
Based on our review of the apps for the nine preparing marketing campaigns, and choosing
selected states, among other qualitative to pilot the app prior to the full release. States
differences between state apps, we noted that chose the Express option generally noted
variation in the depth and scope of guidance shorter development times.
information provided to app users.
Specifically, some states provided more The cost of app development also varied
detailed information on symptoms, testing, according to the information reported to us
and quarantine. We also found that the apps’ by officials from the 11 states. 44 One state
privacy use agreements provided varying reported zero development costs because a
details on how the apps protect privacy, nonprofit organization developed the state’s
including how users can delete their data. 42 app, while another state reported
development costs of $700,000. 45 State
officials noted that their marketing costs also
3.3 States reported that app
varied; costs ranged from $380,000 to $3.2
development time and cost varied million, as of June 2021.
based on several factors
States used federal funding for development
Officials from each of the nine selected states and marketing costs; some used state funding
we interviewed and the two additional states as well. Six of the nine states reported that
that provided written information varied in they used CARES Act funding to support the
their reported app development time frames development of their apps or marketing costs.
and costs. 43 Some public health authorities However, according to the CDC, which
from these states attributed these variations distributes certain CARES Act and
to several factors, including legal review and supplemental COVID-19 relief funds through
marketing efforts. Officials from nine of the
its Epidemiology and Laboratory Capacity for
11 states reported that the time to develop
Prevention and Control of Emerging Infectious
their apps ranged from less than 2 months to
Diseases cooperative agreement, 46 exposure
over 5 months. This time included the
notification apps are allowable expenses
development of the apps, as well as
conducting legal reviews of contracts, through these awards, but the agency does
42 45
Some of the exposure notification apps had embedded links We did not independently verify the states’ reported costs.
to the public health authorities’ websites, which provided 46
access to the state’s privacy use agreement or other As part of the Coronavirus Aid, Relief, and Economic Security
information. Act (CARES), Coronavirus Preparedness and Response
Supplemental Act, and Paycheck Protection Program and
43
We interviewed state public health officials from a non- Health Care Enhancement Act supplements, the cooperative
generalizable sample of nine states that had deployed an agreement awarded approximately $11 billion to support the
exposure notification app as of January 1, 2020: Alabama, domestic response to COVID-19. See CARES Act, Pub. L. No.
Colorado, Connecticut, Minnesota, Nevada, North Carolina, 116-136, 134 Stat. 281 (2020); Coronavirus Preparedness and
Pennsylvania, Virginia, and Washington. We also reached out Response Supplemental Appropriations Act, 2020, Pub. L. No.
to two other states, Louisiana and Utah, which deployed apps 116-123, 134 Stat. 146 (2020); Paycheck Protection Program
in the later stages of our evidence collection about the status and Health Care Enhancement Act, Pub. L. No. 116-139, 134
of their efforts to deploy an app and received written feedback Stat. 620 (2020). An additional award of $19.11 billion from the
to our structured questions. Coronavirus Response and Relief Supplemental Appropriations
44 Act of 2021, Pub. L. No. 116-260, Div. M was awarded to
This includes the nine selected states we interviewed and the continue to shore up domestic response efforts to COVID-19.
other two states that provided written responses to our See Consolidated Appropriations Act, 2021, Div. M, Pub. L. No.
questions. 116-260, 134 Stat. 1182 (2020).
Exposure Notification GAO-21-104622 25
not require recipients to report on use of In addition, different levels of app use were
funds to support exposure notification apps. reported by officials from the nine selected
states we interviewed and the two additional
3.4 Officials reported varying states that provided written information.
Specifically, for two states the number of
download levels and use times their app users received exposure
notifications as of June 2021 were above
Different app download levels (or activations 30,000 (31,000 for one state and 42,000 for
for states using the Express option) were the other), while four states reported
reported by officials from the nine selected notifications that ranged from about 900 to
states we interviewed and the two additional 3,800; the remaining five states did not track
states that provided written information. 47 these data. However, the number of
Specifically, four states reported less than 1 notifications depends on a variety of factors,
million, four states reported 1 to 2 million, including the extent of the app users’
and two states reported more than 2 million contacts.
downloads (or activations), as of June 2021. 48
The other state does not track these data. Further, limited data are available on the
According to Google and Apple extent to which exposure notifications
representatives, states that initially deployed
affected people’s behavior, according to
a custom app and then later added the
public health officials and studies we
Express option, experienced a significant
reviewed. For example, public health
increase in activations. Specifically,
authorities do not know whether app users
representatives stated that the adoption rate
are actually using the app and following
quadrupled for four states that added the
Express option (Nevada, New Jersey, New instructions for next steps contained in the
York, and Virginia). However, there may be alerts. Seven of the nine selected states that
other factors that affect adoption rates. we interviewed said that they did not track
Further, the number of downloads and whether app users actually sought testing or
activations is not an accurate reflection of the medical care based on the receipt of an
number of people using the app. For example, exposure notification from an app; one state
a person could download or activate the app said it was done inconsistently and the other
and not use it, or could download the app remaining state did not respond to our
multiple times. See section 4.5 for additional request for these data.
information on this issue and appendix II for
additional information on state app adoption
rates.
47 48
Download data includes Android and iOS phones in states States with customized apps can calculate download levels
with customized apps; downloads for Android phones in states for Android and iOS smartphones from data obtained from
using the Express option; or “activations” for iOS phones in Google and Apple apps stores. However, for states using the
states using the Express (“app-less”) option. Express option, they can determine downloads for Android
devices but must estimate the number of users that activated
the app on iOS smartphone.
Exposure Notification GAO-21-104622 26
4 Challenges Associated with Exposure Notification Apps
We identified the following five categories the phone is being held or is in a pocket or
of challenges associated with these apps: otherwise obstructed location, and the
position of one phone with respect to the
• Accuracy of measurements
other phone (e.g., if it has a 90 degree
• Privacy and security concerns rotation or is facing down).
• Adoption and use of apps
As a result, a person could receive a
• Verification code delays notification even if that person was far
• Evidence of effectiveness away or separated by a physical barrier
from an infected person. Such a result,
known as a false positive, can lead a person
4.1 Accuracy of measurements who has been notified to take unnecessary
steps, such as getting tested, or self-
The techniques that exposure notification quarantining. Further, false positives could
apps use to measure distance have reduce that person’s confidence in the app,
technical limitations that can result in users which could lead to them not using the app
receiving false exposure notifications. For or using it less often. In addition, without
example, BLE wireless radio technology, accurate measurements, an app could fail
used to measure the distance between two to detect that two people are in close
smartphones, cannot always reliably proximity for a certain amount of time,
measure whether two smartphones are leaving the potentially exposed person with
within 6 feet of each other. In addition, a false sense of security—a false negative.
research has demonstrated that the BLE
signal strength does not always decrease To help address these limitations, various
with distance, and can even increase with
industry experts have highlighted the
distance under certain conditions. 49 For
potential of using other technologies to
example, objects in the environment
perform measurements instead of or in
between a sender and a receiver (e.g.,
addition to BLE, including ultra-wideband
furniture, walls, people) can impact the
signals and ultrasound. 50 Several studies
signal, causing the received signal strength
to vary substantially. Other factors include have found that these other technologies
the type of phone and antenna, whether may be more accurate than BLE. 51 In
49 51
See Douglas Leith & Stephen Farrell, “Coronavirus Contact See, for example, N. Ahmed et al., “A Survey of COVID-19
Tracing: Evaluating the Potential of Using Bluetooth Contact Tracing Apps,” IEEE Access, vol. 8 (July 2020):
Received Signal Strength for Proximity Detection,” ACM 134577-134601, accessed December 1, 2020,
SIGCOMM Computer Communication Review, vol. 50, no.4 https://doi.org/10.1109/ACCESS.2020.3010226; and J.
(2020); 1-11. Meklenburg et al., “SonicPACT: An Ultrasonic Ranging
50 Method for the Private Automated Contact Tracing (PACT)
Like BLE, ultra-wideband is a wireless radio transmission Protocol,” arXiv.org (Dec. 2020): 1-14, accessed November
technology, but it could provide measurements that are 24, 2020, https://arxiv.org/abs/2012.04770.
more accurate. However, ultra-wideband is only available on
certain newer smartphones. Ultrasound refers to the
transmission of inaudible acoustic pulses in the ultrasonic
frequency range between phones.
Exposure Notification GAO-21-104622 27
addition, researchers have suggested that Despite the privacy protections built into
exposure notification apps could use sensor the apps by Google and Apple, the public
technologies to improve distance may lack confidence that their privacy is
estimation based on BLE measurements, protected, in part, due to a lack of
such as by using a gyroscope, an independent assessments and federal legal
accelerometer, or a magnetometer. 52 For protections for the privacy of app data. In
example, these technologies could help to particular, CDC’s guidance on the
detect the position of the phone. However, implementation and use of exposure
notification apps recommends that the
thus far, there has been only limited use of
apps go through independent security and
these technologies. Specifically, while some
privacy assessments, 53 and that the results
entities with an exposure notification app
be made publicly available. 54 However, we
use ultrasound technology for distance
found that none of the nine selected states
estimation, including several U.S.
had fully implemented this guidance.
universities, as of June 2021, we did not
Specifically, officials from five of the nine
find any exposure notification apps that use
selected states reported that security and
ultra-wideband signals, a gyroscope,
privacy assessments were performed;
accelerometer, or a magnetometer. however, the results were not made
publicly available. The remaining four states
4.2 Privacy and security concerns reported that these assessments were not
performed.
Privacy
Currently there is no federal law that
Officials from all nine of the selected states provides the public with clearly applicable
identified privacy as an important factor in privacy protections for the information that
determining whether to implement an exposure notification apps gather.
exposure notification app and in selecting Specifically, in January 2019, we reported
the system used by the app (i.e., Google that the U.S. did not have a comprehensive
Apple system). In particular, officials stated internet privacy law governing the
that users would likely not adopt an app collection, use, and sale or other disclosure
that collected their personal information, of consumers’ personal information.
including location data. Accordingly, we recommended that
Congress consider developing legislation on
52 document detailing the process and the outcome of the
A gyroscope is a device used for measuring or maintaining
orientation and angular velocity. An accelerometer is a analysis. See Office of Management and Budget, Managing
device used to measure acceleration forces. A Information as a Strategic Resource, Circular A-130
magnetometer is a device that measures the strength and (Washington, D.C.: July 2016).
sometimes the direction of magnetic fields. 54
Centers for Disease Control and Prevention, Guidelines for
53 the Implementation and Use of Digital Tools to Augment
An example of such an assessment is a privacy impact
assessment that is used by federal agencies in response to Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.
requirements in the E-Government Act of 2002. Among 15, 2020) and Preliminary Criteria for the Evaluation of
other things, the assessment is an analysis of how personally Digital Contact Tracing Tools for COVID-19, version 1.2
identifiable information is handled to ensure compliance (Atlanta, Ga.: May 17, 2020).
with applicable privacy requirements and manage privacy
risks. Also, a privacy impact assessment includes a formal
Exposure Notification GAO-21-104622 28
internet privacy that, among other things, Security
would enhance consumer protections. 55
Legislation governing the collection and use To ensure that exposure notification apps
of consumers’ personal information—in function as intended and that user privacy
particular for exposure notification apps— is protected, it is important that developers
could help to safeguard their privacy, and build in security protections. However,
provide the public with greater assurance security assessments of these apps are
that its privacy is protected. However, such limited.
legislation has not been enacted. 56
Security considerations should include the
In the absence of such legislation, individual supporting infrastructure—such as central
companies have set their own privacy servers—and address how the data are
requirements for exposure notification stored and maintained, including
apps, including requirements on the
appropriate authentication and access
collection and use of the data. For example,
controls. Security incidents could lead to
Google and Apple have each established
privacy violations (e.g., identifying or
requirements for their exposure notification
tracking users) or disrupt the functioning of
system and their respective app stores
regarding data collection and privacy. These the app (e.g., inserting false data). This
requirements specify that only the would likely result in the public’s loss of
minimum amount of user data that is confidence in the apps, potentially leading
necessary for response efforts should be to decreased use. Researchers have
collected, and that the data may only be identified a variety of potential threats for
used for such efforts. 57 In addition, the
requirements state that the apps cannot
collect any information to identify or track
the precise location of users.
55 specifically address exposure notification apps or the
GAO, Internet Privacy: Additional Federal Authority Could
Enhance Consumer Protection and Provide Flexibility, associated privacy issues. In addition, at least one state
GAO-19-52 (Washington, D.C.: Jan. 15, 2019). Other federal passed a law regarding the use of location data for contact
laws governing health information, including the Health tracing. Specifically, in June 2020 Kansas passed a law stating
Insurance Portability and Accountability Act, may not that contact tracing shall not be conducted through the use
provide consistent, clearly-applicable privacy protections for of any service or means that uses cell phone location data to
the information that likely would be gathered and used in identify or track, directly or indirectly, the movement of
digital contact tracing activities. See Congressional Research persons. See K.S.A. § 48-961 (2021).
Service, COVID-19: Digital Contact Tracing and Privacy Law, 57
See, for example, Google, Google COVID-19 Exposure
LSB10511 (Washington, D.C.: July 9, 2020). Notifications Service Additional Terms, (last modified May 4,
56 2020), accessed May 16, 2021,
Congress has introduced several bills over its last two
sessions that address aspects of exposure notification apps https://blog.google/documents/72/Exposure_Notifications_
or digital contact tracing tools. Of the bills that have been Service_Additional_Terms.pdf; and Apple, Exposure
introduced, one that was enacted into law related to Notification APIs Addendum, (last revised May 4, 2020),
implementing a national strategy for contact tracing and accessed May 16, 2021,
enhancing information technology and data modernization https://developer.apple.com/contact/request/download/Ex
capabilities (American Rescue Plan Act of 2021, Pub. L. No. posure_Notification_Addendum.pdf.
117-2, § 2401, 135 Stat. 4, 40 (2021)). However, it does not
Exposure Notification GAO-21-104622 29
exposure notification apps. 58 Table 3
identifies several examples of these threats
and their potential effects.
Table 3: Examples of threats and their effects for exposure notification apps
Threat Description Potential Effect
Re- Comparing exposure notifications with The identity of an infected app user is revealed.
identification personal logs of a phone owner’s recent
contacts.
Denial of Broadcasting fake encounter messages to Loss of availability of a smartphone due to the
service consume resources of other smartphones. additional battery power, storage, and processing
time required to store and process the fake
messages.
Phone Tracking a user’s location by analyzing the A user’s location and movements is revealed.
tracking information sent in encounter messages,
such as the random identifiers.
Relay (or Re-transmitting captured encounter Smartphone receives exposure notification despite
replay) messages at the same or a different not coming in close contact with an infected person
location. (i.e., false positive).
Source: GAO review of selected literature. | GAO-21-104622
The Google Apple system includes features apps are a relatively new technology, these
intended to mitigate these threats. In assessments have, as of now, limited data
addition, according to representatives from and results. Further, as previously stated,
these companies, they had a third party the selected states have not provided the
perform a security assessment of the results of independent security assessments
system. There are also ongoing assessments in a public format, as recommended by CDC
on the security of exposure notification guidance. 60
apps, including on the Google Apple
system. 59 However, as exposure notification
58 testing and validation services. In April 2021, the company
See, for example, N. Ahmed et al., “A Survey of COVID-19
Contact Tracing Apps”; Massachusetts Institute of reported that it had identified a vulnerability with apps using
Technology Lincoln Laboratory, “Exposure Notification the Google Apple system. Specifically, the company reported
Security Assessment Considerations,” Lexington, that preinstalled apps could gain access to system logs made
Massachusetts. Unpublished Article; and M. Chowdhury et by exposure notification apps on Android devices. According
al., “COVID-19 Contact Tracing: Challenges and Future to the company, these logs could include information, such
Directions,” IEEE Access, vol. 8 (Nov. 2020): 225703-225729, as whether a person had received an exposure notification
accessed February 16, 2021, and the random identifiers that a smartphone device had
https://ieeexplore.ieee.org/document/9252092. sent and received. Google representatives stated that a fix
for this vulnerability was available as of May 5, 2021, and
59
For example, the Massachusetts Institute of Technology’s that there is no evidence that it was exploited.
Lincoln Laboratory has developed security assessment 60
Centers for Disease Control and Prevention, Guidelines for
considerations for the Google Apple system and conducted a
security assessment of one of the apps used in the states. In the Implementation and Use of Digital Tools to Augment
addition, in February 2021, DHS’s Science and Technology Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.
Directorate’s Silicon Valley Innovation Program awarded 15, 2020).
funding to a company (AppCensus) for a project to develop
Exposure Notification GAO-21-104622 30
4.3 Adoption and use of apps users, which followed other incidents of
misuse of personal information. 62 Also,
States have also faced challenges attracting multiple officials noted that the public was
public interest in downloading (or skeptical of the Google Apple system, since
activating) and using an exposure app. State it is a joint initiative between U.S.
public health officials told us that, in spite technology companies and the government
of their marketing and outreach efforts, that involves personal health information.
getting people to download (or activate) In particular, officials from three states
and use their app is difficult for the indicated that the public expressed
following reasons. concerns about the perceived “big brother”
nature of exposure notification.
Lack of trust. Mistrust of governmental
health authorities and technology Lack of understanding of how apps
companies can lead people to forgo using function. Multiple officials said they
apps, according to literature and state frequently had to counter misinformation
officials. For example, officials from six of about how the apps work and the data they
the 11 states cited public concerns about collect. For instance, officials from one state
the use of apps for government surveillance reported that they emphasized the app's
(e.g., using the apps to track users’ location) use as a public health communication tool
as a leading obstacle to app adoption, 61 because of misinformation describing the
even though these apps do not collect exposure notification app as a data
location data. In addition, the public may collection tool used to surveil and track the
not trust big technology companies with public.
their data. These concerns may be
exacerbated by reported vulnerabilities Also, multiple officials said the public had a
with apps using the Google Apple system. limited understanding about the apps’
Specifically, as previously stated, a company privacy-preserving features. Officials from
reported that other apps on a phone could three states said that they believed the
potentially gain access to sensitive public did not understand the technical
information, including whether a person aspects of the apps, which may include how
had received an exposure notification. In the random identifiers do not reveal
addition, the lack of trust regarding the use personal information. 63 Such
of apps may be intensified by other misunderstandings may contribute to public
incidents where technology companies unwillingness to download or use the apps.
potentially misused consumers’ personal Further, such misunderstandings may also
information. For instance, in April 2018, contribute to hesitance to enter verification
Facebook disclosed that a Cambridge codes for people who had downloaded an
University researcher may have improperly app on their device. Specifically, sometimes
shared the personal data of 87 million of its people receive positive COVID-19 test
61 63
The 11 states include nine from our selected sample plus According to Google and Apple documentation, the
two additional states. random identifiers exchanged with other smartphones are
62 not linked to the app user’s identity or phone number and
GAO-19-52. change on a periodic basis (e.g., every 10 minutes).
Exposure Notification GAO-21-104622 31
results long after the app has been reason they chose not to deploy an app was
downloaded. Because people may not have the lack of necessary supporting
initially understood (or have forgotten) how infrastructure or internet service in rural
the app works, including the apps’ built-in areas. To be effective, exposure notification
privacy preserving features, some app users apps need to be downloaded and used by a
many not want to input their verification critical mass of the general public. While
codes to prompt exposure notifications to the levels of adoption needed to achieve
other people, according to officials from certain measures of effectiveness are not
selected states. well established (see section 4.5), increasing
the number of people using the app should
Lack of awareness of the availability of the result in a greater likelihood that users who
apps. Officials from several states noted come in close contact with an infectious
that it was difficult to make people aware of
person will be notified of potential
the apps. For example, officials from
exposure, according to CDC and the
multiple states noted that they thought
Massachusetts Institute of Technology’s
building awareness in closely connected
Lincoln Laboratory.
communities with influential leaders would
encourage people to download and use an
app. However, one state tried, but was 4.4 Verification code delays
unsuccessful, in recruiting support from
some of these groups, including churches States were challenged in distributing
and a college football program. One official verification codes quickly to the public. As
said the lack of support was a lost previously stated, these codes are used to
opportunity to build awareness and confirm that a person had a positive test or
increase that state’s app adoption rate. diagnosis before they are able to upload
Further, some states reported having their recent temporary keys to the National
minimal resources for marketing, which one Key Server. For people to be notified of
official said resulted in low awareness of potential exposure quickly, these
the state’s app. Three states reported that a verification codes need to be distributed in
federally led national marketing campaign a timely manner and users need to
would have helped promote their app and voluntarily decide to use them to notify
drive higher rates of adoption. Similarly, recent contacts.
officials from a national health organization
reported that a national public awareness Officials from several of the selected states
campaign led by the CDC would help reported that their initial process for
encourage adoption and be more cost- distributing the codes required a public
effective than individual state campaigns. health official, such as a contact tracer, to
provide a person with the verification code
Limited access. Another reason it can be via phone after the person had received a
difficult to get people to download and use positive test result or a confirmed diagnosis.
an exposure notifications app is lack of However, states reported that, due to
access to a smartphone, reliable cellular staffing shortages, in particular as cases
coverage, and broadband internet service, surged, it sometimes took several days to
according to selected states and literature. provide the code. As a result, some app
Indeed, officials in a few states told us one users who had tested positive for COVID-19
Exposure Notification GAO-21-104622 32
were delayed in submitting their website with instructions for how to obtain
verification codes to notify others of the code. State officials reported that this
possible exposure, according to officials automated distribution resulted in an
from selected states. 64 In addition, there increase in the number of verification codes
can be delays in test results being available disseminated to app users. For example,
and provided to health care providers who following implementation of the new
report the results to local or state health process, one state's average distribution of
officials. In particular, earlier in the verification codes increased from 15 to 85 a
pandemic, testing availability and day, according to public health officials. In
turnaround time for results could take a addition, officials from a different state
week or more. Following the receipt of test reported that they had seen an increase in
results, the health care provider or
the number of codes redeemed, and
laboratory then reports the results to local
improved the timeliness of code
or state health officials. Any delays in this
redemption following implementation of
process could also contribute to delays in
the new system. However, another state
public health authorities’ distribution of
verification codes to individuals after the noted that, even after automating the
individual has received the confirmed process, it still took 4 days, on average, for
diagnosis. a person to receive a verification code
following a positive test result.
In addition, a few states noted that contact
tracers did not always follow the state’s 4.5 Limited evidence of
processes for providing app users with a effectiveness
verification code to enter COVID-19 test
results in the app. For example, when a We found limited evidence that exposure
contact tracer was conducting an interview notification apps are effective at enhancing
with a person, they were supposed to ask if the speed or reach of manual contact
the person had downloaded the state’s app, tracing or at reducing the spread of disease.
and if so, to provide them with a
One reason for the dearth of evidence is
verification code. However, officials from
that states collect limited data from
one state stated that this was not always
exposure notification apps due to the
performed.
emphasis on data privacy. In addition, little
to no guidance exists on what data to
To help address this challenge, five of the
collect and how to collect the data. As for
nine selected states implemented an
slowing the spread of COVID-19, studies
automated process to distribute the codes.
have not yet sufficiently demonstrated that
Instead of providing the codes entirely
exposure notification apps are having an
through phone calls, some states also send
effect. CDC and others have, therefore,
text messages with the code or a link to a
64
If the user tests positive for COVID-19, a public health
authority generates a verification code and then sends the
user the code to verify the positive test results. A user can
then voluntarily input this code in the app to submit recent
temporary keys.
Exposure Notification GAO-21-104622 33
highlighted the need for additional research diseases like COVID-19, the speed of
into the effectiveness of exposure contact tracing plays a critical role in
notification apps in preventing the spread reducing disease spread, allowing those
of disease. who may have been exposed to take action
more quickly. States could use the speed of
4.5.1 States collect limited data from notification as one indicator or metric of
exposure notification apps app effectiveness. According to Google and
Apple representatives, the time between
The privacy protections that are when a user submits their temporary keys
incorporated into the functionality of the to the key server and when a person would
existing apps limit the data available to be notified is estimated to be between 4
public health authorities, which reduces the and 10 hours. However, a few factors can
ability to measure and improve the delay the submission, including the amount
effectiveness of the apps. For example, of time it takes to receive a test result (e.g.,
officials from all nine selected states said to time for test processing and reporting to
preserve personal privacy, they do not the health care provider and to the local
collect data on who has installed an app, health officials) and when that person
including who has received an exposure receives and uses the verification code.
notification. In addition, states have limited
data on how well the apps are working, 4.5.2 States lack guidance on measuring
including how changes to the formula used
effectiveness
to calculate the level of risk affects the
number of people provided with exposure
notifications. 65 In addition, states do not States lack guidance for measuring the
effectiveness of exposure notification apps.
collect location data, so they are unable to
Officials from nearly all the selected states
identify where disease spread is occurring.
told us they wanted to gauge the impact of
their apps and assess effectiveness, such as
Furthermore, states do not collect data on the enhanced reach through electronic
the speed of exposure notification, notification. Officials from several of the
according to our review of information selected states said that they had reached
provided by selected states. Metrics on the out to CDC for guidance regarding
speed of notification are not provided as recommended approaches and indicators
part of the Google Apple system, though for measuring app effectiveness, which was
these data could be collected by the states confirmed by CDC officials. However, the
individually, should they choose to, requested information was not available.
according to Google and Apple CDC officials indicated that they considered
representatives. With fast-spreading developing additional guidance to evaluate
65
MITRE is planning to enhance the Exposure Notification
Private Analytics portal with additional features, including
the ability to analyze how changes in risk parameters affect
the number of exposure notifications. The portal and the
exposure notification analytics data it provides are only
available to states using the Express option.
Exposure Notification GAO-21-104622 34
app effectiveness. However, they or reach compared to manual contact
acknowledged there are limited app tracing. 67
evaluation strategies available due to the
lack of data from exposure notification apps The lack of standardized metrics was
and, as a result, they are not planning to identified as a challenge in President
develop additional guidance. Because of the Biden’s National Strategy for the COVID-19
lack of federal guidance, officials from Response and Pandemic Preparedness in
states said they were “on their own” and January 2021. The strategy noted that
began reaching out to other states and states use and report different metrics for
countries that had deployed apps for advice tracking COVID-19 response activities,
and best practices, such as metrics for including contact tracing, and called for
measuring effectiveness. Officials from the common federal metrics to evaluate
majority of the selected states said they progress and the identification of areas
wished there had been additional guidance where additional federal resources should
available, including how to measure app be directed. 68
effectiveness; officials from three states
used the analogy of “building the plane In part due to the lack of federal guidance,
while flying it,” to describe their experience selected states varied in the types of data
deploying their apps with limited direction. that they are collecting to measure the
overall effectiveness of their own apps and
CDC has developed general guidance on have developed their own metrics and
exposure notification apps, such as indicators for determining how well the
minimum and preferred characteristics. 66 apps are working. States are using one or
CDC also developed guidance to measure more of these metrics:
the success of manual contact tracing
efforts, including both process and outcome • App downloads or activations 69
metrics, but has not developed specific • Verification codes issued 70
guidance on criteria to use in measuring
app effectiveness, such as increased speed • Verification codes claimed 71
66 68
Centers for Disease Control and Prevention, Preliminary The White House, National Strategy for the COVID-19
Criteria for the Evaluation of Digital Contact Tracing Tools Response and Pandemic Preparedness, (Washington, D.C.,
for COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and Jan. 21, 2021).
Guidelines for the Implementation and Use of Digital Tools to 69
Augment Traditional Contact Tracing, version 1.0 (Atlanta, As previously mentioned, download data includes Android
Ga.: Dec. 15, 2020). CDC’s May 2020 and December 2020 and iOS phones in states with customized apps; downloads
guidance and website information on digital contact tracing for Android phones in states using the Express option; or
tools did not include a definition for effectiveness or any “activations” for iOS phones in states using the Express
standardized metrics for states or indicators for measuring (“app-less”) option.
app effectiveness. 70
Verification codes issued refers to the codes that are
67 disseminated to app users with a positive COVID-19 test
Centers for Disease Control and Prevention, Evaluating
Case Investigation and Contact Tracing Success, (Atlanta, result. The codes may be provided on the phone by contact
Ga.: May 26, 2020), accessed June 9, 2021, tracers or through other methods, such as text message.
https://www.cdc.gov/coronavirus/2019-ncov/php/contact-t 71
Verification codes claimed refers to the codes entered by
racing/contact-tracing-plan/evaluating-success.html. app users with a positive COVID-19 test result that enables
them to send the recent temporary keys to the National Key
Server to notify others that they may be at risk.
Exposure Notification GAO-21-104622 35
• Exposure notifications generated 72 regard to reach, the number of downloads
gives an approximate, but not actual sense
Yet, officials from eight of the nine selected of the number of app users.
states noted that some of these metrics
provide a limited understanding of app The number of downloads is not an
effectiveness. Data on downloads, accurate reflection of app usage. Officials
verification codes, and exposure from one state speculated that some
notifications provide public health people were downloading the apps out of
authorities some information to gauge how curiosity but never enabling them on their
effective they are. However, these metrics smartphone. Further, after a person
do not indicate how quickly people were downloads the app, they need to perform a
notified of exposure or timeliness relative series of actions for it to be used as
to manual contact tracing alone. With intended (see fig. 7). Therefore, a user
72
Exposure notifications are alerts provided to close
contacts of the app users who confirm a positive test or
diagnosis using a verification code they enter into the app.
However, they are only an estimate, as app users voluntarily
provide this information.
Exposure Notification GAO-21-104622 36
could download but not use the app or according to officials from one state.
download it more than once, according to Further, because states do not track who
public health officials from several selected receives an exposure notification, states
states. Also, people could choose to not have a limited understanding of what
enable receiving exposure notifications; impact, if any, these notifications have on
ignore notifications; and if they do test disease spread and the overall effectiveness
positive, not voluntarily provide that of their apps.
information. While download totals may
represent the possible population of app Nonetheless, public health officials from
users that could receive benefits (i.e., seven of the nine selected states said they
notification) from these apps, these believe the exposure notification apps have
limitations hinder states from been effective and that their apps had been
understanding the effectiveness of using worthwhile. Officials from two states said
exposure notification apps. they think adoption even at relatively low
levels would help slow disease spread.
Verification codes claimed by the people Furthermore, exposure notification apps
who have tested positive for COVID-19 and provided a new tool for states to use—at a
who entered the information in the app time of urgent need—to limit the spread of
(which prompts exposure notifications to be COVID-19.
sent to others) may also provide an
indication of app use. Similarly, the number 4.5.3 Evidence of reduced disease
of exposure notifications gives a sense of spread has been limited but additional
how many people are notified of potential studies are underway
exposure to COVID-19, but it may provide
limited insight into the effectiveness of the We reviewed seven selected modeling
public health intervention because users studies that have sought to measure the
need to voluntarily provide this information effects of the use of apps on the spread of
in their apps. Finally, none of the data COVID-19. However, there are important
indicate whether people changed behavior limitations to these studies—such as limited
as a result of the notification. evidence to support assumptions about
behavioral changes—which hinder the
While people seeking testing or medical ability to draw high-confidence conclusions
care could be asked whether they sought about the apps effectiveness. 73 The studies
testing or care due to an exposure we reviewed generally suggested that the
notification from an app, such information use of exposure notification apps can
may violate a user's expectations of privacy, reduce disease transmission. 74 In general,
73 74
Infectious disease models are simplified versions of reality The selected studies we reviewed covered a range of
that help to characterize disease spread (see GAO-20-372 geographic areas, including Washington State, the United
for an overview of infectious disease modeling). Other types Kingdom, Spain, and Switzerland, and were published
of epidemiological studies of contact tracing apps that could between April 2020 and May 2021. Five of the seven papers
be conducted in real world settings, rather than via are peer-reviewed publications. We identified the papers
modeling, face methodological, logistical, and ethical from our interviews with subject matter experts and a
challenges, including the lack of empirical data, confounding search of the literature. Studies we reviewed include:
factors that affect disease spread, and other issues.
Exposure Notification GAO-21-104622 37
the studies suggested that app usage can isolated. In some cases, particularly with
decrease COVID-19 infections and deaths, studies earlier in the pandemic, these
with the size of the estimated effects assumptions were not grounded in research
depending on the level of app adoption, and were not otherwise well supported. For
among other things. For example, one peer- example:
reviewed study estimated that, when 15
percent of the population used an exposure • Assumptions in one study were that
notification app, infections could be everyone notified of a potential
reduced by approximately 8 percent and exposure would self-isolate, with a 2
deaths by about 6 percent. Another peer- percent drop-out rate each day, and
reviewed study in the United Kingdom that 18 percent of infected people
estimated that a 30 percent app uptake remained asymptomatic, with no
averted approximately one infection for
variation in this rate across age
every four infections that arose over a 4½-
groups. 75 These assumptions were not
month period.
grounded in evidence because little to
none was available at the time.
However, there are significant limitations to
these modeling studies. For example, the • A study of three counties in Washington
models estimated outcomes by relying on State assumed in its simulations that it
assumptions about app usage and would take 2 days from symptom onset
behavioral changes associated with to receive a COVID-19 test result, which
notifications. These assumptions covered the authors characterized as a key
factors such as how many people used an assumption underlying the findings.
app, how many app users had a positive However, in the earlier months of the
test result, and how many app users self-
R. Hinch, et al., Effective Configurations of a Digital Contact D. Menges, et al., “A Data-Driven Simulation of the Exposure
Tracing App: A Report to NHSX, (April 16, 2020), accessed Notification Cascade for Digital Contact Tracing of SARS-CoV-
December 9, 2021, 2 in Zurich, Switzerland,” JAMA Network Open, (4
https://cdn.theconversation.com/static_files/files/1009/Rep (4):e218184), (April 30, 2021), accessed July 13, 2021,
ort_-_Effective_App_Configurations.pdf?1587531217. https://jamanetwork.com/journals/jamanetworkopen/fullar
M. Abueg, et al., “Modeling the Effect of Exposure ticle/2779376.
Notification and Non-pharmaceutical Interventions on Massachusetts Institute of Technology Lincoln Laboratory,
COVID-19 Transmission in Washington State,” npj Digital “Simulated Automatic Exposure Notification (SimAEN):
Medicine, (4, 49), (March 12, 2021) accessed March 12, Exploring the Effects of Interventions on the Spread of
2021, https://www.nature.com/articles/s41746-021-00422- COVID,” Private Automated Contact Tracing (PACT)
7. Technical Report #3, (December 8, 2020), accessed March 1,
C. Wymant, et al., “The Epidemiological Impact of the NHS 2021, https://pact.mit.edu/simulated-automatic-exposure-
COVID-19 App,” Nature, Vol. 594, no. 7863 (2021).pp. 408- notification-simaen-exploring-the-effects-of-interventions-
412, accessed February 25, 2021. on-the-spread-of-covid-wlogos/.
75
P. Rodríguez, et al., “A Population-Based Controlled Estimates of the COVID-19 asymptomatic rates vary
Experiment Assessing the Epidemiological Impact of Digital widely by age group, according to information from CDC.
Contact Tracing,” Nature Communications, (January 26, Centers for Disease Control and Prevention, “Estimated
2021), accessed February 22, 2021, Disease Burden of COVID-19,” COVID-19, (Atlanta, Ga.:
https://www.nature.com/articles/s41467-020-20817-6. updated May 19, 2021), accessed July 13, 2021,
S. Marcel, et al., “Early Evidence of Effectiveness of Digital https://www.cdc.gov/coronavirus/2019-ncov/cases-updates
Contact Tracing for SARS-CoV-2 in Switzerland,” Swiss /burden.html.
Medical Weekly, (December 16, 2020), accessed March 9,
2021, https://smw.ch/article/doi/smw.2020.20457.
Exposure Notification GAO-21-104622 38
pandemic, wait times for test results in underway that suggest the use of apps can
U.S. could be a week or more. 76 help mitigate the spread of COVID-19. 77 In
addition, some states are conducting their
• Oxford University researchers
own evaluations of the effectiveness of
associated with the United Kingdom
exposure notification apps in reducing
studies told us that the recent rise in
disease spread.
variant strains and vaccinations has
increased uncertainty in assumptions
about disease transmission. 4.5.4 CDC and others have highlighted
the need for additional research and
In addition to the studies on disease spread, data
some studies have estimated shorter-term
outcomes, such as the number of close Exposure notification apps are a relatively
contacts detected by exposure notification recent public health intervention. As a
apps. In one simulation study, the findings result, additional primary research on the
implied that the app prompted quarantine benefits and effectiveness of exposure
notification apps is needed, according to
recommendations for, at most, an
CDC and other public health researchers. 78
estimated 5 percent more exposed contacts
This includes a need for primary research
than manual contact tracing. However, as
into the use of digital tools in conjunction
with the modeling studies we reviewed on
with manual systems, since public health
disease spread, these studies of shorter-
authorities are unlikely to use digital tools
term outcomes are also subject to
in isolation, according to all selected states
important limitations, such as model inputs and most literature we reviewed.
derived from studies with limited sample Specifically:
sizes or national estimates applied to states
or local regions.
• CDC has noted that more data are
needed from preliminary
Since we originally identified papers for our implementation efforts to quantify the
review, additional studies are now public health value of these apps. 79
76 78
D. Lazer, et al., “Report #8: Failing the Test: Waiting Times A. Anglemyer, et al., “Digital contact tracing technologies
for COVID Diagnostic Tests Across the U.S.” in The State of in epidemics: a rapid review.” Cochrane Database of
the Nation: A 50-State COVID-19 Survey, (OSF Preprints, Systematic Reviews. (2020).
August 2020), accessed July 13, 2021, 79
https://doi.org/10.31219/osf.io/gj9x8. Centers for Disease Control and Prevention, Guidelines for
the Implementation and Use of Digital Tools to Augment
77
See, for example, C. Segal, et al., Early Epidemiological Traditional Contact Tracing, version 1.0 (Atlanta, Ga.: Dec.
Evidence of Public Health Value of WA Notify, a Smartphone- 15, 2020).
based Exposure Notification Tool: Modeling COVID-19 Cases
Averted in Washington State (June 2021), accessed July 1,
2021,
https://www.medrxiv.org/content/10.1101/2021.06.04.212
57951v4; and J. Masel, et al., Quantifying meaningful
adoption of a SARS-CoV-2 exposure notification app at the
campus of the University of Arizona (June 2021), accessed
June 1, 2021,
https://www.medrxiv.org/content/10.1101/2021.02.02.212
51022v6.
Exposure Notification GAO-21-104622 39
Also, the agency has highlighted the effectiveness of exposure notification
need for more studies on the apps is needed, since there is currently
effectiveness of digital tools, including a limited understanding of the extent to
exposure notification apps, to support which apps may have changed the
contact tracing and reduce the spread course of the pandemic in the U.S. 81
of infectious disease. In addition, CDC The organization called for a data
has identified a specific research need driven approach and research on app
to comprehensively compare the efficacy to help app developers and
effectiveness of manual contact tracing others, including the federal
with exposure notification apps and has government, decide whether to
initiated work to study these issues with improve apps for potential future use or
the Massachusetts Institute of abandon the approach if research
Technology’s Lincoln Laboratory, showed that desired outcomes had not
according to CDC officials. been achieved.
• WHO has called for additional research
on the minimum adoption levels Moreover, the need for additional research
required for these apps to be effective on the effectiveness was identified as a
in light of the limited evidence to challenge in President Biden’s National
date. 80 In addition, since some Strategy, issued in January 2021. The
populations have limited access to strategy notes that the federal government
digital technology, WHO identified the should work with public health authorities
potential for the systematic exclusion of and the private sector to collect COVID-19
individuals who cannot access such data on a range of issues, including the
effectiveness of contact tracing. 82
technologies. It called for additional
research and sufficient regulatory
oversight of these issues.
• Linux Foundation Public Health noted
that additional research on the
82
The White House, National Strategy for the COVID-19
80
World Health Organization, Contact Tracing in the Context Response and Pandemic Preparedness, (Washington, D.C.,
of COVID-19, Interim Guidance, February 1, 2021. Jan. 21, 2021).
81
Linux Foundation Public Health was founded in summer
2020 with an initial focus on helping public health
authorities deploy apps based on the Google Apple system.
Exposure Notification GAO-21-104622 40
5 Policy Options That Could Help Address Challenges for Future Use
We identified four policy options that, when result in users receiving false exposure
implemented, could help address the notifications, such as by improving the
challenges we have identified for both accuracy of the distance measurements
current and future use of exposure performed by exposure notification apps.
notification apps. 83 Policymakers could also For example, apps could use additional
choose to maintain the status quo—that is, sensors (e.g., the gyroscope and
allow current efforts to proceed without magnetometer that certain smartphones
intervention. The relevant policymakers already have) or other technologies, such as
ultra-wideband and ultrasound. In addition,
could include Congress, other elected
research could examine methods for
officials, federal agencies, state and local
evaluating other factors that affect the risk
governments, academic research
of disease transmission, such as whether
institutions, and industry. While some
the encounter occurred indoors or
challenges described in this report may be
outdoors.
addressed through current efforts, other
challenges may not be resolved, may be
Policymakers could promote research in
exacerbated, or may take longer to resolve multiple ways, including by providing grants
without intervention. The four policy to academic and research institutions or by
options are in the following areas: research setting up a public-private partnership.
and development, privacy and security, Further, the research could build off of prior
data collection and measurement, and and ongoing research by various entities.
national strategy.
Opportunities
5.1 Policy option: Research and
• Research on technological limitations,
development such as inaccurate distance
measurements, could help increase the
Policymakers could promote research and accuracy and speed of exposure
development to address technological notification apps, incentivizing users to
limitations. download and use them.
• Research on technologies and
Description
architectures other than those used by
U.S. states could also improve the apps,
Research could seek to address the
technical limitations we identified that can for example by increasing the speed
83 that would be needed to fully implement a specific policy
We present policy options that were within the scope of
this technology assessment. This is not an exhaustive list of option or combination of options—for instance, on potential
all potential policy options, nor are policy options intended design and legal issues—nor did we assess how effective the
to be recommendations to federal agencies or matters for options may be. We express no view regarding the extent to
congressional consideration. They are not listed in a specific which legal changes would be necessary to implement them.
rank or order, and we are not suggesting that they be
completed individually or combined in any particular
fashion. We did not conduct the detailed additional analysis
Exposure Notification GAO-21-104622 41
and reach of notifications. Such smartphone technology would require
alternatives include GPS and centralized ongoing research.
or hybrid data architecture.
• Partnerships with technology 5.2 Policy option: Privacy and
companies could help with integrating security standards and best
improvements into smartphone practices
operating systems. These collaborations
could spur further technological Policymakers could promote uniform
innovation. privacy and security standards and best
Considerations practices for exposure notification apps.
• Research into new technologies could
Description
be costly and is generally considered a
long-term investment with uncertain
Policymakers could support the
benefits. development of privacy and security
• The roles for government, the private standards and best practices for exposure
sector, and academia in researching notification apps to ensure that these apps
new technologies for exposure function as intended and that user privacy
notification apps would need to be is protected. One way to do this would be
defined, planned, and coordinated to to specify standards for public health
ensure that research and costs are not authorities to ensure that personal data are
duplicative. encrypted when stored, and to specify
limits on the types of data that can be
• Research may not produce cost- collected and how the data may be used
effective improvements, because and disclosed. In addition, the standards
existing apps may still be sufficiently could specify that the data can only be used
accurate for notifying a person of for disease response efforts and that
potential exposure. Moreover, other personal data cannot be shared with other
alternative technologies also have agencies, law enforcement, or immigration
accuracy limitations, and other data authorities without a user’s consent.
architectures may increase the risk of
revealing sensitive user information. Another action policymakers could take is
to require that standards be developed and
• Research into new technologies based
agreed on by a broad coalition of
on the COVID-19 pandemic may also
stakeholders. Best practices could also be
result in apps that are not functional for developed by government agencies (e.g.,
future outbreaks or pandemics. NIST) or the private sector.
Diseases that are not transmitted
through the air, such as sexually Opportunities
transmitted diseases, would require
apps that use different methods to • Developing and adopting uniform
determine potential exposure. In privacy and security standards and
addition, the continuous changes in related best practices could help
address real and perceived risks that
Exposure Notification GAO-21-104622 42
the public’s data might be misused or Description
otherwise not appropriately protected.
Policymakers could assess the approaches
• Standards developed and agreed on by
that states have used to increase adoption
a broad coalition of stakeholders could
and then develop best practices based on
increase the likelihood of still broader
those results. Best practices could also
stakeholder agreement and buy-in.
include standardization of the metrics
• Independent security and privacy collected and reported to measure
assessments could evaluate apps based effectiveness as well as the procedures for
on these standards and best practices, verification code distribution. The
and these assessments could be made development of best practices could be led
publicly available. by a broad coalition of stakeholders and
result in guidance to states. These efforts
Considerations could help address the challenges we
• Policymakers would need to balance identified related to app adoption,
the need for privacy and security with verification code delays, and efficacy
the direct and indirect costs of determination.
developing and implementing these
standards and practices. Opportunities
• Implementing these privacy
• Best practices could help state public
requirements may require flexibility
health authorities share strategies to
because different jurisdictions could
improve app adoption. For example, if a
use different technologies (e.g., BLE or
state found that translating the app into
GPS) and data architectures to collect
multiple languages improved adoption
and use the data.
among non-English speaking people,
• It could be challenging to determine this information could be shared with
how to oversee and enforce the privacy other states. Also, understanding and
and security standards and practices. appealing to user motivations could
promote app adoption. Further,
5.3 Policy option: Best practices to partnering with trusted sponsors could
measure effectiveness encourage cooperation in COVID-19
contact tracing, as published by the
Policymakers could promote best practices National Academies of Sciences,
to increase adoption and measure the Engineering, and Medicine. 84
effectiveness of exposure notification apps. • Such practices could help state public
health authorities by providing
information on potential methods and
84The National Academies of Sciences, Engineering, and
Medicine, Encouraging Participation and Cooperation in
Contact Tracing: Lessons from Survey Research,
(Washington, D.C.: Aug. 2020).
Exposure Notification GAO-21-104622 43
processes for distributing verification deployment, and use of exposure
codes in a timely manner. notification apps.
• In addition, best practices can help
states to measure the effectiveness and Description
impact of these apps. Best practices
could also leverage outside knowledge Policymakers could evaluate whether to
to promote app adoption. More enhance the current national strategy or a
future pandemic response strategy to
accurate measurement of app
enable a coordinated nationwide approach
effectiveness would help public health
to the development and deployment of
authorities identify opportunities for
exposure notification apps. 85 This could
improvement, both to the technology’s
help address the challenges we identified
function and to its widespread use.
related to the adoption of these apps and
evidence of their effectiveness. An
Considerations enhanced strategy could include specifying
federal, state, and local roles and
• The creation of best practices could coordination efforts. Further, an enhanced
require consensus from many public strategy could identify what other
and private sector stakeholders, which infectious diseases (e.g., tuberculosis,
can be time- and resource-intensive. measles) may be applicable to exposure
notification apps in the future.
• If the best practices are not updated,
they may not be relevant or useful in a
As part of this strategy, the federal
future pandemic.
government could decide to repurpose
• In some cases, stakeholders may lack apps that were developed for state use
sufficient or complete information or during the COVID-19 pandemic, or use
the experience to develop best comparable technology to develop new
practices. If best practices are put in contact tracing solutions. Policymakers
place without sufficient basis, it could could recommend a national exposure
limit further innovation. notification app that public health
authorities could decide to use based on
5.4 Policy option: Enhance the their individual needs, resulting in a generic
exposure notification app tailored to state
national strategy needs or a federally managed exposure
notification app that is made available for
Policymakers could collaborate to enhance states to use.
the national strategy and promote a
coordinated approach to the development,
85
The White House, National Strategy for the COVID-19
Response and Pandemic Preparedness, (Washington, D.C.,
Jan. 21, 2021).
Exposure Notification GAO-21-104622 44
Opportunities Considerations
• Enhanced national coordination could • Implementing a coordinated national
prompt faster deployment of apps in strategy would likely have associated
the future if that coordination builds costs and require a source of sustained
upon the underlying infrastructure and funding during and after the pandemic.
leverages the lessons learned from
• Without clear roles and responsibilities,
COVID-19.
coordination culd be challenging. For
• A federally led national marketing example, coordination of groups with
campaign with cohesive and coherent divergent perspectives and interests
messaging could result in wider may pose challenges to defining
adoption of exposure notification apps. outcomes and measuring performance
Increased federal promotion and and effectiveness of apps.
support of the exposure notification
• It is unclear whether the public would
apps could potentially help with
be more or les likely to trust and use a
increasing public trust in the apps.
national exposre notification app than
• A national app could allow integration one developed by their state
of exposure notification capabilities government. Some states, including
with other disease prevention and Virginia, Colorado, and California, have
response activities, such as test passed state-wide privacy laws. Due to
scheduling or vaccine delivery the absence of a federal privacy law in
coordination. the U.S., the public may be less likely to
trust the federal government’s privacy
protections.
Exposure Notification GAO-21-104622 45
6 Agency and Expert Comments
We provided a draft of this report to the Departments of Health and Human Services (including
CDC and NIH), Homeland Security, and Commerce, Federal Communications Commission, and
Federal Trade Commission for their review. The agencies provided technical comments, which
we incorporated as appropriate. Representatives from Apple, the Association of Public Health
Laboratories, Google, Massachusetts Institute of Technology’s Lincoln Laboratory, and MITRE
Corporation also reviewed a draft of this product; we incorporated their technical comments as
appropriate.
We are sending copies of this report to the appropriate congressional committees and other
interested parties. In addition, the report is available at no charge on the GAO website at
https://www.gao.gov.
If you or your staff have any questions about this report, please contact Karen L. Howard at
(202) 512-6888 or howardk@gao.gov or Vijay A. D’Souza at (202) 512-6240 or
dsouzav@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. GAO staff who made key contributions to this
report are listed in appendix III.
Karen L. Howard, PhD
Director,
Science, Technology Assessment, and Analytics
Vijay A. D’Souza
Director,
Information Technology and Cybersecurity
Exposure Notification GAO-21-104622 46
List of Congressional Addressees
The Honorable Patrick Leahy The Honorable Bennie G. Thompson
Chairman Chairman
The Honorable Richard Shelby The Honorable John Katko
Vice Chairman Ranking Member
Committee on Appropriations Committee on Homeland Security
United States Senate House of Representatives
The Honorable Ron Wyden The Honorable Carolyn B. Maloney
Chairman Chairwoman
The Honorable Mike Crapo The Honorable James Comer
Ranking Member Ranking Member
Committee on Finance Committee on Oversight and Reform
United States Senate House of Representatives
The Honorable Patty Murray The Honorable Richard Neal
Chair Chairman
The Honorable Richard Burr The Honorable Kevin Brady
Ranking Member Republican Leader
Committee on Health, Education, Labor, Committee on Ways and Means
and Pensions House of Representatives
United States Senate
The Honorable Morgan Griffith
The Honorable Gary C. Peters Republican Leader
Chairman Subcommittee on Oversight and
The Honorable Rob Portman Investigations
Ranking Member Committee on Energy and Commerce
Committee on Homeland Security and House of Representatives
Governmental Affairs
United States Senate The Honorable Brett Guthrie
Republican Leader
The Honorable Rosa L. DeLauro Subcommittee on Health
Chairwoman Committee on Energy and Commerce
The Honorable Kay Granger House of Representatives
Ranking Member
The Honorable Michael C. Burgess
Committee on Appropriations
House of Representatives
House of Representatives
The Honorable Frank Pallone, Jr.
Chairman
The Honorable Cathy McMorris Rodgers
Republican Leader
Committee on Energy and Commerce
House of Representatives
Exposure Notification GAO-21-104622 47
Appendix I: Objectives, Scope, and Methodology
Objectives • Federal Communications Commission
(FCC).
We were asked to assess smartphone
applications (apps)—commonly referred to as In addition, we interviewed representatives
exposure notification apps—that are intended from companies involved in the development
to notify persons of potential exposure to of exposure notification apps (Google, Apple,
infectious diseases. This report discusses: and PathCheck Foundation); public health
organizations (Association of Public Health
• the benefits and design of exposure Laboratories, Public Health Informatics
notification apps; Institute, Council of State and Territorial
Epidemiologists, Association of State and
• the current level of deployment in the
Territorial Health Officials); federally funded
U.S.;
research and development centers
• challenges affecting their use; and (Massachusetts Institute of Technology’s
• policy options that could help address key Lincoln Laboratory and MITRE Corporation);
challenges for future use. and academic researchers from Oxford
University’s Big Data Institute, Nuffield
Scope and methodology Department of Medicine. We identified these
entities through our interviews and document
reviews. During our interviews with officials
To address these objectives, we reviewed
and representatives, we discussed topics such
documentation and met with officials from
as exposure notification app functionality;
selected federal agencies and entities
benefits of its use; levels of deployment in the
involved in providing guidance, funding
U.S.; technological limitations; and challenges
research, and other efforts related to
to its development, deployment, and use. We
exposure notification apps. These agencies
also obtained written responses from two
were:
organizations: the National Association of
County and City Health Officials and Linux
• Centers for Disease Control and
Foundation Public Health.
Prevention (CDC) and National Institutes
of Health (NIH) within the Department of
Health and Human Services (HHS), Further, we conducted a literature search for
• Cybersecurity and Infrastructure Security articles regarding exposure notification apps,
Agency, Science and Technology including their benefits, capabilities, and
Directorate, within the Department of challenges, as well as policy options
Homeland Security (DHS), associated with the apps. A research librarian
• National Institute of Standards and conducted searches of various databases
Technology (NIST), Federal Trade including Inspec, Scopus, Policy File,
Commission (FTC), within the Department ProQuest’s COVID-19 Research Database, and
of Commerce, and the Harvard Kennedy School’s Custom Google
Think Tank Search. We used synonyms of the
Exposure Notification GAO-21-104622 48
following search terms to identify relevant We also contacted several individual states to
articles: contact tracing, exposure verify their status in deploying an app. We
notification, Google Apple exposure, analyzed the inventory to identify the extent
application, app, system, platform, digital, to which states and territories had deployed
mobile, smartphone. We paired these search apps, the underlying technologies used (e.g.,
terms with additional synonyms for privacy, Bluetooth Low Energy), and the use of
security, policy, legislation, opportunities, and national servers. 87 States that had an app in a
challenges. We considered articles that met pilot phase at the time of our review were
the following criteria: published from 2016 included in the category of “states that had
through January 2021 in academic journals, not deployed an app as of June 2021.”
working papers, trade journals, legislative
materials, and reports by government To obtain additional information associated
agencies and nonprofit organizations. From with the development and use of the apps,
the results produced by this search, we we interviewed state public health officials
reviewed a selection of articles to provide an from a non-generalizable sample of nine
overview and additional context for our states that had an exposure notification app
research objectives. We also used the results as of January 1, 2021: Alabama, Colorado,
to help inform our development of an Connecticut, Minnesota, Nevada, North
inventory of states by app deployment status, Carolina, Pennsylvania, Virginia, and
among other sources noted below. Washington. 88 We selected this sample based
on deployment date, geographical
To identify the current level of deployment in distribution, the number of COVID-19 cases
the U.S., we developed an inventory of and deaths, and app developer. We aimed for
exposure notification apps that had been a selection of states that would allow for the
deployed by U.S. states and territories as of selected states to provide a broad overview
June 2021. We developed the inventory by: and context for assessing our engagement’s
research objectives. Because the selection
• reviewing inventories that had been was based on a non-generalizable sample, the
developed by other organizations; 86 results cannot be used to make inferences
• reviewing state health department about all states that had deployed an app. We
websites related to COVID-19 to identify also received written feedback to structured
whether they identified an available app, questions from two additional states
or plans to deploy one; (Louisiana and Utah) that deployed apps in
• conducting Google searches; and the later stages of our evidence collection.
• reviewing Android and iPhone app stores. We also conducted a review of each of the
selected states apps on both a phone using
86The inventories included those developed by MIT Technology 88We considered a state as having an app if it had an official
Review, the Association of State and Territorial Health Officials, application available for download or the state officially
and the Ada Lovelace Institute. supported an exposure notification system that exists in a
87Specifically, whether states used the Association of Public smartphone operating system. As of January 1, 2021, we had
identified 20 states with an exposure notification app. After we
Health Laboratories’ National Key Server and Multi-tenant selected our sample, we learned that one additional state—
Verification Server. Wisconsin—had deployed an app in late December 2020.
Exposure Notification GAO-21-104622 49
the iOS (iPhone 6) and Android (Samsung intended to be inclusive of all potential policy
Galaxy S9) operating systems. As a part of this options and are neither recommendations to
review, we reviewed the general functions, federal agencies nor matters for
features, and usability of the apps. To help congressional consideration. They are also not
understand how privacy considerations listed in any specific rank or order. We are not
applied to the apps, we examined and suggesting that they be done individually or
compared each state’s privacy policies with combined in any particular fashion.
recommended practices identified in federal Additionally, we did not conduct work to
guidance, such as CDC’s guidelines for digital assess how well they may lead to a particular
tools. 89 outcome.
In addition, to obtain perspectives from states We conducted our work from November 2020
that had not deployed an app, we collected to September 2021 in accordance with all
information from a non-generalizable sections of GAO’s Quality Assurance
selection of seven states that had not Framework that are relevant to technology
deployed an app at the time of our review assessments. The framework requires that we
(Montana, Nebraska, Oregon, Rhode Island, plan and perform the engagement to obtain
South Carolina, Texas, and West Virginia), sufficient and appropriate evidence to meet
which included an interview with one state our stated objectives and to discuss any
and written responses to a semi-structured limitations to our work. We believe that the
set of questions for the other six. We selected information and data obtained, and the
these states based on geographical analysis conducted, provide a reasonable
distribution, suggestions from stakeholders basis for any findings and conclusions in this
we interviewed, and information we gathered product.
during our review regarding challenges
certain states may have faced.
We identified policy options based on our
literature review and interviews with federal
agencies, the selected states, and other
stakeholders, including national health
organizations and researchers. We assessed
each policy option by identifying potential
benefits and considerations of implementing
them, as identified over the course of our
review. Based on the evidence collected, we
identified four policy options. The list is not
89
Centers for Disease Control and Prevention, Preliminary
Criteria for the Evaluation of Digital Contact Tracing Tools for
COVID-19, version 1.2 (Atlanta, Ga.: May 17, 2020); and
Guidelines for the Implementation and Use of Digital Tools to
Augment Traditional Contact Tracing, version 1.0 (Atlanta, Ga.:
Dec. 15, 2020).
Exposure Notification GAO-21-104622 50
Appendix II: Exposure Notification App Adoption Rates for Selected
U.S. States
As previously discussed, states may use the rate of adoption to measure the success of their
efforts to promote exposure notification apps. However, officials from the 11 states in our
review reported that they used differing methods to calculate adoption rates. App adoption
rates can be determined by dividing the total number of smartphone downloads, or
activations, (numerator) by the size of a given population (denominator). However, states use
different methods for determining the denominator, which affects the adoption rates.
Specifically, two states used the total state population, three used populations aged 18 or
older, and three used the percent of the population with a smartphone (either age 18 or older
or 18 to 65), according to state officials. These inconsistent methods impede comparative
assessments across states. The following table provides information on the apps deployed by
the nine U.S. states selected for our review, plus Louisiana and Utah.
Table 4: Reported exposure notification app adoption rates for 11 U.S. states, as of June 2021
State Type App name Launch Reported Reported Reported
(custom or date downloads and/or population adoption
Express) estimated (denominator) rateb
activationsa
(numerator)
Alabama Custom GuideSafe™ Aug. 280,000 Not provided 20%
2020
Colorado Express COVID Oct. 2,511,070 Not provided 42%
Exposure 2020 The combined
Notifications number of iOS and
Android
activations taken
from several
sources.
Connecticut Express COVID Alert Nov. N/A N/A N/A
CT 2020
Louisiana Custom and COVID Jan. 663,379 3,560,976 19%
Express Defense 2021 This includes the (18 and older)
combined number Based on U.S.
of iOS and Android Census Bureau
app downloads for estimate for the
the custom app, as adult population.
well as the
Android app
downloads and
the estimated
number of
activations for iOS
for the Express
option.
Exposure Notification GAO-21-104622 51
Minnesota Custom and COVIDAware Nov. 1,419,232 5,600,000 25%
Express 2020 This includes the (Total population)
combined number Based on U.S.
of iOS and Android Census Bureau
app downloads for estimates.
the custom app, as
well as the
Android app
downloads and
the estimated
number of
activations for iOS
for the Express
option.
Nevada Custom and COVID Trace Aug. 1,202,874 3,100,000 49%
Express 2020 This includes the (80% of state
combined number population, a
of iOS and Android proxy for the
app downloads for number of people
the custom app, as with
well as the smartphones)
Android app Based on U.S.
downloads and Census Bureau
the estimated estimates.
number of
activations for iOS
for the Express
option.
North Custom SlowCOVIDNC Sept. 854,802 9,472,502 9%
Carolina 2020 The combined (18 and older)
number of iOS and Based on North
Android app Carolina’s Office
downloads. of State Budget
and Management
estimates for the
adult population.
Pennsylvania Custom COVID Alert Sept. 912,863 10,880,000 8%
PA 2020 The combined (18-65 years old)
number of iOS and Based on U.S.
Android app Census Bureau
downloads. estimates for the
adult population
and Pew Research
Center estimates
that 85% of the
adult population
has a smartphone.
Utah Express UT Exposure Feb. N/A N/A N/A
Notifications 2021
Exposure Notification GAO-21-104622 52
Data are not
tracked. However,
officials estimated
there were
600,000
activations in the
initial phase of app
deployment.
Virginia Custom and COVIDWISE Aug. 1,100,338 4,253,335 26%
Express 2020 This includes the (18-65 years old)
combined number Based on U.S.
of iOS and Android Census Bureau
app downloads for estimates for the
the custom app, as adult population
well as the and Google and
Android app Apple statements
downloads and that 80% of the
the estimated adult population
number of has a smartphone.
activations for iOS
for the Express
option.
Washington Express WA Notify Nov. 2,068,916 6,115,500 34%
2020 The number of (18 and older)
Android app Based on
downloads and Washington’s
the estimated Office of Financial
number of Management
activations for iOS. estimates for the
adult population
and Pew Research
Center estimates
that 85% of the
adult population
has a smartphone.
Legend: N/A = A state that does not track the number of app downloads or its adoption rate.
Source: GAO compilation of data from selected states, related documents, interviews and other sources. I GAO-21-104622
Notes: The “as of “date for the data provided by the states ranged from May 31, 2021 to June 11, 2021.
aFor states using the Express option, daily activations for Android are from the Google Play app store, while the
Apple activations numbers are an estimate based on counting the number of times people accessed the app’s logo
image and using a multiplier provided by Google and Apple to estimate the number of people who go on to install
the Express option.
bThe adoption rates were provided to us by the states; we rounded the rates up to the nearest whole number. We
did not calculate the adoption rates; however, the rates can be calculated with the provided numerators and
denominators for the states that provided this information.
Exposure Notification GAO-21-104622 53
Appendix III: GAO Contacts and Staff Acknowledgments
GAO contacts
Karen L. Howard, PhD, (202) 512-6888 or howardk@gao.gov
Vijay A. D’Souza, (202) 512-6240 or dsouzav@gao.gov
Staff acknowledgments
In addition to the contacts named above, Sushil Sharma (Assistant Director), Neela Lakhmani
(Assistant Director), Eric Bachhuber (Analyst in Charge), Scott Borre (Analyst in Charge), Nora
Adkins, Daniel Emirkhanian, Donna Epler, Nancy Glover, Anika McMillon, Melissa Melvin,
Monica Perez-Nelson, Ben Shouse, Amber Sinclair, and Umesh Thakkar made key
contributions to this report. Rebecca Gertler, Tim Kinoshita, Eleni Orphanides, and Ethiene
Salgado-Rodriguez also contributed to this report.
(104622)
Exposure Notification GAO-21-104622 54
GAO’s Mission
The Government Accountability Office, the audit, evaluation, and investigative arm of Congress,
exists to support Congress in meeting its constitutional responsibilities and to help improve the
performance and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make informed oversight, policy, and
funding decisions. GAO’s commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s
website (https://www.gao.gov). Each weekday afternoon, GAO posts on its website newly
released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted
products, go to https://www.gao.gov and select “E-mail Updates.”
Order by Phone
The price of each GAO publication reflects GAO’s actual cost of production and distribution and
depends on the number of pages in the publication and whether the publication is printed in color
or black and white. Pricing and ordering information is posted on GAO’s website,
https://www.gao.gov/ordering.htm.
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money
order. Call for additional information.
Connect with GAO
Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Subscribe to our RSS Feeds or E-mail Updates.
Listen to our Podcasts and read The Watchblog.
Visit GAO on the web at https://www.gao.gov.
To Report Fraud, Waste, and Abuse in Federal Programs
Contact: Website: https://www.gao.gov/fraudnet/fraudnet.htm
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Nikki Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400,
U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548
Public Affairs
Chuck Young, Managing Director, YoungC1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149, Washington, DC 20548
Strategic Planning and External Liaison
Stephen Sanford, Managing Director, spel@gao.gov, (202) 512-9715
U.S. Government Accountability Office, 441 G Street NW, Room 7B37N, Washington, DC 20548
File and source
- File
- gao-21-104622.pdf
- Size
- 4,195,140 bytes
- SHA-256
- efab175a9536d2e250749bdf2371a73256248641c6e2273d423db734d43b5ac0
- Our copy
- gao-21-104622.pdf
- Original
- www.gao.gov