GAO-21-600, COVID-19: HHS’s Collection of Hospital Capacity Data

Summary

GAO-21-600, COVID-19: HHS's Collection of Hospital Capacity Data, a Government Accountability Office report to congressional addressees dated August 2021. GAO describes changes the Department of Health and Human Services (HHS) made to hospital capacity data collection through HHS Protect, a data system HHS created in April 2020, and the challenges reporting entities experienced, including a lack of clarity on the requirements. The report says HHS used the data to provide assistance such as staff resources or supplies in 40 states, while public health stakeholders told GAO they relied on state and local data. It names stakeholder engagement and improved communication as key lessons learned. It says HHS agreed with GAO's January 2021 recommendation but, as of June 2021, has not implemented it.

Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used

Full text

COVID-19
HHS’s Collection of
Hospital Capacity
Data

Report to Congressional Addressees
August 2021

GAO-21-600

United States Government Accountability Office

 United States Government Accountability Office

Highlights of GAO-21-600, a report to
congressional addressees

August 2021
COVID-19
HHS’s Collection of Hospital Capacity Data
What GAO Found
During the COVID-19 pandemic, the Department of Health and Human Services
(HHS) made frequent and significant changes to the collection of hospital
capacity data. In April 2020, HHS created a new data ecosystem—HHS
Protect—to capture, among other things, national- and state-level data on
inpatient and intensive care beds in use, supplies of personal protective
equipment (PPE), and COVID-19 treatments. Subsequently, HHS changed the
methods through which data could be reported to HHS Protect and also changed
reporting requirements. According to HHS officials, this was done to capture
more complete data and to capture more information, such as data on influenza-
related hospitalizations and COVID-19 vaccines administered. Reporting entities
said they experienced multiple challenges implementing the changes, including a
lack of clarity on the requirements and logistical challenges such as having to
adapt their systems to provide the data. As HHS made changes, HHS issued
updated guidance to clarify reporting requirements.
HHS uses hospital capacity data to identify and address resource shortages and
to inform the public. For example, according to HHS officials, HHS has used the
data to provide assistance such as staff resources or supplies in 40 states.
Additionally, HHS has shared the hospital capacity data to inform the public.
However, public health stakeholders told GAO they have relied on state and local
data for their purposes rather than data from HHS Protect. For example,
epidemiological association officials said their members relied on state and local
data for case investigation because they contained more detailed information and
did not use HHS Protect data on hospital capacity. According to HHS officials,
some states that may not be collecting their own data rely on HHS Protect
capacity data to inform their public health response to the pandemic.
HHS agency officials and stakeholders identified the need for stakeholder
engagement and improved communication among key lessons learned to better
ensure the collection of quality hospital capacity data during a public health
emergency. For example, HHS officials told GAO that there is a need for
dialogue and external validation to ensure data quality and accuracy. They also
noted that the need for a system like HHS Protect will continue beyond the
COVID-19 pandemic. Officials GAO interviewed from stakeholder organizations
and selected states noted that increased collaboration and communication—as
well as more time to implement changes—would have facilitated the
implementation of the changes to the data collection process. These lessons
learned are consistent with GAO’s January 2021 recommendation that HHS
engage with stakeholders to review and inform the alignment of ongoing data
collection and reporting standards through establishing an expert committee.
HHS agreed with the recommendation, but as of June 2021, the department has
not implemented it.

View GAO-21-600. For more information,
contact Jessica Farb at (202) 512-7114 or
farbj@gao.gov.
Why GAO Did This Study
The magnitude of the COVID-19
pandemic has underscored the
importance of having quality data to
help the federal government
understand the health care system’s
capacity to provide care and to inform
the allocation of resources. HHS
launched HHS Protect in April 2020 to
capture hospital capacity data.
Throughout the public health
emergency HHS has made changes to
how information is collected and used.
The CARES Act includes a provision
for GAO to report on its ongoing
COVID-19 monitoring and oversight
efforts. GAO was asked to examine
HHS’s implementation of HHS Protect.
In this report, GAO describes (1)
HHS’s implementation of HHS Protect
hospital capacity reporting
requirements and the challenges
experienced by reporting entities; (2)
HHS’s and stakeholders’ use of the
data, if at all; and (3) lessons learned
about ensuring the collection of quality
hospital capacity data during a public
health emergency.
GAO reviewed agency guidance and
HHS Protect hospital capacity
dashboards and reports, and
interviewed HHS officials as well as
officials from three states that report or
reported directly to HHS Protect on
behalf of their hospitals. These states
were selected for variation in
geography and the mix of rural and
non-rural hospital facilities. GAO also
interviewed officials from public health
stakeholder groups including hospital
associations, epidemiological
associations, and local health
organizations.  GAO provided a draft of
this report to HHS for review and
comment. HHS had no comments on
the report.

Page i
GAO-21-600  COVID-19
Letter

1
Background
4
Frequent and Significant Changes to Reporting Requirements
Created Challenges for Reporting Entities
5
HHS Uses Hospital Capacity Data to Address Resource
Shortages and Inform the Public; Public Health Stakeholders
Have Relied on State and Local Data for Their Purposes
11
Key Lessons  Learned Include Need for Better Communication
and Stakeholder Engagement
17
Agency Comments
18
Appendix I
GAO Contact and Staff Acknowledgments
22

Figures
Figure 1: Percentage of Hospitals with Capacity Data Reported to
HHS Protect by State, as of June 2, 2021.
13
Figure 2: National Estimated Inpatient Bed Utilization by State, as
of June 13, 2021
14
Figure 3: National Estimated Intensive Care Unit (ICU) Bed
Utilization by State, as of June 13, 2021
15

Contents

Page ii
GAO-21-600  COVID-19

Abbreviations

ASPR
Office of the Assistant Secretary for Preparedness and
Response
CDC

Centers for Disease Control and Prevention
CMS
Centers for Medicare & Medicaid Services
COVID-19
Coronavirus Disease 2019
HHS

Department of Health and Human Services
ICU

Intensive care unit
NHSN
National Healthcare Safety Network
OCIO
Office of the Chief Information Officer
PPE

Personal protective equipment

This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.

Page 1
GAO-21-600  COVID-19
441 G St. N.W.
Washington, DC 20548
August 5, 2021
Congressional Addressees
More than a year after the Department of Health and Human Services
(HHS)—which is responsible for overseeing the public health response to
the Coronavirus Disease 2019 (COVID-19) pandemic—declared a public
health emergency for the United States, the nation is concurrently
responding to and recovering from the pandemic. As of June 16, 2021,
there were over 33,000,000 reported cases and more than 590,000
reported deaths in the United States.1 In recent months, the number of
cases and hospitalizations have declined largely due to the development
and administration of vaccines, but as recently as February 2021, when
the HHS Office of Inspector General surveyed hospitals regarding their
capacity to care for patients during the COVID-19 pandemic, hospitals
reported being overwhelmed. In particular, hospitals reported
experiencing significant strain during surges in COVID-19 infections, with
some reporting that they operated at over 100 percent capacity during
surges.2
As we have noted in our work throughout the COVID-19 pandemic, the
virus’ rapid spread and magnitude have underscored the importance of
having quality data to help the federal government understand the health
care system’s capacity—including the capacity of hospitals to admit and
treat patients—to provide needed care and to inform timely and

1Data on COVID-19 cases in the United States are based on aggregate case reporting to
the Centers for Disease Control and Prevention (CDC) and include probable and
confirmed cases as reported by states and jurisdictions. CDC COVID-19 counts are
subject to change due to delays or updates in reported data from states and territories.
According to CDC, the actual number of COVID-19 cases is unknown for a variety of
reasons, including that people who have been infected may have not been tested or may
have not sought medical care. CDC’s National Center for Health Statistics COVID-19
death counts in the United States are based on provisional counts from death certificate
data, which do not distinguish between laboratory-confirmed and probable COVID-19
deaths. Provisional counts are incomplete due to an average delay of 2 weeks (a range of
1–8 weeks or longer) for death certificate processing. The data were accessed on June
16, 2021. Data include deaths occurring from January 2020 through the week ending on
June 12, 2021.
2Department of Health and Human Services, Office of Inspector General, Hospitals
Reported that the COVID-19 Pandemic Has Significantly Strained Health Care Delivery,
OEI-09-21-00140 (Washington, D.C.: March 2021).
Letter

Page 2
GAO-21-600  COVID-19
responsive decisions, including allocating resources.3 In addition, we
have noted that accurate, complete, consistent, and timely data are
essential for monitoring trends at the state and regional level, and making
informed comparisons between these areas and assessing the effect of
public health response measures.
Responsibilities for detecting and responding to public health
emergencies are dispersed among federal, state, and local public health
entities and health care facilities throughout the country—which can make
collecting comprehensive and reliable hospital data challenging. Within
the first four months of the COVID-19 pandemic, on April 10, 2020, HHS
launched a new data ecosystem—HHS Protect—for collecting and
sharing national and state-level COVID-19 data.4 Among other things,
HHS Protect captures data on hospital capacity, including the number of
ventilators and inpatient and intensive care beds in use as well as the
supply of personal protective equipment (PPE) and the availability of
therapies to treat the virus. Reporting entities include individual hospitals
as well as states—or state hospital associations—reporting on behalf of
their hospitals.
The Coronavirus Aid, Relief, and Economic Security (CARES) Act
includes a provision for GAO to report on its ongoing monitoring and
oversight efforts related to the COVID-19 pandemic.5 You asked us to
examine HHS’s implementation of HHS Protect and its effect on the
response to COVID-19. In this report, we describe
•
HHS’s implementation of HHS Protect, and the challenges
encountered by reporting entities;
•
how the department and public health stakeholders are using the
data, if at all, and

3We regularly issue government-wide reports on the federal response to COVID-19. For
the latest report, see GAO, COVID-19: Continued Attention Needed to Enhance Federal
Preparedness, Response, Service Delivery, and Program Integrity, GAO-21-551
(Washington, D.C.: July 19, 2021). Our next government-wide report will be issued in
October 2021 and will be available on GAO’s website at https://www.gao.gov/coronavirus.
4HHS Protect is a secure data ecosystem aimed at collecting, sharing, and analyzing near
real-time COVID-19 data—including data on testing, information on supply chains, and
state policies—from over 200 data sets maintained by federal, state, and local
governments and industry.
5Pub. L. No. 116-136, § 19010(b), 134 Stat. 281, 580 (2020).

Page 3
GAO-21-600  COVID-19
•
lessons learned by the department and public health stakeholders
about ensuring the collection of quality hospital capacity data during a
public health emergency.
To describe HHS’s implementation of HHS Protect and the challenges
encountered by reporting entities, we reviewed federal regulations,
guidance, and data collection templates HHS provided to hospitals for
hospital capacity reporting as well as public comments on hospital
capacity reporting. We collected information from officials representing
various HHS offices and agencies throughout the department regarding
the implementation process and HHS’s reporting requirements, either
through interviews or written responses to our questions. These included
officials from the Office of the Chief Information Officer (OCIO), the Office
of the Assistant Secretary for Preparedness and Response (ASPR), and
the Centers for Disease Control and Prevention (CDC).6 We also
interviewed officials from stakeholder organizations representing both
reporting entities and public health stakeholders. These stakeholders
included five hospitals associations, two epidemiological associations,
and an organization of local health officials selected to represent a wide
range of perspectives such as from urban and rural hospitals, and state
and local epidemiologists and health officials. We also interviewed state
officials from a non-generalizable sample of three states—Florida, Iowa,
and New Jersey—that had been certified to report on behalf of hospitals
in their states.7 We selected these states on the basis of their mix of rural
and non-rural hospitals and geographic diversity.
To describe how the department and public health stakeholders are using
the data, we reviewed HHS’s reporting guidance, data collection
templates, and data available on HHS Protect and healthdata.gov. We
interviewed officials from OCIO, ASPR, and CDC about their use of the
data. We also interviewed officials representing reporting entities—our
three selected states and hospital associations—and public health
stakeholders including the two epidemiological associations and the local
health officials about their use of the data.
To describe the lessons learned by the department and public health
stakeholders about ensuring the collection of quality hospital data during

6OCIO manages HHS Protect with input from CDC and ASPR.
7Before reporting directly to HHS Protect on behalf of hospitals, a state must receive
written certification from ASPR affirming that the state has an established functioning data
reporting mechanism that can report data at the required frequency. As of April 2021, 32
states were certified to report to HHS Protect on behalf of hospitals in their states.

Page 4
GAO-21-600  COVID-19
a public health emergency, we interviewed officials from various HHS
offices and agencies described above and officials representing reporting
entities and other public health stakeholders.
We conducted this performance audit from September 2020 to August
2021 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.
HHS has primary responsibility for leading the federal public health and
medical response to emergencies. Various HHS offices and agencies are
involved in collecting data and formulating this federal response.
Specifically
•
ASPR leads the nation’s medical and public health preparedness for,
response to, and recovery from public health emergencies. As part of
this role, ASPR oversees the federal stockpile of vaccines,
pharmaceuticals, and medical supplies and devices and aims to
enhance medical surge capacity by organizing, training, and
equipping federal public health and medical responders, as well as
calling upon and deploying these personnel during responses.
•
CDC is the nation’s lead public health agency, and in this role, it
conducts public health surveillance and provides technical assistance
and guidance to state, territorial, tribal, and local health agencies.
•
OCIO supports the department by leading the development and
implementation of information technology infrastructure across the
department.
HHS relies on information from a variety of federal, state, and local public
health entities and health care facilities throughout the country to detect
and respond to public health emergencies. Typically, clinicians at the
local level—including hospital staff—are the most likely to be the first
ones to detect a public health-related incident, and they are expected to
report such an incident to state and federal authorities. For example,
hospitals and nursing homes have long reported data on certain
infections to CDC’s National Healthcare Safety Network (NHSN), which
Background

Page 5
GAO-21-600  COVID-19
the agency uses to track and analyze certain infections.8 In response to
the COVID-19 pandemic, in March 2020, HHS began to collect data from
hospitals on cases and capacity and subsequently launched HHS Protect
in April 2020. According to HHS officials, prior to the establishment of
HHS Protect, the reporting of hospital capacity data at the national level
was fragmented; and as a data ecosystem that collects data from over
200 sources, HHS Protect represents an effort to integrate data
elements.9
During the COVID-19 pandemic, HHS made frequent and significant
changes to requirements for the collection of hospital capacity data in an
effort to capture more complete and consistent data from hospitals across
the country. According to stakeholders, hospitals and states experienced
multiple challenges including a lack of clarity on the requirements and the
need to adapt systems to provide the data.

As the pandemic unfolded, HHS launched HHS Protect as the primary
COVID-19 hospital data collection system. HHS also made changes to
the methods through which hospital capacity data could be submitted.
According to HHS officials, in addition to making changes to reporting
methods, the department increased the number of required and optional
reporting elements over time as new informational needs were identified.
The changes to reporting methods HHS (or its agencies) introduced
throughout the pandemic included:

8In 2005, CDC replaced its National Nosocomial Infections Surveillance (NNIS) system,
which was introduced in the 1970s with NHSN, through which hospitals and other facilities
report data using a uniform set of definitions on health care-associated infections, i.e.
infections patients may acquire in a health care setting while receiving treatment for other
conditions.
9The 2006 Pandemic and All-Hazards Preparedness Act required HHS to work with state,
local, and tribal public health officials to establish a nationwide public health network to
help manage infectious disease outbreaks or other public health emergencies and provide
situational awareness capabilities. Pub. L. No. 109-417, § 202, 120 Stat. 2831, 2845
(2006) (codified, as amended, at 42 U.S.C. § 247d-4(c)). In our September 2017 report,
we reported that HHS had made little progress in establishing the network of systems to
share public health data. GAO, Public Health Information Technology: HHS Has Made
Little Progress toward Implementing Enhanced Situational Awareness Network
Capabilities, GAO-17-377 (Washington, D.C.: Sept. 6, 2017). We have ongoing work
evaluating the status of HHS’s efforts to establish this capability.
Frequent and
Significant Changes
to Reporting
Requirements
Created Challenges
for Reporting Entities
HHS Made Frequent and
Significant Changes to
Hospital Data Reporting
Requirements to Capture
More Complete and
Consistent Data

Page 6
GAO-21-600  COVID-19
March 29, 2020. Then-Vice President Pence, on behalf of the White
House Coronavirus Task Force, requested hospitals to report data
deemed critical for epidemiological surveillance and public health
decision making to the newly developed Patient Impact and Hospital
Capacity module in NHSN, which is primarily used to track certain
healthcare-associated infections.
April 10, 2020. HHS launched HHS Protect and introduced
TeleTracking as a reporting method, through which hospitals could
report their capacity data to the HHS Protect platform. At that time,
HHS provided five methods through which COVID-19 hospital
capacity data could be reported:
1. States or state hospital associations could report on behalf of
hospitals in that state directly to HHS Protect;
2. Hospitals could report through TeleTracking;
3. Hospitals could report their data to NHSN;
4. Hospitals could authorize their information technology vendor to
share information directly with HHS; or
5. Hospitals could publish their data in a standardized format on their
websites.10
HHS officials told us that the department made the changes to
COVID-19 hospital capacity data reporting methods because it
recognized that different options might work better in certain
situations. For example, they told us that some states were already
collecting this type of information from hospitals, and by allowing
states—or state hospital associations—to report on behalf of hospitals
HHS could ease the burden on hospitals. Also, according to these
officials, since states and territories have differing reporting and data
management capabilities, the various reporting methods provided
flexibility for reporting entities while maintaining data commonality at
the national level.
July 15, 2020. HHS removed NHSN as a method for hospitals to
report their capacity data.11 According to HHS officials, the

10According to HHS officials, as of June 2021, a small number of hospitals were sharing
their data directly through their information technology vendors, and no hospitals were
publishing their data.
11CDC continues to collect COVID-19 infections and related information from nursing
homes and long-term care facilities through NHSN. See 85 Fed. Reg. 27,550, 27,627
(May 8, 2020) (adding 42 C.F.R. § 483.80(g)).

Page 7
GAO-21-600  COVID-19
department opted to remove NHSN as an option in part because its
hospital capacity data were incomplete. Additionally, HHS officials told
us that during this time, the department had asked CDC to add new
data fields in NHSN to capture information on the distribution of
remdesivir—a therapeutic treatment for COVID-19—but that change
would have taken too long to implement.12
In addition to making changes to reporting methods, HHS increased
the number of data elements to be reported to HHS Protect as new
informational needs were identified. Specifically,
July 15, 2020. HHS began asking hospitals to report data on their
inventory of remdesivir.
December 18, 2020. HHS began asking hospitals to report data on
the incidence of influenza cases.
January 8, 2021. HHS began asking hospitals to report in new fields
for the inventory and number of courses administered of the COVID-
19 therapeutics casirivimab/imdevimab and bamlanivimab.
January 12, 2021. HHS began asking hospitals to report on the
number of hospital employees and patients who had received the
COVID-19 vaccine.
April 7, 2021. HHS began asking hospitals to report in new fields for
the inventory and number of courses administered of the COVID-19
therapeutic bamlanivimab/etesevimab.
HHS continues to update these data elements on a periodic basis.
Additionally, effective September 2, 2020, the Centers for Medicare &
Medicaid Services (CMS) began requiring hospitals participating in
Medicare and Medicaid to report certain HHS COVID-19 capacity data
elements as a condition of obtaining payment from Medicare and

12According to HHS officials, the White House Coronavirus Task Force directed CDC to
add new data fields to NHSN within 4 days, but CDC officials told the task force that the
agency would need more time than that to make the requested change if it were to
maintain compliance with the Paperwork Reduction Act of 1995. Pub. L. No. 104-13, 109
Stat. 163. Under the act as amended, federal agencies generally may not collect
information from more than 10 members of the public unless they take certain steps in
advance, such as seeking public comment regarding the burden on respondents. See 44
U.S.C. § 3507; 5 C.F.R. § 1320.5. HHS sought public comment on the burden for
hospitals to report to HHS Protect in January 2021.
HHS Protect Reporting Elements
As of June 2021, hospitals are asked to report
on 116 questions, some of which are optional
and some of which are mandatory. Hospitals
are asked to report on over 52 elements on a
daily basis. These elements include, among
other things,
•
hospital identification information,
•
the number of staffed and occupied
inpatient and intensive care unit beds,
•
the number of COVID-19-related
admissions, and
•
the number of COVID-19-related deaths.
Additionally, hospitals are asked to report on
at least 31 elements on a weekly basis—on
Wednesdays—including
•
the number of personal protective
equipment (PPE) supplies available for a
range of days,
•
the ability to obtain PPE supplies, and
•
therapeutics usage and inventory (for
specified therapeutics).
Source: GAO analysis of HHS guidance.  |  GAO-21-600.

Page 8
GAO-21-600  COVID-19
Medicaid.13 According to HHS officials, the percentage of hospitals
reporting increased after this change. Specifically, these officials told us
that as of November 2020, over 80 percent of hospitals were reporting
their capacity data to HHS Protect.
According to stakeholders we spoke with, the implementation of new
reporting requirements throughout the course of the pandemic resulted in
challenges for hospitals. Specifically, representatives from all five of the
hospital associations we spoke with told us that reporting entities
experienced logistical challenges in adapting staff resources or data
systems to comply with reporting requirements. Stakeholders also told us
that reporting entities were sometimes unclear on what information HHS
was seeking or why the information was being collected. Below are some
of the specific challenges identified by stakeholders:
Staff resources. Representatives from two hospital associations told
us that staffing resources are more limited in smaller and rural
hospitals as compared to larger, urban hospitals, making it difficult for
some of these hospitals to meet the daily reporting requirements. For
example, smaller hospitals may not have staff available for weekend
reporting.
Data systems. Representatives from all five of the hospital
organizations we spoke with told us that hospitals’ existing workflows
often did not align with the requirements of TeleTracking or HHS
Protect, requiring them to either create new data workflows or enter
data manually. Additionally, one hospital association noted that the
way HHS asked hospitals to report on PPE supplies was not
consistent with how these data are collected and maintained by
hospital systems. Specifically, the association noted that hospital
systems’ group purchasing organizations maintained that information,
and the reporting elements did not align with the way hospital systems
obtain and distribute supplies to individual hospitals within the

13Hospitals must meet certain conditions of participation—including health and safety
standards—in order to receive payment from Medicare or Medicaid. Medicare is the
federal health insurance program for persons age 65 or over, certain individuals with
disabilities, and individuals with end-stage renal disease. Medicaid is a joint federal-state
program that finances health care coverage for low-income and medically needy
populations. CMS revised the conditions of participation to require COVID-19 reporting as
specified by the agency. 85 Fed. Reg. 54,820, 54,873 (Sept. 2, 2020) (adding 42 C.F.R. §
482.42(e)(2020)).
According to
Stakeholders, Hospitals
Faced Logistical
Challenges and a Lack of
Clarity on the New
Reporting Requirements

Page 9
GAO-21-600  COVID-19
system.14 Stakeholders also told us that when HHS was rolling out
new reporting requirements, hospitals did not have sufficient time to
modify their collection systems to report on the new data elements.
HHS subsequently built in more lead time for hospitals to adjust to
new reporting requirements.
Clarity of reporting requirements. Various stakeholders told us
there was a lack of clarity around HHS’s requirements initially, but
subsequently HHS issued clarifying guidance, and HHS officials told
us they communicated with stakeholders throughout the
implementation of HHS Protect. According to the various
stakeholders, there was initially a lack of clarity regarding definitions
and a lack of guidance on how to report some data elements. Officials
from one hospital association and one state told us that HHS Protect
did not initially have a data dictionary (i.e., a document that defines
data elements) for how to report hospital capacity data at the time
HHS Protect was implemented in April 2020. Stakeholders told us that
the lack of clear definitions left data elements open to interpretation
and people tasked with entering the data could interpret the data
elements differently. For example, officials from one state told us that
80 percent of questions they received from hospitals about reporting
hospital capacity data were related to the definitions, such as the
definition of intensive care unit (ICU) beds.15 Additionally, officials
from one state we spoke to told us that their hospitals were confused
about whether to include nursery beds in their reports. HHS
subsequently provided guidance that clarified hospitals should include
all ICU, emergency department, observation, neonatal ICU, pediatric
ICU, newborn, and nursery beds in the hospital bed count. HHS
officials told us they worked closely with stakeholders to understand
questions they had about reporting and to provide clarity. For

14Group purchasing organizations are organizations that act as purchasing intermediaries
that negotiate contracts between health care providers and vendors of medical products
and services.
15We previously reported on potential confusion among reporting entities about what data
to include. For example, in January 2021, we reported that as COVID-19 cases increased,
some hospitals were able to reclassify (for the short term) some of their non-ICU beds as
ICU beds. As a result, some hospitals may have included reclassified and traditional ICU
beds in their total counts of ICU beds, but they were unsure how to accurately report the
ICU bed counts to HHS Protect, according to experts interviewed for our January 2021
report. See GAO, COVID-19: Critical Vaccine Distribution, Supply Chain, Program
Integrity, and Other Challenges Require Focused Federal Attention, GAO-21-265
(Washington, D.C.: January 28, 2021).
Data Quality Implications
Some stakeholders raised concerns about
HHS Protect data quality. For example,
several stakeholders noted that they had
more confidence in National Healthcare
Safety Network (NHSN) data quality than
they do for HHS Protect data.  Several
stakeholders raised that HHS Protect and
TeleTracking may lack sufficient data quality
checks.
However, officials from the Department of
Health and Human Services (HHS) told us
that various validation checks are conducted
to ensure data quality. For example,
validation checks include checks for
negative values, inconsistencies and
anomalous data requiring further outreach
or correction. HHS officials told us the
agency also corrects data based on public
input that identifies errors. For example, at
times throughout the pandemic, HHS
Protect data on inpatient and intensive care
unit bed usage showed significantly different
data than some state dashboards for those
measures. According to the officials,
members of the public brought this issue to
the attention of HHS for a particular state
and data quality teams were able to identify
the issue and make corrections.
Source: GAO analysis.  |  GAO-21-600

Page 10
GAO-21-600  COVID-19
example, they held monthly webinars for reporting entities and
provided clarification on the definition of ICU beds from November
2020 through January 2021.
From September 2020 to January 2021, HHS sought public comment on
the implementation of COVID-19 hospital capacity reporting requirements
and HHS’s estimate that the reporting burden imposed by HHS Protect
was 1.5 hours per day.16 Public comments HHS received from technology
vendors, hospitals, and care providers noted that the frequent changes to
the reporting infrastructure, data elements, definitions, and purpose of
reporting resulted in additional burdens for reporting entities. Commenters
also noted that the time burden on hospitals was greater than HHS’s
estimate, with one commenter noting the agency did not take into account
that reporting may require multiple staff, and others noting that it may
require pulling data from multiple sources.

16In addition to the accuracy of the estimated burden, HHS sought public comment on the
necessity and utility of the proposed information collection for the proper performance of
HHS functions, ways to enhance the quality, utility, and clarity of the information to be
collected, and the use of automated collection techniques or other forms of information
technology to minimize the information collection burden.

Page 11
GAO-21-600  COVID-19
HHS uses hospital capacity data to identify and address shortages in
capacity, supplies, and staffing and to inform policymakers and the public
about key pandemic-related information. Public health stakeholders and
the states we spoke with told us they have relied on state and local data
for their purposes rather than data from HHS Protect. According to HHS
officials, other states have relied on HHS Protect data to inform their
responses to the pandemic.

According to early guidance issued by HHS, hospital capacity data
reported to HHS Protect was to be used to facilitate planning, monitoring,
and resource allocation. Initially, HHS used the data to inform its
allocation of remdesivir, a therapeutic treatment for COVID-19 cases. The
allocation strategy evolved over months, but the final allocation process
was based on the state’s share of hospitalized COVID-19 patients as
reported to HHS Protect. Specifically, HHS allocated remdesivir to a state
based on the state’s share of national confirmed or suspected COVID-19
hospitalized patients during a 7-day reporting period. According to HHS,
the federal government had no role in the allocation of remdesivir after
September 30, 2020.17
Subsequently, in October 2020, HHS implemented an interagency effort
intended to provide actionable information to state, local, tribal, and
federal entities, as well as private sector entities. Interagency teams
analyze HHS Protect hospital capacity data to identify shortages in
capacity, staffing or PPE supplies—such as respirators, gowns, and
gloves—which may require federal intervention. Federal teams follow-up
with states to validate the data indicating shortages and, if appropriate,

17According to ASPR, the memorandum of understanding between HHS and remdesivir’s
manufacturer ended on September 30, 2020, after which hospitals were to work directly
with distributors to purchase remdesivir.
HHS Uses Hospital
Capacity Data to
Address Resource
Shortages and Inform
the Public; Public
Health Stakeholders
Have Relied on State
and Local Data for
Their Purposes
HHS Uses Hospital
Capacity Data to Identify
and Address Shortages in
Capacity, Supplies, and
Staffing

Page 12
GAO-21-600  COVID-19
provide resources.18 Since its implementation in October 2020, HHS
officials told us that they have facilitated the distribution of supplies or
staff resources in 40 states. Officials told us that, as of June 2021, there
were about 2,600 instances where shortages were flagged. States were
alerted and resolved the shortage in about 1,500 of these instances, and
additional federal support—such as medical teams or supplies—was
required in about 100 instances, according to HHS officials. In the
remaining instances, the information was validated but no action was
needed.
According to HHS officials, both short- and long-term plans related to data
collection from hospitals are under consideration. Specifically, officials
told us that, in partnership with other stakeholders, the department plans
to re-evaluate the data being collected and streamline reporting
requirements to focus on the most critical information.
HHS has shared hospital capacity data—including data on the
percentage of hospitals reporting and the percentage of hospital beds in
use—publicly on both the HHS Protect and Healthdata websites.19 HHS
officials told us they share these data to keep the public informed about
the status of the pandemic, to help guide a data-driven response to
COVID-19, and to inform local or regional policies, such as re-opening
policies.
There are multiple dashboards shared on the HHS Protect website
presenting national and state-level data on the completeness of
reporting—e.g., the percentage of hospitals in each state reporting—as
well as the estimated number and percentage of inpatient and ICU beds
in use. In addition to the dashboards, the website includes links to the
underlying data reports containing detailed information on the frequency
of individual hospitals reporting on each required element. (See figure 1
for the dashboard on overall completeness of reporting.) As of June 2,

18Officials from two of our three selected states confirmed that HHS conducted outreach to
the state when facility data indicated shortages; however, officials from one state noted
that by the time HHS followed up, the shortages had been resolved; officials from the
other state noted that HHS Protect data may erroneously indicate shortages because it
does not capture hospitals’ access to shared supplies within a hospital system.
19The Healthdata website contains the reported data. HHS, HHS COVID-19 Datasets,
accessed on June 04, 2021 https://healthdata.gov. HHS Protect contains the analysis of
the data. HHS, Hospital Utilization, accessed on May 21, 2021,
https://protect-public.hhs.gov/.
HHS Has Shared Some
Hospital Capacity Data to
Inform the Public and
Policymakers

Page 13
GAO-21-600  COVID-19
2021, at least 95 percent of hospitals in all states except for Alaska and
Colorado were reporting some capacity data to HHS Protect.20
Figure 1: Percentage of Hospitals with Capacity Data Reported to HHS Protect by State, as of June 2, 2021.

Note: The data reflects the percentage of hospitals in each state whose data are reported to HHS
Protect through any method of reporting.

HHS has also developed an interactive map graphic to display national
and state-level data on the total number of inpatient beds and the number
and percent of inpatient beds in use for all patients and COVID-19
patients, respectively. The dashboard also indicates the number of
hospitals reporting on these three specific data elements. These data are
updated daily. Selection of a specific state will show the utilization
numbers and rates as well as the number of hospitals reporting on these
elements for that state (see fig. 2).

20Approximately 87.5 percent of hospitals in Alaska and 93.4 percent of hospitals in
Colorado had reported their capacity data to HHS Protect.

Page 14
GAO-21-600  COVID-19
Figure 2: National Estimated Inpatient Bed Utilization by State, as of June 13, 2021

Note: The number of hospitals reporting each data element varied, with some hospitals not reporting
all three data elements.

Similarly, HHS Protect presents an interactive map graphic to illustrate
national and state-specific ICU bed utilization rates for all patients and
COVID-19 patients, respectively. The dashboard also indicates the
number of hospitals reporting on these specific data elements. These
data are updated daily. Selection of a specific state will show the
utilization for that state. See fig. 3.

Page 15
GAO-21-600  COVID-19
Figure 3: National Estimated Intensive Care Unit (ICU) Bed Utilization by State, as of June 13, 2021

Note: The number of hospitals reporting each data element varied, with some hospitals not reporting
all three data elements.

HHS Protect also presents interactive map graphics to illustrate facility-
level reporting frequency and capacity information. By entering in a zip
code or address, the graphics identify facilities in a user-selected radius,
how frequently the facility reports data to HHS Protect, and provide the
total number of beds, the number of beds occupied, and bed utilization
rates, for inpatient and ICU beds, respectively.

Page 16
GAO-21-600  COVID-19
Representatives from the public health stakeholder organizations we
spoke with told us that they have not used the hospital capacity data in
HHS Protect and instead rely on state and local data sources which they
said were more useful for their purposes.21 Officials from our three
selected states told us that their states had infrastructure in place prior to
the implementation of HHS Protect to collect data on hospital capacity,
and relied on these data to inform the public health response to COVID-
19.22 During the pandemic, state officials from all three states told us they
relied on the data they collected through their own systems. Officials from
two of the states told us that they had developed their data collection
questions through collaboration and coordination with reporting entities,
and that their questions were more useful to state public health decision-
makers than HHS Protect data. For example, HHS Protect requires
hospitals to report on the number of beds in use, and officials from one
state told us they asked instead for the number of beds available, which
provided more pertinent information for their purposes. They also told us
that the state had implemented an automated feed from hospitals to
provide bed utilization data every 10 minutes, giving the state more timely
information on hospital capacity. According to HHS officials, states that
were certified to report to HHS Protect were more likely to have had state-
based hospital capacity data collection efforts to help inform their
responses to COVID-19. However, the officials told us that at least five
states relied on HHS Protect hospital capacity data to inform their
responses, and that they were aware of several hundred users accessing
the data across other states.23
Additionally, representatives from both epidemiological associations and
the local health official organization we spoke with told us their members
rely on data that was already being collected by their state and local
entities to inform their response to COVID-19. For example,

21Hospitals may not need to use HHS Protect data to monitor local capacity because they
have access to capacity data in real time. For example, one way that hospitals could use
capacity data is to inform transfers of patients; however, representatives from one hospital
association noted that their members have not used the hospital capacity data as reported
to HHS Protect data for transferring patients because the data are not a good tool for
making such decisions. These representatives told us these decisions are best handled at
the local level where facilities monitor capacity in real time.
22For example, one of our selected states adapted a system used to collect capacity
information during natural disasters to collect information on COVID-19-related
hospitalizations and supplies on a daily basis.
23Officials told us that they field multiple password reset requests from states, suggesting
there are active users of the data.
Public Health
Stakeholders and Some
States Relied on State and
Local Data Sources to
Inform Their Response to
COVID-19; According to
HHS, Other States Relied
on HHS Protect Data

Page 17
GAO-21-600  COVID-19
representatives from one epidemiological association told us that their
members did not find HHS Protect data useful for their purposes, noting
that they require more granular data for case investigation than is
available from HHS Protect.
When we asked HHS officials and other stakeholders what key lessons
they learned about ensuring the collection of quality hospital capacity data
during a public health emergency, they offered a variety of lessons. HHS
officials identified the need for timely quality data to support the federal
government’s response to pandemics, among other things.
Representatives from hospital associations, epidemiological associations,
and the three states we interviewed offered several lessons learned from
their experiences on the implementation of HHS Protect. Officials from
stakeholder organizations focused on lessons pertaining to collaboration,
communication, and stakeholder buy-in.
According to HHS officials, lessons learned include:
•
Timely, complete, high-quality data on national supplies, staffing, bed
availability, and therapeutics at the facility-level for hospitals across
the country is critical to guide the national response to a pandemic like
COVID-19.
•
Because a system to meet these critical needs did not exist and had
to be stood up in response to the COVID-19 pandemic, it necessitated
the iterative development and implementation of HHS Protect over
time.
•
There is a need for a system that serves the functions HHS Protect
serves and that need will continue beyond the COVID-19 pandemic.
HHS Protect can be leveraged to create a long-term all-hazards
system that can be quickly adapted to support future public health
emergencies.
•
Dialogue and external validation of data is essential to ensure quality
and accuracy of HHS Protect data.
•
CDC officials specifically noted that ensuring the completeness,
quality, and usefulness of HHS Protect data requires the continued
involvement of CDC staff with expertise in public health, health care
infections, and outcomes analysis to contribute to data collection and
analysis.
Stakeholders, including representatives from hospital and epidemiological
associations and officials from our three selected states noted the
following lessons learned:
Key Lessons
Learned Include
Need for Better
Communication
and Stakeholder
Engagement

Page 18
GAO-21-600  COVID-19
•
Implementing significant changes to the data collection process during
the middle of an emergency could have been better facilitated by
more stakeholder engagement.
•
Allowing for lead time for reporting entities to understand the data
request and align workflow processes, such as building interfaces with
existing data systems, could have helped to ensure quality data
collection.
•
Engaging reporting entities and testing data elements and definitions
to get input and buy-in could have helped ensure reporting entities
had a shared understanding of the data elements.
•
Improving communication regarding changes in reporting, and
providing access to subject matter experts who can answer questions
on data content could have led to improved reporting entity
responsiveness and data quality. For example, officials from
stakeholder organizations we spoke with told us that clear guidance
and documentation, such as clear data dictionaries, timely feedback
to hospitals on errors, and transparency around the use of data during
the initial phases of implementation of HHS Protect would have been
helpful.
The lessons learned regarding the need for collaboration around shared
definitions and data collection goals are consistent with our January 2021
recommendation that HHS engage with stakeholders through establishing
an expert committee, to review and inform the alignment of ongoing data
collection and reporting standards related to key health indicators.24 In our
prior work, we found that involving reporting entities can help ensure that
the collection of additional data elements would be useful and feasible
and developing shared definitions through collaboration can help ensure
the collection of standardized quality data to inform the federal
government’s response to the public health emergency. HHS agreed with
our recommendation, and officials told us HHS has engaged in on-going
discussions with stakeholders; however, as of June 2021, HHS had not
yet established an expert committee, citing resource constraints and the
ongoing pandemic.
We provided a draft of this report to HHS for review and comment. HHS
officials told us that they had no comments on the draft report.

24GAO-21-265.
Agency Comments

Page 19
GAO-21-600  COVID-19
We are sending copies of this report to the Secretary of Health and
Human Services, appropriate congressional committees, and other
interested parties. The report is also available at no charge on the GAO
website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact
me at (202) 512-7114 or at farbj@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made key contributions to this report
are listed in appendix I.

Jessica Farb
Director, Health Care

Page 20
GAO-21-600  COVID-19
List of Addressees
The Honorable Patrick Leahy
Chairman
The Honorable Richard Shelby
Vice Chairman
Committee on Appropriations
United States Senate
The Honorable Ron Wyden
Chairman
The Honorable Mike Crapo
Ranking Member
Committee on Finance
United States Senate
The Honorable Patty Murray
Chair
The Honorable Richard Burr
Ranking Member
Committee on Health, Education, Labor, and Pensions
United States Senate
The Honorable Gary C. Peters
Chairman
The Honorable Rob Portman
Ranking Member
Committee on Homeland Security and Governmental Affairs
United States Senate
The Honorable Rosa L. DeLauro
Chairwoman
The Honorable Kay Granger
Ranking Member
Committee on Appropriations
House of Representatives

Page 21
GAO-21-600  COVID-19
The Honorable Frank Pallone, Jr.
Chairman
The Honorable Cathy McMorris Rodgers
Republican Leader
Committee on Energy and Commerce
House of Representatives
The Honorable Bennie G. Thompson
Chairman
The Honorable John Katko
Ranking Member
Committee on Homeland Security
House of Representatives
The Honorable Carolyn B. Maloney
Chairwoman
The Honorable James Comer
Ranking Member
Committee on Oversight and Reform
House of Representatives
The Honorable Richard Neal
Chairman
The Honorable Kevin Brady
Republican Leader
Committee on Ways and Means
House of Representatives
The Honorable Anna G. Eshoo
Chairwoman
Subcommittee on Health
Committee on Energy and Commerce
House of Representatives
The Honorable Diane DeGette
Chair
Subcommittee on Oversight and Investigations
Committee on Energy and Commerce
House of Representatives

Appendix I: GAO Contact and Staff
Acknowledgments

Page 22
GAO-21-600  COVID-19
Jessica Farb at (202) 512-7114 or farbj@gao.gov

In addition to the contact named above, Gerardine Brennan (Assistant
Director), Jasleen Modi (Analyst-in-Charge), and Kendra Sippel-Theodore
made key contributions to this report. Also contributing were Samuel
Amrhein, Eric Peterson, and Jennifer Rudisill.

Appendix I: GAO Contact and Staff
Acknowledgments
GAO Contact
Staff
Acknowledgments
(104559)

The Government Accountability Office, the audit, evaluation, and investigative
arm of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of the
federal government for the American people. GAO examines the use of public
funds; evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make informed
oversight, policy, and funding decisions. GAO’s commitment to good government
is reflected in its core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost is
through our website. Each weekday afternoon, GAO posts on its website newly
released reports, testimony, and correspondence. You can also subscribe to
GAO’s email updates to receive notification of newly posted products.
The price of each GAO publication reflects GAO’s actual cost of production and
distribution and depends on the number of pages in the publication and whether
the publication is printed in color or black and white. Pricing and ordering
information is posted on GAO’s website, https://www.gao.gov/ordering.htm.
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card, MasterCard,
Visa, check, or money order. Call for additional information.
Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts.
Visit GAO on the web at https://www.gao.gov.
Contact FraudNet:
Website: https://www.gao.gov/about/what-gao-does/fraudnet
Automated answering system: (800) 424-5454 or (202) 512-7700
Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400,
U.S. Government Accountability Office, 441 G Street NW, Room 7125,
Washington, DC 20548
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548
Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707
U.S. Government Accountability Office, 441 G Street NW, Room 7814,
Washington, DC 20548
GAO’s Mission
Obtaining Copies of
GAO Reports and
Testimony
Order by Phone
Connect with GAO
To Report Fraud,
Waste, and Abuse in
Federal Programs
Congressional
Relations
Public Affairs
Strategic Planning and
External Liaison
Please Print on Recycled Paper.

File and source

File

21-600-covid-19-hhs-039-s-collection-of.pdf

Size

1,465,060 bytes

SHA-256

49f69aad41340eb2089daaa61e58c1214f9a7042f24b7fa9acf3bbc3716403bc

Our copy

21-600-covid-19-hhs-039-s-collection-of.pdf

Original

www.gao.gov