Bureau Should Assess Significant Data Collection Challenges as It Undertakes Planning for 2030 (GAO-21-365)
Summary
A GAO report to congressional requesters, GAO-21-365, dated March 2021, on the decennial census. It states that in March 2020 the Census Bureau delayed the start of field data collection because of COVID-19 safety and then revised several operational timelines in response to the pandemic and Department of Commerce decisions. Nationally the Bureau reported completing more than 99 percent of nonresponse follow-up cases by October 15, 2020, and it attributes completing the work by that date in part to its use of technology. Its title states that the Bureau should assess significant data collection challenges as it plans for 2030.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
Full text
United States Government Accountability Office
Report to Congressional Requesters
DECENNIAL
March 2021
CENSUS
Bureau Should
Assess Significant
Data Collection
Challenges as It
Undertakes Planning
for 2030
GAO-21-365
March 2021
DECENNIAL CENSUS
Bureau Should Assess Significant Data Collection
Challenges as It Undertakes Planning for 2030
Highlights of GAO-21-365, a report to
congressional requestors
Why GAO Did This Study What GAO Found
The 2020 Census was conducted In March 2020, the Census Bureau (Bureau) delayed the start of field data
under extraordinary circumstances. collection because of COVID-19 safety, and then revised several operational
In response to the COVID-19 timelines in response to the pandemic and Department of Commerce
pandemic and related Commerce (Commerce) decisions. Nationally the Bureau reported completing more than 99
decisions, the Bureau made a series percent of nonresponse follow-up cases (households that have not responded to
of late changes to the design of the the census) by October 15, 2020. The Bureau attributes the use of technology as
census. As GAO previously among the reasons it completed the work by this date.
reported, these changes introduced
risks to the quality of data that the The Bureau, however, had lower completion percentages ranging between 94
Bureau provides for congressional and 99 for 10 local geographic areas, in part because of natural disasters and
apportionment and redistricting COVID-19. For example, according to the Bureau, in Shreveport, Louisiana,
purposes. short-term closures stemming from the hurricane impacted data collection for
GAO was asked to review the 82,863 housing units. As a mitigation strategy, the Bureau shifted the Shreveport
Bureau’s implementation of the 2020 operation to telephone enumeration and brought in more than 1,200 enumerators
Census. This report assesses the from travel teams. Despite these efforts, the Bureau was unable to complete
Bureau’s implementation of the: (1) 22,588 cases in Shreveport before data collection ended. For these cases the
nonresponse follow-up operation, (2) Bureau will need to rely on alternate methods including imputation, which draws
group quarters enumeration, and (3) data from similar nearby households to determine whether a housing unit exists,
plans to assess those operations. To whether it is occupied, and, if so, by how many people.
address these objectives, GAO
In addition to the challenges brought on by natural disasters, the Bureau
conducted a series of surveys of all
248 census offices during the encountered other difficulties during nonresponse follow-up, such as, the inability
collection of data for those of supervisors to reassign open cases in a timely fashion. GAO found that
operations. GAO also monitored the census field supervisors did not have the authority to reassign cases and had to
cost and progress of operations and wait for the field manager to make those reassignments. Bureau officials told
interviewed census field supervisors GAO it would consider the reassignment of cases as it moves towards planning
for each operation. for the 2030 Census.
What GAO Recommends To monitor nonresponse follow-up, the Bureau used quality control procedures,
such as real-time monitoring of enumerator activities by supervisors and training
GAO is recommending that the assessments. However, GAO found the Bureau did not have proper controls in
Bureau inform 2030 Census place, allowing some enumerators to work without having passed the required
planning by evaluating the effects of training assessment. The Bureau agreed that additional controls were necessary.
major operational changes made
during data collection and
The Bureau planned to count individuals living in group quarters, such as skilled-
challenges encountered for
nursing and correctional facilities, between April 2, 2020, and June 5, 2020, but
nonresponse follow-up and group
quarters enumerations. In its
revised those dates to July 1, 2020, through September 3, 2020. The pandemic
comments, the Department of made it difficult to count group quarters. For example, Bureau staff found it
Commerce agreed with GAO’s challenging to locate a point of contact at some group quarters because facilities
findings and recommendation. The were closed due to the pandemic. Bureau officials told us that in December 2020
Bureau also provided technical they decided to re-contact more than 24,000 out of approximately 272,000 group
comments, which GAO incorporated quarter facilities to collect data, and that imputation would be used to count
as appropriate. individuals at the remaining facilities still reporting a zero population count.
The Bureau is updating plans to assess operations and identify resulting lessons
learned from the 2020 Census. As part of its planning for 2030, it will be
View GAO-21-365. For more information,
contact J. Christopher Mihm, 202-512-6806, or important for the Bureau to assess the impact of the 2020 late design changes
mihmj@gao.gov and the operations’ challenges that arose.
United States Government Accountability Office
Contents
Letter 1
Background 3
The Bureau Generally Met Its Nonresponse Follow-up Operational
Goals, but Faced Data Collection and Monitoring Challenges 5
During Pandemic, the Bureau Had Successes and Challenges
Counting People in Group Quarters and Service-Based
Locations 18
Bureau Needs to Assess Design Changes and Data Collection
Challenges as It Begins Planning for 2030 28
Conclusions 29
Recommendations for Executive Action 29
Agency Comments and Our Evaluation 30
Appendix I Objectives, Scope, and Methodology 33
Appendix II Comments from the Department of Commerce 36
Appendix III GAO Contact and Staff Acknowledgments 37
Tables
Table 1: Area Census Offices That Completed Less than 99
Percent of Their Nonresponse Follow-up Workload 11
Table 2: Example of Group Quarter Facility Population Overcount 24
Table 3: Interviewed Census Field Supervisors from 25 Area
Census Offices 34
Figures
Figure 1: Census Bureau COVID-19 Related Safety Protocols 5
Figure 2: 2020 U.S. Census Bureau’s Operations Timeline 6
Figure 3: 2020 Census Nonresponse Follow-up Operation
Cumulative Productivity 10
Figure 4: Group Quarters Operation Generally Progressed
According To Revised Schedule 19
Figure 5: Satisfaction with Personal Protection Equipment
Provided for the Group Quarters Operation among Area
Census Office Managers Improved over Time 21
Page i GAO-21-365 Decennial Census
Figure 6: Screenshot of Group Quarters eResponse Excel Data
Entry Template 26
Abbreviations
ACOs Area Census Offices
Bureau Census Bureau
CFS Census Field Supervisors
COVID-19 Coronavirus Disease 2019
Commerce Department of Commerce
eResponse Electronic Response Data Transfer
PPE Personal Protective Equipment
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.
Page ii GAO-21-365 Decennial Census
Letter
441 G St. N.W.
Washington, DC 20548
March 22, 2021
Congressional Requesters
The decennial census is a complex and costly undertaking under ordinary
conditions. The 2020 Decennial Census was taken under extraordinary
circumstances. In response to the Coronavirus Disease 2019 (COVID-19)
pandemic and related Department of Commerce (Commerce) decisions,
the Census Bureau (Bureau) made a series of late changes to the design
of the census. These changes affected the schedule and way the Bureau
collected census responses in the field. The changes also introduced
risks to the quality of data that the Bureau provides for congressional
apportionment and redistricting purposes. As we testified in December
2020, for these reasons it will be important for the Bureau to conduct a
robust evaluation and assessment process to determine the quality of
data collection operations for the 2020 Census. 1
The largest field data collection operation for the 2020 Census was
nonresponse follow-up. Nonresponse follow-up is an operation where
enumerators personally visit households that do not respond to the
census in order to count the household. The Bureau also conducts group
quarters enumeration which is a count of residents of group facilities
(such as skilled nursing facilities and correctional facilities). Another
enumeration operation that generally is conducted at about the same time
as group quarters enumeration is service-based enumeration. For this
operation, the Bureau counts individuals experiencing homelessness
where they receive services such as at soup kitchens and shelters, or at
non-sheltered outdoor locations where they sleep.
You asked us to review how the 2020 Census Nonresponse Follow-up
and Group Quarters Enumeration operations performed. This report
assesses (1) the Bureau’s implementation of the 2020 Census
Nonresponse Follow-up; (2) the Bureau’s implementation of Group
Quarters Enumeration and Service-Based Enumeration operations; and
(3) the Bureau’s plans to assess these operations.
To accomplish these objectives, we reviewed key documents including
the 2020 Census Operational Plan that discussed the goals and
1GAO, 2020 Census: Census Bureau Needs to Ensure Transparency over Data Quality,
GAO-21-262T (Washington, D.C.: Dec. 3, 2020).
Page 1 GAO-21-365 Decennial Census
objectives for each of the operations included in this review, weekly
updates on cost and progress of operations, as well as training manuals
and e-training modules and other related documents for nonresponse
follow-up, group quarters enumeration, and service-based enumeration.
To obtain an on-the-ground perspective of the conduct of nonresponse
follow-up, group quarters enumeration, and service-based enumeration,
we interviewed 40 Census Field Supervisors (CFS) from 20 Area Census
Offices (ACO) for nonresponse follow-up, 20 CFSs from 10 ACOs for
group quarters enumeration, and six CFSs from the service-based
enumeration operation. ACOs were selected based on several factors,
including workload size, expected response rate, and recruitment
capabilities. Information collected from CFSs are not generalizable to the
experiences of all CFSs nationally.
We also surveyed ACO managers at all 248 ACOs, regularly interviewed
Census Headquarters officials, and reviewed data from the Bureau’s
information technology systems to monitor progress throughout peak
operations. We examined data from the Bureau’s unified tracking systems
to check for logical errors and inconsistencies, and followed up with
agency officials knowledgeable about the data in cases where we had
questions about potential errors or inconsistencies, and to inquire about
the accuracy and completeness of the data. Based on these efforts, we
determined those data to be sufficiently reliable for the purposes of our
reporting objectives. Finally, we compared the information we gathered
from the interviews and surveys to the Bureau’s Operational Plans,
including plans revised due to COVID-19. Appendix I presents a more
detailed description of our scope and methodology.
We conducted this performance audit from March 2020 to March 2021 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Page 2 GAO-21-365 Decennial Census
For the 2020 Census, the Bureau was required by law to count the
Background population as of April 1, 2020 (Census Day) and deliver state population
counts to the President by December 31, 2020, to determine the number
of congressional seats apportioned to each state. 2 The Bureau is also
required by law to deliver population counts to the states within 1 year of
Census Day for redistricting purposes—March 31, 2021, in the case of
the 2020 Census. Even in normal circumstances, conducting the count to
meet these deadlines is an enormous and complex undertaking, for which
the Bureau spends years planning. This decennial, the COVID-19
pandemic made the effort all the more challenging for the Bureau.
While the cost of the decennial census has steadily increased over the
past several decades, with enumeration costs rising from about $16 per
household in 1970 to around $92 in 2010 (all in constant 2020 dollars),
the percentage of households self-responding to mailed census
questionnaires has generally declined. The self-response rate declined
from 78 percent in 1970 to 63 percent in 2010. However, for the 2020
Census, the Bureau saw an increase in the self-response rate to 67
percent. According to Bureau Officials, this was due in part to the
inclusion of an internet response option and the extension of the deadline
for self-response from July 31, 2020, to October 15, 2020, due to COVID-
19.
To ensure nonrespondents are included in the decennial count the
Bureau conducts the nonresponse follow-up operation—where
enumerators personally visit nonresponding households. Nonresponse
follow-up is labor intensive and is the largest and costliest operation that
the Bureau undertakes. During the nonresponse follow-up operation,
enumerators confirm vacant and nonexistent addresses and make
repeated visits to occupied homes to capture full information from each
household. Enumerators also leave a notice of visit form if they cannot
reach a household, encouraging a member of the household to go online
or call a Census Questionnaire Assistance Center to complete a self-
response.
Happening at about the same time as nonresponse follow-up, the group
quarters enumeration operation counts individuals living or staying in
group housing, such as college/university student housing, correctional
facilities, and nursing/skilled-nursing facilities. To count people during this
2On January 28, 2021, the Bureau released a statement that its current schedule points to
April 30, 2021, for the completion of the apportionment counts.
Page 3 GAO-21-365 Decennial Census
operation, enumerators may visit the facility to administer the census
questionnaires; pick up completed questionnaires or paper listings of
census response data for each person who is served or staying at the
facility on April 1, 2020; or swear in and train group quarters
administrators to conduct the enumeration of their own facility. Group
quarters facilities also had the option to self-enumerate electronically
through a new Electronic Response Data Transfer (eResponse) option.
Additionally, the service-based enumeration and targeted non-sheltered
outdoor location operations are used to count persons experiencing
homelessness. To conduct these operations, the Bureau sent census
takers to count people at shelters, soup kitchens, and locations previously
identified by the Bureau where people are known to sleep outdoors.
Enumerators completed in-person interviews or, in response to the
pandemic, shelters were given the option to have enumerators drop off
and pick up completed paper listings from administrators.
To adapt to the COVID-19 pandemic, the Bureau acquired personal
protective equipment (PPE) for all field and office staff. Specifically, the
Bureau reported that it provided more than 3.2 million masks, 14 million
gloves, and 3.3 million individually sized bottles of hand sanitizer to field
staff. Also, most training was conducted remotely due to COVID-19
concerns. The Bureau provided 2 hours of in-person enumerator training
where enumerators also received equipment. The original plan had been
for 2 days of in-person training. Training discussed avoiding contact and
remaining outside at a 6-foot distance (see fig. 1).
Page 4 GAO-21-365 Decennial Census
Figure 1: Census Bureau COVID-19 Related Safety Protocols
In March 2020, the Bureau delayed the start of nonresponse follow-up
The Bureau Generally because of COVID-19 safety concerns, and then revised operational
Met Its Nonresponse timelines in response to the pandemic and Commerce decisions. Figure 2
shows a timeline of these events.
Follow-up Operational
Goals, but Faced
Data Collection and
Monitoring
Challenges
Page 5 GAO-21-365 Decennial Census
Figure 2: 2020 U.S. Census Bureau’s Operations Timeline
Initial suspensions. As part of its initial COVID-19 response and to
promote the safety of its workforce and the public, the Bureau delayed its
field operations multiple times. On March 18, 2020, the Bureau
announced it would suspend operations for 2 weeks. On March 28, 2020,
the Bureau announced another 2-week suspension. In recognition of the
pandemic’s community spread, on April 13, 2020, the Bureau announced
that it would extend the suspension of nonresponse follow-up to a total of
3 months.
Page 6 GAO-21-365 Decennial Census
Plan for delayed delivery of population counts. On April 13, 2020, the
Bureau and Commerce announced plans to deliver population counts for
apportionment purposes by April 30, 2021, and for redistricting purposes
by July 31, 2021—4 months later than the respective statutory deadlines.
According to the announcement, Commerce had requested that
Congress grant a 120-day extension to the Bureau’s reporting deadlines.
The Bureau also announced an extension to the self-response period and
planned to conduct the nonresponse follow-up operation through October
31, 2020.
Gradual resumption of operations. The Bureau gradually resumed
nonresponse follow-up operations, announcing a phased approach on
May 4, 2020. Between July 16, 2020, and August 9, 2020, the Bureau
began nonresponse follow-up in areas that the Bureau deemed
sufficiently safe. Nonresponse follow-up was fully launched in all areas by
August 9, 2020.
Time frames revised again to meet original statutory deadlines. On
August 3, 2020, the Bureau announced that it would accelerate its
timeline, as directed by the Secretary of Commerce, to deliver population
counts for apportionment and redistricting purposes by the statutory
deadlines—4 months earlier than the plan the Bureau announced on April
13, 2020. To meet this timeline, the Bureau announced that nonresponse
follow-up and the self-response period would now end on September 30,
2020—1 month earlier than previously announced.
August 3 re-plan of nonresponse follow-up. The Bureau developed a
nonresponse follow-up “re-plan” that modified certain procedures to meet
the shortened time frame. For example, to meet production goals the
Bureau offered enumerators awards, encouraged increased enumerator
travel to areas in need, and instituted phone enumeration as an additional
contact method. To reduce the workload of nonresponse follow-up, the
Bureau also made a set of changes to quality control including reducing
the scope of the quality control operation and reducing the number of
contact attempts for a quality control interview.
The nonresponse follow-up operation continued under the re-plan for
approximately 1 month before the U.S. District Court for the Northern
District of California issued a temporary restraining order, followed by a
preliminary injunction, which barred the Bureau from operating under the
August 3 re-plan. Due to the injunction, the Bureau ceased re-plan
operational changes and the court required that nonresponse follow-up
operations continue past the Bureau’s September 30, 2020, deadline. On
Page 7 GAO-21-365 Decennial Census
October 13, 2020, the Supreme Court granted a stay of the injunction,
allowing the Bureau to cease data collection, and on October 15, 2020,
the Census Bureau ended all data collection operations, including
nonresponse follow-up.
Processing census data. On November 20, 2020, senior Bureau
officials told us that they ran into what the Bureau referred to as
anomalies as they processed responses for the apportionment count. For
example, as they integrated data in group quarter counts they found
mismatches and duplicates. In addition, as they changed the year for the
date of birth to align with a person’s age they found that the system was
sometimes aging people incorrectly. According to Bureau officials,
processing anomalies are not unexpected, in that they occur with each
census, and time is typically built into the schedule to identify and address
them. As a result of addressing the anomalies to ensure quality data, as
of March 2021 apportionment data have not yet been delivered to the
President. 3
The Bureau Met National The Bureau reported that nonresponse follow-up came in under budget
Nonresponse Follow-up and completed more than 99 percent of cases by the end of the
operation. The Bureau reported that as of March 5, 2021, it had spent
Completion Goals Overall,
nearly $1.43 billion dollars, more than $180 million under its planned
but Struggled in Some budget of $1.61 billion.
Areas
By October 15, the Bureau reported that it had completed 99.93 percent
of its national nonresponse follow-up workload (almost 64.1 million
cases), which included occupied, vacant, nonexistent housing units, as
well as cases selected for quality control and other addresses added for
verification. According to Bureau documentation, a nonresponse follow-up
case is considered completed under the following circumstances:
• a field interview of a household member is conducted;
• the household self-responds via phone, mail, or internet after being
added to the nonresponse follow-up workload;
• a proxy (i.e., neighbor, landlord, caretaker, etc.) provides information
on the household;
3On January 28, 2021, the Bureau released a statement that its current schedule points to
April 30, 2021, for the completion of the apportionment counts.
Page 8 GAO-21-365 Decennial Census
• after visiting the household at least once, the Bureau uses
administrative records such as Internal Revenue Service or Social
Security records to enumerate the household;
• the address is identified as vacant or nonexistent by a field visit or a
proxy interview; or
• after the maximum attempts (typically six visits) the case is flagged as
closed. 4
Bureau officials attribute high productivity as one of the factors that led to
the Bureau completing 99.93 percent of its nonresponse follow-up by
October 15. 5
Census data presented in figure 3 show reported productivity throughout
the operation exceeded the Bureau’s projections. Specifically, according
to Bureau reporting, as of October 15, a cumulative 1.92 cases were
completed per hour, exceeding the Bureau’s planned 1.55 cases. Bureau
officials told us that increased use of technology in the 2020 Census,
including the use of software on handheld devices to assign cases,
helped increase productivity.
4Maximum attempt cases were reopened near the end of nonresponse follow-up in an
attempt to collect the data. The Bureau imputes data for cases still open at the end of
nonresponse follow-up.
5The Bureau rounds to whole numbers due to disclosure avoidance rules and its
published numbers may differ from ours as a result.
Page 9 GAO-21-365 Decennial Census
Figure 3: 2020 Census Nonresponse Follow-up Operation Cumulative Productivity
Under the August 3 re-plan, the Bureau employed other methods to
complete nonresponse follow-up cases within the shortened time frame.
Specifically, the Bureau offered enumerators financial awards for working
additional hours while completing a minimum number of cases, and
removed restrictions on overtime. Of the area census office managers we
surveyed, 48.3 percent of respondents stated that they were satisfied with
the effectiveness of awards to speed up nonresponse follow-up, retain
staff, or increase the number of hours worked.
The Bureau also increased the use of travel teams to complete cases in
hard-to-count areas, sending approximately 26,000 enumerators to other
areas or regions during nonresponse follow-up. In some circumstances,
the Bureau also offered incentive awards to enumerators willing to travel
to particular areas. In March 2021, the Bureau reported $98.4 million in
expenses for enumerator awards, $795,000 of which was for travel
awards.
Page 10 GAO-21-365 Decennial Census
Areas Where Census Although the Bureau completed more than 99 percent of nonresponse
Completion Rates Were follow-up cases nationwide, some geographic areas were less complete.
Specifically, 10 Area Census Offices (ACOs) completed less than 99
Less than the National
percent of their nonresponse follow-up workloads (see table 1). The
Average Shreveport area in Louisiana had the lowest completion rate with 93.54
percent of workload completed. Bureau officials attributed lower
completion rates in some geographic areas to challenges such as natural
disasters (hurricanes) and difficulties accessing some locations, such as
tribal lands due to COVID-19 restrictions.
For example, Louisiana was hit by a category 4 hurricane in August.
Evacuations and short-term closures stemming from the hurricane
impacted data collection for the remaining uncounted 82,863 housing
units in Shreveport and 25,062 in Jefferson Parish, according to the
Bureau. As a mitigation strategy, the Bureau shifted the Shreveport
operation to telephone enumeration and brought in more than 1,200
enumerators from travel teams to the area once it was safe to return.
Despite these efforts, the Bureau was unable to complete 22,588 cases in
Shreveport before the nonresponse follow-up operation ended. For these
cases the Bureau will need to rely on alternate methods including
imputation, which draws data from similar nearby households to
determine whether a housing unit exists, whether it is occupied, and, if so,
by how many people.
Table 1: Area Census Offices That Completed Less than 99 Percent of Their Nonresponse Follow-up Workload
Percent of Nonresponse Follow- Number of
Area census office State up workload complete open cases
Shreveport Louisiana 93.54% 22,588
Manhattan 2 New York 98.49% 4,192
Window Rock Arizona 98.74% 1,129
San Francisco California 98.75% 1,999
Chicago South Illinois 98.86% 1,929
Quincy Massachusetts 98.88% 1,104
Sunnyvale California 98.89% 1,803
New Haven Connecticut 98.93% 1,876
Vista California 98.94% 1,430
Colorado North Colorado 98.96% 2,823
Source: GAO analysis of Census Bureau reports. | GAO-21-365
Notes: The data above are as of October 19, 2020, which accounts for delays in reporting. However,
the nonresponse follow-up operation ended on October 15, 2020. The data source used here does
not count cases with a status of “Max Attempts – Unresolved” as completed.
Page 11 GAO-21-365 Decennial Census
Under the August 3 re-plan, in areas struggling to resolve cases or facing
challenges to accomplish in-person interviews because of COVID-19 and
natural disasters, enumerators were able to enumerate households by
phone. 6 According to Bureau data, enumerators made more than 10
million calls and completed nearly 1.2 million cases by phone. This new
option was made available at the discretion of regional leadership
beginning August 12 through the end of nonresponse follow-up. Phone
enumeration allowed enumerators to attempt an interview without being
physically present at the location. Enumerators could also leave a
voicemail encouraging the resident to call the Census phone number.
However, one CFS we spoke to told us that phone enumerations were
difficult to conduct remotely due to nonresponse follow-up rules that
required an in-person proxy rather than over the phone after three contact
attempts. After we notified the Bureau of this concern, the Bureau clarified
for enumerators how they could use the handheld device to indicate that
they could not complete an in-person proxy, or to complete the proxy
interviews later when they returned to the field. According to Bureau
officials, while many phone numbers were not always current or were
nonresidential, they were, nevertheless, satisfied with its success to make
contact and complete census responses during nonresponse follow-up.
Bureau Faced Challenges We identified other factors that may have contributed to the Bureau’s
Completing Nonresponse difficulty completing cases, including restricted access to large multi-unit
buildings. According to 24 of 40 CFSs we interviewed, enumerators
Follow-up Cases
experienced difficulties completing cases due to building access issues.
Some CFSs we interviewed (16 out of 40) attributed these access issues
to COVID-19 related concerns. In our late September to early October
survey of area census office managers, less than half (41.5 percent) of
responding managers were satisfied with procedures for enumerating
large multi-unit structures.
According to Bureau officials, the ACO officials along with partnership
staff were to help enumerators gain access to multi-unit buildings.
However, the pandemic made communication with the building managers
difficult. Specifically, CFSs told us that enumerators were often turned
6Enumerators called nonresponding households and offered to complete their
questionnaire for them over the phone. In cases where the enumerator did not reach the
respondent, the enumerator left a voicemail to remind the respondent of the ways in which
they could complete the census questionnaire. Telephone contact also included calls
made by enumerators when they located a phone number in the field, such as for a real
estate agent or building manager. Phone numbers provided to enumerators were from
other surveys administered by the Bureau, including the American Community Survey.
Page 12 GAO-21-365 Decennial Census
away from accessing multi-unit buildings because of the pandemic.
Bureau officials at headquarters also told us that some building managers
worked remotely during the pandemic, often leaving enumerators with no
point of contact. Bureau officials told us that even before COVID-19,
access to multi-units had been an issue and that the pandemic made
access to multi-unit buildings more challenging, and that they will
continue to look for ways to address access issues. If enumerators could
not get access to conduct an interview, then the Bureau had to rely on
alternative sources of data collection such as administrative records or
proxy data.
ACO managers surveyed expressed dissatisfaction with the technological
capability used by the Bureau to assign and route cases (a capability
known as the optimizer). The 2020 Census was the first decennial to use
electronic case assignment. Only 21.6 percent of ACO managers who
responded to our final survey reported satisfaction with the accuracy and
efficiency of the optimizer to assign cases. CFSs we interviewed
mentioned specific frustrations with the optimizer, such as what they
determined was the illogical assignment of cases that led to routes that
sent enumerators to households in a confusing order, the enumeration
start times late in the day, and the uneven distribution of cases among
available enumerators. These CFSs expressed concern that the optimizer
negatively affected nonresponse follow-up efficiency.
When we asked Bureau officials about these concerns, they said that the
optimizer’s algorithm works effectively. Specifically, Bureau officials
pointed to the increased productivity for the 2020 nonresponse follow-up
operation as an indicator of the optimizer’s effectiveness. Officials also
clarified that the routes planned by the optimizer can appear
counterintuitive because it calculates the best route and time frame to
maximize in-person interviews. However, Bureau officials acknowledged
that they might have done a better job of explaining the optimizer during
the CFS and enumerator training. Bureau officials said they plan to
evaluate the effectiveness of new technological tools such as the
optimizer, as well as the training.
CFSs also said they experienced some challenges related to case
reassignments. The Bureau’s protocol positioned Census Field
Managers, who supervise the CFSs, to assign cases. Some CFSs we
interviewed reported difficulty with the timely reassignment of cases,
especially in multi-unit buildings, due to the Field Managers’ volume of
work and other responsibilities. Bureau officials told us that the
reassigning of cases occurred more often than expected because of the
Page 13 GAO-21-365 Decennial Census
increased need for enumerator travel, but understood the need to
reassign cases in a timely manner for routine multi-unit visits, and would
reconsider whether the CFS should have responsibility for reassigning
cases.
Perspective on Case Assignments
“I would like to have more power to assign cases. I have been in touch with
luxury building management and, for example, one said that enumerators
can come in the next day from 10 AM to 3 PM but as a CFS I can’t assign
enumerators these addresses for this requested timeframe. The CFM
[Census Field Manager] can turn requests around but they are overwhelmed.”
Source: GAO interview with Census Field Supervisor. | GAO-21-365
The Bureau Faced The Bureau used several quality control procedures to monitor
Challenges Monitoring nonresponse follow-up in near real time that posed challenges.
Quality during
Supervisory Alerts. To effectively manage nonresponse follow-up, the
Nonresponse Follow-up Bureau provides data-driven tools for the CFSs to manage enumerators,
including system alerts that identify issues that require the supervisor to
follow up. Operational issues such as enumerators not working assigned
hours or falling behind schedule need to be resolved quickly because of
the tight time frames. For the nonresponse follow-up operation, the
system generated codes that covered a variety of operational issues such
as unusually high or low productivity (which may be a sign of fraud or
failure to follow procedures) and administrative issues such as the
completion of expense reports and time cards.
During the operation, more than 10 million alerts were sent to census field
supervisors, with the highest number of alerts (2.2 million) in the Atlanta
region. Each alert requires the supervisor to take action and then record
how the alert was resolved. According to census field supervisors we
interviewed, a little more than half (21 out of 40) said they received 20 or
fewer alerts per day. Other supervisors (11) reported receiving 21 or more
alerts per day, with two reporting circumstances where they received
hundreds of alerts a day.
To assist supervisors, these alerts need to be reliable and properly used.
However, 17 out of 40 census field supervisors we spoke to indicated the
alerts were not always useful. For example, CFSs told us that alerts
would be triggered for enumerators standing too far from the door of the
household (inaccurate location) or if an enumerator’s start time was
Page 14 GAO-21-365 Decennial Census
outside of the approved time range. According to Bureau officials,
enumerators did not always stand in front of the door because of COVID-
19 and this often triggered the inaccurate location alert. Bureau officials
also told us that enumerators also logged on early to view their cases for
the day and that this would trigger the enumerator start time (too early)
alert.
In addition, our area census office manager survey results collected
toward the end of the nonresponse follow-up operation show that 41.3
percent of responding managers were satisfied with the usefulness of
CFS alerts. Moreover, we previously reported during the address
canvassing operation for the 2020 Census that CFSs also found that
alerts were not always useful. 7 According to Bureau officials, they plan to
review new technology, which includes CFS alerts, as part of its
nonresponse follow-up assessment plan.
Further, Bureau officials determined that one type of alert (“No Proxy
Attempt”) did not trigger properly in the Los Angeles, Dallas, and Chicago
regions. This type of alert is triggered when the enumerator is prompted
to contact a proxy but fails to record that an attempt was made. According
to Bureau documentation, alerts are important because they notify
supervisors to enumerators not following procedures. According to
Bureau officials, they will be looking into why the no proxy alert failed to
trigger in some parts of the country.
Enumerator and census field supervisor evaluation after initial
training. Ensuring that the material learned during the initial training of
enumerators and CFSs is understood and retained is critical. At the end
of training, enumerators and CFSs are evaluated using automated
software that replicates various situations and prompts them for the
correct action to take. Not only does this post-training knowledge
assessment reiterate what the enumerators and CFSs have already
learned, but also more training can be provided for anyone who did not
retain or understand the training. For example, the enumerators who
score less than 70 percent on this assessment are supposed to receive
additional on-the-job training led by their CFS (referred to as an
observation).
7GAO, 2020 Census: Bureau Generally Followed Its Plan for In-Field Address
Canvassing, GAO-20-415 (Washington, D.C.: Mar. 12, 2020).
Page 15 GAO-21-365 Decennial Census
According to Bureau quality control reports, about 2.45 percent (nearly
8,000) of active enumerators failed the assessment and were not
observed, yet continued to work. According to the Bureau this is an
improvement from an earlier census test. During the Bureau’s 2018 End-
to-End Address Canvassing Test the Commerce Inspector General
identified 26 out of 237 listers (11 percent) who either had no score
recorded to show they took the training assessment, or failed the
assessment and had no record of successful observation. 8
According to Bureau officials, for the 2020 Census they actively
monitored enumerators who had failed the training assessment. For
example, a regional quality working group monitored enumerator
assessment scores and tracked whether needed observations by the
census field supervisor had occurred. According to Bureau officials,
regional operations staff were notified of enumerators who continued to
work but had not yet passed the assessment or had an observation.
We were told that some enumerators were given up to three chances to
pass the assessment; however, the system that tracks assessment
scores is not integrated with the system that assigns work to
enumerators. Therefore enumerators who had failed still received work.
Bureau officials told us that enumerators were not always deactivated and
attributed this, in part, to disruptions caused by COVID-19, and the rapid
nature of the operation, which made scheduling and completing training
observations challenging for census field supervisors. Bureau officials
also said that paper observation forms were not always entered into the
training system in a timely fashion, thus inflating the number of
enumerators flagged by quality control as working without a completed
observation.
Nevertheless, enumerators completing cases without having passed the
assessment or without having been observed introduced risk that cases
could be improperly completed. Bureau officials agreed that enumerators
should not be working if they have not passed the assessment and that
additional controls were necessary. Officials also told us that they intend
to automate the submission of forms for the 2030 Census and improve
the ability to monitor and correct enumerators working (after having failed
the initial assessment) without proper training or observation.
8U.S. Department of Commerce, Office of Inspector General, 2020 Census: Issues
Observed During the 2018 End-to-End Census Test’s Address Canvassing Operation
Indicate Risk to Address List Quality, OIG-19-008-A (Washington, D.C.: February 2019).
Page 16 GAO-21-365 Decennial Census
Quality control reinterviews. The Bureau conducts a sample of
reinterview cases to ensure enumerators follow proper procedures and
that results were not falsified. For the 2020 Census, the Bureau
conducted approximately 1.57 million reinterview cases. Once a case is
selected for reinterview, it is assigned to an enumerator who did not
complete the original interview. Under the original plan, enumerators had
up to six attempts to conduct the reinterview. The reinterview operation
compares the reinterview data to the original nonresponse follow-up
interview to identify enumerator errors and possible falsification.
The Bureau selects cases for reinterview using random, analytic, or
supplemental assignment. Random reinterviews are a selected sample of
cases from each enumerator’s workload. For analytic reinterviews, the
Bureau uses statistical calculations to flag enumerators whose work
differs significantly from others. Cases completed by these enumerators
are chosen so that an analytic reinterview can be used to further
investigate these enumerators to determine if they are following proper
enumeration procedures. Quality control staff at the Bureau’s National
Processing Center can select cases for supplemental reinterview if they
suspect that an enumerator may not be following procedures. The Bureau
also conducts “re-work” reinterviews to verify or correct all eligible case
information collected by enumerators who the Bureau confirms to have
falsified information. For the 2020 Census, the Bureau reported that
587,000 random reinterviews were conducted, compared to 970,000
analytic reinterviews. Less than 12,000 supplemental cases were
conducted.
Because of the August 3 re-plan, the Bureau eliminated the random
selection of reinterview cases and reduced the number of contact
attempts for conducting a reinterview from six to three. According to
Bureau officials, these steps were taken to meet the new time frames for
completing nonresponse follow-up, which moved from October 31 to
September 30. In making these changes the Bureau estimated that it
would reduce the number of random reinterviews by an estimated
800,000 cases and increase productivity, but noted that there could be a
potential reduction in the quality of enumerator work. Subsequently, on
September 5, 2020, a temporary restraining order was issued and the
Bureau reversed the changes it made to reinterview as part of the August
3 re-plan. As such, it resumed making six visits to obtain a reinterview
and resumed the random selection of cases for reinterview.
Bureau officials told us that with the temporary elimination of random
reinterviews before the restraining order 23.5 percent of (nearly 70,000)
Page 17 GAO-21-365 Decennial Census
enumerators did not have cases selected. The Bureau also stated that,
nevertheless, it believes that its analytic selection techniques sufficiently
detect unusual patterns, and thus for 2030 it may consider not randomly
selecting cases for reinterview.
During Pandemic, the
Bureau Had
Successes and
Challenges Counting
People in Group
Quarters and Service-
Based Locations
The Bureau Completed The Bureau had planned to count individuals living in group quarters
Group Quarters According between April 2 and June 5, 2020, but it delayed the in-field operation due
to the pandemic. The in-field operation progressed according to a revised
to a Revised Schedule
schedule. The Bureau began the in-field group quarters operation on July
and under Budget 1, 2020, and completed field work September 3, 2020 (see fig. 4).
The Bureau collected information from 272,164 group quarter facilities.
This included group quarters from military enumeration, maritime/military
vessel enumeration, service-based enumeration, and other group
quarters enumeration operations. Most facilities (62 percent, or
approximately 168,741) were of three types: noncorrectional adult group
homes, college/university student housing, and nursing/skilled-nursing
facilities.
Page 18 GAO-21-365 Decennial Census
Figure 4: Group Quarters Operation Generally Progressed According To Revised
Schedule
Progress after data processing was completed.
a
The Bureau completed the group quarters operation under budget. In
March 2021, the Bureau reported that the operation cost $60.2 million,
while the budgeted cost was $72.4 million—a difference of 17 percent.
The Bureau stated many group quarters administrators opted for low-
contact enumeration methods, which required fewer or no enumerators
and lowered costs.
The service-based enumeration operation, designed to count individuals
experiencing homelessness, was scheduled for March 30, 2020, to April
1, 2020, but was also delayed due to the pandemic. After consulting with
stakeholders, including direct service providers, advocates, and local
government officials, the Bureau conducted the operation between
September 22, 2020 and September 30, 2020. The Bureau reported that
it enumerated more than 53,000 service-based locations, including almost
37,000 temporary non-sheltered outdoor locations.
Page 19 GAO-21-365 Decennial Census
The Bureau budgeted for an estimated 77,000 service-based
enumeration locations and identified 53,701 service-based locations to be
enumerated at the start of the service-based enumeration operation.
Bureau officials told us some service-based sites shut down due to the
pandemic and natural disasters. They said people fleeing disasters and
experiencing homelessness who were housed in hotels due to the
pandemic would be counted as part of a separate transitory locations
operation. 9
In March 2021, Bureau officials stated the cost of the service-based
enumeration operation was $9.7 million, compared to an anticipated cost
of $14 million. Similar to group quarters, the Bureau stated low-contact
enumeration methods at emergency and transitional shelters required
fewer enumerators and reduced the cost of the operation.
The Bureau Adapted To count group quarters during the pandemic, the Bureau implemented
Operations in Response to new low-contact enumeration options and COVID-19 safety measures.
For example, the Bureau encouraged administrators at group quarters
the Pandemic
facilities to switch their method of enumeration from in-person
enumeration to eResponse (whereby facility administrators can
electronically submit enumeration data at a date of their choosing within
operational time frames) and offered group quarter facilities the chance to
mail back paper listings, rather than have them be picked up by an
enumerator. These changes helped reduce person-to-person contact. By
contrast, in 2010, census workers visited all group quarters in person.
Field staff responsible for conducting group quarters enumeration
reported they were generally satisfied with the Bureau’s COVID-19 safety
measures. These measures included providing staff with PPE and
requiring enumerators to practice social distancing and wear masks
during field operations. The majority (16 out of 18) of group quarters
CFSs who we interviewed said they were very or generally satisfied with
the Bureau’s response during the pandemic.
Our survey of area census office (ACO) managers also found that they
grew more satisfied with the adequacy of PPE provided over time. When
surveyed in mid-May, 48 percent of the ACO managers who responded
9The Enumeration at Transitory Locations operation enumerates respondents living in
transitory locations such as recreational vehicle parks, campgrounds, hotels, motels,
marinas, racetracks, circuses, and carnivals, and those who do not have a usual home
elsewhere.
Page 20 GAO-21-365 Decennial Census
said they were satisfied with the PPE provided; by August the percent
satisfied increased to 77 percent.
Figure 5: Satisfaction with Personal Protection Equipment Provided for the Group
Quarters Operation among Area Census Office Managers Improved over Time
The Bureau Experienced
Challenges with Group
Quarter Data Collection
Data Collection in the The Bureau experienced group quarter and service-based enumeration
Pandemic data collection challenges due to the pandemic and at group quarter
facilities with multiple sites. Prior to the pandemic, group quarters were
given the option to:
• respond via eResponse,
• have census field staff conduct in-person enumeration,
• have census field staff drop off individual census questionnaires for
the administrator to distribute, or
Page 21 GAO-21-365 Decennial Census
• complete a paper response data listing that would be picked up by
census field staff.
However, when the pandemic hit, 2020 group quarter facilities were
encouraged to move away from in-person enumeration and opt for low
contact enumeration methods, such as eResponse and mail back of
paper listings.
Group quarter and service-based enumeration field staff found it
challenging to locate a point of contact at some group quarters because
facilities were closed due to the pandemic. Interviews with CFSs and in
open-ended comments on our ACO managers survey (see text box)
confirmed this challenge. Identifying and communicating with points of
contact were important because census field staff needed to encourage
facility administrators to not only switch to low-contact enumeration
methods, but to also set new enumeration appointment times after the
operational timeline was moved, as well as encourage nonresponding
group quarters to provide their data.
Perspective on Impact of the Pandemic on Group Quarters Operation
“[The Group Quarters Operation] is greatly affected by COVID. Many group
quarters are closed and staff was furloughed which made collection very,
very difficult.”
Source: GAO survey of area census office managers. | GAO-21-365
Particularly difficult to enumerate were colleges and university student
housing. Because of COVID-19, many colleges and universities ceased
in-person instruction in March 2020 and students left college campuses.
According to Bureau guidance college students are to be counted as of
April 1, 2020, where they live and sleep most of the time, which would be
at school if it had not been for the pandemic. The Bureau communicated
with colleges and universities in an attempt to obtain census data for
college and university housing. Bureau officials stated they received data
from 88 percent of student housing facilities. However, colleges and
universities did not always provide the data to the Bureau. 10 Bureau
officials told us that:
10Bureau officials told us that detailed information on why universities did not always
provide data will not be known until assessments are completed.
Page 22 GAO-21-365 Decennial Census
• data privacy and confidentiality concerns resulted in some colleges
not providing data about their student population to the Bureau;
• many school administrators said they planned to submit data to the
Bureau, but completing the census was not a priority for
administrators and they subsequently failed to submit the data to the
Bureau; and
• fraternity and sorority houses in particular presented a challenge,
because it was difficult to locate a point of contact for these sites.
In December 2020, as the Bureau processed group quarter data, Bureau
officials told us they found some census tracts with group quarters
population data that were inconsistent with benchmark data derived from
other Bureau data sources. These sources included the 2010 Census, the
Bureau’s American Community Survey data, and the 2020 Census Group
Quarters Advance Contact operation. Early analysis identified multiple
census tracts where the group quarters population count was either too
high, too low, or misallocated across tracts. 11 Specifically, the Bureau
found that about 24,000 group quarter locations had returned their
enumeration data with zero population counts or were not enumerated.
According to the Bureau, group quarter facilities (such as a university or
federal correctional facility) with multiple locations may have also
contributed to preliminary data on group quarters being over and under
counted. Specifically, Bureau officials stated that group quarters have
complicated structures given the relationship between group quarters’
overall facility (university) and the respective group quarters units
(dormitories).
According to Bureau officials, group quarters can be over counted when
the individual units submit the same data for all units within the facility or
under counted when neither the unit nor the facility submits data due to
confusion over the responsible reporting entity. For example, according to
Bureau officials they found some universities submitted the total dormitory
population for the entire university to each individual residence hall (see
table 2).
11A census tract usually covers a contiguous area and its boundaries generally follow
visible and identifiable features. The primary purpose of census tracts is to provide a
stable set of geographic units for the presentation of statistical data. Census tracts
generally have a population size between 1,200 and 8,000 people, with an optimum size
of 4,000 people.
Page 23 GAO-21-365 Decennial Census
Table 2: Example of Group Quarter Facility Population Overcount
University Expected 2020 Count
Housing Population Count Submitted
Residence Hall 1 250 1,250
Residence Hall 2 250 1,250
Residence Hall 3 250 1,250
Residence Hall 4 250 1,250
Residence Hall 5 250 1,250
Total 1,250 6,250
Source: GAO analysis of Census Bureau reports. | GAO-21-365
According to Bureau officials, enumeration data submitted with zero
population counts primarily affected college/university student housing,
correctional facilities, nursing/skilled-nursing group quarters, and military
quarters. 12 For example, Bureau officials said some college/university
student housing contact persons were no longer located on campus at
the time enumeration occurred because schools had closed, correctional
facilities were relocating and releasing people due to the pandemic, and
nursing facilities frequently would not allow enumerators on the premises
because of concerns about spreading COVID-19.
Bureau officials also noted that group quarter contact persons responsible
for several group quarters at the facility oftentimes submitted data for all
the group quarters under a single group quarter identification. This
resulted in a single group quarter being over counted and the remaining
group quarters having a zero population.
Concerned about the accuracy and completeness of group quarter data,
in mid-December 2020, Bureau officials began making phone calls to
group quarters that were not enumerated, and in many cases
successfully gathered additional data. For example, Bureau officials said
that they had population data for 88 percent of the college/university
student housing prior to the follow-up phone calls and after the phone
calls were able to resolve a total of approximately 97 percent.
Nevertheless, Bureau officials told us that some group quarters that
remain unresolved with a zero population count would go through count
imputation for group quarters, which according to Bureau officials was
12Bureau officials told us that data broken down by group quarters that submitted a zero
population count will not be known until all the post-processing of data is complete.
Page 24 GAO-21-365 Decennial Census
specifically developed to address this problem and will be the first time
the Bureau plans to use count imputation for group quarter facilities with
zero population counts. 13 The Bureau has previously used characteristic
imputation for individuals in group quarters where data for individuals
were missing for characteristics such as sex or race and ethnicity.
eResponse The share of group quarters that took advantage of the eResponse option
was greater than anticipated due to the pandemic, an innovation since the
2010 Census. Before the operation began, the majority of group quarters
(67.4 percent) chose in-person enumeration methods versus by
eResponse (32.6 percent), according to the Bureau. Later, the Bureau
contacted group quarters administrators to encourage them to shift
enumeration methods to low-contact options like eResponse.
According to the Bureau, overall approximately 42 percent of responses
were received through eResponse and 58 percent were enumerated by
in-field methods. Bureau officials pointed to the availability of eResponse,
and the fact that it began as planned on April 2, 2020, as successes.
Nonetheless, Bureau officials said some group quarters facilities found
eResponse challenging to complete. They said that some group quarters
administrators who were in charge of multiple facilities had difficulty
submitting eResponse data. The eResponse template required group
quarters administrators to provide a unique ID for each group quarters
facility, but some administrators submitted all group quarters under one
ID. Figure 6 shows an example of the template administrators were to fill
out for eResponse.
13The Bureau uses imputation to create records for housing units that appear occupied,
but for which no other information is available. In previous censuses, the Bureau has used
count imputation for occupied households with no census data, but has not used count
imputation for group quarter facilities.
Page 25 GAO-21-365 Decennial Census
Figure 6: Screenshot of Group Quarters eResponse Excel Data Entry Template
Source: U.S. Census Bureau. | GAO-21-365
Also, Bureau officials said many of the group quarter’s administrators
struggled to parse out the date of birth field into three separate fields
because their systems stored this data item in one single field. Bureau
officials also pointed to other challenges, including a facility’s lack of
access or unfamiliarity with Excel, which was the preferred format to
complete eResponse. The Bureau accepted eResponse data submitted
by group quarters administrators in other formats.
In open-ended comments on our survey of ACO managers, some said
eResponse was complicated and confusing for group quarters
administrators. While others saw value in the ability of eResponse to
collect the data (see text box), Bureau officials said testing eResponse
specifically would be helpful for 2030.
Page 26 GAO-21-365 Decennial Census
Perspectives on Group Quarters eResponse Option
“I think [group quarters] was poorly conceived, and [eResponse] poorly
designed. [eResponse] is clunky, confusing, and requires too much on the
part of respondents to adequately participate, and has required far too much
of my staff’s time to walk them through the process.”
“Many of the group quarter and service-based organizations were slow, even
reluctant, at responding, but there were many that transitioned to e-Response
and paper drop off/pick up and that expedited the operation.”
Source: GAO survey of area census office managers. | GAO-21-365
We previously reported that during the 2018 End-to-End Census test,
only 25 of the 75 facilities that selected the eResponse enumeration
option submitted responses by the deadline. The Bureau pointed to
issues with the required format for the submission of response files as a
possible explanation for these challenges during the End-to-End test.
According to Bureau officials, a set of adjustments was made after the
2018 End-to-End-Test including adding a detailed instructions tab to the
eResponse template. Nevertheless, problems persisted with the
eResponse option. Bureau officials acknowledge that additional
adjustments and testing will help to improve and simplify this data
collection process for group quarters administrators prior to 2030.
Enumeration of Military Group In addition to the challenges described above with regard to zero
Quarters population counts for military group quarters, Bureau officials told us that
the military group quarters enumeration was the slowest to respond. As of
August 24, 2020, military group quarters had the lowest completion rate
at 63.5 percent (3,783 of 5,955) complete, followed by military disciplinary
barracks and jails at 77.8 percent (28 of 36) complete. 14 According to
Bureau officials, even though they had consistent communication with
military personnel from the Department of Defense Joint Services
Working Group collecting enumeration data proved to be difficult because
of the pandemic—similar to other group quarter types.
Specifically, some military installations did not allow access to
installations until late summer because of the pandemic, and some
barracks were being used to house military COVID-19 patients. However,
14Due to the way the Bureau calculates completion rates all group quarter types are
reported as reaching 100 percent completion at the end of the operation. However, due to
the zero population counts the Bureau is unable to provide final numbers until the post-
processing of data is complete.
Page 27 GAO-21-365 Decennial Census
for the most part, Joint Services Working Group members encouraged
the use of other methods such as eResponse and once the installations
lifted restrictions, they made sure enumerators had access to barracks
when notified of access issues.
Bureau officials said they also experienced challenges with military
barrack points-of-contact. According to Bureau officials, a common issue
was the assigned point of contact would often change or would be
delegated to someone unfamiliar with the Census requirements. In 2010,
the Bureau also experienced issues with coordinating the enumeration
with the military installation points of contact. The 2010 Census
assessment included a number of recommendations for improving the
enumeration of military sites in the group quarters workload, including
continued advanced phone calls to establish military points of contact, to
ensure a timely enumeration.
The Bureau made operational changes in response to COVID-19 and
Bureau Needs to Commerce decisions that for example, included shortening the time
Assess Design frames for implementing operations and reducing the scope of quality
checks on field data collection.
Changes and Data
Collection Challenges The Bureau has numerous planned assessments and evaluations of 2020
operations which, in conjunction with its post-enumeration survey—a
as It Begins Planning survey conducted independently of each census to determine how many
for 2030 people were missed or counted more than once—help determine the
overall quality of the census and document lessons for future censuses.
In December 2020, we reported that the Bureau was updating its study
plans to examine the range of operational modifications made in
response to COVID-19, including the August 3 re-plan and later
changes. 15 In that report, we recommended that the Bureau update and
implement its assessments to address data quality concerns, as well as
any operational benefits. Commerce agreed with that recommendation.
In this report, we have discussed a number of changes that were made
for the 2020 Census including a set of late design changes as well as
challenges identified with new methods for collecting data for
nonresponse follow-up and eResponse for group quarters enumeration.
In January 2021 we spoke to Bureau officials about the status of those
15GAO, 2020 Census: Census Bureau Needs to Assess Data Quality Concerns Stemming
from Recent Design Changes, GAO-21-142 (Washington, D.C.: Dec. 3, 2020).
Page 28 GAO-21-365 Decennial Census
study plans and they told us they were in the process of updating those
plans and therefore they were not yet available for review. Standards for
Internal Control in the Federal Government indicates that agencies should
analyze and respond to identified risks, including by considering relevant
changes that have taken place within the agency’s environment.
Accordingly, as part of updating its plans, it is important that the Bureau
includes assessing the risks and impacts of late design changes and
challenges raised in this report during nonresponse follow-up and group
quarters enumeration. The resulting study plans can then lead to more
comprehensive and essential lessons learned for 2030 Census planning.
Conducting a census amid a global pandemic posed unprecedented
Conclusions challenges to the Bureau. The Bureau delayed the start of nonresponse
follow-up and group quarters and service-based enumerations, and was
later required to revise operational timelines.
Given these disruptions, it is critical for the Bureau to ensure that it can
learn as much as possible about the effects of design and schedule
changes on data collection and quality control operations, as well as
challenges encountered implementing new methods for collecting data.
Moreover, actions taken by the Bureau out of necessity to meet
compressed deadlines may be considered as options in future censuses.
As the Bureau formalizes its updated approach, it will be important that it
includes in its plans a thorough evaluation of issues stemming from late
design changes and challenges encountered in trying to collect data.
Doing so will help inform future census efforts.
We are making one recommendation to the Department of Commerce:
Recommendations for
Executive Action The Secretary of Commerce and the Director of the U.S. Census Bureau
should, to inform 2030 planning, evaluate how major operational changes
in response to data collection challenges affected the quality and
completeness of nonresponse follow-up and group quarters
enumerations. At a minimum, this evaluation should address:
• late design changes;
• procedures for accessing multi-unit buildings;
• the process for reassigning cases;
• supervisory alerts used to manage enumerators;
• quality control over training assessments; and
Page 29 GAO-21-365 Decennial Census
• the eResponse option for group quarter data collection.
(Recommendation 1)
We provided a draft of this report to the Department of Commerce. In its
Agency Comments written comments, reproduced in appendix II, the Department of
and Our Evaluation Commerce agreed with our findings and recommendation. The Bureau
also provided technical comments, which we incorporated as appropriate.
We are sending copies of this report to the Secretary of Commerce, the
Undersecretary of Economic Affairs, the Acting Director of the U.S.
Census Bureau, and the appropriate congressional committees. In
addition, the report is available at no charge on the GAO website at
http://www.gao.gov.
If you or your staff have any questions about this report please contact J.
Christopher Mihm at (202) 512-6806 or by email at mihmj@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who made
key contributions to this report are listed in appendix III.
J. Christopher Mihm
Managing Director, Strategic Issues
Page 30 GAO-21-365 Decennial Census
List of Requesters
The Honorable Patrick Leahy
Chairman
Committee on Appropriations
United States Senate
The Honorable Gary C. Peters
Chairman
The Honorable Rob Portman
Ranking Member
Committee on Homeland Security and Governmental Affairs
United States Senate
The Honorable Ron Johnson
Ranking Member
Permanent Subcommittee on Investigations
Committee on Homeland Security and Governmental Affairs
United States Senate
The Honorable Jeanne Shaheen
Chairwoman
Subcommittee on Commerce, Justice, Science, and Related
Agencies Committee on Appropriations
United States Senate
The Honorable Carolyn B. Maloney
Chairwoman
The Honorable James Comer
Ranking Member
Committee on Oversight and Reform
House of Representatives
The Honorable Jamie Raskin
Chairman
Subcommittee on Civil Rights and Civil Liberties
Committee on Oversight and Reform
House of Representatives
The Honorable Gerald E. Connolly
Chairman
The Honorable Jody Hice
Ranking Member
Subcommittee on Government Operations
Committee on Oversight and Reform
House of Representatives
Page 31 GAO-21-365 Decennial Census
List of Requesters Continued
The Honorable Christopher Coons
United States Senate
The Honorable Dianne Feinstein
United States Senate
The Honorable Joe Manchin III
United States Senate
The Honorable Jack Reed
United States Senate
The Honorable Brian Schatz
United States Senate
The Honorable Chris Van Hollen
United States Senate
The Honorable Karen Bass
House of Representatives
The Honorable Joaquin Castro
House of Representatives
The Honorable Judy Chu
House of Representatives
The Honorable Steven Horsford
House of Representatives
The Honorable Jim Jordan
House of Representatives
The Honorable Chip Roy
House of Representatives
Page 32 GAO-21-365 Decennial Census
Appendix I: Objectives, Scope, and
Appendix I: Objectives, Scope, and
Methodology
Methodology
This report assesses (1) the Census Bureau’s (Bureau) implementation of
nonresponse follow-up; (2) the Bureau’s implementation of group quarters
enumeration and service-based enumeration operations; and (3) the
Bureau’s plans to assess these operations.
To meet objectives one and two, we used a combination of approaches
and methods to examine the conduct of these operations. These included
surveys of the Bureau’s 248 area census office (ACO) managers
administered at six key points during Census operations; analysis of
Bureau data on case completion methods; interviews with key Bureau
Headquarters officials, directors of Regional Census Centers, and ACO
census field supervisors (CFS); and reviews of relevant documentation
including from our prior work on the 2010 Census and the planning and
implementation of the 2020 Census.
To examine how conditions in the field affected the implementation of
nonresponse follow-up, group quarters enumeration, and service-based
enumeration operations, and how changes to time frames and plans in
response to the Coronavirus Disease 2019 (COVID-19) pandemic and
Department of Commerce decisions affected the field operations, we
interviewed CFSs from each operation.
• For nonresponse follow-up, we conducted 40 phone interviews with
CFSs from 20 different ACOs from August 6, 2020, through October
1, 2020. We selected ACOs to capture responses from areas with low
response rates in 2010, recruiting challenges, high workload, those
who participated in the soft launch of nonresponse follow-up, or those
whom we had previously contacted during the Bureau’s address
canvassing operation. In selecting ACOs we also considered those
with unique attributes, such as tribal territory or congressional interest.
See below for a complete list of the ACOs with CFSs we interviewed.
• For group quarters enumeration, we conducted 20 phone interviews
with CFSs from 10 different ACOs from July 6, 2020, through August
19, 2020. We selected these ACOs to capture responses from areas
with recruiting challenges, high workload, those who participated in
the soft launch of nonresponse follow-up, or those whom we had
previously contacted during the Bureau’s address canvassing
operation. Interviews were conducted by phone, rather than in person,
due to health and safety concerns related to COVID-19. The selected
ACOs are listed below.
• We also conducted six phone interviews with CFSs from six different
ACOs involved with the service-based enumeration (SBE) and the
Page 33 GAO-21-365 Decennial Census
Appendix I: Objectives, Scope, and
Methodology
temporary non-sheltered outdoor locations (TNSOL) operations from
September 25, 2020, through October 1, 2020. We limited our
interviews to six CFSs due to the short window of employment for
CFSs once they complete SBE and TNSOL operations. We selected
ACOs with the highest workload for SBE and TNSOL operations. The
six selected site are listed below.
Table 3: Interviewed Census Field Supervisors from 25 Area Census Offices
Area Census Nonresponse Group Quarters Service-Based
Office Follow-up Enumeration Enumeration
Atlanta, GA x x ─
Beckley, WV x ─ ─
Boston, MA x x x
Bronx, NY x x x
Brooklyn, NY ─ x x
Brooklyn [2], NY x ─ ─
Chicago, IL x ─ ─
Washington, DC x ─ ─
Denver, CO x x x
Detroit, MI x ─ ─
Fairfax, VA ─ x ─
Hagerstown, MD ─ x ─
Inglewood, CA x ─ ─
Las Vegas, NV x ─ ─
Lexington, KY x ─ ─
Milwaukee, WI x ─ ─
Oakland, CA x x x
Philadelphia, PA ─ x x
Providence, RI x ─ ─
San Antonio, TX x ─ ─
Savanna, GA x ─ ─
Seattle, WA x ─ ─
State College, PA x ─ ─
Trenton, NJ ─ x ─
Window Rock, AZ x ─ ─
Total: 25 20 10 6
Source: GAO. | GAO-21-365
Page 34 GAO-21-365 Decennial Census
Appendix I: Objectives, Scope, and
Methodology
Additionally, to obtain a ground-level perspective on the conduct of the
nonresponse follow-up, group quarters enumeration, and SBE operations,
we surveyed the Bureau’s entire population of 248 ACO managers six
times during the 2020 Census, including in late February to early March,
early April, late May, late June to early July, late August, and early
October. The response rates were 71, 75, 76, 72, 67, and 69 percent,
respectively. We also reviewed open-ended responses provided by the
ACO managers as part of this survey.
To track the progress of the nonresponse follow-up operation, we
reviewed daily reports that tracked progress by ACO and case completion
at the ACO and national level. These reports also included data such as
the number of proxy responses collected and the number of cases closed
as vacant housing units. To assess the reliability of these Bureau-
provided data, we reviewed Bureau electronic documentation to gain
information about the data and their sources. We examined data from the
Bureau’s Unified Tracking System to check for logical errors and
inconsistencies, and followed up with agency officials knowledgeable
about the data in cases where we had questions about potential errors or
inconsistencies, and to inquire about the accuracy and completeness of
the data. Values are updated by the Bureau throughout the operations,
and may be revised by the Bureau even after the operations close. On
the basis of our efforts, we determined that the data were sufficiently
reliable for the purposes of this engagement.
Finally, to address our third objective to review the Bureau’s plans to
assess the quality of these operations, we interviewed Bureau officials to
discuss whether operational changes made to the design would be
included in the assessments of operations. We were unable to review
those study plans because they have not been finalized due to changes
being made as a result of the pandemic.
We conducted this performance audit from March 2020 until March 2021
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audits to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Page 35 GAO-21-365 Decennial Census
Appendix II: Comments from the Department
Appendix II: Comments from the Department
of Commerce
of Commerce
Page 36 GAO-21-365 Decennial Census
Appendix III: GAO Contact and Staff
Appendix III: GAO Contact and Staff
Acknowledgments
Acknowledgments
J. Christopher Mihm at (202) 512-6806 or mihmj@gao.gov
GAO Contact
In addition to the contact named above, Lisa Pearson (Assistant Director),
Staff Timothy Wexler (Analyst-in-Charge), Mark Abraham, Anna Blasco,
Acknowledgments Michael Bechetti, Devin Braun, Arpita Chattopadhyay, Alexandra
Edwards, Margaret Fisher, Sam Gaffigan, Rob Gebhart, Kirsten Lauber,
Paulina Maqueda Escamilla, Kerstin Meyer, Ty Mitchell, Emmy Rhine
Paule, Rachel Schultz, Kate Sharkey and Jon Ticehurst made key
contributions to this report.
(104161)
Page 37 GAO-21-365 Decennial Census
The Government Accountability Office, the audit, evaluation, and investigative
GAO’s Mission arm of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of the
federal government for the American people. GAO examines the use of public
funds; evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make informed
oversight, policy, and funding decisions. GAO’s commitment to good government
is reflected in its core values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of through our website. Each weekday afternoon, GAO posts on its website newly
GAO Reports and released reports, testimony, and correspondence. You can also subscribe to
GAO’s email updates to receive notification of newly posted products.
Testimony
Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and
distribution and depends on the number of pages in the publication and whether
the publication is printed in color or black and white. Pricing and ordering
information is posted on GAO’s website, https://www.gao.gov/ordering.htm.
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card, MasterCard,
Visa, check, or money order. Call for additional information.
Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts.
Visit GAO on the web at https://www.gao.gov.
Contact FraudNet:
To Report Fraud,
Website: https://www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in
Automated answering system: (800) 424-5454 or (202) 512-7700
Federal Programs
Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400,
Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125,
Relations Washington, DC 20548
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548
Stephen J. Sanford, Acting Managing Director, spel@gao.gov, (202) 512-4707
Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814,
External Liaison Washington, DC 20548
Please Print on Recycled Paper.
File and source
- File
- gao-21-365.pdf
- Size
- 1,678,075 bytes
- SHA-256
- 46de34a5772932c94f28e49e326e441e5b96ddad91196161d98062482067ad07
- Our copy
- gao-21-365.pdf
- Original
- www.gao.gov