Nonresponse Bias Report for the 2020 Household Pulse Survey
Summary
A U.S. Census Bureau report, Nonresponse Bias Report for the 2020 Household Pulse Survey, Version 1.0, dated March 24, 2021, from the Demographic Statistical Methods Division. It evaluates potential nonresponse bias in the 2020 Household Pulse Survey, which produced estimates for 21 data collection periods in three phases, using survey response data and American Community Survey estimates. The report describes the sampling frame of 66 sampling areas, seven key measures such as food scarcity and difficulty paying household expenses, and the weighting adjustments. Its executive summary states that response patterns differ across demographic domains and that weighting adjustments appear to mitigate some nonresponse bias. The report closes with a table of planning database quartile definitions drawn from 2014-2018 American Community Survey estimates.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
Cite this document
“Nonresponse Bias Report for the 2020 Household Pulse Survey.”
Original at www2.census.gov. Archived at Pandemic Darlings, https://pandemicdarlings.com/source-documents/expansion-of-child-tax-credits-and-mental-health-of-parents-with-low-i-2677857e49/
, SHA-256 2f8ab20e9a9308633b32409574da879a20a5f18e5930a60568aadf74036c18c0.Full text
Demographic Statistical Methods Division
Sample Design and Estimation
Nonresponse Bias Report for the 2020
Household Pulse Survey
Version 1.0
March 24, 2021
Sandra Peterson
Norilsa Toribio
James Farber, ADC
David Hornick, Lead Scientist
Cleared for public release
Table of Contents
Executive Summary .............................................................................................................................. 1
1. Introduction ................................................................................................................................... 2
1.1 Overview of the 2020 Household Pulse Survey .................................................................. 2
1.2 Discussion of Nonresponse Bias in the HPS ......................................................................... 4
2. Methodology ................................................................................................................................. 5
2.1 Correlation of Key Estimates and Key Domains .................................................................. 6
2.2 Nonresponse Bias in First Interviews Based on Sampling Geography ............................... 7
2.3 Nonresponse Bias in Second Interviews Based on First Interview Responses.................. 8
3. Assumptions and Limitations ....................................................................................................... 9
3.1 Assumptions........................................................................................................................... 9
3.2 Limitations............................................................................................................................10
4. Results ..........................................................................................................................................11
4.1 Correlation of Key Estimates and Key Domains ................................................................12
4.2 Nonresponse Bias in First Interviews Based on Sampling Geography .............................15
4.3 Nonresponse Bias Based on HPS Response data ..............................................................18
5. Conclusions ..................................................................................................................................22
6. References ...................................................................................................................................23
HPS Key Estimate Definitions ..................................................................................24
Planning Database Quartile Definitions .................................................................25
Table of Tables
Table 1: Percent of Weeks with Significant Correlation between Demographics and Household
Pulse Survey Estimates .......................................................................................................................14
Table 2: 2020 Household Pulse Survey Average Weighted Response Rates by Domain ..............16
Table 3:2020 Household Pulse Survey Average Weighted Respondent/Nonrespondent
Distributions by Domain .....................................................................................................................17
Table 4: Average Household Pulse Survey Response Measures by Domain ..................................19
Table 5: Household Pulse Survey Average Domain Distributions by Response Status .................20
Table 6: Household Pulse Survey Average Estimates by Response Status .....................................21
Table 7: Household Pulse Survey Key Estimate Definitions.............................................................24
Table 8: Planning Database Quartile Definitions ..............................................................................25
Cleared for public release
Executive Summary
The 2020 Household Pulse Survey (HPS) was developed to measure the impacts of the
coronavirus pandemic on households within all fifty states, the District of Columbia, and the
fifteen largest metropolitan statistical areas. All data collection was done virtually, with sample
units contacted by text message and/or email. Overall response rates were low, which raised
concerns about nonresponse bias. This report used HPS response data and American
Community Survey estimates to evaluate potential nonresponse bias in the HPS estimates.
The 2020 HPS produced estimates for 21 different data collection periods in three phases.
Phase 1 produced weekly estimates (weeks 1-12, from April 23 to July 21) where respondents
from one week were included in the following week sample for up to three weeks. Phase 2
introduced new questions and consisted of two-week data collection periods (weeks 13-17,
from August 19 to October 26) where each sample was independent of the previous week.
Phase 3 also consisted of two-week data collection periods (weeks 18-21, from October 28 to
December 21), with the same sampling methods and questionnaire as Phase 2. Phase 3
continued into 2021, but this report only analyzes data from 2020 HPS.
The level of nonresponse bias varied across data collection periods, and all individual
nonresponse bias measures are posted to the HPS technical documentation page (census.gov).
This report provides average measures across a subset of weeks in the 2020 HPS data
collection. It shows evidence that response patterns differ across demographic domains that
are correlated with key estimates, which could result in biased estimates. This research also
shows that weighting adjustments appear to mitigate some of this nonresponse bias by
controlling weights to independent totals by domain.
This analysis was limited by the data available for all HPS sample units, so the results should be
interpreted with caution. There is no way of knowing what the HPS estimates would be if all
sampled cases responded to the survey. This report also does not address other sources of
survey error such as frame coverage. This analysis shows that the weighting adjustments help
mitigate nonresponse bias, but it does not prove that there was no bias, by nonresponse or
coverage, in the HPS estimates.
Cleared for public release 1
1. Introduction
The 2020 Household Pulse Survey (HPS) was developed to measure the impacts of the
coronavirus pandemic on households within all fifty states, the District of Columbia (DC), and
the fifteen largest metropolitan statistical areas (metro areas). This survey was developed
quickly in response to the pandemic and required data collection methods that have never
been implemented before by the U.S. Census Bureau. The 2020 HPS was conducted completely
by web, and all sampled addresses were contacted by email and/or text message.
This report presents discussion about, and analysis of, potential nonresponse bias in the 2020
HPS estimates. The Census Bureau plans to conduct more analysis than what is presented here,
however this report is limited to the analysis that could be completed quickly using data already
available to us.
The HPS produced 21 different collection cycles in three phases during 2020, all of which were
approved for release to the public1. Each data release included tables of estimates, a source
and accuracy statement, and a public use file so data users may produce their own estimates.
This report was released with additional tables of nonresponse bias measures, labeled Tables
A1-A8 and B1-B2, which are posted to the HPS technical documentation page (census.gov). This
report analyzes measures of bias for a subset of weeks, as identified by each table. Any
questions about these results may be sent to dsmd.source.and.accuracy@census.gov.
1.1 Overview of the 2020 Household Pulse Survey
The HPS sampling frame consisted of all housing units on the Master Address File (MAF)
where at least one email address or cell phone number was known. Geocoded units on the
MAF contain census block codes that identified the location of each address. Ungeocoded
units on the MAF do not contain census block codes but do contain county codes and ZIP
codes. The HPS sampling frame was divided into 66 different areas for the 51 states
(including DC) and 15 metro areas. All sampling areas had the same sample sizes to ensure
adequate estimate precision within all areas 2. Variations in population across the different
sampling areas led to different sampling base weights across the sampling areas.
The HPS selected a sample of housing units (HUs) from the MAF, as most other
demographic surveys do. Most other demographic surveys then attempt to contact persons
1
Cycle 21 approval number was CBDRB-FY21-POP001-005.
2
Some small states had smaller sample sizes because the sampling frame did not contain enough addresses to
support all HPS samples.
Cleared for public release 2
at the sampled address through telephone and/or personal visits. Personal visits were not
possible for the HPS, however, so we continued to follow-up with the emails and/or cell
phones associated with the address until we received a response. Response was measured
by how far into the questionnaire someone got before exiting the instrument. Phase 1
respondents were set at 30 percent of the questions, and Phase 2 and 3 respondents were
set at 23 percent of the questions. The Phase 2 and 3 cutoff was around the same point in
the questionnaire as Phase 1, but the percentage was different because more questions
were added to the questionnaire.
Phase 1 included the first twelve weeks of HPS (April 23 – July 21), during which
respondents from one week were included in the following week’s sample for up to three
weeks total. Therefore, we had more information about the nonrespondents among the
sample cases on their second or third interviews. Phase 2 was conducted weeks 13 through
17 (August 19 – October 26) where samples were independent and each sample case was
only interviewed once. The 2020 Phase 3 data collection periods included weeks 18 through
21 (October 28 – December 21) and continued the same sampling methods as Phase 2.
Phase 2 and 3 data collection periods were two weeks long, but we continued to refer to
them as weeks to be consistent with Phase 1. Due to the data collection and sampling
methods, very little was known about the nonrespondents in the Phase 2 and Phase 3 HPS.
Key measures (and the HPS detailed tables that contain them 3) were the percentages of
respondents who reported:
1. Expected loss in employment income (Employment Table 1)
2. Food scarcity (Food Sufficiency and Food Security Table 2)
3. Housing insecurity for owners (Housing Table 2a)
4. Housing insecurity for renters (Housing Table 2b)
5. Likelihood of foreclosure for owners (Housing Table 3a)
6. Likelihood of eviction for renters (Housing Table 3b)
7. Difficulty paying for usual household expenses (Spending Table 1)
Key domains were:
• Race/ethnicity
• Household income
For more information about the HPS, refer to the Technical Documentation at census.gov
(U.S. Census Bureau, 2020b).
3
Available on the HPS Data Tables page (census.gov).
Cleared for public release 3
1.2 Discussion of Nonresponse Bias in the HPS
Some degree of nonresponse bias and variance is a normal feature of almost all statistical
surveys. The HPS produces pandemic impact estimates using the answers from responding
persons. These estimates may be biased if answers from respondents differ from the
potential answers of nonrespondents. The magnitude of nonresponse bias is a function of
the response rate and differences between respondents and nonrespondents.
The HPS attempts to minimize nonresponse bias by adjusting weights for potential
differences between respondents and nonrespondents. However, these weighting
adjustments are limited by a lack of information about nonresponding units. Therefore, the
nonresponse adjustment cells are defined only by the sampling area from the MAF. To
account for population mobility, respondents are allocated to new sampling areas using
address information provided in the response data. Then the weights are controlled to the
American Community Survey (ACS) estimates of occupied HUs within those assigned
sampling areas. Whenever address information is not provided, or does not match to a
sampling area, the case is assigned to the sampling area in which they were sampled.
During Phase 1, respondents who did not provide an address in their second or third
interviews were assigned to the same sampling area as their first interview.
The HU-level sampling weights are converted to person weights by multiplying the HU-level
weight by the number of adults within the household. This means the demographics (such
as age, sex, Hispanic origin, and race) of the respondent are applied to all adults within the
household even though it is possible -- in fact, probable -- that all persons within a
household do not have identical demographic characteristics. The person weights are then
controlled to independent population estimates from the Census Bureau’s Population
Division. The adjustment cells are initially defined within each state by age, sex, Hispanic
origin, race, and educational attainment, and cells are collapsed together if there are not at
least 30 respondents within a cell for a given state. These demographic characteristics are
imputed for any respondents who fail to provide this information.
Despite the measures taken to reduce nonresponse bias, there is likely some amount of
nonresponse bias that cannot be corrected without knowing the pandemic impacts of the
nonrespondents. Additionally, the survey design limits our ability to measure nonresponse
bias for multiple reasons:
1. Population mobility – it is possible for people to move to a different county, state,
and country without changing their email address or cell phone number. This means
Cleared for public release 4
there is no guarantee that respondents and nonrespondents still live in the
geographic areas where they were sampled.
2. Nonresponse reason – due to the method of data collection, we do not know
whether a sample case failed to respond because the contact information is invalid
or because they chose not to respond to the survey. This forces us to treat all
noninterviews as valid units that did not respond.
3. Nonrespondent characteristics – without field representative observations we may
obtain from personal visit attempts, we have no way of knowing the characteristics
of nonresponding cases.
The nonresponse bias analysis in this report is not a perfect measure of the bias in the HPS
estimates but is an early approximation of bias that may exist based on observations in
response patterns across demographic groups that are correlated with pandemic impacts.
2. Methodology
This section describes the methods and data used for the analysis in this report. For analysis of
domains that were collected in the HPS, the domain categories were determined by the
response options and domain groupings used in weighting adjustments.
For domains that use ACS estimates from the block-group4 the case was sampled within, the
domain categories are four quartiles for each ACS estimate. We calculated the quartiles using
all block-groups on the planning database (U.S. Census Bureau, 2020a), initially placing 25
percent of all block groups within each of four buckets. For some ACS estimates, like the
percent of housing units that are mobile homes, more than 25 percent of the block-groups had
a value of zero. In this case, we assigned all block-groups with a value of zero to the lowest
quartile and evenly divided the remaining block-groups across the remaining three quartiles.
Refer to Appendix B for the resulting quartile definitions.
After defining the quartiles using the planning database, we merged the HPS response data to
the domain data by block-group code. Ungeocoded cases did not have a block-group code and
were excluded from all analysis by ACS estimate domains.
4
Block-groups are a collection of census blocks within the same census tract. This is the lowest level for which the
ACS creates estimates. More information is available at the Census Bureau Glossary (census.gov).
Cleared for public release 5
2.1 Correlation of Key Estimates and Key Domains
Most 2020 HPS measures have never been collected before, so there is no historical
knowledge about what demographic characteristics are correlated to the key estimates.
Therefore, before conducting nonresponse analysis, it seemed appropriate to identify which
demographic groups show significant differences in the key HPS estimates.
This analysis used data from the HPS responses as well as ACS estimates of demographic
properties of the census block-group that sampled addresses belong to. The ACS estimates
were available on the Planning Database (U.S. Census Bureau, 2020a) and allowed us to
analyze demographic characteristics for all geocoded sample cases in the HPS, even though
this may not match the current location of the respondents (and nonrespondents) at the
time of HPS data collection.
The HPS produced estimates of total persons and percent of all persons who experienced
each specific pandemic impact. In order to compare estimates of populations of different
sizes, all analysis in this report was based on the percent estimate. The key estimates were
calculated as the sum of final person weights for all respondents who reported each
pandemic impact divided by the sum of person weights for all respondents who answered
the question about the pandemic impact. All key estimates excluded HPS respondents for
which the key estimate did not apply or who did not provide a valid answer to the key
question. The formula for HPS estimates was:
∑𝑖∈s 𝑤𝑖 𝐼𝑖
𝐸𝑆𝑇 =
∑𝑖∈s 𝑤𝑖
where s is the set of all respondents within the domain of interest who answered the key
question, 𝑤𝑖 is the final person weight for respondent i, and 𝐼𝑖 is an indicator (0 or 1) for
whether the respondent reported experiencing the pandemic impact (refer to Appendix A
for criteria).
Variances were calculated using Fay’s Balance Repeated Replication (BRR) method with 80
replicate weights (Fay, 1989). We used these variances to test the significance of estimate
differences across subgroups within each demographic characteristic. Any characteristics
that showed significant estimate differences are later analyzed for response differences to
evaluate the potential nonresponse bias impacts.
Cleared for public release 6
Key estimates five, six, and seven were added to the HPS questionnaire in week 13 and
could not be produced for any weeks before then. Some domain variables were also not
available on early weekly data files and are, therefore, excluded from this analysis for those
specific weeks.
2.2 Nonresponse Bias in First Interviews Based on Sampling Geography
The HPS sampling frame did not contain demographic information that could be used to
classify nonrespondents. However, the geographical location of the sample address, when
known, was used to determine demographic characteristics about the blocks containing the
address. To accomplish this, we matched the HPS sample data to the ACS block-group level
estimates by block-group codes.
This analysis measured levels of response within different demographic groups determined
by the ACS estimates for the block-group that contained the sample address. Ungeocoded
cases did not have block-group codes in the HPS sampling frame and were excluded from
the analysis.
Weighted response rates showed the percent of each demographic subgroup that
responded to the HPS and were calculated as:
∑𝑖∈𝑠 𝑤𝑖 𝑅𝑖 𝐷𝑖
𝑅𝑅 =
∑𝑖∈𝑠 𝑤𝑖 𝐷𝑖
where:
𝑠 = the set of all sample cases within the sampling area
𝑤𝑖 = the sampling base weight for sample case i
𝑅𝑖 = the response indicator (1 for respondents, 0 for nonrespondents) for sample case i
𝐷𝑖 = the domain indicator (1 if within the domain of interest, 0 otherwise) for sample
case i
We used replicate weights to calculate a variance of the response rates and test the
significance of response rate differences between all subgroups of a demographic
characteristic, using a Tukey adjustment for multiple comparisons.
Weighted respondent and nonrespondent distributions showed the demographic
distribution of respondents and nonrespondents separately, and respondent distributions
were calculated as:
∑𝑖∈𝑠 𝑤𝑖 𝑅𝑖 𝐷𝑖
𝑅𝐷 =
∑𝑖∈𝑠 𝑤𝑖 𝑅𝑖
Cleared for public release 7
using the same eligibility criteria, weights, and indicators as the response rate calculation.
Nonrespondent distributions used the same formula, but with the 𝑅𝑖 variable indicating
nonrespondents instead of respondents. We tested for differences between respondent
and nonrespondent distributions within domain characteristics using Rao-Scott chi-square
tests and replicate weights.
2.3 Nonresponse Bias in Second Interviews Based on First Interview Responses
This analysis used HPS response data to evaluate response patterns among demographic
groups identified by the demographic data provided in the HPS. For Phase 1 interviews,
first-time respondents for a given week were divided into those who also responded in the
following week, and those who did not respond in the following week. For Phase 2 and 3
interviews, all respondents were divided into those who first accessed the on-line
questionnaire in the first six days of the 13-day data collection period and those who did
not. We used a six-day cutoff because the Phase 1 data collection period was six days, so
any Phase 2 and 3 respondents who accessed the instrument after that point would have
been nonrespondents in Phase 1. Some respondents started the interview in the first six
days of data collection and finished the interview in the last seven days, but we could not
determine when the interview was completed so we divided them by when the interview
was started.
Weighted response measures showed the percent of HPS respondents within each
demographic subgroup that belonged to the response status that indicated more
cooperation. For Phase 1, this subgroup was respondents who provided responses in more
than one week of data collection. For Phases 2 and 3, this subgroup was respondents who
accessed the on-line instrument in the first six days of data collection. The response
measures were calculated as:
∑𝑖∈r 𝑤𝑖 𝑅𝑖 𝐷𝑖
𝑅𝑀 =
∑𝑖∈r 𝑤𝑖 𝐷𝑖
where:
𝑟 = the set of all respondents
𝑤𝑖 = the final person weight for respondent i
𝑅𝑖 = indicator (0 or 1) for whether respondent i is in the response group of interest
𝐷𝑖 = indicator (0 or 1) for whether respondent i is within the domain of interest
We used replicate weights to calculate a variance of the response measures and test the
significance of differences between all subgroups of a demographic characteristic, using a
Tukey adjustment for multiple comparisons.
Cleared for public release 8
Weighted domain distributions by response status showed the demographic distributions
within each subgroup of respondents, and were calculated as:
∑𝑖∈𝑟 𝑤𝑖 𝑅𝑖 𝐷𝑖
𝐷𝐷𝑅𝑆 =
∑𝑖∈r 𝑤𝑖 𝑅𝑖
using the same eligibility criteria, weights, and indicators as the response measure
calculation. We tested for domain distribution differences between response status
subgroups using Rao-Scott chi-square tests and replicate weights.
The key estimates by response status showed how response differences among the
different demographic groups potentially impacted the estimates. The formula was the
same as the HPS estimates produced in Section 2.1, where the domain of interest was
determined by response status. We calculated a 90 percent confidence interval around the
full sample estimate to determine if the estimate from only respondents who responded at
least twice (for Phase 1) or accessed the instrument in the first six days (for Phase 2 and 3)
was outside that confidence interval.
3. Assumptions and Limitations
This report analyzes potential nonresponse bias in the HPS, but there are many other sources of
error that are not examined. We presented HPS data observations that may indicate bias, but
please keep in mind the assumptions made and the limitations of this analysis.
3.1 Assumptions
This report presents HPS estimates and response rates by ACS demographic estimates for
the block-group containing the sample address. It is possible that HPS respondents no
longer live at the address where they were sampled, so this analysis assumes that
households who move away from the address where they were sampled would move to an
area with similar characteristics to the area they left.
When calculating overall response rates by ACS estimates of the geographical area
containing the address, we assumed that all sample cases were valid units within those
geographical areas. In-person surveys typically locate new sample addresses and exclude all
cases that no longer exist as occupied residences. The HPS data collection methods did not
allow for this type of determination, so we must assume that all sample units were
occupied residences and the contact information belonged to persons living within those
residences. In reality, the unit may be vacant or demolished, and the contact information
Cleared for public release 9
may not be in use because the original contact person passed away or changed their phone
and email. Therefore, the HPS response rates by geographic area were actually participation
rates that are typically lower than response rates because they include invalid units as
nonresponse.
Analysis of response patterns using HPS respondent data allowed us to analyze response
based on known demographics of the potential HPS respondents. However, the universe of
cases for this analysis consisted of only sample units that responded to the HPS at some
point. For Phase 1 analysis, we assumed that HPS respondents who did not respond to a
second interview request were similar to sample cases that did not respond to the first
request. For Phase 2 and 3 analysis, we assumed that HPS respondents who first accessed
the instrument in the second half of the two-week data collection period were similar to
sample cases that did not respond at all.
3.2 Limitations
This analysis focused on nonresponse bias but with little knowledge of the nonrespondent
characteristics. We had ACS demographic estimates for the block-group each address
belongs to but did not know the demographic characteristics of the HPS sample cases that
did not respond. We also did not know demographic characteristics for respondents who
did not provide valid answers to the demographic questions, so they are excluded from this
analysis.
This analysis also did not cover other sources of error in the HPS. For example:
• Frame coverage – the HPS sampling frame included only MAF units for which we had
a cell phone number or email address. The MAF is a reliable source for all addresses
in the United States, but around 19 percent of addresses on the MAF were excluded
from the HPS sampling frame due to missing contact information.
• Population coverage – The HPS collected responses from one person within each
household, and their answers represented all adults in the household. This assumed
everyone in the household had the same demographics as the respondent, which
potentially impacted the representation of different demographic subgroups. One
example is the educational attainment demographic, because some parents with
college degrees represented adult children living in the home who had not yet
completed their education.
• Processing – During post-data collection processing, we assigned current geography
codes (state and 15 largest metro areas) based on an address provided by the
respondent. There was potential error in this process due to item nonresponse for
Cleared for public release 10
the address questions, typos in the address provided, and respondents who no
longer lived in the United States.
We had limited time to complete this analysis, so there were some aspects that we did not
have time to research. Some potential further research is:
• Analysis of estimates and response rates while controlling for demographic
characteristics that were used in weighting adjustments. Many of the demographic
characteristics presented in this report could be correlated with each other. If a
demographic variable was correlated with a characteristic that was accounted for in
the weighting, then the weighting adjustments could mitigate potential nonresponse
bias for that demographic subgroup.
• Use of administrative records to identify characteristics of nonrespondents.
• Item nonresponse for key estimates and domains. Responses were considered
sufficient partials based on how far into the questionnaire they got before exiting.
This meant any questions after that cutoff were missing more often. Most
demographic questions were in the first section of the questionnaire, but household
income and address were the last section of the questionnaire. For key estimate
items, any respondents that did not reach the question were simply excluded from
the estimates. This likely resulted in downward bias of total estimates for these
items, and the potential bias of rate estimates is unknown.
• State or metro area. This analysis does not evaluate potential nonresponse bias at
those lower geographic levels. State-level coverage ratios and response rates are
available in the weekly source and accuracy statements (U.S. Census Bureau, 2020b).
4. Results
This section presents overall results of the HPS nonresponse bias analysis. The HPS released
weekly or biweekly (depending on survey phase) estimates on the HPS website (census.gov).
Response rates and coverage ratios by key domain groups were also provided in the
weekly/biweekly source and accuracy statement on the HPS technical documentation page
(census.gov). Those estimates and quality measures used all HPS response data, including
respondents for which some key domain questions were not answered. The analysis for this
report, summarized in the following subsections, is limited to only respondents who provided
all information needed for each measure. Therefore, the measures discussed in this report are
different from the estimates reported for the survey and the response rates in the source and
accuracy statement. Unless stated otherwise, all comparisons are significant at a ten percent
significance level.
Cleared for public release 11
4.1 Correlation of Key Estimates and Key Domains
To analyze the correlation of key estimates and key domains, we calculated all key
estimates by all key domains. The full estimates by week are in Table A1. Some observations
from weeks 1, 4, 8, 11, 13, 15, 18, and 21 are:
• HPS respondent age – persons age 65 and older were economically affected by the
pandemic less than younger age groups. The oldest age group had lower
percentages of respondents with expected loss in employment income and difficulty
paying for usual household expenses for all weeks. Other estimates and age groups
showed significant differences but not in all weeks.
• HPS respondent educational attainment – all weeks showed that college graduates
were least likely, and persons with no high-school diploma were most likely, to
expect loss in employment and have difficulty paying for usual household expenses.
College graduates were also least likely to have housing insecurity as owners in all
weeks.
• HPS respondent race/ethnicity – Hispanic persons were impacted more than non-
Hispanic white-only persons for almost all5 weekly estimates of expected loss in
employment income, food scarcity, housing insecurity, and difficulty paying for usual
household expenses. When looking only at race (ignoring ethnicity), black-only and
Asian-only persons both had higher rates of difficulty paying for usual household
expenses than white-only and other races and were not significantly different from
each other.
• HPS respondent household income – Significant comparisons across all groups and
estimates differed by week, but all weeks had the lowest income group higher than
the highest income group for estimates of expected loss in employment income,
food scarcity, housing insecurity (owners and renters), and difficulty paying for usual
household expenses.
• ACS estimates for sample unit block-group – Estimates 1 and 7 (expected loss in
employment income and difficulty paying for usual household expenses) both had
first quartile rates different from fourth quartile rates in all weeks for characteristics:
median age, percent age 65 and older, percent below poverty, percent Hispanic,
percent black-only, percent white-only, percent receiving public assistance, percent
without health insurance, percent owner occupied, and percent renter occupied.
Other characteristics had significant differences for some estimates in some weeks,
as seen in Table A1.
5
Hispanic and non-Hispanic white-only rates were not significantly different for week food scarcity.
Cleared for public release 12
For each week of data collection, we also calculated the respondent distributions across the
different domain variables for all respondents, only respondents who reported being
impacted by each estimate, and only respondents who reported not being impacted by
each estimate. The Rao-Scott Chi-Square test indicated whether the impacted respondent
distributions were significantly different from the nonimpacted respondent distributions for
each domain variable and estimate. Table 1 provides the percent of weeks where the
correlation between the demographic group and each key estimate were significant (p-
value below 0.1). Refer to Tables A2.1 and A2.2 for all weekly distributions and Chi-Squared
results.
Key items 5 & 6 (likelihood of foreclosure/eviction) were not significant for 100 percent of
weeks for any domain estimates. This was likely because the variances of those particular
estimates tended to be higher than the other estimates. Across all weeks and domains, the
average coefficients of variation for estimates 5 and 6 were 38.5 percent and 28.7 percent,
respectively. The average coefficients of variation for all other estimates ranged from 2.7
percent to 14.2 percent.
Excluding key items 5 & 6, demographic variables that were significant for all weeks and key
items were household income, highest educational attainment, race, race/ethnicity, and
percent of population with no health insurance.
Key item 7 (difficulty paying for usual household expenses) was the only estimate
significantly correlated with HPS respondent gender in all weeks. In all Phase 2 and 3 weeks,
females reported higher rates for key item 7 than males.
All domains appeared to have some level of significant correlation with at least some key
estimates, so we analyzed all domains for the analysis that follows.
Cleared for public release 13
Table 1: Percent of Weeks with Significant Correlation between Demographics and Household Pulse Survey Estimates
Household Pulse Survey Key Estimate
1 2 3 4 5* 6* 7*
Domain Loss Food Housing Housing Foreclosure Eviction Difficult to Pay
Employ Scarcity Insecure Insecure Likely Likely Household
Income Owners Renters Expenses
HPS respondent gender 50% 50% 0% 25% 75% 0% 100%
HPS respondent ethnicity 100% 75% 100% 100% 0% 0% 100%
HPS respondent household income 100% 100% 100% 100% 75% 67% 100%
HPS respondent age category 100% 88% 100% 100% 50% 0% 100%
HPS respondent highest educational
100% 100% 100% 100% 25% 50% 100%
attainment
HPS respondent race/ethnicity 100% 100% 100% 100% 25% 25% 100%
HPS respondent race 100% 100% 100% 100% 67% 0% 100%
ACS Average persons per occupied
100% 13% 63% 63% 25% 25% 100%
household
ACS Housing unit density 100% 38% 25% 25% 25% 50% 100%
ACS Median household income 88% 88% 100% 75% 25% 50% 100%
ACS Median house value 88% 88% 75% 63% 75% 25% 100%
ACS Median age 100% 38% 63% 50% 25% 25% 100%
ACS Percent of population that is female 38% 63% 13% 13% 25% 0% 75%
ACS Percent of population that is
100% 50% 100% 75% 0% 0% 100%
Hispanic
ACS Percent of housing units within
100% 38% 38% 25% 0% 25% 75%
multi-unit structures of at least 10 units
ACS Percent of housing units that are
75% 25% 13% 13% 25% 0% 50%
mobile homes
ACS Percent of population that is non-
100% 88% 100% 75% 50% 25% 100%
Hispanic and black-only
ACS Percent of population that is non-
100% 75% 100% 88% 25% 25% 100%
Hispanic and white-only
ACS Percent of population with no
100% 100% 100% 100% 25% 0% 100%
health insurance
ACS Percent of housing units that are
100% 63% 63% 50% 25% 25% 100%
owner occupied
ACS Percent of population age 18-24 88% 25% 38% 13% 50% 50% 75%
ACS Percent of population age 65 and
100% 0% 50% 63% 25% 25% 100%
older
ACS Percent of population below the
100% 88% 100% 75% 25% 50% 100%
poverty level
ACS Percent of population receiving
100% 38% 38% 38% 25% 0% 100%
public assistance income
ACS Percent of housing units that are
100% 63% 63% 50% 25% 25% 100%
renter occupied
ACS Percent of housing units that are
0% 88% 25% 13% 50% 50% 100%
vacant
ACS total housing units within block-
63% 25% 38% 63% 50% 25% 100%
group
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 4, 8, 11, 13, 15, 18, and 21
* Percentages do not include weeks 1, 4, 8, or 11 because key items 5-7 were added to the survey in week 13
Cleared for public release 14
4.2 Nonresponse Bias in First Interviews Based on Sampling Geography
The effect of nonresponse cannot be measured directly, but one indication of its potential
effect is the nonresponse rate by domains that are correlated with key estimates. Table 2
presents the average weighted response rates for key domains across the weeks included in
this analysis. See Table B1 for the complete set of response rates, standard errors, and
significance groupings.
Table 2 shows the average response rates for the population in each quartile, with the last
column of the table summarizing the magnitude of the difference in these response rates.
Since there are four weeks included in this analysis and six comparisons within each
domain, the percent of comparisons that are significant are calculated based on a total of
24 comparisons for each key item.
The most differences in response rates were observed in five domains, namely, median
household income, median house value, percent of population with no health insurance,
percent of population below the property level, and percent of housing units that are
vacant. For these key items, the responses in the four quartiles were significantly different
from each other every week. For the domains median household income and median house
value, the population in the fourth quartile had significantly higher response rates than the
rest of the population. For the domains percent of population with no health insurance,
percent of population below the property level, and percent of housing units that are
vacant, the population in the first quartile had significantly higher response rates than the
rest of the population.
The least differences in response rates were seen in the domains percent of housing units
within multi-unit structures of at least 10 units and percent of the population age 65 and
older.
Cleared for public release 15
Table 2: 2020 Household Pulse Survey Average Weighted Response Rates by Domain
Average Response Rate Percent of
Comparisons
Domain Quartile Quartile Quartile Quartile that are
1 2 3 4 Significant*
Average total persons in occupied households 6.5% 6.6% 6.8% 6.9% 50%
Housing unit density 6.0% 7.0% 7.1% 6.7% 79%
Median household income 5.0% 6.0% 6.9% 8.4% 100%
Median house value 5.0% 6.1% 7.2% 8.3% 100%
Median age 6.3% 6.9% 6.9% 6.7% 67%
Percent of population that is female 6.6% 6.9% 6.9% 6.4% 67%
Percent of population that is Hispanic 6.1% 7.0% 7.2% 6.3% 88%
Percent of housing units within multi-unit
6.8% 6.6% 6.8% 6.6% 21%
structures of at least 10 units
Percent of housing units that are mobile homes 7.1% 6.7% 6.1% 5.4% 96%
Percent of population that is non-Hispanic and
6.9% 7.1% 7.0% 5.8% 79%
black-only
Percent of population that is non-Hispanic and
5.7% 7.1% 7.3% 6.5% 96%
white-only
Percent of population with no health insurance 7.6% 7.0% 6.4% 5.7% 100%
Percent of housing units that are owner
6.0% 6.5% 6.7% 7.5% 96%
occupied
Percent of population age 18-24 7.1% 6.9% 6.6% 6.3% 88%
Percent of population age 65 and older 6.8% 6.8% 6.7% 6.6% 29%
Percent of population below the poverty level 7.9% 7.0% 6.2% 5.3% 100%
Percent of population receiving public assistance
7.1% 6.8% 6.5% 5.8% 92%
income
Percent of housing units that are renter occupied 7.5% 6.7% 6.5% 6.0% 96%
Percent of housing units that are vacant 7.5% 7.0% 6.4% 5.7% 100%
Total housing units within block-group 6.4% 6.6% 6.6% 6.9% 63%
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 8, 13, and 18
* How often response rates for one quartile were significantly different from response rates for the other three
quartiles within the same variable and data collection period.
Table 3 presents the respondent and nonrespondent distributions averaged across the
weeks for persons within the different domain quartiles. Respondent and nonrespondent
distributions show the relative percent contribution of members of a domain subset to
respondent and nonrespondent populations separately. This is different than the response
rates, which are the relative percent of respondents within the different domain subsets.
The last column of Table 3 shows the percent of weeks (out of a total of four weeks) where
the respondent and nonrespondent distributions were significantly different for the given
domain. See Table B2 for the weekly respondent/nonrespondent distributions and
associated statistics.
Cleared for public release 16
Table 3:2020 Household Pulse Survey Average Weighted Respondent/Nonrespondent Distributions by Domain
Distribution Quartiles Percent of Weeks with
Domain
Type 1 2 3 4 Different Distributions
Average total persons in occupied Respondent 24.2% 24.2% 26.7% 24.9%
75%
households Nonrespondent 25.0% 24.7% 26.2% 24.1%
Respondent 20.5% 30.4% 26.5% 22.6%
Housing unit density 100%
Nonrespondent 23.1% 28.8% 25.2% 23.0%
Respondent 15.4% 21.6% 27.6% 35.5%
Median household income 100%
Nonrespondent 20.9% 24.4% 26.8% 27.9%
Respondent 14.5% 22.6% 31.5% 31.4%
Median house value 100%
Nonrespondent 20.1% 25.5% 29.2% 25.3%
Respondent 23.3% 27.3% 25.7% 23.8%
Median age 100%
Nonrespondent 25.1% 26.4% 24.9% 23.6%
Respondent 22.0% 27.2% 27.9% 22.9%
Percent of population that is female 100%
Nonrespondent 22.4% 26.5% 27.1% 24.0%
Respondent 20.5% 28.3% 29.9% 21.3%
Percent of population that is Hispanic 100%
Nonrespondent 22.5% 27.0% 27.5% 23.1%
Percent of housing units within multi- Respondent 44.5% 17.6% 18.5% 19.4%
75%
unit structures of at least 10 units Nonrespondent 44.1% 17.8% 18.4% 19.8%
Percent of housing units that are Respondent 63.5% 15.0% 11.6% 9.9%
100%
mobile homes Nonrespondent 59.8% 14.9% 12.9% 12.4%
Percent of population that is non- Respondent 25.2% 27.1% 27.7% 19.9%
100%
Hispanic and black-only Nonrespondent 24.7% 25.4% 26.5% 23.4%
Percent of population that is non- Respondent 18.0% 28.9% 30.0% 23.1%
100%
Hispanic and white-only Nonrespondent 21.7% 27.3% 27.3% 23.8%
Percent of population with no health Respondent 28.3% 27.6% 24.4% 19.8%
100%
insurance Nonrespondent 24.6% 26.3% 25.6% 23.5%
Percent of housing units that are Respondent 21.9% 23.6% 25.4% 29.1%
100%
owner occupied Nonrespondent 24.6% 24.5% 25.3% 25.6%
Respondent 25.4% 27.4% 25.5% 21.7%
Percent of population age 18-24 100%
Nonrespondent 24.0% 26.5% 26.0% 23.4%
Percent of population age 65 and Respondent 26.4% 26.0% 24.1% 23.5%
50%
older Nonrespondent 26.1% 25.7% 24.4% 23.9%
Percent of population below the Respondent 31.7% 29.1% 23.2% 16.0%
100%
poverty level Nonrespondent 26.5% 27.7% 25.1% 20.7%
Percent of population receiving public Respondent 43.1% 24.5% 18.8% 13.7%
100%
assistance income Nonrespondent 40.7% 23.9% 19.4% 16.0%
Percent of housing units that are Respondent 29.1% 25.4% 23.6% 21.9%
100%
renter occupied Nonrespondent 25.6% 25.3% 24.5% 24.6%
Percent of housing units that are Respondent 27.9% 30.1% 24.4% 17.7%
100%
vacant Nonrespondent 24.6% 28.8% 25.6% 21.0%
Total housing units within block- Respondent 12.6% 18.4% 24.8% 44.1%
100%
group Nonrespondent 13.3% 18.9% 25.3% 42.5%
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 8, 13, and 18
All domains except average total persons in occupied households, percent of housing units
within multi-unit structures of at least 10 units, and percent of population age 65 and older
had significantly different respondent/nonrespondent distributions every week.
Cleared for public release 17
There are significant differences in response rates and respondent distributions across most
domains, and the amount of nonresponse bias depends on the pandemic impact differences
between respondents and nonrespondents within each domain. Of course, we do not know
the total degree of bias because we do not know the pandemic impacts for nonresponders.
However, HPS reduces potential bias by applying nonresponse weighting adjustments and
controlling to independent population controls by various demographics.
4.3 Nonresponse Bias Based on HPS Response data
The previous section presented measures of nonresponse bias for overall HPS response
rates by ACS demographic estimates of the block-groups containing the sample addresses.
The analysis in this section used only HPS response data to analyze potential nonresponse
bias based on demographic data reported in the HPS. The universe for this analysis was a
small portion of the overall HPS sample, but this was the only way to know the
demographics of the nonresponders.
Table 4 presents the average response measures across a subset of weeks for Phase 1 and
Phase 2 and 3 data collection separately. Refer to Tables A3 and A4 for the weekly Phase 1
and Phase 2 and 3 response measures, respectively. The Phase 1 response measures were
the percent of all first-time respondents who also responded a second time. The Phase 2
and 3 response measures were the percent of all respondents who first accessed the
instrument in the first six days of data collection. Phase 1 measures were generally lower
than Phase 2 and 3 measures because they were defined differently. The Phase 2 and 3
measures were also high because the data only allowed us to determine the first time a
respondent accessed the instrument and not necessarily when they completed their
response.
The “Percent of comparisons significant” column in Table 4 indicates how often the
response measure for each domain group is significantly different from another group
within the same domain variable. The number of comparisons in the percent is determined
by how many different groups are within the domain variable. For example, each individual
age group was part of 32 comparisons for the eight other age groups and four weeks
analyzed. Similarly, the domain variables with only two categories had only four
comparisons across the four weeks analyzed.
In all weeks, Hispanic response measures were lower than non-Hispanic response
measures. In Phase 1, average response measures were lowest for persons age 18-24 with
93.8 percent of all comparisons significant.
Cleared for public release 18
Table 4: Average Household Pulse Survey Response Measures by Domain
Phase 1 Phase 2 and 3
Average percent Percent of Average Percent of
Domain
of response a comparisons percent of comparisons
second time significant* early response significant*
HPS respondent age group
Age 18-25 10.2% 93.8% 64.8% 9.4%
Age 25-29 15.2% 56.3% 63.6% 12.5%
Age 30-34 16.6% 43.8% 63.7% 9.4%
Age 35-39 18.1% 37.5% 63.1% 12.5%
Age 40-44 18.3% 34.4% 63.3% 9.4%
Age 45-49 19.3% 37.5% 63.2% 9.4%
Age 50-54 20.1% 31.3% 64.5% 15.6%
Age 55-64 22.6% 53.1% 65.5% 12.5%
Age 65+ 25.0% 81.3% 67.0% 46.9%
HPS respondent education
No high school diploma 16.4% 33.3% 62.7% 16.7%
High school diploma 17.4% 33.3% 64.0% 16.7%
Some college 18.1% 41.7% 64.7% 16.7%
College degree 21.9% 91.7% 66.1% 50.0%
HPS respondent race/ethnicity
Hispanic 15.5% 50.0% 60.9% 41.7%
Non-Hispanic white-only 20.4% 66.7% 67.1% 91.7%
Non-Hispanic black-only 15.7% 41.7% 59.1% 50.0%
Non-Hispanic other races 18.8% 41.7% 63.1% 50.0%
HPS respondent gender
Male 18.4% 25.0% 65.6% 50.0%
Female 19.0% 25.0% 63.9% 50.0%
HPS respondent ethnicity
Non-Hispanic 19.5% 100.0% 65.6% 100.0%
Hispanic 15.5% 100.0% 60.9% 100.0%
HPS respondent household income
Income <$25K 19.1% 14.3% 64.0% 17.9%
Income $25K to <$35K 20.9% 7.1% 64.3% 0.0%
Income $35K to <$50K 18.8% 7.1% 64.6% 0.0%
Income $50K to <$75K 19.9% 7.1% 65.2% 10.7%
Income $75K to <$100K 21.1% 3.6% 67.2% 7.1%
Income $100K to <$150K 20.7% 0.0% 66.8% 3.6%
Income $150K to <$200K 22.1% 17.9% 65.9% 3.6%
Income $200K+ 21.4% 21.4% 66.9% 7.1%
HPS respondent race
White-only 19.8% 33.3% 66.3% 75.0%
Black-only 15.8% 33.3% 59.0% 50.0%
Asian-only 14.6% 16.7% 62.3% 25.0%
Other races 18.7% 16.7% 63.0% 50.0%
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 4, 8, 11, 13, 15, 18, and 21
* How often response measures for a domain category were significantly different from response measures
for other categories within the same variable and data collection period.
Cleared for public release 19
Table 5 shows the average domain distributions by response status from the weekly
distributions and Chi-Square values in Tables A5.1, A5.2, A6.1, and A6.2.
Table 5: Household Pulse Survey Average Domain Distributions by Response Status
Phase 1 Phase 2 and 3
Domain Two or more One Accessed Accessed
Responses response Instrument Early Instrument Later
HPS respondent age group 100% of weeks different 100% of weeks different
Age 18-25 6.8% 13.7% 9.3% 9.3%
Age 25-29 9.1% 11.8% 8.4% 8.8%
Age 30-34 9.4% 10.9% 9.5% 10.0%
Age 35-39 8.7% 9.1% 8.7% 9.4%
Age 40-44 8.5% 8.8% 8.3% 8.8%
Age 45-49 8.1% 7.8% 7.8% 8.4%
Age 50-54 8.3% 7.7% 8.3% 8.4%
Age 55-64 18.2% 14.4% 17.6% 17.0%
Age 65+ 22.9% 15.9% 22.1% 20.0%
HPS respondent education 75% of weeks different 50% of weeks different
No high school diploma 8.8% 10.2% 8.1% 8.8%
High school diploma 31.1% 33.8% 30.4% 31.4%
Some college 29.3% 30.6% 30.5% 30.6%
College degree 30.8% 25.4% 31.0% 29.2%
HPS respondent race/ethnicity 100% of weeks different 100% of weeks different
Hispanic 16.0% 20.1% 16.2% 19.2%
Non-Hispanic white-only 63.7% 57.1% 65.1% 58.8%
Non-Hispanic black-only 11.0% 13.5% 10.0% 12.8%
Non-Hispanic other races 9.4% 9.3% 8.6% 9.3%
HPS respondent gender 25% of weeks different 50% of weeks different
Male 48.3% 49.3% 49.0% 47.2%
Female 51.7% 50.7% 51.0% 52.8%
HPS respondent ethnicity 100% of weeks different 100% of weeks different
Non-Hispanic 84.1% 80.0% 83.8% 80.9%
Hispanic 15.9% 20.0% 16.2% 19.1%
HPS respondent household income 50% of weeks different 75% of weeks different
Income <$25K 16.3% 17.4% 13.8% 14.8%
Income $25K to <$35K 12.7% 12.1% 10.8% 11.4%
Income $35K to <$50K 12.4% 13.6% 12.6% 13.1%
Income $50K to <$75K 17.7% 18.1% 17.7% 18.0%
Income $75K to <$100K 13.5% 12.7% 14.1% 13.1%
Income $100K to <$150K 14.3% 13.9% 15.8% 14.9%
Income $150K to <$200K 6.6% 5.9% 7.2% 7.1%
Income $200K+ 6.6% 6.2% 8.1% 7.6%
HPS respondent race 75% of weeks different 100% of weeks different
White-only 74.8% 70.9% 76.8% 72.4%
Black-only 12.7% 15.7% 11.5% 14.8%
Asian-only 1.5% 2.0% 1.7% 1.9%
Other races 11.0% 11.3% 10.0% 10.9%
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 4, 8, 11, 13, 15, 18, and 21
Cleared for public release 20
The domain distributions in Table 5 tell a similar story as the response measures in Table 4.
Age group distributions were different in all weeks, primarily due to the differences in
response for the older age groups. Race/ethnicity distributions were also different in all
weeks, likely because the non-Hispanic white-only group tended to respond more than the
other race/ethnicities. Household income and gender groups were not consistently
different across all weeks. This may be misleading, however, because the universe for this
analysis was all cases that responded to the HPS at least once, which was not necessarily
the same distribution as the total sample. For more information about respondent
representation among these demographic groups, refer to the coverage ratios in the HPS
source and accuracy statement (U.S. Census Bureau, 2020b).
The analysis has shown some significant correlations between estimates and domains as
well as some significant correlation between response rates and domains. This leads to the
conclusion that there was potential nonresponse bias in the overall estimates. To evaluate
how the different response rates by domains might impact the key estimates, Table 6
presents overall estimates by response status for phase 1 and phase 2 and 3 separately.
Refer to Tables A7 and A8 for the weekly Phase 1 and Phase 2 and 3 estimates, respectively.
Table 6: Household Pulse Survey Average Estimates by Response Status
Phase 1 Phase 2 and 3
Two or Percent of Accessed Percent of
Estimate All All
More weeks Instrument weeks
Respondents Respondents
Responses different* Early different*
1-Expected income loss 37% 36% 0% 27% 27% 0%
2-Food scarcity 2% 2% 0% 3% 3% 0%
3-Housing insecurity-owners 5% 6% 25% 4% 5% 0%
4-Housing insecurity-renters 13% 13% 25% 12% 13% 0%
5-Eviction-owners n/a n/a n/a 4% 4% 0%
6-Eviction-renters n/a n/a n/a 15% 17% 0%
7-Difficulty paying expenses n/a n/a n/a 15% 15% 25%
Source: U.S. Census Bureau Household Pulse Survey data from weeks 1, 4, 8, 11, 13, 15, 18, and 21
* How many weeks, out of four total, the All Respondents estimate was significantly different from the Two or
More Responses (for Phase 1) or Accessed Instrument Early (for Phase 2 and 3) estimate.
Only three weeks found any significant difference between the response subset estimate
and the estimate with all respondents, and all three weeks were for different estimates.
This indicates that the HPS weighting adjustments reduced nonresponse bias by controlling
the weights for domains that are correlated with response. However, this could also simply
be because this analysis was limited to only cases that responded to the HPS at least once.
Cleared for public release 21
5. Conclusions
The 2020 HPS data had many potential sources of error, including coverage, processing, unit
nonresponse, and item nonresponse. The analysis in this report focused on the potential bias
due to unit response and was not an evaluation of total survey error.
We found evidence that two specific key estimates, expected loss in employment income and
difficulty paying for usual household expenses, were strongly correlated with respondent age,
race/ethnicity, educational attainment, and household income. These same estimates were
also correlated with neighborhood ACS rates by age, race/ethnicity, poverty-level, public
assistance, health insurance, and owner/renter occupied.
Estimates of likelihood of foreclosure or eviction did not have many significant results because
they are a relatively rare event with higher variances than the other key estimates. The
remaining three estimates, food scarcity and housing insecurity for owners/renters, showed
some correlation with domains but were not consistent across all HPS data collection periods.
The HPS sampling frame did not contain demographic information for nonrespondents, so our
analysis of overall HPS response used the geographical location of the sample address to obtain
demographic estimates from the ACS for respondents and nonrespondents participating in their
first interview. This analysis found that response rates were significantly different every week
for the domains median household income, median house value, percent of population with no
health insurance, percent of population below the property level, and percent of housing units
that are vacant. In addition, respondent and nonrespondent distributions were significantly
different for almost all domains of interest.
To analyze response patterns based on person-specific demographics, we also produced
domain response measures (how many times they responded in Phase 1, and how early they
accessed the instrument in Phase 2 and 3), domain distributions by response status, and overall
key estimates by response status that may provide insights into characteristics of the
nonrespondents. This analysis found response measures were most correlated with respondent
age, education, and race/ethnicity. However, the overall key estimates generally did not detect
nonresponse bias, most likely because many of the most correlated domains (age,
race/ethnicity, education) are accounted for in the HPS weighting adjustments.
Our results show evidence of potential nonresponse bias in the 2020 HPS. Our results also show
evidence that the weighting adjustment methods help to mitigate that nonresponse bias in the
final estimates. However, this does not mean there is no nonresponse bias in the HPS
Cleared for public release 22
estimates. The weighting adjustments do not incorporate any geographical ACS estimates, such
as percent of population with no health insurance, that appear to be correlated with response
and key estimates.
All results in this analysis should be interpreted with caution. We found many domains that
appear to be correlated with response and the HPS estimates, and domains that are not
accounted for in the weighting adjustments indicate potential nonresponse bias. However, the
weighting adjustments may still mitigate this bias if these domains are correlated with other
domains that are accounted for in the weights. Further research is needed to determine if there
are any sources of nonresponse bias that are not currently accounted for in the final HPS
weights and estimates.
6. References
Fay, Robert E. (1989). Theory and Application of Replicate Weighting for Variance Calculations.
In Proceedings of the Survey Research Methods Section, American Statistical Association,
212-217
U.S. Census Bureau. (2020a). 2020 Planning database, released on June 22, 2020.
https://www.census.gov/topics/research/guidance/planning-databases.2020.html
U.S. Census Bureau. (2020b). Household Pulse Survey Technical Documentation (census.gov)
Cleared for public release 23
HPS Key Estimate Definitions
Table 7: Household Pulse Survey Key Estimate Definitions
Estimate Description Criteria Universe
Key estimate 1: expected loss in employment income EXPCTLOSS=1 EXPCTLOSS=1 or 2
Key estimate 2: food scarcity CURFOODSUF=4 CURFOODSUF=1,2,3, or 4
TENURE=2 and TENURE=2 and
Key estimate 3: housing insecurity for owners
MORTCONF=1 MORTCONF=1,2,3,4, or 5
TENURE=3 and TENURE=3 and
Key estimate 4: housing insecurity for renters
MORTCONF=1 MORTCONF=1,2,3,4, or 5
TENURE=2 and TENURE=2 and
Key estimate 5: likelihood of foreclosure for owners
MORTCUR=2 and MORTCUR=2 and
(week 13 and later only)
FORCLOSE=1 FORCLOSE=1,2,3, or 4
TENURE=3 and TENURE=3 and
Key estimate 6: likelihood of eviction for renters
RENTCUR=2 and RENTCUR =2 and
(week 13 and later only)
FORCLOSE=1 EVICT=1,2,3, or 4
Key estimate 7: difficulty paying for usual household expenses
EXPNS_DIF=4 EXPNS_DIF=1,2,3, or 4
(week 13 and later only)
Source: U.S. Census Bureau 2020 Household Pulse Survey response data
Cleared for public release 24
Planning Database Quartile Definitions
Table 8: Planning Database Quartile Definitions
Number of
Variable Description Quartile Value range
block-groups
1 54,523 1.00 - 2.25
2 54,232 2.26 - 2.57
Average total persons in occupied households
3 55,728 2.58 - 2.95
4 54,279 2.96 - 20.76
1 55,620 0 - 188.02
2 54,704 188.03 - 1,129.10
Housing unit density
3 54,703 1,129.11 - 2,574.38
4 54,705 2,574.47 - 487,500.00
1 53,170 0 - 41,135
2 53,170 41,136 - 57,373
Median household income
3 53,175 57,375 - 80,956
4 53,165 80,957 - 250,001
1 51,552 0 - 111,200
2 51,571 111,300 - 176,500
Median house value
3 51,558 176,600 - 315,600
4 51,560 315,700 - 2,000,001
1 55,327 0 - 33.6
2 54,428 33.7 - 39.6
Median age
3 54,518 39.7 - 46.3
4 54,877 46.4 - 89.5
1 55,043 0 - 47.52
2 54,704 47.53 - 50.91
Percent of population that is female
3 54,711 50.92 - 54.45
4 54,756 54.46 - 100.00
1 54,684 0 - 1.38
2 54,822 1.39 - 6.47
Percent of population that is Hispanic
3 54,895 6.48 - 21.36
4 54,813 21.37 - 100.00
1 108,050 0 - 0
Percent of housing units within multi-unit 2 36,920 0.03 - 6.43
structures of at least 10 units 3 36,928 6.44 - 22.98
4 36,920 22.99 - 100.00
1 134,374 0 - 0
2 28,138 0.05 - 4.98
Percent of housing units that are mobile homes
3 28,159 4.99 - 17.34
4 28,147 17.35 - 100.00
1 65,478 0 - 0
Percent of population that is non-Hispanic and 2 51,154 0.01 - 3.20
black-only 3 51,237 3.21 - 15.23
4 51,345 15.24 - 100.00
1 54,845 0 - 37.43
Percent of population that is non-Hispanic and 2 54,895 37.44 - 72.02
white-only 3 54,767 72.03 - 89.94
4 54,707 89.95 - 100.00
Cleared for public release 25
Number of
Variable Description Quartile Value range
block-groups
1 55,097 0 - 3.00
2 54,745 3.01 - 6.79
Percent of population with no health insurance
3 54,685 6.80 - 12.86
4 54,687 12.87 - 100.00
1 54,695 0 - 47.44
2 54,682 47.45 - 70.26
Percent of housing units that are owner occupied
3 54,688 70.27 - 85.12
4 54,697 85.13 - 100.00
1 54,794 0 - 4.81
2 54,673 4.82 - 7.63
Percent of population age 18-24
3 54,743 7.64 - 11.09
4 55,004 11.10 - 100.00
1 55,067 0 - 9.88
2 54,747 9.89 - 15.08
Percent of population age 65 and older
3 54,699 15.09 - 21.14
4 54,701 21.15 - 100.00
1 54,730 0 - 4.83
2 54,676 4.84 - 11.06
Percent of population below the poverty level
3 54,670 11.07 - 21.73
4 54,726 21.74 - 100.00
1 94,310 0 - 0
Percent of population receiving public assistance 2 41,405 0.05 - 2.24
income 3 41,594 2.25 - 4.83
4 41,424 4.84 - 100.00
1 54,696 0 - 14.87
2 54,689 14.88 - 29.73
Percent of housing units that are renter occupied
3 54,682 29.74 - 52.55
4 54,695 52.56 - 100.00
1 54,718 0 - 3.14
2 54,686 3.15 - 8.69
Percent of housing units that are vacant
3 54,726 8.70 - 16.37
4 54,688 16.38 - 100.00
1 56,070 0 - 394
2 54,985 395 - 546
Total housing units within block-group
3 54,544 547 - 766
4 54,734 767 - 26,436
Source: U.S. Census Bureau 2014-2018 American Community Survey Estimates
Cleared for public release 26
File and source
- File
- expansion-of-child-tax-credits-and-mental-health-of-parents-with-low-income-in-2021.pdf
- Size
- 553,449 bytes
- SHA-256
- 2f8ab20e9a9308633b32409574da879a20a5f18e5930a60568aadf74036c18c0
- Original
- www2.census.gov