Shuttered Venue Operators Grant Program Evaluation — 2M Research, U.S. Small Business Administration (October 27, 2023)
Summary
The Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report, dated October 27, 2023, prepared by 2M Research for the U.S. Small Business Administration under Contract No. 73351022F0099. The executive summary states that the Shuttered Venue Operators Grant (SVOG) program was established in 2021 and included over $16 billion in grants, with eligible applicants able to qualify for grants equivalent to 45 percent of their gross earned revenue, up to a maximum single grant amount of $10 million. It says the evaluation used SVOG applicant data, grantee closeout survey information, a grantee web survey, grantee interviews and Dun & Bradstreet data. Its key findings include that 19 percent of SVOG grantees (and 29 percent of non-recipients) were underserved businesses and that grantees usually spent most of their funds on payroll.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
- Issuer
- Source documents
Cite this document
“Shuttered Venue Operators Grant Program Evaluation — 2M Research, U.S. Small Business Administration (October 27, 2023).”
Original at www.sba.gov. Archived at Pandemic Darlings, https://pandemicdarlings.com/source-documents/svog-evaluation-peer-difference-in-difference/
, SHA-256 cf7bcb4877947ebdc5b2a122a1a68eeba6afbb1caaca1ca87d53a108c1066c73.Full text
Shuttered Venue Operators Grant Program
Evaluation: Final Evaluation Report
Submitted to:
Sana Ahmed Wilder
U.S. Small Business Administration
October 27, 2023
Contract No. 73351022F0099
Prepared By:
James Murdoch, Arpita Chakravorty, Emma Ward, Lan Hu, Kowther Elqutob, and Paul
Ruggiere
2M Research
1521 N Cooper St, Suite 600
Arlington, TX 76011
www.2mresearch.com
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Table of Contents
Executive Summary ............................................................................... ii
Introduction ......................................................................................... 1
Domain 1: Characteristics of SVOG Grantees and Non-Recipients ................ 7
Domain 2: Implementation of SVOG....................................................... 23
Domain 3: Impacts of SVOG .................................................................. 25
Domain 4: Perceptions of SVOG and Remaining Pandemic Recovery Needs 42
Conclusion.......................................................................................... 51
Technical Appendix............................................................................. A-1
Contract No. 73351022F0099 | 2M Research | i
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Executive Summary
The Shuttered Venue Operators Grant (SVOG) program was established in 2021 to
support the ongoing operations of eligible live venues and operators, live venue
promoters, theatrical producers, talent representatives, live performing arts
organization operators, museums, and motion picture theaters that were financially
affected by social distancing and other prevention measures due to the COVID-19
pandemic. Established by the 2020 Economic Aid to Hard-Hit Small Businesses,
Nonprofits, and Venues Act, the program was administered by the U.S. Small Business
Administration (SBA) Office of Disaster Assistance (now named the Office of Disaster
Recovery and Resilience) and included over $16 billion in grants to eligible venues.
Eligible applicants to the program could qualify for grants equivalent to 45 percent of
their gross earned revenue, up to a maximum single grant amount of $10 million. 1
This report describes the key findings from a program evaluation conducted by 2M
Research (2M) to better understand grantee characteristics, the allocation and
spending of their funds, and the key outcomes of SVOG. The evaluation used a
combination of SVOG applicant data, grantee closeout survey information, data
collected by 2M from a grantee web survey and grantee interviews, and extant data
from Dun & Bradstreet.
The figures below present key findings from the evaluation.
1
For more information on the SVOG program, please see:
https://www.sba.gov/funding-programs/loans/covid-19-relief-options/shuttered-venue-operators-grant/about-svog.
Contract No. 73351022F0099 | 2M Research | ii
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 1: SVOG grantees and non-recipients varied in terms of venue type.
Key Finding 2: Most SVOG grantees were small businesses with less than 50
employees, but small businesses also made up a large share of non-recipients.
Contract No. 73351022F0099 | 2M Research | iii
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 3: Awards went to businesses throughout the United States, but
award rates were lower in Southern states.
Note: Award rate is defined as the number of grantees in the state divided by the total
number of applications in the state, multiplied by 100.
Contract No. 73351022F0099 | 2M Research | iv
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 4: 19 percent of SVOG grantees (and 29 percent of non-recipients)
were underserved businesses.
Note: The evaluation team used “underserved” to capture businesses identified in Dun
& Bradstreet data as any of the following: disabled-owned, veteran-owned, women-
owned, minority-owned, or small disadvantaged. The evaluation team retained the
same naming convention from Dun & Bradstreet’s data categories. We recognize that
the underserved category does not fully capture all businesses that have historically
experienced unequal treatment by the government.
Contract No. 73351022F0099 | 2M Research | v
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 5: Grantees usually spent most of their funds on payroll.
Contract No. 73351022F0099 | 2M Research | vi
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 6: Many grantees reported that SVOG was important for the survival
of their business during the pandemic.
▪ “This grant specifically allowed us to stay open. We are big fans of it. It was a
very complex application. There was a lot of support on the website, and all of
the webinars and the YouTube videos were so helpful that SBA put out… yes,
very, very grateful.” – SVOG Grantee Interview Respondent
▪ “This kind of support was absolutely amazing and made it able for us to survive
and to come back and begin getting back-to-normal business paying artists and
so on. If only some form of this support [was] always in place. Everybody's lives
would be better!” – SVOG Grantee Survey Respondent
Key Finding 7: Grantees requested additional funding to address pandemic
recovery needs and described industry-wide issues that still impact their
businesses. About one-fourth of grantees indicated no remaining needs.
Contract No. 73351022F0099 | 2M Research | vii
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 8: SVOG grantees experienced an increase in average employee
counts compared to non-recipients after the disbursement of SVOG funds, but the
increase was not statistically significant and cannot conclusively be attributed to
SVOG funds.
Note: The difference in the employee count trends between grantees and non-
recipients was not statistically significant in the difference-in-differences regression
model.
Key Finding 9: SVOG grantees experienced increased annual revenues compared
to non-recipients after the disbursement of SVOG funds. There is conclusive
evidence that SVOG had a positive impact on the annual revenue of grantees.
Note: The difference in the revenue trends for grantees and non-recipients was
statistically significant (p <0.01) in the difference-in-differences regression model.
Contract No. 73351022F0099 | 2M Research | viii
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Finding 10: Grantees experienced a significant improvement in average
business viability, with lower scores indicating better viability, compared to non-
recipients after the disbursement of SVOG funds. There is conclusive evidence
that SVOG had a positive impact on the business viability of grantees.
Note: The business viability score ranges from 1 to 9, with 9 representing businesses
most likely to cease operations and 1 representing businesses least likely to cease
operations. The difference in the viability trends between grantees and non-recipients
was statistically significant (p < 0.05) in the difference-in-differences regression model.
Overall, the key findings from the SVOG Program Evaluation indicate that program
funds went to a diverse set of businesses across the United States. Most grantees spent
funds on payroll and general operating expenses related to their businesses. The
grantee survey and interviews found evidence that grantees felt SVOG prevented their
businesses from closure. The evaluation found conclusive evidence that SVOG funds
increased annual revenues and improved the viability of grantees.
The report findings suggest several recommendations to consider when implementing
similar emergency response and/or industry-specific grant programs in the future.
Contract No. 73351022F0099 | 2M Research | ix
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Recommendation 1: Regularly conduct analysis of client characteristics,
comparing to those of applicants that were not admitted to the program.
▪ Evidence. The evaluation found smaller businesses, underserved businesses,
businesses in some regions of the U.S., and certain venue types were less likely
to receive an SVOG award.
▪ Implementation. To better understand differences or potential bias in award
and lending patterns, SBA programs can implement real-time, interactive
dashboards that display topline measures that compare clients to all
applicants. If there are clear differences, programs can consider implementing
outreach efforts to target groups that are receiving fewer awards, loans,
certifications, and more. See Recommendation 3 for more evidence supporting
outreach. A dashboard may also support track measures of impact on
surrounding industries and economic impacts, to better facilitate evidence-
building activities.
Recommendation 2: Support outreach that can identify needs and challenges
from the perspective of the industry or area targeted by the grant program.
▪ Evidence. SVOG targeted the performing arts industry. The grantee survey
responses and interviews found many grantees face larger industry-wide issues
that the SVOG funds could not address on their own. These issues included
decreases in attendance at in-person events and fewer theatrical releases in the
era of streaming (an issue specific to motion picture theater operators).
▪ Implementation. Outreach efforts can be restricted by grant management
practices which can be outside of a program’s control, as was the case for SVOG.
It is possible to design programs with integrated outreach and service resources
to inform continuous quality improvement, however. SBA programs can further
opportunities for client support and improved client experiences with built-in
outreach mechanisms. Using support surveys, listening sessions, and other
outreach, SBA programs can proactively identify issues and solutions to inform
program implementation.
Contract No. 73351022F0099 | 2M Research | x
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Recommendation 3: Support grantee connections to other sustainable sources of
funding.
▪ Evidence. The evaluation found almost all grantees spent all their SVOG funds,
and, in the interviewees, several grantees mentioned needs for more funding
and their current challenges identifying sustainable sources of funding for their
businesses.
▪ Implementation. Using improved mechanisms for outreach and understanding
client characteristics, SBA programs can organize convenings that bring
together customers based on eligibility for additional funding sources. At the
convenings, SBA programs can make connections to other SBA resources, and
facilitate discussions to help customers create networks from which they may
learn about best practices to access available, sustainable funding.
Recommendation 4: Consider ways to streamline and ease the burden of the
application process, improving customer experience.
▪ Evidence. In the web survey, a small number of grantees indicated a negative
experience with the SVOG application process.
▪ Implementation. Often grant program staff face constraints in the degree to
which they can interact with applicants and influence application requirements,
especially in the case of grant programs established through legislation, like
SVOG. That said, when flexibilities exist, SBA programs can dedicate staff to
support questions or technical issues that arise during the application process;
a critical component to providing meaningful support is to ensure collaboration
between policymakers and communications personnel to develop guidance in
plain language. In addition, SBA should look for ways to develop a centralized
application that can determine eligibility for more than one program, so that
applicants are not required to submit multiple applications.
Contract No. 73351022F0099 | 2M Research | xi
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Introduction
Background
In December 2020, the Shuttered Venue Operators Grant (SVOG) was established by
Section 324 of the Economic Aid to Hard-Hit Small Businesses, Nonprofits and Venues
Act (15 U.S.C. 9009a), as part of H.R. 133 Consolidated Appropriations Act, and was
amended by Section 5005 of the American Rescue Plan Act of 2021. The program was
designed to support the ongoing operations of eligible live venues and operators, live
venue promoters, theatrical producers, talent representatives, live performing arts
organization operators, museums, and motion picture theaters that were financially
affected by social distancing and other prevention measures due to the COVID-19
pandemic. The program was administered by the U.S. Small Business Administration
(SBA) Office of Disaster Assistance, now the Office of Disaster Recovery and Resilience,
and included over $16 billion in grants to eligible venues. Eligible applicants to the
program could quality for grants equivalent to 45 percent of their gross earned
revenue, up to a maximum single grant amount of $10 million.
As of September 2022, the SVOG program awarded funds to 12,944 businesses. After
the initial round of awards, the SBA invited grantees that received less money than
they requested to apply for a reconsideration of their award amount. Grantees who
accepted reconsideration invitations were required to provide supporting
documentation that demonstrated sustained revenue loss of 70 percent or greater, as
well as documentation that may have been missing from their initial application. In
August 2021, SVOG grantees were encouraged to apply for a supplemental award. This
award provided additional funding up to 50 percent of the original award amount with
a cap at $10 million in total grant awards. Of the 12,944 businesses that received funds,
3,739 received higher award amounts after reconsideration and 10,581 received
supplemental awards. The funds were distributed from May 2021 through August 2022,
with most funds (95 percent) distributed by December 2021.
Contract No. 73351022F0099 | 2M Research | 1
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 1. Most SVOG funds were distributed in 2021.
Month and Year of Award Number of Awards Cumulative Percent
May 2021 31 0.24
June 2021 3,097 24.17
July 2021 6,871 77.25
August 2021 1,205 86.56
September 2021 487 90.32
October 2021 369 93.17
November 2021 255 95.14
December 2021 44 95.48
January to August 2022 192 96.96
Missing award date 393 100
N = 12,944
Source: SVOG Application Data.
Evaluation Objectives and Questions
To better understand grantee characteristics, the allocation and spending of their
funds, and the key outcomes of SVOG, the SBA Office of Program Performance,
Analysis, and Evaluation contracted with 2M Research (2M), an SBA HUBZone-certified,
minority-owned, and small disadvantaged business, to conduct a mixed methods
evaluation of the program. 2M’s evaluation team designed the evaluation to answer
the following evaluation questions which are organized into four key domains:
Domain 1: Characteristics of SVOG Grantees and Non-Recipients.
▪ What are the characteristics of SVOG grantees? How do SVOG grantees compare
to business owners who applied but failed to receive SVOG funds?
Domain 2: Implementation of SVOG.
▪ How have SVOG grantees spent their grant funds?
▪ How did the actual use of funds compare to the approved allowable costs
following budget review(s)?
Contract No. 73351022F0099 | 2M Research | 2
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Domain 3: Impacts of SVOG.
▪ What is the change in number of jobs for SVOG venues before and after the
receipt of grant funds? What is the change for venues that applied and failed to
receive SVOG funds during the same time frame? How does the change in
number of jobs vary by venue type, industry, geography, number of employees,
socioeconomic designation, and owner demographics, among other variables?
▪ What is the average change in revenue earned by SVOG venues before and after
the receipt of grant funds? What is the change for venues that applied and failed
to receive SVOG funds during the same time frame? How does the average
change in revenue vary by key business characteristics?
▪ How many SVOG grantees are still in operation three months, six months, and
twelve months after the final disbursement of all funds (including funds from
reconsiderations and supplementations)?
▪ What are the characteristics of SVOG grantees still in operation today and how
do they compare to applicants who failed to receive SVOG funds? How do they
compare to SVOG grantees that are no longer in operation? How does operating
status vary by key business characteristics?
Domain 4: Perceptions of SVOG and Remaining Pandemic Recovery Needs.
▪ What are grantees’ perceptions of the SVOG grant’s relationship to business
survival? Do SVOG grantees associate the grant with business outcomes?
▪ What are the remaining pandemic recovery needs of SVOG grantees?
Contract No. 73351022F0099 | 2M Research | 3
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Evaluation Data Sources
The SVOG Program Evaluation includes five data sources:
1. SVOG Application Data: Data provided by the SBA that included information on
the characteristics of program applicants, identified which businesses received
a grant (grantees) and those that applied but failed to receive a grant (non-
recipients), and included details on approved award amounts of all grantees.
2. SVOG Closeout Survey: Data provided by SBA that included responses to key
questions about grantees’ perceptions of the program and its relationship to
business survival. Only 82 grantees had responded to the closeout survey by the
time the evaluation team received the data.
3. Grantee Survey: Data collected by the evaluation team through a web survey
delivered to a random sample of 4,000 SVOG grantees. The survey included
information on how grantees spent the funds they received and an open-ended
question about the remaining pandemic recovery needs of grantees. The
evaluation team received 1,829 survey responses, a response rate of 46 percent.
4. Follow-On Interviews: Data collected by the evaluation team through follow-
on semi-structured interviews with 36 grantees that responded to the grantee
survey; these interviews gathered more in-depth information about the
remaining pandemic recovery needs of grantees.
5. External Data from Dun & Bradstreet: Data purchased from Dun & Bradstreet
that included information on trends for number of employees, sales, and
business viability of both grantees and non-recipients from 2018 to 2022. The
evaluation team planned to use this data to also capture trends in annual
revenue, but the information source was unavailable for SVOG applicants. The
evaluation team estimated annual revenue based on revenue figures from the
SVOG application data for 2019 and 2020 and the available data from Dun &
Bradstreet using a multiple imputation process.
Contract No. 73351022F0099 | 2M Research | 4
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Evaluation Analysis Methods
The evaluation team studied each data source with different analytical methods:
1. Descriptive Statistics: The evaluation team analyzed characteristics of
grantees and non-recipients in the SVOG application data; information on
program perceptions from the closeout survey and grantee survey; and trends
in revenue, employee counts, and business survival for grantees and non-
recipients from Dun & Bradstreet using descriptive statistics.
2. Logistic Regression: The evaluation team used logistic regression to
understand the key characteristics of successful applications that received
SVOG funds.
3. Difference-in-Differences (DID) Regression Modeling with Propensity Score
Matching (PSM): The evaluation team matched grantees to non-recipients
based on their key characteristics using PSM and then estimated a DID that
determined if grantees experienced larger increases in revenue, employee
counts, and business viability compared to non-recipients. The evaluation team
estimated a DID overall and a DID for key business characteristics to determine
if findings changed for different business types.
4. Topic Modeling: The evaluation team performed topic modeling of open-ended
grantee survey data to identify the remaining pandemic recovery needs of
grantees.
5. Thematic Analysis: The evaluation team conducted thematic analysis of the
follow-on interview transcripts to identify additional information on the
pandemic recovery needs of grantees.
Organization of the Report
The remainder of this report is organized into sections for each of the four domains of
the SVOG Program Evaluation. Domain 1 provides a summary of the characteristics of
SVOG grantees and non-recipients. Domain 2 discusses how grantees spent their
funds. Domain 3 reports on the impact of the SVOG program on business survival,
revenue, and number of employees. Domain 4 presents a discussion on the
perceptions of the SVOG program and the remaining pandemic recovery needs of
grantees. Finally, the conclusion highlights recommendations based on the study’s key
findings and discusses the study’s limitations.
Contract No. 73351022F0099 | 2M Research | 5
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 2. The evaluation team used a mixed methods approach to implement the SVOG Program Evaluation.
Data Sources Analysis Methods
Grantee Survey
DID Regression
Topic Modeling
SVOG
Application
Follow-On Descriptive Logistic
Domains and Evaluation Questions
Data Dun & Thematic
with PSM
SVOG Closeout
Survey Interviews Bradstreet Statistics Regression Analysis
Domain 1: Characteristics of SVOG Grantees and Non-Recipients ✓ ✓ ✓ ✓
What are the characteristics of SVOG grantees? How do SVOG grantees compare
✓ ✓ ✓ ✓
to business owners who applied but failed to receive SVOG funds?
Domain 2: Implementation of SVOG ✓ ✓
How have SVOG grantees spent their grant funds? ✓ ✓
How did the actual use of funds compare to the approved allowable costs
✓ ✓
following budget review(s)?
Domain 3: Impacts of SVOG ✓ ✓ ✓ ✓ ✓
What is the change in number of jobs for SVOG venues before and after the
receipt of grant funds? What is the change for venues that applied and failed to
receive SVOG funds during the same time frame? How does the change in ✓ ✓ ✓ ✓
number of jobs vary by venue type, industry, geography, number of employees,
socioeconomic designation, and owner demographics, among other variables?
What is the average change in revenue earned by SVOG venues before and after
the receipt of grant funds? What is the change for venues that applied and failed
✓ ✓ ✓ ✓
to receive SVOG funds during the same time frame? How does the average
change in revenue vary by key business characteristics?
How many SVOG grantees are still in operation three months, six months, and
twelve months after the final disbursement of all funds (including funds from ✓ ✓
reconsiderations and supplementations)?
What are the characteristics of SVOG grantees still in operation today and how
do they compare to applicants who failed to receive SVOG funds? How do they
✓ ✓ ✓ ✓
compare to SVOG grantees that are no longer in operation? How does operating
status vary by key business characteristics?
Domain 4: Perceptions of SVOG and Remaining Pandemic Recovery Needs ✓ ✓ ✓ ✓ ✓ ✓
What are grantees’ perceptions of the SVOG grant’s relationship to business
✓ ✓ ✓ ✓ ✓ ✓
survival? Do SVOG grantees associate the grant with business outcomes?
What are the remaining pandemic recovery needs of SVOG grantees? ✓ ✓ ✓
Contract No. 73351022F0099 | 2M Research | 6
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Domain 1: Characteristics of SVOG Grantees and Non-
Recipients
This section presents information on the award amounts and characteristics of
grantees and non-recipients of the SVOG program, including venue type, number of
employees, annual revenue in 2020, geography, industry, and underserved business
status.2 The primary data source for this domain is the SVOG application data. The
evaluation team also used information from Dun & Bradstreet to capture the industry
and underserved business status of each grantee and non-recipient. The evaluation
team applied descriptive statistics and a logistic regression to develop the findings
presented in this section. More information on the data sources and analytical
methods is available in the Technical Appendix.
What are the characteristics of SVOG grantees? How do SVOG grantees compare to
business owners who applied but failed to receive SVOG funds?
Key Insights
▪ SVOG grantees and non-recipients varied in terms of venue type.
▪ Most SVOG grantees were small businesses with less than 50 employees, but
small businesses also made up a large share of non-recipients.
▪ Awards went to businesses across the United States, but businesses in Southern
states like Louisiana, Mississippi, Alabama, Georgia, and Florida were less
successful in securing SVOG funds than businesses in other parts of the country.
▪ 19 percent of SVOG grantees (and 29 percent of non-recipients) identified as an
underserved business.
Award Amount
SVOG provided awards of different sizes to American businesses during the pandemic.
Most businesses (75 percent) received an award of less than $1 million. A smaller
number (23 percent) received an award of $1 million or more (but less than $10
million). Only about two percent of grantees received an award of $10 million or more.
2
The evaluation team uses “underserved” to capture businesses identified in Dun & Bradstreet data as any of the following:
disabled-owned, veteran-owned, women-owned, minority-owned, or small disadvantaged. The evaluation team retained
the same naming convention from Dun & Bradstreet’s data categories. We recognize that the underserved category does not
fully capture all businesses that have historically experienced unequal treatment by the government.
Contract No. 73351022F0099 | 2M Research | 7
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 3. Most SVOG grantees received an award of less than $1 million.
N = 12,944
Note: Reconsidered and supplemental award amounts were used when available.
Source: SVOG Application Data.
Venue Type
SVOG grantees and non-recipients had a range of venue types. Although many
grantees were live venue promoters, SVOG application data shows that 53 percent of
non-recipients were also live venue promoters. This suggests that live venue
promoters were more likely to be unsuccessful than other venue types at securing
SVOG funds. This also holds true of talent representatives although the finding is less
pronounced (11 percent of grantees compared to 18 percent of non-recipients). Motion
picture theater operators, by contrast, were most successful at securing SVOG funds.
Although the overall number of grantees is small (12 percent), the percent of grantees
is six times the percent of non-recipients (2 percent), which is the largest difference of
any venue type.
Contract No. 73351022F0099 | 2M Research | 8
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 4. SVOG grantees and non-recipients varied in terms of venue type.
Note: Differences between grantees and non-recipients by venue type are statistically
significant based on a chi-squared test.
Source: SVOG Application Data.
Number of Employees
SVOG grantees were most often small businesses with less than 50 employees (90
percent), and many were very small with less than five employees (43 percent). This
highlights an important success of the SVOG program as SBA aims to support the
“smallest of the small” businesses3; however, about 55 percent of non-recipients were
very small businesses as well. While many very small businesses received SVOG funds,
this business size was also the least successful at securing funds.
3
Please refer to Objective 1.1: Ensure all entrepreneurs have access to capital to start and grow their business in
SBA Strategic Plan FY 2022-2026
Contract No. 73351022F0099 | 2M Research | 9
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 5. Most SVOG grantees and non-recipients were small businesses with less
than 50 employees.
Note: Differences between grantees and non-recipients by number of employees are
statistically significant based on a chi-square test.
Source: SVOG Application Data.
Annual Revenue in 2020
Annual revenue findings mirror those for number of employees above. Many
businesses with small annual revenues (less than $100,000 in 2020) received SVOG
funds (41 percent of grantees), but these businesses also made up a large share of
non-recipients (61 percent).
Contract No. 73351022F0099 | 2M Research | 10
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 6. Most SVOG grantees and non-recipients had 2020 annual revenues less
than $1 million.
Note: Differences between grantees and non-recipients by annual revenue are
statistically significant based on a chi-square test.
Source: SVOG Application Data.
Geography
SVOG grantees were geographically dispersed across the country in all 50 states, the
District of Columbia, Puerto Rico, the U.S. Virgin Islands, and Guam. Most grantees’
businesses were in large metropolitan areas like New York, Chicago, and Los Angeles,
and fewer were in more rural areas and smaller metropolitan areas and cities. Non-
recipient businesses shared a similar distribution.
Contract No. 73351022F0099 | 2M Research | 11
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 7. SVOG grantee and non-recipient businesses were dispersed throughout
the country, with higher concentrations in major metropolitan areas.
Contract No. 73351022F0099 | 2M Research | 12
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Notes: In the heat map, a lack of color indicates areas of the country with few or no
grantees/non-recipients. Darker shades indicate more density, with the darkest blue
indicating locations that have the largest density of grantees/non-recipients.
Sources: SVOG Application Data, Cartographic Boundary Files from the U.S. Census
Bureau.
About 90 percent of both grantees and non-recipients were in metropolitan areas. The
percentage of grantees in smaller micropolitan and rural areas was slightly higher than
non-recipients, though not by a substantial amount (7 percent compared to 5 percent
for micropolitan and 4 percent compared to 2 percent for rural).
Exhibit 8. Almost all SVOG grantees and non-recipients were in metropolitan
areas.
Notes: This evaluation uses U.S. Census Bureau definitions for Metropolitan and
Micropolitan. For more information, please visit:
https://www.census.gov/programs-surveys/metro-micro/about/glossary.html. Rural is
any location outside a metropolitan or micropolitan area as defined by the Census.
Differences between grantees and non-recipients by urbanicity are statistically
significant based on a chi-square test.
Source: SVOG Application Data and Census Bureau metropolitan definitions.
Grantees and non-recipients were also dispersed across SBA’s 10 regions. There were
larger percentages of non-recipients, however, in the Pacific and Southeast regions.
Contract No. 73351022F0099 | 2M Research | 13
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 9. SVOG grantees and non-recipients were in all 10 SBA regions.
Notes: SBA regions are collections of states. They are defined here:
https://advocacy.sba.gov/regional-advocates/. Differences between grantees and non-
recipients by SBA region are statistically significant based on a chi-square test.
Source: SVOG Application data.
When examining the award rate by state, differences between grantees and non-
recipients become clearer. In Southern states like Louisiana, Mississippi, Alabama,
Georgia, and Florida, the percentage of awarded applications is smaller (between 50
and 70 percent) than in other parts of the country. This also holds true in Nevada,
Arizona, Illinois, and the U.S. Virgin Islands. In contrast, the SBA awarded funds to over
90 percent of applications in Iowa and South Dakota. Award rates in the rest of the
states and territories ranged from 70 to 90 percent.
Contract No. 73351022F0099 | 2M Research | 14
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 10. Southern states had lower award rates than other parts of the country.
Note: Award rate is defined as the number of grantees in the state divided by the total
number of applications in the state, multiplied by 100.
Sources: SVOG Application Data; State hexagon shapefile obtained from
https://vizpainter.com/wp-content/uploads/Hex-States-Shapefile.zip
Industry
SVOG grantees operated in many different industries as well. Most were in the Arts,
Entertainment, and Recreation (44 percent) or the Information (13 percent) industry.
Non-recipients had a smaller percentage in these two industries and in most cases had
larger percentages in other industries. Since SVOG targets arts and cultural businesses
(which align well with the Arts, Entertainment, and Recreation and the Information
industries), this finding may indicate that non-recipients were sometimes unsuccessful
because they came from outside the program’s target industries.
Contract No. 73351022F0099 | 2M Research | 15
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 11. SVOG grantees and non-recipients varied by industry, though most
grantees were in either the Arts, Entertainment, and Recreation or the
Information industry.
Note: Percentages in the figure were rounded to the nearest whole number. The
industry was defined from the first two digits of the business NAICS in the SBA
application data. For more information on NAICS, please visit
https://www.census.gov/naics/?58967?yearbck=2022. The Other Industries category
includes 10 industries, which are Mining; Utilities; Transportation and Warehousing;
Agriculture, Forestry, Fishing, and Hunting; Construction; Wholesale Trade;
Management of Companies and Enterprises; Manufacturing; Finance and Insurance;
and Health Care and Social Assistance. Differences between grantees and non-
recipients by industry are statistically significant based on a chi-square test.
Source: SVOG Application Data and Dun & Bradstreet.
Contract No. 73351022F0099 | 2M Research | 16
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Underserved Business Types
In addition to very small businesses discussed above, the SBA also has a specific focus
on businesses that historically have received fewer federal supports than other types
of businesses. In this evaluation, the evaluation team used “underserved” to capture
businesses identified in Dun & Bradstreet data as any of the following: disabled-
owned, veteran-owned, women-owned, minority-owned, or small disadvantaged. The
evaluation team retained the same naming convention from Dun & Bradstreet’s data
categories.4 Nineteen percent of SVOG grantees identified as at least one of these
business types. Through SVOG, the SBA provided funds to 2,486 underserved
businesses during the pandemic. This highlights an important finding given the SBA’s
mission to support America’s underserved businesses, but the SVOG Application data
also shows that about 29 percent of non-recipients identified as an underserved
business. Comparing these figures for grantees and non-recipients, underserved
applicants were more likely to be rejected than businesses that did not identify as one
of the categories within our underserved definition.
Exhibit 12. More non-recipients identified as an underserved business than SVOG
grantees.
Note: Differences between grantees and non-recipients by underserved business
status are statistically significant based on a chi-square test.
Sources: SVOG Application Data and Dun & Bradstreet.
4
The evaluation team recognizes that the underserved category does not fully capture all businesses that have historically
experienced unequal treatment by the government.
Contract No. 73351022F0099 | 2M Research | 17
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Among the underserved businesses that received SVOG funds, about 54 percent
identified as women-owned, 46 percent identified as minority-owned, 18 percent
identified as small disadvantaged, 9 percent identified as veteran-owned, and 1
percent identified as disabled-owned. Among the underserved businesses that
applied but failed to receive funds, the percentages identifying as each type were
similar. The largest differences were for small disadvantaged and minority-owned
businesses; these two types were less likely to successfully apply compared to other
types of underserved businesses.
Contract No. 73351022F0099 | 2M Research | 18
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 13. Among SVOG grantees and non-recipients that were underserved businesses, most were women-
owned, minority-owned, and small disadvantaged businesses.
Note: The denominator for all percentages is the number of underserved businesses that received (grantees) or failed to
receive (non-recipients) funds from SVOG. Percentages across the business types do not add up to 100 because a
business can identify as more than one type. Businesses may also identify as HUBZone or 8(a); however, virtually no
SVOG grantees or non-recipients identified as these types and therefore are not included in the exhibit.
Sources: SVOG Application Data and Dun & Bradstreet.
Contract No. 73351022F0099 | 2M Research | 19
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Key Characteristics
Exhibit 14 below summarizes findings for Domain 1 by the average probability of
successfully applying and receiving funds from SVOG for businesses with the different
characteristics examined in this section. Overall, the most important characteristic for
increasing the probability of securing an award was being a motion picture theater
operator. This venue type had a 95 percent average probability of award. The exhibit
shows other characteristics that were important (85 percent probability or greater)
highlighted in blue. These include being a large business with 250 or more employees;
being in a rural location; being in the Atlantic SBA region; and being in the Arts,
Entertainment, and Recreation or the Public Administration industries. The exhibit
shows characteristics with decreased probability (75 percent probability or lower) of
successfully applying for SVOG funds highlighted in yellow. Characteristics that
decrease the probability of an award include being a live venue operator or promoter
or talent representative; having a small 2020 annual revenue of less than $100,000;
and being in a non-target industry such as Educational Services.
Contract No. 73351022F0099 | 2M Research | 20
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 14. Motion picture theater operators, larger businesses with more than
250 employees, businesses with large annual reviews, rural businesses, and
businesses located in the Atlantic SBA region all had higher probabilities of
receiving an award when applying for SVOG funds.
Average Probability of
Category Subcategory
Successful SVOG Application
Venue Types Live performing arts organization operator 84%
Live venue operator or promoter 74%
Motion picture theater operator 95%
Museum Operator 80%
Talent representative 72%
Theatrical producer 82%
Number of Less than 5 80%
Employees 5-9 78%
10-49 80%
50-249 84%
250 or more 86%
2020 Annual Less than $100,000 73%
Revenue $100,000 to $999,999 85%
$1,000,000 to $9,999,999 84%
$10,000,000 or more 85%
Urbanicity Metropolitan 80%
Micropolitan 83%
Rural 85%
SBA Region Atlantic 85%
Great Lakes 81%
Great Plains 84%
Mid-Atlantic 80%
New England 84%
Pacific 78%
Pacific Northwest 82%
Rocky Mountains 82%
South Central 79%
Southeast 77%
Industry Accommodation and Food Services 80%
Administrative and Support and Waste Management
74%
and Remediation Services
Arts, Entertainment, and Recreation 89%
Educational Services 69%
Information 81%
Other Services 73%
Professional, Scientific, and Technical Services 62%
Public Administration 92%
Real Estate Rental and Leasing 70%
Retail Trade 62%
Other Industries 54%
Unclassified 68%
Underserved No 81%
Business Status Yes 77%
Contract No. 73351022F0099 | 2M Research | 21
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
N = 17,042
Notes: Blue shading indicates a probability of 85 percent or higher. Yellow shading
indicates a probability of 75 percent or lower. The overall average probability of
receiving SVOG funding was 80 percent. The evaluation team estimated the
probabilities based on results from a logistic regression where grantee status (yes/no)
was the dependent variable and the characteristics above were independent variables.
Source: SVOG Application Data, Dun & Bradstreet, and U.S. Census Bureau
metropolitan definitions.
Contract No. 73351022F0099 | 2M Research | 22
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Domain 2: Implementation of SVOG
This section presents information from the grantee survey about how grantees spent
their SVOG funds. More information on this survey is available in the Technical
Appendix.
How have SVOG grantees spent their grant funds?
Key Insights
▪ Payroll costs and other ordinary business expenses were the most common
uses of SVOG funds.
▪ On average, businesses spent over half of their SVOG funds on payroll.
▪ Virtually all grantees that responded to the survey spent all their funds; only
one percent indicated that they had leftover funds.
SVOG grantees used the funds they received to address a variety of business expenses.
The grantee survey included a question that asked grantees whether they used their
funds for the following categories (respondents could check more than one category):
▪ Payroll costs.
▪ Rent payments.
▪ Utility payments.
▪ Scheduled mortgage payments.
▪ Scheduled debt payments on any indebtedness incurred in the ordinary course
of business prior to February 15, 2020.
▪ Worker protection expenditures (for example, costs incurred to implement
health and safety measures to protect employees from COVID-19 exposure,
including face masks, gloves, sanitization supplies, touchless payment devices,
and plexiglass shields).
▪ Payments to independent contractors.
▪ Other ordinary and necessary business expenses such as maintenance costs;
administrative fees and service subscriptions; state and local taxes and licensing
fees; operating leases as of February 15, 2020; and insurance payments.
▪ Expenditures related to producing a theatrical or live performing arts
production, concert, exhibition, or comedy show.
▪ Other (respondents were asked to specify what funds were used for if they
selected this option).
▪ Grantees could also select “I still have some SVOG funds to spend.”
Contract No. 73351022F0099 | 2M Research | 23
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
The most common expense categories were payroll costs (82 percent); other ordinary
and necessary business expenses (64 percent); payments to independent contractors
(57 percent); utility payments (57 percent); rent payments (48 percent); and
expenditures related to producing live performances/shows (48 percent). Almost all
grantees that responded to the survey spent all their funds; only one percent indicated
that they had leftover funds.
Grantees who responded to the survey also indicated the percentage of funds they
dedicated to each selected category below (as illustrated in Exhibit 15). On average,
grantees spent over half of their funds on payroll expenses; 17 percent on other
ordinary and necessary business expenses; 10 percent on payments to independent
contractors; 5 percent on utilities; 7 percent on expenditures related to producing a
live performance; and 5 percent on rent. Almost no funds were spent on worker
protection, scheduled debt payments, or scheduled mortgage payments, on average.
Exhibit 15. Grantees typically spent most of their SVOG funds on payroll.
Contract No. 73351022F0099 | 2M Research | 24
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
N = 1,829
Source: SVOG Grantee Survey, Item 2: How did you use the SVOG grant funds? (Check
all that apply) (chart on the left) and Item 3: Estimate, as best you can, the percentage
of total SVOG funds used on each of the categories below. Your entries will total at the
bottom. The total across all spent and unspent funds should add up to 100% (chart on
the right).
How did the actual use of funds compare to the approved allowable costs
following budget review(s)?
Key Insights
▪ The survey did not find any evidence that grantees used their funds for
unapproved costs.
When responding to the survey, about 12 percent of grantees selected the “Other”
category, which could potentially indicate unapproved uses of funds. Upon closer
examination, however, the descriptions matched the approved allowable costs. The
evaluation team recoded almost all responses to the approved costs (see Exhibit A.9
in the Technical Appendix for more detail). The survey did not find any evidence that
grantees used their funds for unapproved costs.
Domain 3: Impacts of SVOG
This section presents the impacts of SVOG on the number of employees, annual
revenue, and business survival by venue type, geography, industry, and underserved
business status. The data source for the findings in this section included SVOG
application data and Dun & Bradstreet data. The evaluation team used trends in the
number of employees as a proxy for jobs. Since virtually all businesses (both grantees
and non-recipients) remained in or resumed operations according to the SVOG
Grantee Survey data, the evaluation team used the Dun & Bradstreet viability score as
a proxy to examine differences in business survival. The evaluation team also used
propensity score matching (PSM) to match the grantees to non-recipients based on
their characteristics and evaluated the impact of SVOG using a difference-in-
differences (DID) analysis. More information on the data sources and analysis methods
is available in the Technical Appendix.
Contract No. 73351022F0099 | 2M Research | 25
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
What is the change in number of jobs for SVOG venues before and after the receipt
of grant funds? What is the change for venues that applied and failed to receive
SVOG funds during the same time frame?
Key Insights
▪ Employee count did not vary significantly for grantees and non-recipients after
the SVOG program was launched.
The evaluation team did not find any substantial differences between grantees and
non-recipients regarding employee counts. Furthermore, the evaluation did not find
any evidence that SVOG increased the number of employees and by extension the
number of jobs among grantees’ businesses. Since grantees stated they spent most of
their funds on payroll, it is likely that SVOG supported employee retention rather than
workforce growth.
Exhibit 16. SVOG grantees experienced an increase in average employee counts
compared to non-recipients after the disbursement of SVOG funds, but the
increase was not statistically significant and cannot conclusively be attributed to
SVOG funds.
Note: The difference in the employee count trends between grantees and non-
recipients was not statistically significant in the difference-in-differences regression
model.
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 26
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
How does the change in number of jobs vary by venue type, number of
employees, annual revenue, geography, industry, and socioeconomic status,
among other variables?
Key Insights
▪ There was no substantial variation in the employee count by venue type,
geography, industry, or underserved business status.
The DID model estimated that grantees experienced an average increase of 17 more
employees because of SVOG funds (though the finding was not statistically significant).
The average change in the number of employees varied by some characteristics of
venue type, region, industry, and underserved business status, but the differences
were minor and not statistically different from the overall average increase in the
number of employees.
Exhibit 17. The average increase in the number of employees for grantees
compared to non-recipients did not vary significantly by venue type.
Note: The dashed blue line shows grantees had, on average, 17 more employees than
non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles show
the increase in employees after disbursement of SVOG funds for grantees compared to
non-recipients of each venue type. Negative values indicate grantees experienced a
decrease in employees compared to non-recipients. For every venue type, the increase
in the number of employees for grantees compared to non-recipients was not
statistically different than the overall increase (the confidence interval line crosses the
shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 27
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 18. The average increase in the number of employees for grantees
compared to non-recipients did not vary substantially by region.
Note: The dashed blue line shows grantees had, on average, 17 more employees than
non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles show
the increase in employees after disbursement of SVOG funds for grantees compared to
non-recipients in each SBA region. Negative values indicate grantees experienced a
decrease in employees compared to non-recipients. For every SBA region, the increase
in the number of employees for grantees compared to non-recipients was not
statistically different than the overall increase (the confidence interval line crosses the
shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 28
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 19. The average increase in the number of employees for grantees compared to non-recipients varied
some by industry, but findings were not statistically different than the overall average increase.
Note: The dashed blue line shows grantees had, on average, 17 more employees than non-recipients after disbursement
of SVOG funds (i.e., in 2022). The red circles show the increase in employees after disbursement of SVOG funds for
grantees compared to non-recipients in each industry. Negative values indicate grantees experienced a decrease in
employees compared to non-recipients. For every industry, the increase in the number of employees for grantees
compared to non-recipients was not statistically different than the overall increase (the confidence interval line crosses
the shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 29
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 20. The average increase in the number of employees for grantees
compared to non-recipients did not vary substantially by underserved business
status.
Note: The dashed blue line shows grantees had, on average, 17 more employees than
non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles show
the increase in employees after disbursement of SVOG funds for grantees compared to
non-recipients by underserved business status. Underserved business are businesses
identified in Dun & Bradstreet data as any of the following: disabled-owned, veteran-
owned, women-owned, minority-owned, or small disadvantaged. For underserved
and not underserved businesses, the increase in the number of employees for
grantees compared to non-recipients was not statistically different than the overall
increase (the confidence interval line crosses the shaded confidence interval of the
dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 30
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
What is the average change in revenue earned by SVOG venues before and after
the receipt of grant funds? What is the change for venues that applied and failed
to receive SVOG funds during the same time frame?
Key Insights
▪ SVOG grantees experienced increased annual revenues compared to non-
recipients after the disbursement of SVOG funds.
▪ There is conclusive evidence, based on the DID regression results, that SVOG
increased the annual revenues of grantees more than what they would have
without the program.
Annual revenue for grantees was about the same as non-recipient revenue between
2018 and 2021, but grantee revenue increased significantly from 2021 to 2022. The DID
model estimated that grantees experienced a 28 percent greater increase in annual
revenue compared to non-recipients. Since the DID model matches non-recipients to
grantees based on key characteristics and identifies non-recipients as a comparison
group that represented what would have happened to grantees in the absence of
SVOG funds, there is conclusive evidence that the revenue increase for grantees from
2021 to 2022 would not have occurred in the absence of SVOG. The grantee survey
data shows these funds helped to offset further losses for grantees and kept them from
closing operations.
Exhibit 21. SVOG grantees experienced increased annual revenues compared to
non-recipients after the disbursement of SVOG funds.
Contract No. 73351022F0099 | 2M Research | 31
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Note: The difference in the revenue trends for grantees and non-recipients was
statistically significant (p <0.01) in the difference-in-differences regression model.
Source: Dun & Bradstreet data; SVOG Application data.
How does the average change in revenue vary by venue type, number of
employees, annual revenue, geography, industry, and socioeconomic status?
Key Insights
▪ There was no substantial variation in the revenue by venue type, geography,
industry, or underserved business status.
Exhibit 22. The average percent change in annual revenue did not vary
substantially by venue type.
Note: The dashed blue line shows grantees had, on average, 28% more annual revenue
than non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles
show the percent change in revenue after disbursement of SVOG funds for grantees
compared to non-recipients of each venue type. For every venue type, the percent
change in annual revenue for grantees compared to non-recipients was not
statistically different than the overall increase (the confidence interval line crosses the
shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 32
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 23. The average percent change in annual revenue did not vary
substantially by SBA region.
Note: The dashed blue line shows grantees had, on average, 28% more annual revenue
than non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles
show the percent change in revenue after disbursement of SVOG funds for grantees
compared to non-recipients of each SBA region. For every SBA region, the percent
change in annual revenue for grantees compared to non-recipients was not
statistically different than the overall increase (the confidence interval line crosses the
shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 33
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 24. The average percent change in annual revenue did not vary by industries.
Note: The dashed blue line shows grantees had, on average, 28% more annual revenue than non-recipients after
disbursement of SVOG funds (i.e., in 2022). The red circles show the percent change in revenue after disbursement of
SVOG funds for grantees compared to non-recipients in each industry. For every industry, the percent change in annual
revenue for grantees compared to non-recipients was not statistically different than the overall increase (the confidence
interval line crosses the shaded confidence interval of the dashed blue line).
Source: Dun & Bradstreet data; SVOG Application.
Contract No. 73351022F0099 | 2M Research | 34
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 25. The average percent change in annual revenue did not vary
substantially by underserved business status.
Note: The dashed blue line shows grantees had, on average, 28% more annual revenue
than non-recipients after disbursement of SVOG funds (i.e., in 2022). The red circles
show the percent change in revenue after disbursement of SVOG funds for grantees
compared to non-recipients by underserved business status. Underserved business
are businesses identified in Dun & Bradstreet data as any of the following: disabled-
owned, veteran-owned, women-owned, minority-owned, or small disadvantaged. For
underserved and not underserved businesses, the percent change in annual revenue
for grantees compared to non-recipients was not statistically different than the overall
increase (the confidence interval line crosses the shaded confidence interval of the
dashed blue line).
Source: Dun & Bradstreet data; SVOG Application data.
Contract No. 73351022F0099 | 2M Research | 35
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
How many SVOG grantees are still in operation three months, six months, and
twelve months after the final disbursement of all funds (including funds from
reconsiderations and supplementations)?
Key Insights
▪ Very few businesses (one percent of the grantee survey sample) closed
operations after the receipt of SVOG funds.
▪ Most businesses that closed stayed open at least one year after the receipt of
SVOG funds.
Only 22 grantees that responded to the grantee survey indicated they were no longer
in operation. In the Dun & Bradstreet data, there were also very few businesses that
had evidence of closure. Of the 22 grantees that indicated they closed operations,
most (64 percent) did not close until one year or more after the receipt of SVOG funds.
Exhibit 26. Of the grantees that closed operations, most stayed open a year or
longer after the receipt of SVOG funds.
Source: SVOG Grantee Survey, Item 1a: About how long after you received the last of
the SVOG funds did your business close?
What are the characteristics of SVOG grantees still in operation today and how do
they compare to applicants who failed to receive SVOG funds? How do they
compare to SVOG grantees that are no longer in operation?
Key Insights
▪ SVOG significantly improved the viability of grantee businesses.
Contract No. 73351022F0099 | 2M Research | 36
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
The administrative and collected data did not show a substantial number of business
closures, so statistical comparisons between grantees whose businesses remained
open and those that closed could not be made. The Dun & Bradstreet viability rating
provided a means to compare grantees with non-recipients. The rating or viability
score predicts the likelihood that a company would go out of business, become
inactive, or file for bankruptcy over the next 12 months. The viability score is a rating
from 1-9 where 1 is the lowest probability of going out of business or becoming
inactive over the next 12 months and 9 is the highest probability of going out of
business or becoming inactive. Reductions in the score indicate an improvement in
business viability.
After the SVOG program was launched, the viability score of grantees improved
significantly for grantees compared to non-recipients. Overall, the DID regression
predicted that grantees improved their viability by 0.3 points more than they would
have experienced without SVOG funds.
Exhibit 27. Grantees experienced a significant improvement in average business
viability, with lower scores indicating better viability, compared to non-recipients
after the disbursement of SVOG funds.
Note: The business viability score ranges from 1 to 9, with 9 representing businesses
most likely to cease operations and 1 representing businesses least likely to cease
operations. The difference in the viability trends between grantees and non-recipients
was statistically significant (p < 0.05) in the difference-in-differences regression model.
Source: Dun & Bradstreet data.
Contract No. 73351022F0099 | 2M Research | 37
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
How does operating status vary by venue type, number of employees, annual
revenue, geography, industry, and socioeconomic status?
Key Insights
▪ Average improvement in viability score did not vary significantly by venue type,
SBA region, industry, or underserved business status.
Exhibit 28. The average improvement in the viability score for SVOG grantees
compared to non-recipients did not vary by venue type.
Note: The dashed blue line shows grantees had, on average, a decrease of 0.3 points
more in the viability score than non-recipients after disbursement of SVOG funds (i.e.,
in 2022). Decreases in the score indicate better business viability. The red circles show
the change in the score after disbursement of SVOG funds for grantees compared to
non-recipients of each venue type. For every venue type, the change in the viability
score for grantees compared to non-recipients was not statistically different than the
overall change (the confidence interval line crosses the shaded confidence interval of
the dashed blue line).
Source: Dun & Bradstreet data.
Contract No. 73351022F0099 | 2M Research | 38
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 29. The average improvement in the viability score for SVOG grantees
compared to non-recipients did not vary substantially by SBA region.
Note: The dashed blue line shows grantees had, on average, a decrease of 0.3 points
more in the viability score than non-recipients after disbursement of SVOG funds (i.e.,
in 2022). Decreases in the score indicate better business viability. The red circles show
the change in the score after disbursement of SVOG funds for grantees compared to
non-recipients in each SBA region. For every region, the change in the viability score
for grantees compared to non-recipients was not statistically different than the overall
change (the confidence interval line crosses the shaded confidence interval of the
dashed blue line).
Source: Dun & Bradstreet data.
Contract No. 73351022F0099 | 2M Research | 39
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 30. The average improvement in the viability score of grantees compared to non-recipients did not vary
substantially by industry.
Note: The dashed blue line shows grantees had, on average, a decrease of 0.3 points more in the viability score than
non-recipients after disbursement of SVOG funds (i.e., in 2022). Decreases in the score indicate better business viability.
The red circles show the change in the score after disbursement of SVOG funds for grantees compared to non-recipients
in each industry. For every industry, the change in the viability score for grantees compared to non-recipients was not
statistically different than the overall change (the confidence interval line crosses the shaded confidence interval of the
dashed blue line).
Source: Dun & Bradstreet data.
Contract No. 73351022F0099 | 2M Research | 40
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 31. The average improvement in the viability score of grantees compared
to non-recipients did not vary by underserved business status.
Notes The dashed blue line shows grantees had, on average, a decrease of 0.3 points
more in the viability score than non-recipients after disbursement of SVOG funds (i.e.,
in 2022). Decreases in the score indicate better business viability. The red circles show
the change in the score after disbursement of SVOG funds for grantees compared to
non-recipients by underserved business status. Underserved business are businesses
identified in Dun & Bradstreet data as any of the following: disabled-owned, veteran-
owned, women-owned, minority-owned, or small disadvantaged. For underserved
and not underserved businesses, the change in the viability score for grantees
compared to non-recipients was not statistically different than the overall change (the
confidence interval line crosses the shaded confidence interval of the dashed blue
line).
Source: Dun & Bradstreet data.
Contract No. 73351022F0099 | 2M Research | 41
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Domain 4: Perceptions of SVOG and Remaining
Pandemic Recovery Needs
This section presents findings on perceptions of SVOG and remaining pandemic
recovery needs. The source for these findings comes primarily from the grantee
closeout survey, grantee web survey, and follow-on interviews. To develop these
findings, the evaluation team applied topic modeling and thematic analysis of text-
based data (from the open-ended survey item and interview transcripts) and
descriptive statistics of closeout survey data. More information on the data sources
and analysis methods is available in the Technical Appendix.
What are grantees’ perceptions of the SVOG grant’s relationship to business
survival? Do SVOG grantees associate the grant to business outcomes?
Key Insights
▪ Almost all (93 percent) of the closeout survey respondents, a third (33 percent)
of the grantee survey respondents, and virtually all interview respondents
reported that SVOG helped their business survive during the pandemic.
▪ A small number (1 percent or less) of grantee survey respondents indicated a
negative experience with the SVOG application process.
Closeout Survey Responses
SVOG grantees responding to the closeout survey agreed that the funds were very
important in keeping their business open or reopening their business.
Contract No. 73351022F0099 | 2M Research | 42
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 32. Almost all closeout survey respondents said SVOG played a very
important role in keeping their business open or reopening their business.
N = 82
Source: SVOG Closeout Survey, Item 1: In your opinion, how important was the SVOG
grant to keeping your business open or reopening your business?
Closeout survey respondents (33 percent) also reported that SVOG helped them to
reopen their business sooner.
Contract No. 73351022F0099 | 2M Research | 43
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 33. Of the closeout survey respondents that said SVOG helped them to
reopen sooner, more than half said the grant helped them to reopen between 1
and 4 months sooner.
N = 27
Source: SVOG Closeout Survey, Follow-Up to Item 3: If yes, how much sooner did your
business reopen?
Finally, 98 percent of the closeout survey respondents agree, some strongly, that SVOG
addressed their needs.
Contract No. 73351022F0099 | 2M Research | 44
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 34. Most closeout survey respondents agreed that SVOG addressed their
needs.
N = 82
Source: SVOG Closeout Survey, Item 5: My need was addressed by SVOG.
Grantee Survey and Interview Discussions
In the web survey and follow-on interviews, grantees noted the importance of
supporting art in the community and expressed gratitude that the SBA steered funding
to the arts industry. Some interviewees noted frustrations with the close-out and
auditing processes, describing them as unclear and time-consuming. Others preferred
a longer period during which they could have spent their funds. Nevertheless,
respondents expressed gratitude for the funds and the speed of the program’s launch.
“This grant specifically allowed us to stay open. We are big fans of it. It
was a very complex application. There was a lot of support on the
website, and all of the webinars and the YouTube videos were so helpful
that SBA put out… yes, very, very grateful.” - Interview Respondent
Contract No. 73351022F0099 | 2M Research | 45
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Virtually all interview respondents believed that SVOG was instrumental to their
businesses’ survival. Many respondents expressed their gratitude and described how
SVOG funds kept their businesses operating during the pandemic and shutdown in a
challenging economic climate of lower revenues, decreasing donations, and increasing
costs. These funds helped to compensate for lost revenue and fewer donations,
allowing businesses to continue to pay their bills during the economic shutdown.
Respondents noted that they likely would have sunk into greater debt or closed their
businesses altogether without SVOG funds. Most respondents stated that SVOG funds
were generally sufficient to meet their needs and that their businesses’ remaining
recovery needs stemmed from the prolonged effects of the pandemic on the arts
industry, including a need for ongoing financial support over time.
“This kind of support was absolutely amazing and made it able for us to
survive and to come back and begin getting back to normal business
paying artists and so on. If only some form of this support [was] always
in place. Everybody’s lives would be better!”- Grantee Survey
Respondent
“We would like to see the federal government elevate the arts and
culture sector. This is not a bail out… it would both stimulate the
economy… and reduce acts of crime. We are extremely thankful for this
program and are impressed with the platform that the SBA put together
in such a short period of time. - Grantee Survey Respondent
About 26 percent5 of survey respondents indicated no or few remaining pandemic
recovery needs after receipt of SVOG funds. Several respondents used the open-ended
survey item as an opportunity to thank the SBA and emphasized its importance for
their businesses’ survival. Many respondents also reflected on SVOG’s effectiveness
and expressed hope that future government grants (federal, state, or local) would
support their venue should the need ever arise again. Overall, these respondents
associated the role, influence, and impact of government programs providing funding
and supports with the vibrant arts scene and thriving cultural life in American society.
5
The percentage reported is an estimate based on results from topic modeling of the survey data. For more information on
the topic modeling, please see the Technical Appendix.
Contract No. 73351022F0099 | 2M Research | 46
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Despite the praise, some grantee survey respondents shared their frustration with
SVOG and the process of applying for and receiving grant funds. Fewer than one
percent of survey respondents indicated a negative experience with the SVOG process.
“I had to appeal multiple times ... It was a tedious, horrifying process,
and I think the most egregious thing is I wasn’t able to actually talk to
the person working on the account.”- Grantee Survey Respondent
“We asked a very specific question that wasn’t answered in the FAQs and
the person pointed us to the FAQs. I missed a deadline, and the person
keeps reaching out to me to go back and do it, but the system won’t let
me.” - Grantee Survey Respondent
What are the remaining pandemic recovery needs among SVOG grantees?
Key Insights
▪ SVOG grantees requested additional funding to cover decreased revenues;
debts incurred during the pandemic; general operating expenses, payroll, and
other costs; and worker protections.
▪ SVOG grantees also noted key issues and changing behaviors impacting their
recovery, including reduced attendance of in-person events and fewer theatrical
film releases.
Topic modeling of the grantee survey data identified six different categories of
remaining pandemic recovery needs. The first four categories relate to needs which
grantees felt additional funding could address. These include general operating
expenses, payroll, and other costs; debts incurred during the pandemic; worker
protection needs; and decreased revenues due to fewer ticket sales and reduction in
earned income. The last two are larger issues that continue to impact grantee
industries. These include lingering reluctance to attend in-person performances and
events and, specific to the film industry, fewer theatrical releases.
Contract No. 73351022F0099 | 2M Research | 47
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit 35. Grantee survey respondents requested additional funding to address
pandemic recovery needs and described industry-wide issues that still impact
their businesses.
N = 1,518
Notes: The percentages reported in Exhibit 35 above are estimates based on topic
modeling results of the survey data. For more information on the topic modeling,
please see the Technical Appendix. The evaluation team removed grantees that did not
respond to the open-ended item prior to conducting the topic modeling analysis.
Source: Grantee survey, Item 4: What are the needs, related to recovery from the
COVID-19 pandemic, your business is currently experiencing?
Funding Needs
Many survey and interview respondents discussed challenges they continue to face
given decreased revenues from fewer ticket sales, fewer donations and sponsors, and
drops in other sources of income. Respondents stated that without additional support
from programs like SVOG, they will continue to struggle to cover general operating
expenses, payroll costs, and rent payments, among other costs. Several survey
respondents also expressed their need for maintenance and capital improvements like
repairs to HVAC systems, carpet replacements, and equipment upgrades, particularly
for venues that were shuttered for a long period during the pandemic. Finally, many
respondents highlighted the need for greater worker protections, such as better
ventilation systems, masks, and other personal protective equipment.
Contract No. 73351022F0099 | 2M Research | 48
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
In follow-on interviews, respondents said they have seen increases in staff turnover
and payroll costs since the pandemic. While there exists a clear need for businesses to
hire qualified and reliable staff, current funding levels are insufficient to cover the cost
of these human resources with businesses are still recovering financially. In the
interviews, respondents also mentioned that some expenses have increased in price
due to inflation associated with the pandemic. They saw a substantive increase in the
price of goods and materials, rent, utilities, insurance costs, and rates for vendors and
performers. Taken together, these costs substantiate significant remaining needs.
“We just had to transfer money from our cash reserve fund or very
carefully hoarded cash reserve fund for the first time. For us … even
when we sell out with the rising cost of doing business, our profit margin
or profit, our net is much smaller, and that’s the net that we take to pay
everyone else ... It’s really scary. I’m not sure what we’re going to do and
it’s really difficult right now, to be honest with you.” - Interview
Respondent
Survey and interview respondents also expressed a need for relief from paying off
emergency loans that were taken out during the pandemic such as the Economic
Injury Disaster Loan (EIDL). Some felt unsure about how they would meet needs
without federal support. SVOG funds were critical in ensuring their business survival
during the pandemic shutdown, but without ongoing support, they would likely need
to take on more debt, access their personal savings to cover costs, or ultimately close
their business.
“Prior to receiving funds from SVOG, I like so many hundreds or
thousands of my colleagues in the entertainment industry took out SBA
EIDL loans. On top of our performance income not yet returning to
normal, our SVOG funds are now low or gone and now the payments
have come due on our EIDL loans. I am sure some will struggle, even
with the best of intentions, to make these payments with their continued
reduced income. It would be helpful to see a loan forgiveness program
introduced to congress for consideration or at the very least a payment
moratorium on EIDL loans.” - Grantee Survey Respondent
Contract No. 73351022F0099 | 2M Research | 49
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Other respondents leaned into their creativity to manage their remaining pandemic
recovery needs without federal support. Their strategies for securing financial support
included seeking out local and state grants and increasing fundraising efforts to find
new partners and/or donors. To stay afloat against the rising costs of running their
businesses, respondents often had to cut costs through measures like reducing
employee hours, reducing artist and performer pay, and scaling back programming by
hosting fewer shows, canceling events, or postponing programs.
Larger Issues Impacting Grantees’ Industries
Both survey and interview respondents frequently mentioned that their audience at
in-person events and performances reduced significantly since the pandemic. They
observed that their customers’ entertainment habits were changing with many now
preferring to stream movies and performances from the comfort of their own home.
The respondents also recognized a growing reluctance among their customers to
attend events and performances in person due to health concerns.
“I know streaming has definitely affected my business in a hard way…
It’s costing me a lot more money to rent the movie, but it’s also
streaming for a one pay price… Between the inability to get to my
market to advertise… and also having to deal with the streaming, it’s
just making it really difficult to survive.” - Interview Respondent
Along with changing preferences, many respondents noted that movie studios are
releasing films at a slower rate compared to pre-pandemic levels. Many movie theaters
are therefore struggling to remain open and often lack a sufficient pipeline of films to
generate enough revenue that sustains their business.
In response to these challenges, interview respondents expressed the importance of
identifying successful marketing strategies to increase audience levels. Despite this
recognition, they struggle to adapt as technology and advertising have changed over
time and shifted from traditional media like TV, radio, or newspaper to social media.
As a result, their usual means of advertising through traditional media are no longer
effective and they are struggling to navigate the nuances of social media marketing.
Contract No. 73351022F0099 | 2M Research | 50
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Some interview respondents described how they were now trying to strategically plan
shows and programming to increase audience turnout. For example, several theatres
have shifted to focus only on popular, well-known titles to entice new audiences.
Additionally, some respondents were able to ramp up their marketing and advertising
efforts by hosting events, bolstering their community involvement, and increasing
their spending on marketing and advertising.
Conclusion
Summary
Overall, the SVOG evaluation found that grantees felt the funds were instrumental to
the survival of their business. The evaluation team found conclusive evidence that the
annual revenue and viability of grantees’ businesses improved due to the receipt of
SVOG funds. SVOG funds went to a diverse range of businesses, both large and small,
with different key characteristics across various locations in the United States. Very
small businesses experienced more difficulty securing awards, although the grant
awarded funds to many businesses with fewer than five employees. Certain venue
types and industries (for example, motion picture theaters and arts, entertainment,
and recreation businesses) had different likelihoods of securing an award.
Finally, the grantee survey showed that most grantees spent their funds on payroll and
general operating expenses. Only one percent had leftover funds at the time of the
survey. Most grantees also indicated the presence of remaining pandemic recovery
needs resulting from decreased revenues, increased operating costs, worker
protections, and debts incurred during the pandemic. They requested additional funds
from the government to meet these financial needs. Grantees also observed key
trends that are negatively impacting the arts and entertainment industry, specifically
less in-person attendance at events and fewer theatrical film productions.
Contract No. 73351022F0099 | 2M Research | 51
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Recommendations
The report findings suggest several recommendations to consider when implementing
similar emergency response and/or industry-specific grant programs in the future.
Recommendation 1: Regularly conduct analysis of client characteristics,
comparing to those of applicants that were not admitted to the program.
▪ Evidence. The evaluation found smaller businesses, underserved businesses,
businesses in some regions of the U.S., and certain venue types were less likely
to receive an SVOG award.
▪ Implementation. To better understand differences or potential bias in award
and lending patterns, SBA programs can implement real-time, interactive
dashboards that display topline measures that compare clients to all
applicants. If there are clear differences, programs can consider implementing
outreach efforts to target groups that are receiving fewer awards, loans,
certifications, and more. See Recommendation 3 for more evidence supporting
outreach. A dashboard may also support track measures of impact on
surrounding industries and economic impacts, to better facilitate evidence-
building activities.
Recommendation 2: Support outreach that can identify needs and challenges
from the perspective of the industry or area targeted by the grant program.
▪ Evidence. SVOG targeted the performing arts industry. The grantee survey
responses and interviews found many grantees face larger industry-wide issues
that the SVOG funds could not address on their own. These issues included
decreases in attendance at in-person events and fewer theatrical releases in the
era of streaming (an issue specific to motion picture theater operators).
▪ Implementation. Outreach efforts can be restricted by grant management
practices which can be outside of a program’s control, as was the case for SVOG.
It is possible to design programs with integrated outreach and service resources
to inform continuous quality improvement, however. SBA programs can further
opportunities for client support and improved client experiences with built-in
outreach mechanisms. Using support surveys, listening sessions, and other
outreach, SBA programs can proactively identify issues and solutions to inform
program implementation.
Contract No. 73351022F0099 | 2M Research | 52
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Recommendation 3: Support grantee connections to other sustainable sources of
funding.
▪ Evidence. The evaluation found almost all grantees spent all their SVOG funds,
and, in the interviewees, several grantees mentioned needs for more funding
and their current challenges identifying sustainable sources of funding for their
businesses.
▪ Implementation. Using improved mechanisms for outreach and understanding
client characteristics, SBA programs can organize convenings that bring
together customers based on eligibility for additional funding sources. At the
convenings, SBA programs can make connections to other SBA resources, and
facilitate discussions to help customers create networks from which they may
learn about best practices to access available, sustainable funding.
Recommendation 4: Consider ways to streamline and ease the burden of the
application process, improving customer experience.
▪ Evidence. In the web survey, a small number of grantees indicated a negative
experience with the SVOG application process.
▪ Implementation. Often grant program staff face constraints in the degree to
which they can interact with applicants and influence application requirements,
especially in the case of grant programs established through legislation, like
SVOG. That said, when flexibilities exist, SBA programs can dedicate staff to
support questions or technical issues that arise during the application process;
a critical component to providing meaningful support is to ensure collaboration
between policymakers and communications personnel to develop guidance in
plain language. In addition, SBA should look for ways to develop a centralized
application that can determine eligibility for more than one program, so that
applicants are not required to submit multiple applications.
Limitations
It is possible that the grantees that responded to both the web survey and follow-on
interviews had more positive experiences with SVOG than those that declined to
respond. Respondents may also have felt the need to agree with statements about the
positive impacts of SVOG regardless of their personal options (i.e., acquiescence bias).
This bias could cause findings to be more positive than the reality. The evaluation
team did attempt to mitigate this bias by encouraging frank answers, ensuring the
anonymity of responses, and selecting a large survey sample (4,000 grantees).
Contract No. 73351022F0099 | 2M Research | 53
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
A second limitation is that the evaluation lacked data on an extended period of time
after the SVOG program. It is possible that some program effects may continue to
materialize into 2023 and beyond. In this case, the evaluation findings may potentially
underreport the full long-range impact of the SVOG program.
Thirdly, the SVOG program gave an award preference to businesses with significant
declines in revenue. Although the evaluation team attempted to correct for differences
between grantees and non-recipients using PSM, it is possible that not all variables
correlated with revenue decline were included in the matching process. In this case, it
is possible that the evaluation underreported the program’s effects as grantees should
have been worse off than the non-recipients. While less likely, it is also possible that
bias in the matching process caused the evaluation team to miss unanticipated
negative effects of SVOG, though none of the evaluation team’s data supports this
conclusion.
Despite these limitations, the SVOG Program Evaluation found that the program
generally helped businesses during the pandemic and played a key role in keeping
businesses from closure. Future research might consider examining whether the SVOG
impacts were better realized in 2023 or later over time. Future research might also seek
to better understand why some businesses failed to receive funds. Understanding
whether these businesses were legitimately ineligible and/or better off than their
grantee counterparts or not may inform lessons learned for future industry-specific,
disaster recovery programs that the SBA or other federal/state agencies implement.
Contract No. 73351022F0099 | 2M Research | 54
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Technical Appendix
Data Sources
SVOG Application Data
The SVOG application data included a list of DUNS numbers that served as unique
identifiers of businesses that applied for funds from the SVOG. The data also identified
whether each business ultimately received an award (grantee) or not (non-recipient).
All information the evaluation team used from this data source is presented in the
table below.
Exhibit A.1 SVOG application data variables used by the evaluation team.
Variables Description/Notes
DUNS The unique identifier of all businesses in the data.
Applicant Name,
Identifying information needed for the Grantee Survey administration.
Email, Phone
The venue type of each business, which included live performing arts organization
Venue operators, live venue operators or promoters, motion picture theater operators, museum
operators, talent representatives, and theatrical producers.
Street 1, Street 2, The evaluation team used this information to determine the geography of the business,
City, State, Zip specifically the SBA region, and whether the business was in a metropolitan area or not.
The evaluation team used the number of employees reported at the time of application to
Number Of
develop business size categories including less than 5 employees, 5 to 9 employees, 10 to
Employees
49 employees, 50 to 249 employees, and 250 or more employees.
The evaluation team used the annual earned revenue in 2020 to create categories
including less than $100,000; $100,000 to 999,999; $1,000,000 to $9,999,999; and
Annual Revenue
$10,000,000 or more. The application data also included annual revenue for 2019 and
partial revenue data for 2021 (through the first quarter).
The evaluation team used the approved award amount to create categories including less
Award Amount than $100,000; $100,000 to 999,999; $1,000,000 to $9,999,999; and $10,000,000 (awards
were capped at $10 million).
The evaluation team used the award date to determine when awards were given to the
Award Date grantees. This variable helped to identify 2022 as the period “after” the SVOG in the DID
regression models, as most awards were given from May to December of 2021.
SVOG Closeout Survey
At the conclusion of their grant, the SVOG grantees were asked to complete a closeout
survey that included questions about their perception of the program and its
relationship to business survival. The table below describes the items the evaluation
team used from the closeout survey.
Contract No. 73351022F0099 | 2M Research | A-1
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.2 SVOG closeout survey items used by the evaluation team.
SVOG Closeout Survey Items Response Options
1. In your opinion, how important was the SVOG Very important; Somewhat important; Neither
grant to keeping your business open or reopening important nor unimportant; Somewhat unimportant;
your business? Not at all important (and an “Explain” text box)
2. Before you received the grant, did you have a Yes, but we had not yet reopened; Not applicable, we
timeline for reopening? were already open; No
3. Did the SVOG grant help reopen your business
Yes; No
sooner than you would have otherwise?
3. If yes, how much sooner did your business reopen? 1-2 months; 3-4 months; 5-6 months; 6+ months
4. Was the amount of the SVOG grant sufficient to
Yes; No
help meet your obligations?
Strongly Agree; Agree; Neutral; Disagree; Strongly
5. My need was addressed by the SVOG
Disagree
Only 82 grantees responded to the closeout survey at the time the evaluation team
received the data. The closeout survey sample thus differed some from the overall
sample of SVOG grantees; however, the differences were not extreme.
Contract No. 73351022F0099 | 2M Research | A-2
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.3. Comparison of all SVOG grantees to the grantees that responded to
the closeout survey.
Percent of all Percent of closeout
Category Subcategory
grantees survey respondents
Live performing arts organization
26% 21%
operators
Live venue operators or promoters 37% 23%
Venue type Motion picture theater operators 12% 27%
Museum operators 6% 17%
Talent representatives 11% 6%
Theatrical producers 7% 6%
Less than 5 43% 37%
5 to 9 19% 22%
Number of
10 to 49 29% 30%
employees
50 to 249 8% 5%
250 or more 2% 6%
Less than $100,000 41% 37%
Annual revenue $100,000 to $999,999 45% 51%
in 2020 $1,000,000 to $9,999,999 13% 10%
$10,000,000 or more 1% 2%
Atlantic 14% 11%
Great Lakes 14% 15%
Great Plains 4% 11%
Mid-Atlantic 8% 5%
New England 6% 4%
SBA Region
Pacific 19% 11%
Pacific Northwest 5% 4%
Rocky Mountains 4% 7%
South Central 9% 16%
Southeast 16% 17%
Metropolitan 89% 79%
Urbanicity Micropolitan 7% 12%
Rural 4% 9%
Accommodation and Food Services 6% 9%
Administrative and Support and Waste
7% 4%
Management and Remediation Services
Arts, Entertainment, and Recreation 44% 46%
Educational Services 4% 2%
Information 13% 16%
Other Services 6% 7%
Industry
Professional, Scientific, and Technical
3% 1%
Services
Public Administration 2% 0%
Real Estate Rental and Leasing 2% 2%
Retail Trade 2% 1%
Other Industries 3% 4%
Unclassified 9% 7%
Underserved Not a underserved business 84% 84%
business status Underserved business 16% 16%
Contract No. 73351022F0099 | 2M Research | A-3
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
SVOG Grantee Survey
The evaluation team administered a web survey to a random sample of 4,000 SVOG
grantees to collect information on how grantees spent funds they received. The survey
also included an open-ended question about the remaining pandemic recovery needs
grantees face. The evaluation team designed the survey to take about 10 minutes to
complete. The table below shows the items included in the survey.
Exhibit A.4. SVOG grantee survey items.
SVOG Grantee Survey Item Response Options
1. First, is the business that was supported by
Yes; No
the SVOG funds still in operation?
1a. [if 1 was “no”] About how long after you
Less than 3 months; Between 3 and 6 months; Between 6
received the last of the SVOG funds did your
months and 1 year; 1 year or more
business close?
Payroll costs; Rent payments; Utility payments; Scheduled
mortgage payments; Scheduled debt payments on any
indebtedness incurred in the ordinary course of business
prior to February 15, 2020; Worker protection expenditures;
Payments to independent contractors; Other ordinary and
necessary business expenses, including maintenance costs,
2. For which of the following expense
administrative costs (including fees and licensing costs);
categories did you use SVOG grant funds?
state and local taxes and licensing fees; operating leases in
(Check all that apply)
effect as of February 15, 2020, and insurance payments;
Advertising, production transportation, and capital
expenditures related to producing a theatrical or live
performing arts production, concert, exhibition, or comedy
show; Other 1; Other 2; Some portion of unspent funds
remain
3. Estimate, as best you can, the dollar
distribution of funding to the uses listed in the
table below. Your entries will total at the
bottom. Your objective in answering the Categories that were selected for question 3
question is to include the percentage of all
spent and unspent funds so that the total at
the bottom is 100 percent.
4. What remaining pandemic recovery needs
Open text
do you have for your business?
2M received 1,829 responses to the survey, a response rate of 46 percent. The survey
sample had similar characteristics to grantees overall.
Contract No. 73351022F0099 | 2M Research | A-4
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.5 SVOG grantee survey sample characteristics.
Percent of all Percent of grantee
Category Subcategory
grantees survey respondents
Venue type Live performing arts organization
26% 32%
operators
Live venue operators or promoters 37% 32%
Motion picture theater operators 12% 11%
Museum operators 6% 8%
Talent representatives 11% 9%
Theatrical producers 7% 7%
Number of Less than 5 43% 38%
employees 5 to 9 19% 21%
10 to 49 29% 30%
50 to 249 8% 9%
250 or more 2% 2%
Annual revenue in Less than $100,000 41% 34%
2020 $100,000 to $999,999 45% 49%
$1,000,000 to $9,999,999 13% 16%
$10,000,000 or more 1% 1%
SBA Region Atlantic 14% 13%
Great Lakes 14% 15%
Great Plains 4% 5%
Mid-Atlantic 8% 7%
New England 6% 6%
Pacific 19% 19%
Pacific Northwest 5% 7%
Rocky Mountains 4% 5%
South Central 9% 9%
Southeast 16% 14%
Urbanicity Metropolitan 89% 87%
Micropolitan 7% 8%
Rural 4% 6%
Industry Accommodation and Food Services 6% 5%
Administrative and Support and Waste
7% 6%
Management and Remediation Services
Arts, Entertainment, and Recreation 44% 49%
Educational Services 4% 7%
Information 13% 12%
Other Services 6% 6%
Professional, Scientific, and Technical
3% 2%
Services
Public Administration 2% 2%
Real Estate Rental and Leasing 2% 2%
Retail Trade 2% 1%
Other Industries 3% 2%
Unclassified 9% 6%
Underserved Not an underserved business 84% 81%
business status Underserved business 16% 19%
Contract No. 73351022F0099 | 2M Research | A-5
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
SVOG Grantee Interviews
The evaluation team conducted follow-on semi-structured interviews with 36 grantees
who responded to the grantee survey to collect more in-depth information on the
remaining pandemic needs of the grantees. The interviews lasted between 15 and 30
minutes. During the interview, grantees were asked to expand on the answer they
provided to the web survey’s open-ended question and discuss whether the felt they
could meet their remaining needs without additional Federal support. They were also
asked what needs they felt the SVOG addressed well.
The interview respondents reflected diversity in terms of venue type, business size,
and underserved business status.
Exhibit A.6. Grantee interview sample characteristics.
Category Subcategory Percent of interviewees
Live performing arts organization operators 36%
Live venue operators or promoters 39%
Motion picture theater operators 11%
Venue Type
Museum operators 0%
Talent representatives 6%
Theatrical producers 8%
Less than 5 39%
5 to 9 0%
Number of employees 10 to 49 25%
50 to 249 11%
250 or more 3%
Not a underserved business 69%
Underserved business status
Underserved business 31%
Dun & Bradstreet Data
The evaluation team received annual data from 2018 to 2022 on total sales, employee
count, socioeconomic status and viability score from Dun & Bradstreet for the 17,042
businesses that applied for SVOG. Dun & Bradstreet produces a multi-dimensional
rating called the Viability Rating that includes two predictive components and two
descriptive components. The predictive components include a viability score and a
portfolio comparison, while the descriptive components include a data-depth
indicator and a company profile indicator. The viability score is a predictive rating of 1-
9 where 1 is the lowest probability of going out of business or becoming inactive over
the next 12 months compared to approximately 30 million U.S. businesses in the Dun &
Bradstreet Data Cloud and 9 is highest probability of going out of business or
becoming inactive. The portfolio comparison is a predictive rating of 1-9 where 1 is the
Contract No. 73351022F0099 | 2M Research | A-6
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
lowest probability of going out of business or becoming inactive over the next 12
months compared to other businesses within the same model segment and 9 is the
highest probability of going out of business or becoming inactive.
Exhibit A.7. D&B Business Characteristics.
Average
Variable Description
2018 2019 2020 2021 2022
The details of the total
Annual income attributable to
$6,111,312 $7,069,670 $6,392,859 $4,110,340 $3,775,261
Sales product and services sold by
an organization annually.
The number of individuals
engaged by a business to
perform its various purposes,
Employee
e.g., business operations, 68 74 68 52 46
Count
and can include directors,
principals and full or part
time employees.
A score ranging from 1-9 that
predicts the likelihood that a
company will go out of
business, become inactive, or
file for bankruptcy over the
Viability
next 12 months. A score of 1 2 2 3 4 4
Score
suggests a lower probability
of a company going out of
business, and 9 indicates the
highest probability of going
out of business.
Contract No. 73351022F0099 | 2M Research | A-7
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
SVOG Grantee Survey Sampling
The evaluation team developed a simple random sample of 4,000 grantees from a
contact list with grantee phone numbers and emails provided by SBA. After drawing
the initial sample, the evaluation team had to draw replacements for some records.
Specifically, the evaluation team drew replacements for the following:
1. 590 records that demonstrated duplicate contact data.
a. In the initial survey random sample, 590 of the records had duplicates on
phone number, email address, or business names. For these duplicated
records, the evaluation team randomly selected one to represent that
contact. Since some contacts had more than one duplicate, this resulted
in keeping 235 of the 590 records, leaving 355 records to replace.
2. 131 records that did not have a match the application dataset.
3. 13 records that had duplicate DUNS in the applicant data (see more on the
process to handle duplicates in the applicant data in the Data Cleaning and
Process section below).
The evaluation team randomly selected 499 (355+131+13) replacement records from
the valid record sampling frame (after removing the 4,000 drawn in the initial sample
and any duplicated records based on contact information). The final survey sample
included 4,000 records, 3,501 from the initial random sample and 499 replacement
records.
Data Cleaning and Processing
SVOG Application Data
SBA provided the SVOG application data to the evaluation team in four separate
datasets:
1. Business characteristics and award amounts of grantees from the SVOG portal.
2. Business characteristics of non-recipients from the SVOG portal.
3. Earned annual revenue information on grantees provided by SVOG program
staff.
4. Earned annual revenue information on non-recipients provided by SVOG
program staff.
Contract No. 73351022F0099 | 2M Research | A-8
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Each of these data sources included a DUNS, which uniquely identifies a business;
however, each source also had duplicated DUNS. To avoid issues merging the
application data with the Dun & Bradstreet data and to ensure a consistent sample
across all analyses of SVOG grantees and non-recipients, records with duplicate DUNS
were removed from the data prior to analysis. In addition, a small number of DUNS in
the SVOG portal data and the revenue data (for both grantees and non-recipients) did
not match. These records were also removed prior to analysis. Exhibit A.8 shows the
number of cases dropped at each stage of the data processing of the application data.
The final dataset included 17,042 records (12,944 grantees and 4,098 non-recipients).
Contract No. 73351022F0099 | 2M Research | A-9
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.8. Process to remove duplicates from the SVOG application data.
Contract No. 73351022F0099 | 2M Research | A-10
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Finally, in the case of the revenue data, most grantees had more than one revenue
value for each year, as SBA asked them to update their revenue when applying for
reconsideration and/or supplemental awards. Following guidance from SVOG program
staff, the evaluation team used the reconsideration 2.0 revenue when available as that
was the most recent estimated provided by businesses. If that was not available, the
team used the updated earned revenue provided at the time grantees applied for a
supplemental award. If that was not available, the third option the team used was the
earned revenue from the first round of reconsideration. When all three of those
options were not available for a specific business, the team used the earned revenue
provided at the time of the original application.
SVOG Closeout Survey
SBA provided the evaluation team with a cleaned dataset with information from the
closeout survey. The only operation required was to reshape the data from long to
wide format (in long format, each survey item represents a row in the data for each
grantee; in wide format, there is only one row per grantee and each column represents
a different survey item response).
SVOG Grantee Survey
The evaluation team closed the grantee survey on May 19, 2023. At the time of closing,
the team had a total of 1,745 completed responses and another 505 responses that
began the survey but did not finish it. Of the 505 in progress responses, the evaluation
team removed 374 responses that did not answer the first three questions in the
survey. The evaluation team considered the remaining 131 responses as complete
because the only question missing was the open-ended item in the survey.
Contract No. 73351022F0099 | 2M Research | A-11
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
The SVOG grantee web survey included an item asking grantees select expense
categories that they used to spend the SVOG funds. The survey item included
categories of allowable costs described by the SVOG FAQ.6 The item also included two
“other” options that grantees could write in expenses that did not fit into the
categories provided in the survey item. The purpose of the “other” options was to
identify additional expenses that grantees spent funds on. After examining the
responses to the “other” option, the evaluation team recoded most of them to
allowable costs. Exhibit A.9 below provides the responses to the “other” category and
how they were recoded.
Exhibit A.9. Recoding of the “Other” funding expenses.
“Other” item responses (provided exactly as they were inputted into the
Recoded category
survey)
Payroll costs Payroll Fees, Bank Fees, Dividend*
Utility payments Utilities
Worker protection workers protection
expenditures
Payments to independent Contract Workers
contractors
Other ordinary and ▪ 8f Other Business Expenses - depreciation
necessary business ▪ acquisition of furniture and equipment
expenses ▪ Admin Costs, Property Taxes, Credit Card Fees, etc
▪ Administrative
▪ Administrative Costs
▪ Administrative costs & insurance
▪ Administrative Costs & Insurance
▪ Administrative Costs-Professional Fees, Taxes and Licenses for business
▪ Administrative, taxes, fees, etc..
▪ Administrative/Taxes
▪ All other production costs (costumes, set, light & sound rental)
▪ Amortization
▪ Benefits
▪ Building Improvements
▪ Business Travel
▪ Charter Party Agreements With the Venues (Cruise Ship)
▪ Computer equipment
▪ Concessions Supplies
▪ Construction Costs
▪ construction supplies
▪ Construction to get space usable
▪ Contracted Artistic Services and Supplies
6
See page 9 of https://www.sba.gov/sites/default/files/2022-02/2022-2-
16%20SBA%20SVOG%20Post%20Award%20Frequently%20Asked%20Questions-508.pdf.
Contract No. 73351022F0099 | 2M Research | A-12
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
“Other” item responses (provided exactly as they were inputted into the
Recoded category
survey)
▪ contractual
▪ Critical Equipment Costs
▪ Depreciation
▪ Depreciation
▪ Depreciation Expense
▪ Depreciation Expense
▪ Employee benefit
▪ Employee Health Insurance
▪ Equipment
▪ equipment
▪ Equipment - A Van and a snowblower
▪ Equipment (computers); travel
▪ Equipment- after being off so long, many things just didn't work anymore, we
also pivoted our business model- more outdoors, and needed different gear
than we had
▪ equipment and supplies
▪ Equipment and Supplies
▪ Equipment for facility
▪ Equipment Maintenance/Copier - Postage/Shipping - Office Maintenance -
Professional Expense (Accounting, Payroll, Computer)
▪ equipment purchase
▪ Equipment Purchase
▪ equipment purchases and installations of them using construction supplies
▪ Equipment rental to produce events
▪ Equipment Replacement
▪ Equipment to increase the quality of the shows we put on.
▪ Equpment
▪ Expand seating
▪ Facility usage fees, cafeteria fees
▪ Fringe
▪ Fringe Benefits
▪ fringe benefits
▪ Fringe Benefits (Health Insurance)
▪ Funds were also spent towards Insurance expenses, custodial services, waste
removal expenses
▪ General operating expenses
▪ Grounds maintenance, Insurance expense, Janitorial expense
▪ health insurance
▪ HR Fees
▪ Ice Machine, capital expenditure
▪ Insurance
▪ insurance
▪ Insurance Payment
▪ Insurance Payments
▪ Insurance payments
▪ Insurance Payments and Supplies
▪ Insurance Payments, Equipment, Supplies, Other Business Expenses
▪ Insurance, IS licenses, Computer Services, Supplies
▪ Insurance/Business Costs
Contract No. 73351022F0099 | 2M Research | A-13
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
“Other” item responses (provided exactly as they were inputted into the
Recoded category
survey)
▪ Legal fees to get the SVOG
▪ Lighting Equipment
▪ Maintaining our ticket and donation platform, Patron Manager CRM.
▪ Materials & Supplies
▪ Materieals
▪ movie rental expenses
▪ Music rental
▪ needed maintenance from lack of use during shutdown
▪ New equipment for the business
▪ Office Supplies
▪ office supplies and artist development tools
▪ Operating Leases
▪ Other Ordinary Maintenance Costs
▪ Payed the state of MN because they taxed me on this grant
▪ Payroll Fees, Bank Fees, Dividend*
▪ pos system so we could do order by phone for contactless ordering
▪ Production costs: backline rentals, tech fees, movie rights costs,
materials/supplies
▪ Production Supplies
▪ Production Transportation
▪ Professional services
▪ Program costs
▪ Programming costs for artwork exhibitions and education programs, which
are ordinary and necessary costs for the mission of our non profit.
▪ Property Taxes
▪ purchase of lighting for stage
▪ Purchases
▪ Rental fees for lighting, backdrops
▪ Replacement of perishable inventory (beer, etc.) that spoiled while closed
during COVID.
▪ sound equipment
▪ sound equipments
▪ Sound system improvements
▪ Sound System upgrade
▪ Supplies
▪ supplies
▪ Supplies - small equipment
▪ Supplies such as music stands, purchasing/renting music, stand lights, etc
▪ Supplies & Advertising
▪ supplies costs
▪ Supplies Expenses
▪ Supplies such as Concession Purchases
▪ Transportation
▪ transportation
▪ Travel
▪ travel
▪ Travel (Transportation)
▪ Travel / Transportation
▪ Travel costs for attending music Industy conferences
Contract No. 73351022F0099 | 2M Research | A-14
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
“Other” item responses (provided exactly as they were inputted into the
Recoded category
survey)
▪ Travel(Transportation)
▪ Travel/Supplies
▪ union fees
▪ Visa expenses
Expenditures related to ▪ artist costs, venue rent, stagehands, security
producing a theatrical or ▪ Artist fees and guarentees,
live performing arts ▪ Artist guarantees
production, concert, ▪ artist guarantees
exhibition, or comedy ▪ Artists/Production expenses
show ▪ As a promoter, we had to cancel or reschedule 109 performances when the
venues were dark. That means communicating, re-tooling and re-launching
advertising campaigns for ALL shows. Communicating with ALL ticket holders
and accounts on either new dates or refunds. Re-contracting with ALL
services for every show including venue, labor, artist, production, ticketing
and media. With no revenues coming in, the SVOG funds allowed our staff to
continue this tremendous amount of work until the events were back.
▪ Capital
▪ Capital expenditures
▪ Capital Expenditures - Sound Board
▪ Capital expenditures related to producing a theatrical or live performing arts
production (Upgrade lighting Concert Hall)
▪ Capital Improvements
▪ Event expenses
▪ Event Refunds
▪ Event related expenses including janitorial, event staffing and security
▪ General Advertising
▪ Livestreaming Costs
▪ Marketing
▪ New concert productions
▪ Online SEO, Wed advertising
▪ performance fees for artists
▪ Performing Artists
▪ Production Cost
▪ Production costs for a stage production
▪ production costs for our opera
▪ Refund patrons for tickets they bought before Covid-19.
▪ Refunded tickets for cancelled shows
▪ Refunds
▪ Refunds for non concerts
▪ Refunds for shows that were cancelled
▪ Refunds to ticket holders due to COVID related event cancellation
▪ Reimbursement for refunded income due to the pandemic
▪ Show expenses like sets, costumes and props
▪ Stagehand expenses
▪ The money was used extensively for marketing purposes and for travel, to
personally visit venue operators and to go to trade shows.
▪ Ticket purchase reimbursements
▪ Ticket Refunds
Contract No. 73351022F0099 | 2M Research | A-15
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
“Other” item responses (provided exactly as they were inputted into the
Recoded category
survey)
▪ ticket refunds
▪ ticket refunds for cancelled shows
▪ Venue rent
▪ Venue rentals
*Item recoded to more than one category
Prior to delivery of the data to SBA, the evaluation team developed labels for all
responses to the survey items and removed grantee identifying information.
SVOG Grantee Interviews
The evaluation team recorded and transcribed all grantee interviews. Prior to delivery
of the data to SBA, the evaluation team removed all identifying information from the
interview transcripts.
Dun & Bradstreet Data
The evaluation team merged the Dun & Bradstreet data to the SVOG application data
using DUNS. After completing the merge, the team conducted multiple imputation of
missing data values and propensity score matching of grantees and non-recipients.
Both processes are described in more detail below.
Multiple Imputation of Missing Data
Dun & Bradstreet provided sales, employee count, and viability score data for
businesses from 2018 to 2022. Dun & Bradstreet also collects information on revenue
of businesses; however, all 17,042 business in the SVOG applicant data had missing
revenue information in Dun & Bradstreet. In addition, many businesses had missing
info for employee counts, sales, and viability. Exhibit A.10 reports the number of cases
and percent of missing for each of these variables in the data. Importantly, the data
was in long format, with records for each of the 17,042 business repeated by year, for a
total of 85,210 (17,042 X 5) businesses. While revenue data was missing from Dun &
Bradstreet, revenue was available in the SBA application data for 2019 and 2020 and
partial data (first quarter only) was available for 2021.
Contract No. 73351022F0099 | 2M Research | A-16
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.10. Number and percent missing by variable.
Variable Number of cases missing Percent missing
Number of employees 30,679 36
Annual sales 30,679 36
Business viability score 2,990 4
Annual revenue 34,084 40
Exhibit A.11 shows the pattern of missingness in the data. About 34 percent of the
data was not missing any of the key variables needed for analysis. Another 30 percent
was missing only the revenue variable. About 24 percent was missing both number of
employees and sales. Smaller percentages were missing other variables. Importantly,
1 percent of the data is missing all four key variables. For these observations, the
variables were imputed based on the key business characteristics (e.g., venue type)
which are available for all observations.
Exhibit A.11. Pattern of missingness in the data.
Number of
Number of Frequency Percent of
Viability Sales Revenue Variables
Employees of Cases Cases
Missing
Not Missing Not Missing Not Missing Not Missing 0 29,261 34
Not Missing Not Missing Not Missing Missing 1 25,265 30
Not Missing Missing Missing Not Missing 2 20,071 24
Not Missing Missing Missing Missing 3 7,623 9
Missing Not Missing Not Missing Not Missing 1 3 0
Missing Not Missing Not Missing Missing 2 2 0
Missing Missing Missing Not Missing 3 1,791 2
Missing Missing Missing Missing 4 1,194 1
To correct for any bias in the analysis of the employee count and sales data and to
make the analysis of trends in revenue from 2018 to 2022 possible, the evaluation
team and SBA decided to use a multiple imputation process. The imputation occurred
in two phases. First, the evaluation team imputed missing data for viability, number of
employees, and sales. In this step, the team reshaped the data to wide format for
imputation, with the DUNS number as the unique identifier and columns for each year
of data for viability, number of employees, and sales. The wide format allowed the
team to capture any between-year variation in the data. Then the evaluation team
used the complete imputed information from the first set of variables to impute the
revenue information in the long format. The second step was necessary because
Contract No. 73351022F0099 | 2M Research | A-17
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
revenue is completely missing for the years 2018, 2021, and 2022. 7 In the wide format,
columns for these years for revenue had all missing data, and it is not possible to
impute a column that has all missing data; it must have at least some observed values
that the model can use to infer the values for missing observations. Reshaping the
data to long facilitated a dataset with a single column for revenue, which had observed
values for some years. This two-phase approach to missing data is discussed more in
the article by Harel & Schafer (2003):
https://nces.ed.gov/FCSM/pdf/2003FCSM_Harel.pdf.
There is always error in the imputed values. To account for this, the multiple
imputation employs a Bayesian estimation process that imputes values across several
iterations. In the first step, the evaluation team specified 100 iterations. In the second
step, the evaluation team did one additional imputation for revenue on each of the 100
imputed datasets from the first step. After imputation, the evaluation team thus had
100 imputed datasets with complete information for all variables. The evaluation team
ran all DID analysis (see analysis methods below) on each imputed dataset separately
and then pooled findings using Rubin’s rules.
The evaluation team used the PcAux package in R to perform the two-stage multiple
imputation, under the guidance of subject matter expert, Dr. Todd D. Little8, who
helped develop the PcAux package. Further details on the software package are
available at https://github.com/PcAux-
Package/PcAux/blob/master/documentation/PcAux_Field_Guide.pdf . The PcAux
package first expands the data to include nonlinear terms (e.g., squared and cubic
terms) and all possible interactions for all variables in the dataset. Then using the
expanded dataset, the package conducts a single imputation to fill in all missing
information. Next, the package extracts principal component scores from the
expanded, single-imputed dataset. The program informs the user how many linear and
non-linear principal components are required to explain at least 60 percent and 50
percent of the data, respectively. The user then includes those components as the
predictors of the variables with missing information, which in the case of the SVOG
Program Evaluation, included employee counts, revenue, and business viability. The
package then imputes the values of each variable separately using multiple imputation
7
Partial information for revenue was available in 2021, but SBA informed the team this information was only for the first
quarter. We thus imputed revenue for 2021 as if it were missing for all businesses. The partial information was, however,
used to inform the multiple imputation of missing values.
8
https://www.depts.ttu.edu/education/our-people/Faculty/todd_little.php
Contract No. 73351022F0099 | 2M Research | A-18
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
by chained equations (MICE). The evaluation team forced PcAux to use predictive
mean matching to impute values, to avoid imputation of negative revenue, employee
count, and viability values; however, we compared results using the package default as
well and found no substantial difference in the results.
Importantly, multiple imputation is unbiased only if the systematic difference between
the missing values and the non-missing values can be modeled in the data (or if there
is no systematic difference between missing values and non-missing values, which is
not very common). There should certainly have been a systematic difference between
the complete annual revenue values from 2019 and 2020 and annual revenue in 2021
and 2022 due to the COVID-19 pandemic. The year 2018 may have been more
comparable to 2019 and 2020, but since the evaluation team used multiple imputation
for 2021 and 2022, it was not additional effort to include 2018. Although there was a
systematic difference, it was likely modeled in the data with the sales information and
employee counts from Dun & Bradstreet and partial annual revenue data from SBA in
2021 because these values should also have been impacted by COVID in a similar way
as revenue was. In addition, other information in the data, such as venue type, was
also likely informative. For example, if certain venue types were more or less likely to
be impacted than others by the pandemic, this variable contained useful information
that allowed the evaluation team to better estimate annual revenue.
Exhibits A.12, A.13, and A.14 show density plots of the imputed vs. observed data for
revenue, employee counts, and viability, respectively. Each density plot shows
minimal differences between imputed and observed data. The evaluation team thus
had confidence in the robustness of the DID results, which were produced using the
multiply imputed datasets.
Contract No. 73351022F0099 | 2M Research | A-19
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.12. Density plot of imputed and observed annual revenue values.
Sources: SVOG Application Data and Dun & Bradstreet.
Contract No. 73351022F0099 | 2M Research | A-20
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.13. Density plot of imputed and observed employee count values.
Note: The x-axis was restricted to a maximum of 50 employees to increase readability
of the figure; there are a small number of businesses with very large employee counts.
There is no discernable difference between imputed values and observed values for
businesses with more than 50 employees.
Sources: SVOG Application Data and Dun & Bradstreet.
Contract No. 73351022F0099 | 2M Research | A-21
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.14. Density plots of imputed and observed viability score values.
Sources: SVOG Application Data and Dun & Bradstreet.
Propensity Score Matching
The evaluation team used Propensity Score Matching (PSM) to ensure that the
grantees and non-recipients included in analyses of the impact of SVOG shared similar
characteristics and baseline outcomes. The goal of the PSM was to match grantees to
non-recipients so that the outcome trends of the grantees would have been the same
as the non-recipients in the absence of SVOG. Any difference in the trends is thus
attributable to receipt of the SVOG.
Contract No. 73351022F0099 | 2M Research | A-22
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
The PSM process used logistic regression to calculate the probability of each applicant
to receive the SVOG grant based on venue type, SBA region, industry, and socio-
economic status (i.e. underserved status) of the business. Then grantees were
matched to the non-recipient with the most similar predicted value from the logistic
model. This process was done on each imputed dataset. The PSM was performed in R
using the package MatchThem. Details on the R package are available on
https://cran.r-project.org/web/packages/MatchThem/MatchThem.pdf.
The evaluation team examined if the covariates using in the matching process were
balanced, using a balance test. The across-imputation balance summary pools the raw
difference in proportions between the grantees and non-recipients and produces
summary statistics like the average, smallest, and largest balance statistic across all
imputations. Ideally the balance is close to zero, indicating little difference between
the grantees and non-recipients. The statistics below shows the balance on average
across the imputations.
Exhibit A.15. Balance test between matched grantee and non-recipient
characteristics.
Summary Statistic Average Balance
Minimum -8%
1st Quartile 0%
Median 0%
Mean 0%
3rd Quartile 1%
Maximum 4%
Analysis Methods
This section describes the analysis methods used by the evaluation team to produce
the results for the SVOG Program Evaluation. The evaluation team used R to produce
all statistical results except for the topic modeling, which was conducted using the
statistical software Stata. The thematic analysis was performed using the qualitative
data analysis software NVivo.
Descriptive Statistics
The evaluation team produced descriptive statistics such as tabulations of
frequencies, percentages, means, and medians for variables in the SVOG application
data, grantee survey, SVOG closeout survey, and Dun & Bradstreet data. The test
differences between category frequencies/percentages, the evaluation team used a
chi-square test.
Contract No. 73351022F0099 | 2M Research | A-23
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Logistic Regression
The evaluation team used a logistic regression to examine the key characteristics from
the SVOG application data and Dun & Bradstreet data associated with successfully
applying for funding from the SVOG. The dependent variable in the model was the
grantee status (1 = grantee; 0 = non-recipient) for each business in the data. The
independent variables were key business characteristics (see Exhibit 14 in the main
body of the report for a full list). For ease of interpretation, the evaluation team
transformed the log odds coefficients into probabilities for reporting.
Topic modeling
The evaluation team conducted topic modeling of the open-ended grantee survey
item to identify remaining pandemic recover needs described by the grantee survey
respondents. To implement the topic modeling, the team followed several steps:
1. Change all text to lowercase and remove special characters. The evaluation
team ensured no special characters impacted the analysis and also avoided
differences in capitalization, which, if not corrected, can affect the results of the
topic model.
2. Define global stop words. This step forced the topic model to exclude certain
frequently occurring words (i.e., stop words) in the survey responses. First the
evaluation team excluded the following frequent words that virtually all topic
models exclude: a, able, about, across, after, all, almost, also, am, among, an,
and, any, are, as, at, be, because, been, but, by, can, cannot, could, dear, did, do,
does, either, else, ever, every, for, from, get, got, had, has, have, he, her, hers,
him, his, how, however, i, if, in, into, is, it, its, just, least, let, like, likely, may, me,
might, most, must, my, neither, no, nor, not, of, off, often, on, only, or, other, our,
own, rather, said, say, says, she, should, since, so, some, than, that, the, their,
them, then, there, these, they, this, through, tis, to, too, twas, us, wants, was,
we, were, what, when, where, which, while, who, whom, why, will, with, would,
yet, you, your. Next, after examining the most frequently occurring words in the
item responses, the team further excluded the following words: funds, funding,
still, pandemic, business, pre-pandemic, levels, covid, continue, return,
recovery, during, increased, normal, returned, additional, grant. Finally, the
team excluded all words less than 5 characters from the analysis.
3. Drop records with no text information and records that indicated no
remaining needs. This step removed grantees that did not respond to the
open-ended question as well as those that replied to the question with
“nothing”, “none”, “n/a”, or “no.”
Contract No. 73351022F0099 | 2M Research | A-24
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
4. Implement topic models, beginning with a model with 3 topics and
increasing the number of topics in each additional model until a maximum
of 10 topics. To implement the topic models, the evaluation team used the
user-written package in Stata ldagibbs
(https://journals.sagepub.com/doi/10.1177/1536867X1801800107). The team
used the default settings of the package regarding the number of burn-in
iterations, the alpha and beta priors, and the number of samples and sample
iterations. Interested readers may review the ldagibbs help file in Stata for more
information.
5. Examine the 20 most frequent words in each topic from each model and
identify the most consistent topics across all models. Select one model in
which the topics seem most clearly distinct and defined. In this step, the
evaluation team compared the frequent words of each topic across all models
(ranging from a model with only 3 topics to a model with 10 topics). The same
frequent words appeared across multiple models, indicating a consistent set of
topics. Ultimately the team selected the model with 7 topics because this model
had clearly distinct topics and the models with more topics did not.
6. Estimate a probability of association with each topic for each survey
respondent and categorize each respondent into the topic with the highest
probability. The evaluation team organized each respondent into the topic
with the highest probability and reported the frequency and percent of
respondents assigned to each topic. One topic in the model identified
responses that indicated no remaining pandemic recovery needs. The
evaluation team added grantees that responded to the survey item with
“nothing,” “none,” “n/a,” or “no” to this topic (these responses were previously
excluded in step 3 above).
7. Have a separate analyst code 10 percent of the survey responses in NVivo
and compare the topic model results to the NVivo analysis. Two separate
analysts not involved with the topic modeling coded 10 percent of the survey
data in NVivo and identified very similar categories as the topic modeling,
including themes related to reduced revenue, building audience, payroll
expenses, increased general operating expenses, debt/loan relief, costs
associated with continued COVID-19 restrictions (i.e., worker protections), and
no remaining needs. The analysts also identified themes related to decreased
tourism and labor and artist shortages, which the topic model did not explicitly
identify. However, these themes had significant overlap with the themes in the
topic model and were likely captured within the existing categories. The results
of the coding served as confirmation that the topic model found the common
themes represented in the open-ended survey item.
Contract No. 73351022F0099 | 2M Research | A-25
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.16 shows the 20 most frequently occurring words in each topic of the model
with 7 topics, which the evaluation team selected as the final model.
Exhibit A.16. Most frequent words in each topic for the final topic model.
General
Debts operating Fewer
Reluctance to
Decreased incurred expenses, Worker theatrical
attend in- No needs
revenue during the payroll, protections film
person events
pandemic and other releases
costs
revenue years costs staff audience industry support
sales building expenses audiences attendance people needs
ticket months payroll costs audiences money thank
support income cover programming people trying grateful
income loans employees testing patrons movies recovered
earned recover inflation artists numbers coming community
audiences recovering labor needs theatre movie operations
costs capital higher organizations marketing theatre program
operating being challenge support events being thanks
expenses another difficult ongoing getting available event
inflation forced needs returning theater theater assistance
financial close events organization advertising things allowed
donations maintenance production performing performances again received
fully those space reduced rebuilding shows survive
ongoing received prices continued going enough financial
revenues struggling increases related years studios helped
continued getting making health entertainment recover without
budget slowly everything safety returning really fully
operations before supplies services large think needed
significant closed rental programs venue streaming point
Thematic Analysis
The evaluation team conducted thematic analysis of the interview transcripts using
NVivo. The team imported all transcripts into an NVivo server file and then, using an
open coding system, coded text in the transcripts to nodes in the file. Throughout the
coding process, the analysts maintained close contact to ensure consistency across
nodes. After completing the coding, the analysts met with the evaluation team lead
and discussed how to combine the nodes into larger themes to present in the report.
Contract No. 73351022F0099 | 2M Research | A-26
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Difference in Differences
Using the matched data produced using PSM, the evaluation team used a Difference-
in-Differences (DID) regression model to evaluate the causal impact of the SVOG
program on the outcome variables of interest, namely, number of employees, gross
revenue and business survival (represented as Y in the equation below). To prepare the
data for analysis, the evaluation team constructed a variable ‘after’ (where 1 indicates
time after SVOG funds disbursed in 2022; 0 indicates time before SVOG funds were
disbursed, i.e. between 2018 and 2021). The estimation equation is provided below -
Y= β0 + β1×[After] + β2×[SVOG Program Participation] + β3×[After×SVOG Program Participation] + ε
Exhibit A.17. Interpretation of each coefficient in the DID model.
Coefficient Interpretation
β0 The average of the outcome for non-recipients (comparison group) before the SVOG program
β1 Time trend of non-recipients (comparison group)
β2 Difference in the outcome average between the two groups before program participation
β3 Difference in changes in the outcome average over time = Impact of the SVOG program
The exhibits below detail the DID regression findings for each outcome.
Exhibit A.18. Difference-in-differences estimation for employee count.
Term Coefficient Estimate p-value
Intercept (β0) 54 0.0000
After SVOG (β1) -21 0.2108
Grantee
(0 = Non-recipient; 6 0.5582
1 = Grantee) (β2)
After SVOG×Grantee (β3) 17 0.4395
Note: The table above provides difference-in-differences estimation for Employee
count, where β3 provides the change in employee count for grantees compared to
non-recipients after the SVOG program.
Contract No. 73351022F0099 | 2M Research | A-27
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.19. Difference-in-differences estimation for revenue (in logs).
Term Coefficient Estimate p-value
Intercept (β0) 10.75 0.00E+00
After SVOG (β1) 0.10 0.0748
Grantee
(0 = Non-recipient; 1.41 0.000
1 = Grantee) (β2)
After SVOG×Grantee (β3) 0.25 0.0008
Note: The table above provides difference-in-differences estimation for revenue,
where β3 pro vides the change in revenue (in logs) for grantees compared to non-
recipients after the SVOG program.
Exhibit A.20. Difference-in-differences estimation for viability score.
Term Coefficient Estimate p-value
Intercept (β0) 2.9 0.0000
After SVOG (β1) 1.6 0.0000
Grantee
(0 = Non-recipient; -0.2 0.0000
1 = Grantee) (β2)
After SVOG×Grantee (β3) -0.3 0.0000
Note: The table above provides difference-in-differences estimation for Viability score
of businesses, where β3 provides the change in employee count for grantees
compared to non-recipients after the SVOG program.
The evaluation team also extended the DID model to examine if the impact of SVOG
(β3) varied for key business characteristics. The extension of the model is provided
below –
Y= β0 + β1×[After] + β2×[SVOG Program Participation] + β3×[After×SVOG
Program Participation] + β4[Key Characteristic] + β5×[After×Key Characteristic]
+ β6×[SVOG Program Participation×Key Characteristic] + β7×[After×SVOG
Program Participation×Key Characteristic] + ε
Each key characteristic is a categorical variable. Each category, excluding the reference
category, which is captured in the other coefficients of the model, had its own β4, β5, β6,
and β7.
Contract No. 73351022F0099 | 2M Research | A-28
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.21. Interpretation of each coefficient in the extended DID model.
Coefficient Interpretation
The average of the outcome for non-recipients in the reference category (comparison group)
β0
before the SVOG program
β1 Time trend of non-recipients in the reference category (comparison group)
Difference in the outcome average between the non-recipients and grantees in the reference
β2
category before program participation
Difference in changes in the outcome average over time for businesses in the reference
β3
category = Impact of the SVOG program for the reference category
Difference in the average of the outcome for non-recipients in each characteristic category
β4
compared to the reference category before the SVOG program (i.e., difference from β0)
Difference in the time trend of non-recipients in each characteristic category compared to the
β5
reference category (i.e., difference from β1)
Difference in the difference in the outcome average between the non-recipients and grantees in
β6 each characteristic category compared to the reference category before program participation
(i.e., difference from β2)
Difference in the difference in changes in the outcome average over time for businesses in each
β7 characteristic category (i.e., difference from β3, or how the impact of SVOG varies by the
different categories)
The results below show the findings from the extended DID models. The coefficients
for β4, β5, β6 are not reported as they are not important for understanding the varying
effect of the SVOG.
Exhibit A.22. Difference-in-differences Estimation for Employee Count by Venue
Type.
Term Coefficient Estimate p-value
Intercept (β0) 16.76 0.3446
After SVOG (β1) -5.73 0.8848
Grantee
(0 = Non-recipient; 8.49 0.7487
1 = Grantee) (β2)
After SVOG × Grantee (β3) 27.09 0.6079
After SVOG×Grantee×Live venue operator or promoter (β7_1) -6.37 0.9167
After SVOG×Grantee×Motion picture theater operator (β7_2) -39.61 0.7838
After SVOG×Grantee×Museum Operator (β7_3) -44.05 0.7263
After SVOG× Grantee×Talent representative (β7_4) -23.19 0.7627
After SVOG×Grantee×Theatrical producer (β7_5) -25.01 0.8121
Note: The table above provides difference-in-differences estimation for employee
count, where β7_i + β3 (i=1 to 5) provides the change in employee count for grantees
compared to non-recipients after the SVOG program. The reference category for this
estimation is a life performance venue.
Contract No. 73351022F0099 | 2M Research | A-29
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.23. Difference-in-differences Estimation for Employee Count by SBA
Region.
Term Coefficient Estimate p-value
Intercept (β0) 16.94 0.4619
After SVOG (β1) -2.66 0.9587
Grantee
(0 = Non-recipient; 37.95 0.2607
1 = Grantee) (β2)
After SVOG×Grantee (β3) -11.16 0.8682
After SVOG×Grantee×Great Lakes (β7_1) 34.28 0.7008
After SVOG×Grantee×Great Plains (β7_2) 43.20 0.7596
After SVOG×Grantee×Mid-Atlantic (β7_3) -55.26 0.6048
After SVOG×Grantee×New England (β7_4) -16.55 0.9002
After SVOG×Grantee×Pacific (β7_5) 87.12 0.2951
After SVOG×Grantee×Pacific Northwest (β7_6) 6.67 0.9552
After SVOG×Grantee×Rocky Mountains (β7_7) 23.80 0.8657
After SVOG×Grantee×South Central (β7_8) 32.41 0.7355
After SVOG×Grantee×Southeast (β7_9) 12.60 0.8790
Note: The table above provides difference-in-differences estimation for employee
count, where β7_i + β3 (i=1 to 9) provides the change in employee count for grantees
compared to non-recipients after the SVOG program. The reference group for this
estimation is the Atlantic region.
Contract No. 73351022F0099 | 2M Research | A-30
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.24. Difference-in-differences Estimation for Employee Count by
Industries.
Coefficient
Term p-value
Estimate
Intercept (β0) 429.92 0.0000
After SVOG (β1) -228.67 0.0011
Grantee (β2) -398.98 0.0000
After SVOG×Grantee (β3) 221.41 0.0158
After SVOG×Grantee×Administrative and Support and Waste Management
-220.68 0.0512
and Remediation Services (β7_1)
After SVOG×Grantee×Arts, Entertainment, and Recreation (β7_2) -213.80 0.0389
After SVOG×Grantee×Educational Services (β7_3) -230.89 0.0731
After SVOG×Grantee×Information (β7_4) -217.88 0.0754
After SVOG×Grantee×Other Industries (β7_5) -208.65 0.0944
After SVOG×Grantee×Other Services (β7_6) -238.92 0.0446
After SVOG×Grantee×Professional, Scientific, and Technical Services
-219.04 0.0787
(β7_7)
After SVOG×Grantee×Public Administration (β7_8) -492.52 0.0936
After SVOG×Grantee×Real Estate Rental and Leasing (β7_9) -221.52 0.1895
After SVOG×Grantee×Retail Trade (β7_10) -220.73 0.1363
After SVOG×Grantee×Unclassified (β7_11) -223.02 0.0352
Note: The table above provides difference-in-differences estimation for employee
count, where β7_i + β3 (i=1 to 11) provides the change in employee count for grantees
compared to non-recipients after the SVOG program. The reference category in this
estimation is the accommodation and food services industry.
Exhibit A.25. Difference-in-differences Estimation for Employee Count by
Underserved Business Status.
Term Coefficient Estimate p-value
Intercept(β0) 64.91 9.1E-13
After SVOG (β1) -21.80 2.7E-01
Grantee (β2) 13.65 3.1E-01
After SVOG×Grantee (β3) -37.79 2.2E-02
After SVOG×Grantee×Underserved business (β7) 17.25 5.1E-01
Note: The table above provides difference-in-differences estimation for employee
count, where β7 + β3 provides the change in employee count for grantees compared
to non-recipients after the SVOG program.
Contract No. 73351022F0099 | 2M Research | A-31
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.26. Difference-in-differences Estimation for Revenue (in logs) by Venue
Type.
Term Coefficient Estimate p-value
Intercept (β0) 10.75 0.0000
After SVOG (β1) 0.05 0.7028
Grantee (β2)
(0 = Non-recipient; 1.04 0.0000
1 = Grantee)
After SVOG×Grantee (β3) 0.14 0.0370
After SVOG×Grantee×Live venue operator or promoter (β7_1) 0.99 0.0000
After SVOG×Grantee×Motion picture theater operator (β7_2) 1.19 0.0000
After SVOG×Grantee×Museum Operator (β7_3) -0.41 0.0000
After SVOG× Grantee×Talent representative (β7_4) -1.00 0.0000
After SVOG×Grantee×Theatrical producer (β7_5) 0.29 0.0863
Notes: The table above provides difference-in-differences estimation for revenue (in
logs), where β7_i + β3 (i=1 to 5) provides the change in revenue for grantees
compared to non-recipients after the SVOG program. The reference category for this
estimation is a life performance venue.
Contract No. 73351022F0099 | 2M Research | A-32
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.27. Difference-in-differences Estimation for Revenue (in logs) by SBA
Region.
Term Coefficient Estimate p-value
Intercept (β0) 10.80 0.0E+00
After SVOG (β1) 0.01 9.3E-01
Grantee (β2)
(0 = Non-recipient; 1.43 9.7E-39
1 = Grantee)
After SVOG×Grantee (β3) -0.73 3.8E-12
After SVOG×Grantee×Great Lakes (β7_1) -0.17 3.0E-01
After SVOG×Grantee×Great Plains (β7_2) -0.27 2.1E-02
After SVOG×Grantee×Mid-Atlantic (β7_3) 0.09 5.5E-01
After SVOG×Grantee×New England (β7_4) 0.44 1.9E-06
After SVOG×Grantee×Pacific (β7_5) -0.33 1.7E-02
After SVOG×Grantee×Pacific NorthWest (β7_6) 0.27 9.0E-02
After SVOG×Grantee×Rocky Mountains (β7_7) -0.26 1.4E-02
After SVOG×Grantee×South Central (β7_8) -0.01 8.7E-01
After SVOG×Grantee×Southeast (β7_9) 0.17 4.4E-01
Note: The table above provides difference-in-differences estimation for revenue (in
logs), where β7_i + β3 (i=1 to 9) provides the change in revenue for grantees
compared to non-recipients after the SVOG program. The reference group for this
estimation is the Atlantic region.
Contract No. 73351022F0099 | 2M Research | A-33
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.28. Difference-in-differences Estimation for Revenue (in logs) by
Industries.
Coefficient
Term p-value
Estimate
Intercept (β0) 11.44 0.0000
After SVOG (β1) 0.24 0.2690
Grantee (β2) 1.51 0.0000
After SVOG×Grantee (β3) -0.85 0.0000
After SVOG×Grantee×Administrative and Support and Waste
-0.43 0.0000
Management and Remediation Services (β7_1)
After SVOG×Grantee×Arts, Entertainment, and Recreation (β7_2) -0.63 0.0000
After SVOG×Grantee×Educational Services (β7_3) -0.20 0.1088
After SVOG×Grantee×Information (β7_4) -0.61 0.0000
After SVOG×Grantee×Other Industries (β7_5) -1.02 0.0000
After SVOG×Grantee×Other Services (β7_6) -0.66 0.0000
After SVOG×Grantee×Professional, Scientific, and Technical Services
0.25 0.4969
(β7_7)
After SVOG×Grantee×Public Administration (β7_8) 0.73 0.0000
After SVOG×Grantee×Real Estate Rental and Leasing (β7_9) -1.01 0.0000
After SVOG×Grantee×Retail Trade (β7_10) -1.38 0.0000
After SVOG×Grantee×Unclassified (β7_11) 0.03 0.9152
Note: The table above provides difference-in-differences estimation for revenue (in
logs), where β7_i + β3 (i=1 to 11) provides the change in revenue for grantees
compared to non-recipients after the SVOG program. The reference category in this
estimation is the accommodation and food services industry.
Exhibit A.29. Difference-in-differences Estimation for Revenue (in logs) by
Underserved Business Status.
Term Coefficient Estimate p-value
Intercept (β0) 10.87 0.0000
After SVOG (β1) 0.11 0.0888
Grantee (β2) 1.49 0.0000
After SVOG×Grantee (β3) -0.39 0.0000
After SVOG×Grantee×Underserved business (β7) 0.17 0.0474
Note: The table above provides difference-in-differences estimation for revenue,
where β7 + β3 provides the change in revenue for grantees compared to non-
recipients after the SVOG program.
Contract No. 73351022F0099 | 2M Research | A-34
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.30. Difference-in-differences Estimation for Viability Score by venue
type.
Term Coefficient Estimate p-value
Intercept (β0) 2.99 0.0000
After SVOG (β1) 1.36 0.0000
Grantee (β2)
(0 = Non-recipient; -0.33 0.0000
1 = Grantee)
After SVOG×Grantee×Live venue operator or promoter (β7_1) -0.02 0.6692
After SVOG×Grantee×Motion picture theater operator (β7_2) -0.43 0.0008
After SVOG×Grantee×Museum Operator (β7_3) -0.12 0.2809
After SVOG×Grantee×Talent representative (β7_4) -0.39 0.0000
After SVOG×Grantee×Theatrical producer (β7_5) -0.08 0.3562
Note: The table above provides difference-in-differences estimation for viability score,
where β7_i + β3 (i=1 to 5) provides the change in employee count for grantees
compared to non-recipients after the SVOG program. The reference category for this
estimation is a life performance venue.
Exhibit A.31. Difference-in-differences Estimation for Viability Score by SBA
Region.
Term Coefficient Estimate p-value
Intercept (β0) 2.89 0.0E+00
After SVOG (β1) 1.72 8.1E-42
Grantee (β2)
(0 = Non-recipient; -0.24 3.6E-03
1 = Grantee)
After SVOG×Grantee (β3) -0.17 3.0E-02
After SVOG×Grantee×Great Lakes (β7_1) 0.04 7.7E-01
After SVOG×Grantee×Great Plains (β7_2) 0.05 5.5E-01
After SVOG×Grantee×Mid-Atlantic (β7_3) 0.11 2.9E-01
After SVOG×Grantee×New England (β7_4) 0.13 5.6E-02
After SVOG×Grantee×Pacific (β7_5) -0.07 5.3E-01
After SVOG×Grantee×Pacific NorthWest (β7_6) -0.11 3.6E-01
After SVOG×Grantee×Rocky Mountains (β7_7) -0.10 2.3E-01
After SVOG×Grantee×South Central (β7_8) 0.01 8.6E-01
After SVOG×Grantee×Southeast (β7_9) -0.04 8.1E-01
Contract No. 73351022F0099 | 2M Research | A-35
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Note: The table above provides difference-in-differences estimation for viability score,
where β7_i + β3 (i=1 to 9) provides the change in viability score for grantees
compared to non-recipients after the SVOG program. The reference group for this
estimation is the Atlantic region.
Exhibit A.32. Difference-in-differences Estimation for Viability Score by Industries.
Coefficient
Term p-value
Estimate
Intercept (β0) 3.23 0.0E+00
After SVOG (β1) 1.43 4.2E-19
Grantee (β2) -0.17 1.5E-01
After SVOG×Grantee (β3) -0.96 4.9E-27
After SVOG×Grantee×Administrative and Support and Waste
-0.39 2.3E-06
Management and Remediation Services (β7_1)
After SVOG×Grantee×Arts, Entertainment, and Recreation (β7_2) -0.09 4.1E-01
After SVOG×Grantee×Educational Services (β7_3) -0.15 1.3E-01
After SVOG×Grantee×Information (β7_4) 0.13 1.6E-01
After SVOG×Grantee×Other Industries (β7_5) -0.10 2.7E-01
After SVOG×Grantee×Other Services (β7_6) -0.08 4.2E-01
After SVOG×Grantee×Professional, Scientific, and Technical Services
-1.63 1.3E-08
(β7_7)
After SVOG×Grantee×Public Administration (β7_8) -0.27 5.8E-02
After SVOG×Grantee×Real Estate Rental and Leasing (β7_9) 0.02 8.9E-01
After SVOG×Grantee×Retail Trade (β7_10) -0.66 3.6E-15
After SVOG×Grantee×Unclassified (β7_11) -0.07 7.5E-01
Note: The table above provides difference-in-differences estimation for viability score,
where β7_i + β3 (i=1 to 11) provides the change in viability score for grantees
compared to non-recipients after the SVOG program. The reference category in this
estimation is the accommodation and food services industry.
Exhibit A.33. Difference-in-differences Estimation for Viability Score by
Underserved Business Status.
Term Coefficient Estimate p-value
Intercept (β0) 2.84 0.0000
After SVOG (β1) 1.54 0.0000
Grantee (β2) -0.31 0.0000
After SVOG×Grantee (β3) 0.16 0.0001
After SVOG×Grantee×Underserved business (β7) -0.28 0.0000
Contract No. 73351022F0099 | 2M Research | A-36
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Note: The table above provides difference-in-differences estimation for viability score,
where β7 + β3 provides the change in viability score for grantees compared to non-
recipients after the SVOG program.
Finally, to understand the full trends in each of the outcomes, the evaluation team
examined DID models with dummy variables for each time point (with 2018 as the
reference). The exhibits below provide the findings from those models.
Exhibit A.34. Estimated number of employees between 2018 and 2022.
Variable Coefficient Estimate p-value
(Intercept) 48.81 0.0011
2019 18.38 0.3921
2020 16.03 0.4546
2021 -14.49 0.4939
2022 -15.92 0.4504
Grantee
(0 = Non-recipient; 10.11 0.6261
1 = Grantee)
Grantee×2019 -13.94 0.6445
Grantee×2020 -17.02 0.5824
Grantee×2021 16.54 0.5989
Grantee×2022 13.46 0.6333
Exhibit A.35. Estimated Revenue between 2018 and 2022.
Variable Coefficient Estimate p-value
Intercept (β0) 1443873.03 0.0000
2019 (β1_1) 377438.05 0.0000
2020 (β1_2) -398936.22 0.0000
2021 (β1_3) -96193.52 0.3147
2022 (β1_4) -225866.10 0.0128
Grantee (β2)
(0 = Non-recipient; 39603.99 0.0000
1 = Grantee)
Grantee×2019 (β3_1) 387599.95 0.0003
Grantee×2020 (β3_2) -484249.02 0.0216
Grantee×2021 (β3_3) -22401.26 0.0293
Grantee×2022 (β3_4) 83908.18 0.2704
Note: The table above provides the trends in revenue between 2018 and 2022. The
estimated viability score for non-recipients in a specific year is given by the sum of β0
and β1_i (i=1 to 4). The estimated viability score for grantees in a specific year is given
by the sum of β0, β1_i, β2 and β3_i. 2018 is the reference category.
Contract No. 73351022F0099 | 2M Research | A-37
Shuttered Venue Operators Grant Program Evaluation: Final Evaluation Report
Exhibit A.36. Estimated Viability Score between 2018 and 2022.
Variable Coefficient Estimate p-value
Intercept (β0) 1.88 0.0000
2019 (β1_1) 0.57 0.0000
2020 (β1_2) 1.26 0.0000
2021 (β1_3) 2.20 0.0000
2022(β1_4) 2.61 0.0000
Grantee (β2)
(0 = Non-recipient; -0.10 0.0427
1 = Grantee)
Grantee×2019 (β3_1) 0.03 0.6459
Grantee×2020 (β3_2) 0.03 0.6944
Grantee×2021 (β3_3) -0.10 0.1626
Grantee×2022 (β3_4) -0.41 0.0000
Note: The table above provides the trends in viability score between 2018 and 2022.
The estimated viability score for non-recipients in a specific year is given by the sum of
β0 and β1_i (i=1 to 4). The estimated viability score for grantees in a specific year is
given by the sum of β0, β1_i, β2 and β3_i. 2018 is the reference category.
Contract No. 73351022F0099 | 2M Research | A-38
File and source
- File
- svog-evaluation-peer-difference-in-difference.pdf
- Size
- 4,140,569 bytes
- SHA-256
- cf7bcb4877947ebdc5b2a122a1a68eeba6afbb1caaca1ca87d53a108c1066c73
- Original
- www.sba.gov