Economic Recovery Learning Agenda (November 2023) — U.S. Department of the Treasury Office of Recovery Programs
Summary
The Treasury Department's Office of Recovery Programs issued this economic recovery learning agenda in November 2023; a note says the office was later renamed the Office of Capital Access. It sets out evaluation questions for the programs Treasury ran under the 2020 and 2021 relief laws: Emergency Rental Assistance, the Homeowner Assistance Fund, the State and Local Fiscal Recovery Funds, the Capital Projects Fund, the Emergency Capital Investment Program and the State Small Business Credit Initiative. The agenda states that the SLFRF program provides $350 billion, the Capital Projects Fund $10 billion and HAF $9.961 billion. Under ECIP, Treasury had provided more than $8.5 billion in capital to CDFIs and minority depository institutions.
Summary drafted by a model from the document's text below and checked by script against that text before publication. It is a navigation aid, not a reading of what the document proves. Where AI is used
Full text
Text recognized from a scanned document; may contain errors. Page 3 of 39 is not shown because the text could not be recognized reliably.
U.S. Department of the Treasury Office of Recovery Programs Economic Recovery Learning Agenda November 2023 U.S. Department of the Treasury Office of Recovery Programs Economic Recovery Learning Agenda This learning agenda was issued by Treasury in November 2023 prior to the Office of Recovery Programs being renamed to the Office of Capital Access. All references in this learning agenda to the Office of Recovery Programs should be understood as referring to the Office of Capital Access. Introduction The U.S. Department of the Treasury’s Office of Recovery Programs (ORP) is designed to efficiently, effectively, and equitably implement Treasury’s economic recovery programs, including those that were authorized by Congress in 2020 and 2021." This document describes the Office of Recovery Programs’ economic recovery learning agenda. As defined by the White House Office of Management and Budget (OMB), the purpose of a learning agenda is to “identify, prioritize, and establish strategies to develop evidence to answer important short- and long-term strategic questions (i.e., questions about how the agency meets its mission(s), including about how programs, policies, and regulations function both individually and in combination) and operational questions (i.e., questions about the agency’s operations like human resources, grant-making procedures, financial systems and tracking, and internal processes).”2 In this vein, the purpose of the Office of Recovery Program’s economic recovery learning agenda is to: HT HIGHLIG SHARE Share lessons about highlight the how to implement Identify and outcomes of recovery programs ORP programs during future economic crises Build on Treasury’s agency-wide learning agenda to learn about how ORP programs can be implemented effectively and equitably Context The Treasury Department released an agency wide learning agenda in April 2022 along with the FY. 2022-2026 Treasury Strategic Plan. As a complement to the Treasury Learning Agenda, the Office of Recovery Programs economic recovery learning agenda explores in more detail the following questions from the Treasury Learning Agenda and Strategic Plan that are related to economic recovery: Strategic Objective 1.3: Economically Resilient Communities: To what extent are American Rescue Plan (ARP) programs being implemented equitably? What is the impact and/or outcomes of ARP programs on households, businesses, and governments? Strategic Objective 1.4: Resilient Housing Market: What strategies deployed in the recovery from COVID-19 best prevented evictions and foreclosures? How can we track evictions nationwide? 1 These recovery programs were authorized in the Coronavirus Aid, Relief, and Economic Security (CARES) Act, the Consolidated Appropriations Act of 2021, and the American Rescue Plan Act, as well as other legislation. 2 OMB M-19-23, Appendix B: Further Guidance on Learning Agendas 1 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Executive Summary This ORP economic recovery learning agenda is designed to identify the important research questions to understand the impact the United States’ domestic economic recovery programs. These areas for evaluation can be used to guide specific research by Treasury, other federal partners, external researchers, or recipient governments. The current version of this ORP economic recovery learning agenda is based on extensive stakeholder feedback that Treasury received in response to a draft version of the learning agenda that was released in 2023. Equity is at the heart of Treasury’s implementation of economic recovery programs and a key part of the Department’s research activities. Thus, the learning agenda identifies a variety of evaluation designs that could be used to understand equity in program implementation and outcomes. This ORP economic recovery learning agenda is complementary to Treasury’s broader Learning Agenda and the White House American Rescue Plan Equity Learning Agenda. The high-level evaluation questions for the Office of Recovery Programs’ learning agenda are summarized below and delineated in more detail later in this learning agenda document. These questions include both overarching ORP-wide questions and related program specific questions that are designed to tangibly examine ORP’s equitable recovery goals for the country. Treasury is particularly interested in exploring with outside researchers the key questions bolded in the chart below. Overarching ORP Learning Agenda Questions Office of Recovery Programs (Treasury Learning Agenda and Strategic Plan Objective 1.3 and 1.4) 1. How did ORP funds stimulate an equitable economic recovery? 2. How were funds from Treasury programs braided together with each other or with funds from other federal programs to support equitable economic growth? 3. Where did ORP funds go and what are the characteristics of the households, organizations, communities, and governments who received support from ORP programs? 4. How equitable was the impact of recovery funds on Americans most in need of assistance? How did Treasury’s program design choices affect who received support from ORP programs? What have we learned from ORP that can be applied to the development of future economic recovery programs? Learning Agenda Questions by Program Capital Projects Fund (Treasury Learning Agenda and Strategic Plan Objective 1.3) 1. How are recipients using CPF funds to close gaps in broadband access? 2. What approaches are states using to address broadband affordability? 3. How are CPF investments in multi-purpose community facilities supporting digital equity? Emergency Capital Investment Program (Treasury Learning Agenda and Strategic Plan Objective 1.3) 1. After receiving ECIP funds, what are the initial patterns of investment and institutional changes across different types of financial institutions? 2. Over the longer term, how have the Emergency Capital Investment Program, Rapid Response Program, and Equitable Recovery Program changed the capacity of the participating institutions and what broader implications might that have for sector and field level approaches to serving low- and moderate-income communities and populations? U.S. DEPARTMENT OF THE TREASURY 2 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Emergency Rental Assistance (Treasury Learning Agenda and Strategic Plan Objective 1.4) 1. How equitable is the distribution of ERA funds to tenants most in need of assistance? 2. How has the use of promising practices that Treasury encouraged grantees to adopt affected the equitable distribution of ERA funds and the housing stability of tenants? 3. What have we learned from ERA about the development of a national eviction prevention and housing stability infrastructure? 4. How did ERA change tenants’ housing stability and other measures of well-being? Homeowner Assistance Fund (Treasury Learning Agenda and Strategic Plan Objective 1.4) 1. How did HAF’s structure, flexibilities, and state plan review process encourage states to adopt impactful approaches to assist homeowners? 2. How equitable is the distribution of HAF funds to homeowners most in need of assistance? 3. How are HAF funds changing the housing and fiscal stability of homeowners? State and Local Fiscal Recovery Funds (Treasury Learning Agenda and Strategic Plan Objective 1.3) 1. What strategies and supports can improve recipient capacity and program implementation? 2. How are SLFRF funds being distributed in ways that promote an equitable economic recovery? 3. What is the short-term and long-term impact on households, organizations, communities, and governments from specific SLFRF projects in priority policy areas such as affordable housing, workforce, and public safety? 4. What is the impact of SLFRF funds on key economic or fiscal indicators for state and local governments? State Small Business Credit Initiative (Treasury Learning Agenda and Strategic Plan Objective 1.3) 1. How does SSBCI technical assistance impact the ability of underserved businesses to access funds? 2. To what extent does capital from SSBCI create jobs, increase capital access, and provide other measurable community and economic development benefits? 3. To what extent did SSBCI strengthen the resilience and growth of recipient businesses and in particular, minority-owned, women-owned, and otherwise underserved businesses? 4. SSBCI has six common program formats, does one particular program format better reach small businesses, including small businesses in underserved communities? How did jurisdictions use SSBCI funds to support emerging venture capital ecosystems, and to what extent were those efforts successful? * Bold = Areas where Treasury is particularly interested in catalyzing additional studies by outside researchers, including state and local governments 3 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Learning Agenda Development and Feedback In the spring of 2023, ORP released a draft learning agenda that summarized evaluation activities that began in FY22 and provided an overview of research priorities for future years. Treasury then conducted extensive stakeholder engagement to receive external feedback on the draft learning agenda. As part of this feedback process Treasury conducted approximately a dozen focus group sessions with different stakeholders, gathered feedback through an online comment form, and participated in numerous meetings and events hosted by key stakeholders. All told, Treasury engaged with well over 100 individuals and organizations, including funding recipients (state, local, and Tribal governments), other federal agencies, members of the public, government associations, researchers, philanthropy, academics, think tanks, community organizations, and program beneficiaries. The feedback from stakeholders focused on several major themes including a desire for a specific definition of equity, more details on ORP programs’ evaluation priorities, and additional information about the potential data sources available for researchers. This and other valuable feedback received through this stakeholder engagement process is reflected throughout this updated ORP learning agenda, issued in fall 2023. This ORP economic recovery learning agenda is designed to be a living document and outlines questions and learning priorities based on feedback, analysis, and evaluations done to date. As a result, Treasury recognizes that some of these questions may be removed or deprioritized as new questions of interest might be identified in the future. As OMB notes, “Learning agendas should be iterative, flexible, transparent, and tailored to both meet an individual agency’s needs and address agency-specific challenges to developing evidence.” An Emphasis on Collaboration Not all of the questions on the learning agenda will be answered directly by Treasury; some questions are intended to signal Treasury’s interest in a particular area of research and in exploring collaborations with external researchers to examine the relevant questions. As described in more detail in programmatic evaluation sections below, this collaboration could consist of a variety of models including matchmaking between researchers and recipients of Treasury funding to do research in commons areas of interest or sharing additional Treasury data with researchers for the purposes of performing evaluations (as Treasury has already done for the Emergency Rental Assistance program with support from the Department of Housing and Urban Development). Research collaborations are important because Treasury and the Office of Recovery Programs’ capacity for performing evaluations is limited by available resources. In that regard, the General Services Administration’s (GSA) Office of Evaluation Sciences (OES) has already provided invaluable support in helping to design the evaluation questions in this learning agenda. In close partnership with Treasury, OES has also performed a significant number of evaluations through an Inter-Agency Agreement with Treasury as well as through funding from GSA’s Technology Transformation Services. In addition, the White House Office of Management and Budget Evidence Team and OES are managing a national evaluation of the American Rescue Plan, which has the potential to include additional evaluations of the many Treasury programs that were authorized under this legislation. U.S. DEPARTMENT OF THE TREASURY 4 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA A Focus on Equity In addition to building off the Treasury agency wide evaluation questions above, a significant portion of ORP’s early evidence-building work was a result of the equity review process required by Executive Order 13985 (Advancing Racial Equity and Support for Underserved Communities Through the Federal Government) and the resulting agency equity action plan. In this vein, the ORP economic recovery learning agenda is premised on the idea that equity and outcomes are not mutually exclusive but rather inextricably linked— programs will not reach their true goals unless they are advancing more equitable outcomes. f EQUITABLE [ OUTCOMES “Equity and outcomes are not mutually exclusive but rather inextricably linked— programs will not reach their true goals unless they are advancing more equitable outcomes.” This learning agenda uses the definition of equity from Executive Order 13985.3 The important role of equity in evaluation is also featured in the American Rescue Plan Equity Learning Agenda, which contains sample evaluation questions for ORP programs that correspond to many of the evaluation questions in this ORP learning agenda as well as the following overarching question, which aligns with the overall ORP learning agenda: “To what extent are Treasury’s ARP-funded programs (including Capital Projects Fund, Emergency Capital Investment Program, Emergency Rental Assistance, Homeowner Assistance Fund, State and Local Fiscal Recovery Fund, State Small Business Credit Initiative) being implemented equitably, and what barriers to equitable implementation exist?” 3 Executive Order 13985 states that: “The term ‘equity’ means the consistent and systematic fair, just, and impartial treatment of all individuals, including individuals who belong to underserved communities that have been denied such treatment, such as Black, Latino, and Indigenous and Native American persons, Asian Americans and Pacific Islanders and other persons of color; members of religious minorities; lesbian, gay, bisexual, transgender, and queer (LGBTQ+) persons; persons with disabilities; persons who live in rural areas; and persons otherwise adversely affected by persistent poverty or inequality.” 5 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA ORP Learning Agenda Approach This learning agenda also serves as an ORP evaluation plan because studies that Treasury has conducted since FY22 or plans to conduct in the future are identified in relevant programmatic learning agendas that compose the bulk of this learning agenda document. In line with Treasury’s interim evaluation policy, ORP is committed to communicating evaluation objectives, methods, and findings with key stakeholders in contextually appropriate ways. Treasury is dedicated to using a diverse set of evaluation designs in its work. This variety of evaluation designs provides Treasury with valuable information throughout the lifecycle of economic recovery programs. This includes research that informs program development, identifies ways to help recipients make effective use of program funds, improves recipient compliance, measures the impact of programs, and identifies lessons learned for future economic recovery programs. As such, in developing this learning agenda, Treasury considered which of the following methods best matched with its learning agenda goals: Process evaluations: Examining the process by which a program was implemented and the potential effects of program design choices® Descriptive evaluations: Describing the characteristics of those that received services or funds but not necessarily cause and effect* Impact evaluations: Measuring the causal impact of the program on beneficiaries® Other Tools: Data analysis, performance metrics, qualitative methods, and additional tools for examining the implementation and outcomes from programs Building on the ORP-wide research questions, following are program specific evaluation questions that are designed to serve as the component parts to answer the broader ORP wide questions and identify how individual recovery programs contributed to Treasury’s efforts to stimulate an equitable economic recovery from the Coronavirus pandemic. 4 |InM-20-12, OMB states “Descriptive Studies can be quantitative or qualitative in nature, and seek to describe a program, policy, organization, or population without inferring causality or measuring effectiveness. While not all descriptive studies are evaluations, some may be used for various evaluation purposes, such as to understand relationships between program activities and participant outcomes, measure relationships between policies and particular outcomes, describe program participants or components, and identify trends or patterns in data.” 5 Also known as a formative evaluation, which OMB defines in M-20-12: “Formative Evaluation is typically conducted to assess whether a program, policy, or organizational approach-or some aspect of these-is feasible, appropriate, and acceptable before it is fully implemented. It may include process and/or outcome measures. However, unlike outcome and impact evaluations, which seek to answer whether the program, policy, or organization met its intended goals or had the intended impacts, a formative evaluation focuses on learning and improvement and does not aim to answer questions of overall effectiveness.” 6 In M-20-12, OMB states that an “Impact Evaluation assesses the causal impact of a program, policy, or organization, or aspect thereof, on outcomes relative to those of a counterfactual. In other words, this type of evaluation estimates and compares outcomes with and without the program, policy, or organization, or aspect thereof. Impact evaluations include both experimental (i.e., randomized controlled trials) and quasi-experimental designs. An impact evaluation can help answer the question, ‘does it work, or did the intervention lead to the observed outcomes?” U.S. DEPARTMENT OF THE TREASURY 6 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Capital Projects Fund Learning Agenda The Capital Projects Fund (CPF) program provides $10 billion to states, territories, freely associated states, and Tribal governments to fund critical capital projects that enable work, education, and health monitoring in response to the public health emergency. More than 30 million Americans live in areas where there is no broadband infrastructure that provides minimally acceptable speeds and according to the latest OECD data, among 35 countries studied, the United States has the second highest broadband costs.’ As such, a key priority of the CPF program is to make funding available for reliable, affordable broadband infrastructure and other digital connectivity technology projects. CPF represents a significant piece of the large federal investment in broadband connectivity over the coming years through programs managed by Treasury, the U.S. Department of Commerce, Federal Communications Commission (FCC), and U.S. Department of Agriculture. Data There are a number of data sources that could be used to study CPF and other federal programs focused on broadband connectivity. As a starting point, CPF will be regularly collecting data from states about their CPF funded projects. In addition, there is a variety of other federal broadband data which could be paired with data about CPF investments to support the research questions below. Sources of federal broadband data include the Federal Communication Commission’s Broadband Data Collection, the Department of Commerce’s National Telecommunications and Information Administration data hub, and information about internet use from the U.S. Census. 1) How are recipients using CPF funds to close gaps in broadband access? The COVID-19 pandemic revealed and reinforced that communities without access to high-quality modern infrastructure, including broadband, face impediments to fully participating in aspects of daily life, such as remote work, education, and telehealth. One of the key goals of CPF is to increase broadband access across the country. As of October 2023, CPF had awarded funding to connect an estimated 2 million locations to high-speed internet. Potential evaluations questions to examine related to how CPF is closing gaps in broadband access include: * To what extent are CPF recipients using funds to pay for different types of broadband Background infrastructure (including last-mile grant and line extension programs)? + How are CPF resources being used to complement other federal broadband investments (such as those from the Department of Commerce’s Broadband Equity, Access, and Deployment program and the Federal Communications Commission)? + What are the different strategies CPF recipients are using to increase broadband access? + What are the characteristics of places where recipients are using CPF funding to improve broadband access? Using data from CPF reporting as well as other sources of federal broadband data such as broadband data from the Federal Communication Commission, researchers could examine how CPF is being used to meet broadband needs across the country. Specifically, this could be done through a descriptive study of how recipients are using CPF funds to address gaps in broadband access (e.g., last-mile grant programs, line extension programs, connecting MDUs), the technologies used, and broadband speeds achieved with these investments. Potential Evaluation Designs 7 Guidebook to the Bipartisan Infrastructure Law | Build.gov | The White House 7 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Treasury is collecting location data for CPF-funded projects on a quarterly basis. This data includes information on projects funded, obligations, expenditures, project status, outputs, and performance Work Already | indicators. In addition, CPF recipients are required to file Annual Reports that include narrative overviews Underway of projects information related to program outputs and outcomes against the stated objectives in the approved program plans. The first Annual Reports from CPF recipients were due July 31, 2023. These quarterly and annual reports will provide valuable sources of data for future research. Moving forward, Treasury will continue to receive and analyze the annual and quarterly reports submitted by recipients. As more broadband projects continue to be planned and implemented, the data from these Next Steps y recip Pro) P P regular reports will allow for research into the ways in which CPF is being used by recipients to address gaps in broadband access across the country. 2) What approaches are states using to address broadband affordability? Background While too many areas of the country lack high-speed internet, there are also many places where high-speed internet is available but prohibitively expensive, particularly for low-income Americans. Treasury requires that providers that receive CPF funds participate in the Federal Communications Commission’s Affordable Connectivity Program (ACP). The ACP, funded by President Biden’s Bipartisan Infrastructure Law, helps ensure that households can afford high-speed internet by providing a discount of up to $30 per month (or up to $75 per eligible household on Tribal lands). To further lower costs, the Administration secured commitments from 20 leading internet service providers—covering more than 80% of the U.S. population—to offer all ACP-eligible households high-speed, reliable internet plans for no more than $30 per month. As a result of this agreement and the ACP, many eligible households can receive internet access at no cost. As such, research could examine questions such as: * Inimplementing their CPF funded programs, what strategies are states using to make high- speed broadband more affordable? * To what extent have states used innovative approaches to increase the affordability of high- speed broadband? Potential Evaluation Designs Data from CPF reporting, additional information from states, and information from the ACP could allow for research into how more people are able to access affordable broadband service. CPF further encourages that recipients require that broadband projects include at least one low-cost option offered at speeds that are sufficient for a household with multiple users to simultaneously telework and engage in remote learning. A potential evaluation would be to perform a descriptive study of the approaches CPF recipients are taking to address affordability of broadband service and determine effective approaches for ensuring that broadband infrastructure projects provide service that is affordable to end users. Work Already Underway Every state must submit project plans to Treasury for review and approval. Part of Treasury’s review of CPF plans includes analysis of program materials (e.g., grant program guidelines and scoring criteria) to identify how recipients are addressing affordability. Next Steps Treasury is conducting ongoing outreach to states and monitoring of CPF plan implementation to ensure compliance with the broadband affordability commitments contained in the Treasury approved state plans. In the future, external researchers could use this data and potentially partner with CPF to conduct an analysis of the impact of these different approaches on service offerings and uptake of service. U.S. DEPARTMENT OF THE TREASURY 8 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 3) How are CPF investments in multi-purpose community facilities supporting digital equity? Background In addition to broadband projects, CPF is also funding projects that focus on multi-purpose community facilities. Specifically, CPF funds may be used for the construction or improvement of buildings designed to jointly and directly enable work, education, and health monitoring located in communities with critical need for the project. For example, a multi-purpose community facility could be a building, such as a library or community center providing the public with access to computers with high-speed internet service. When determining the need of potential communities to be served by Multi-Purpose Community Facility Projects, CPF recipients may choose to consider any data such as the Census American Community Survey or the U.S. Department of Housing and Urban Development’s Qualified Census Tracts. Potential topics for research in this area include: « What are the characteristics of the locations where CPF recipients are funding multi-purpose community facilities? + What are best practices for using multi-purpose community facilities to support increased digital equity? Potential Evaluation Designs Using data from CPF reporting, additional data from states, and federal data about digital equity would allow for research into the role of multi-purpose community facilities. Researchers could conduct a descriptive study to understand how investments in multi-purpose community facilities are helping to advance digital equity through both basic Internet access as well as device access, trainings, or other offerings that enable work, education, and health monitoring. Work Already Underway Treasury has approved over $1 billion for multi-purpose community centers. Recipients are using CPF funds to support a range of projects, from library renovations to newly constructed technology centers to community health centers. For example, Maine will invest in new “Connectivity Hubs,” which may include libraries, telehealth centers, and food banks, to provide programming on workforce training, education, and telehealth. In Tennessee, the Connected Communities Facilities program will construct or improve facilities that provide public internet access. Those facilities will partner with the Tennessee Department of Economic and Community Development to offer workforce development programs, educational courses, and health services. Next Steps Treasury continues to conduct ongoing outreach to states to monitor implementation of approved multi-purpose community center plans. This includes information that states are required to include in CPF Annual Reports. A major focus will continue to be how these facilities impact digital equity and the degree to which different models and approaches produce different outcomes. Moving forward, Treasury could partner with outside researchers to evaluate the impact of different digital equity approaches in multi-purpose community facility settings to evaluate their effectiveness at addressing critical needs in communities served by these projects. U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Emergency Capital Investment Program Learning Agenda The Emergency Capital Investment Program (ECIP) was created to encourage low- and moderate- income (LMI) community financial institutions to strengthen their efforts to increase access to capital for small businesses and consumers in their communities. Under the program, Treasury has provided more than $8.5 billion in capital to federally-insured depository institutions that are certified Community Development Financial Institutions (CDFls) or minority depository institutions (MDIs) to, among other things, provide loans, grants, and forbearance for small businesses, minority-owned businesses, and consumers, especially in low-income and underserved communities, that may be disproportionately impacted by the economic effects of the COVID-19 pandemic. In the immediate term, a key research priority for the Emergency Capital Investment Program is to identify the best data sets that can be used to understand the program’s outcomes and operations. In the longer term, the program is interested in assessing its outcomes on financial institutions, communities, individuals, and businesses. Data Treasury collects data quarterly and annually from ECIP participants through the Quarterly Supplemental Reports (QSR). The reports collect aggregated loan origination data across a wide range of depository loan products and in certain cases is disaggregated by categories including borrower demographics, loan purpose and the geographic location of the borrower. This is a level of data collection that has not been previously available to Treasury, in part due to legal restrictions on collection of certain demographic data by banks and credit unions. Previously, Treasury was limited to collection of race and ethnicity data based on “proxies”, which are estimates of race and ethnicity data rather than demographic data self- reported by borrowers. However, Congress established in the Consolidated Appropriations Act, 2021, that the institutions participating in the program could collect data otherwise prohibited from collection, under the Equal Credit Opportunity Act, for purposes of ECIP program compliance. 1) After receiving ECIP funds, what are the initial patterns of investment and institutional changes across different types of financial institutions? The dividend or interest rate that participating institutions pay Treasury on their ECIP investments will be reduced if they achieve specified benchmarks for increasing their “Qualified Lending,” which includes lending to minority, rural, and urban low-income and underserved communities, LMI borrowers, and other similar lending as outlined in the ECIP Rate Reduction Incentive Guidelines. When ECIP participants engage in “Deep Impact Lending,” Treasury will provide additional credit toward this rate reduction. Deep Impact Lending includes loans to Low-income Borrowers and Underserved Small Businesses, for Deeply Affordable Housing, and in Persistent Poverty Counties. The additional credit recognizes the fact that the kind of lending that will be most impactful in achieving the statutory purpose of the program often requires more time and resources from the lender. This approach is designed to help level the playing field for borrowers that face the greatest barriers to accessing capital and will provide greater transparency into the impact of the program. Background ECIP will document categories of Qualified Lending and Deep Impact Lending through analysis of Quarterly Supplemental Report (QSR) data. Along with related information, this data could be used to investigate research questions such as: + What characteristics of CDFls and MDIs are associated with certain policy outcomes such as positive impacts for low-income borrowers and communities? For example, are large institutions likely to increase investment at a different or faster rate? U.S. DEPARTMENT OF THE TREASURY 10 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps Treasury will gather data on ECIP Participant lending through the QSRs and may make available public updates on the total lending, qualified lending, and deep impact lending by ECIP participants. Treasury will also analyze the geographic and demographic distribution of lending, to the extent that available QSR data allows such disaggregated analysis. This information could be used by researchers to perform a descriptive evaluation that examines how ECIP institutions are using the funds. The first sets of QSRs were due from ECIP participants in August and September 2023, covering lending from each participant’s closing date to June 2023. Topline preliminary Qualified and Deep Impact Lending data for 2022 was released to the public in connection with the 2023 Freedman’s Bank Forum. The data collected for 2022 showed that ECIP participants originated a total of approximately $26 billion in loans in a little more than six months in 2022. Approximately 75%, or $19.4 billion, went to low to moderate income (LMI) borrowers, borrowers in rural communities, and other categories of Qualified Lending. Approximately one third of the total originations, or $8.6 billion, were Deep Impact loans made to the hardest-to-serve borrowers, including those that are low-income, residents on Tribal lands and in US Territories, and owners of underserved businesses. Work is underway to verify the QSR data through a series of logic tests and risk tolerances performed by the ECIP team, with an expectation that additional data will be released to the public in Q2 FY24. 2) Over the longer term, how have the Emergency Capital Investment Program, Rapid Response Program, and Equitable Recovery Program changed the capacity of the participating institutions and what broader implications might that have for sector and field level approaches to serving low- and moderate-income communities and populations? Background 11 The community financial institutions that received investments through ECIP include banks, holding companies, and credit unions that are designated as community development financial institutions (CDFls) or minority depository institutions (MDIs). These mission-driven financial institutions specialize in delivering responsible capital, credit, and financial services to underserved communities. ECIP provides an incentive for institutions to make qualified loans in minority, rural, and urban low-income and underserved communities and to low- and moderate-income borrowers. Further research in this area could include examining questions such as: » What changes have occurred to CDFls and MDis as a result of Treasury funds? + How have Treasury funds changed the way that financial institutions provide capital to low- income and moderate-income communities? * To what extent can the approaches used through the Emergency Capital Investment Program, Rapid Response Program, and Equitable Recovery Program be applied to other parts of the financial services sector? U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway A long-term program evaluation will be conducted by an outside evaluator. This evaluation will consider the CDFI Equitable Recovery Program, the CDFI Rapid Recovery Program, and ECIP together. The selected evaluation contractor will deliver a final methodology and approach to evaluation for Treasury approval within the first six months of the evaluation contract. Evaluation may involve a variety of analytical and program evaluation techniques to assess award recipients/participants’ loans and investments, organizational performance, and social impacts and returns. In order to conduct the evaluation, the evaluation contractor(s) may need to: (1) analyze existing data provided by the CDFI Fund and Treasury from award recipients/participants; (2) analyze existing data provided by the CDFI Fund and Treasury on CDFls and MDIs that are not award recipients/participants in these programs; (3) review CDFI Fund and Treasury project files and conduct staff interviews; (4) identify a representative sample of projects from each program to evaluate and analyze; (5) perform statistical analysis on the three programs; (6) identify and utilize publicly available data sources that may help to contextualize or enrich the evaluation methodology (which could include data from the U.S. Census Bureau and the U.S. Department of Labor as well as data from federal regulators of financial institutions related to the Home Mortgage Disclosure Act or Community Reinvestment Act, or other sources); and (7) conduct surveys and/or interviews and visit award recipients/participants (subject to Paperwork Reduction Act requirements and timelines). ECIP has started work on a literature review to better understand past approaches to program evaluation. The program seeks to evaluate the limitations of prior studies to determine whether and how the additional data collection on qualified and deep impact lending can be used to provide unique insights into lending by CDFls and MDls, when capitalized through ECIP and the concurrent CDFI Rapid Response Program and CDFI Equitable Recovery Program. ECIP is working to procure a well-qualified evaluation team to develop a long-term research plan and schedule for specific research products and releasing data to stakeholders and the public. Next Steps ECIP anticipates completing the procurement of an evaluation partner by the first quarter of 2024. U.S. DEPARTMENT OF THE TREASURY 12 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Emergency Rental Assistance Learning Agenda The Emergency Rental Assistance® (ERA) program provides funding to assist eligible households experiencing financial hardship with rent, rental arrears, utilities and home energy costs, utilities and energy costs arrears, other expenses related to housing, housing stability services, and other affordable rental housing and eviction prevention activities. Funding was allocated under the Consolidated Appropriations Act of 2021 ($25 billion) and the American Rescue Plan Act of 2021 ($21.55 billion) and distributed to states, U.S. territories, local governments, and Tribal governments. As of October 2023, ERA has made more than 12.3 million payments to households. Since the program’s inception, a key research priority for the Emergency Rental Assistance program has been understanding how its funds are being distributed to tenants, especially low-income and traditionally underserved tenants. Additional information about the equity elements of ERA is available in the White House report on Advancing Equity through the American Rescue Plan, issued in May 2022. As the program matures, Treasury is interested in studying the medium and long-term outcomes to strengthen the evidence base on eviction prevention and housing stability programs and to inform implementation of future federal programs. This includes understanding the outcomes for tenants that received rental assistance as well as studying how different program design features affected who received ERA assistance. Data Treasury is regularly releasing robust ERA data on the Department’s website. This data contains a wealth of information about the households being served by ERA grantees, including the number of households served, the income level of ERA beneficiaries, demographic data (including race, ethnicity, and gender) for households receiving ERA assistance, program expenditure levels, and the amount of ERA funds going towards other affordable rental housing and eviction prevention activities. In addition, through a data sharing collaboration with the Department of Housing and Urban Development, Treasury has provided researchers the ability to use anonymized household level data to study ERA’s impact on households across the country. Treasury will continue to explore other forms of research partnerships. Furthermore, additional complementary data sources include eviction statistics from the Eviction Lab, Census Household Pulse Survey data about housing stability over the course of the pandemic, and Census American Community Survey data which contains tract level statistics for renter households. ERA grantees may also engage in partnerships with researchers to analyze administrative data, for instance to evaluate specific program strategies. Other research efforts through ERA grantee groups, landlord affiliated organizations, and advocates may be explored as well. 8 Two separate ERA programs have been established: the ERA1 program was authorized by the Consolidated Appropriations Act, 2021 providing $25 billion, which ended in late 2022 and the ERA2 program was authorized by the American Rescue Plan Act of 2021 providing $21.55 billion, which remains ongoing. ERA funds are provided directly to states, U.S. territories, local governments, and, in the case of ERAI, Indian Tribes or their Tribally Designated Housing Entities. Please note that only ERA2 grantees that meet the statutory criteria may use their ERA2 award funds for other affordable rental housing and eviction prevention activities (see 15 U.S.C.9058c.(d)(1)(D)). 13 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 1) How equitable is the distribution of ERA funds to tenants most in need of assistance? Background Potential Evaluation Designs Work Already Underway The ERA program provided direct cash assistance to renters and landlords to assist with rent, rental arrears, utilities and home energy costs, utilities and home energy costs arrears, and other expenses related to housing. The program was designed with equity at the forefront. Treasury’s guidance and outreach has emphasized the importance of grantee outreach to communities, particularly to those that traditionally have lower access to federal programs and to individuals more likely to be experiencing financial hardship and housing instability during and due to the pandemic. Treasury is interested in understanding whether the program funds were distributed equitably, and specifically whether the characteristics of the program’s beneficiaries reflected the target populations for the program. In addition to studying tenants, another area of research would be examining the distribution of ERA funds to different types of landlords, including small landlords who often have similar demographics to the renters they serve. As such, potential research questions could include: + What were the traits of renters that received ERA funds and how do these beneficiaries compare to traits of renters that were eligible for ERA? + What are the characteristics of landlords that received ERA funds; how equitable was the distribution of funds to landlords? + What barriers to completing the application process existed? Were there differences in the characteristics of applicants that did and did not receive ERA resources? * Are there geographic or other factors that affected which tenants and landlords received assistance? To explore the equity dimensions of ERA funding distribution, researchers could use data about ERA beneficiaries and their characteristics from Treasury or individual ERA grantees. Using this data could allow researchers to conduct descriptive studies of tenants and landlords that received ERA funds. This could include comparing the demographic characteristics of beneficiaries and landlords in the same geographic areas or studying the distribution of ERA funds in different geographies (rural, urban, suburban) and among small and large landlords. Further, an evaluation using disaggregated race data could provide critical insights on the distribution of ERA funds and equity implications, particularly among Asian American, Native Hawaiian and Pacific Islander populations. On a quarterly basis, Treasury releases demographic data about ERA grantees at the national and state level. The most recent figures as of publication (through Q2 2023) demonstrated the overwhelming majority of ERA funds went to low-income households and communities of color with more than 60% going to communities of color and more than 66% to female-headed households. In addition, GSA’s Office of Evaluation Sciences (OES) conducted an evaluation of the demographics of ERA beneficiaries in relation to the demographics of those eligible for ERA. This study found that relative to eligible renters, a higher share of actual ERA beneficiaries were Black, women, extremely low-income, and American Indian or Alaska Native, Pacific Islander, or Hawaiian Native. This finding that ERA funding went overwhelmingly to these most vulnerable renters indicates that ERA outreach efforts were successful in reaching those renters that most needed assistance: tenants with the lowest incomes and those racial groups most at risk of eviction. On the other hand, Asian renters were underrepresented among beneficiaries of ERA. This finding is consistent with other similar benefits programs and highlighted an opportunity for additional ERA outreach efforts. U.S. DEPARTMENT OF THE TREASURY 14 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Researchers from the Treasury Office of Economic Policy and the Federal Reserve Board are collaborating on new research about the equity dimensions ERA. This new research will underscore the degree to which ERA funds went to the households that most needed assistance at the time when they were most in need. As previewed in Treasury’s report on Equitable Recovery in the United States, among other findings this research will find that ERA funds went to communities with higher shares of Black or Hispanic renters as ERA dollars received per renting household were higher in census tracts with larger shares of Black or Hispanic renters. On average, a renting household in a census tract with a renting population was 75 percent Black or Hispanic received over $375 more than a census tract that has a renting population that is 25 percent Black or Hispanic. Next Steps In addition, Treasury will also continue to explore options to expand on previous equity studies to ensure that funds continue to reach those households that most need assistance. This includes doing further analysis related to equitable distribution of ERA funds to tenants and landlords. 2) How has the use of promising practices that Treasury encouraged grantees to adopt affected the equitable distribution of ERA funds and the housing stability of tenants? Treasury provided guidance to ERA grantees that outlined eligibility criteria for renters seeking financial assistance through the program, but ERA grantees had some flexibility to develop rental assistance programs that suited the needs of their local communities. Treasury has engaged with ERA grantees to identify strategies that promise to speed up program implementation, more efficiently deliver program benefits, enhance program integrity, and improve tenant and landlord access to programs—particularly for vulnerable and harder to reach populations. Treasury has encouraged grantees to adopt these promising practices and to continue sharing strategies that have contributed to effective and equitable implementation of their programs. Specific promising practices include use of fact-specific proxies to simplify documentation requirements, intentional landlord engagement (which may include operational strategies to reduce participation burden on landlords), categorical eligibility of tenants based on their participation in other benefit programs, direct payments to tenants, integration of housing stability and eviction prevention services, as well as data-driven, culturally, and linguistically competent outreach. In addition, a significant number of ERA grantees employed strategies that prioritized certain ERA applicants based on factors such as income, threat of imminent eviction, presence of children in the household, and other factors.® Background Research and evaluation related to these practices and their effects on program implementation speed, delivery of benefits, integrity, and equitable access to these program resources would be valuable contributions to the evidence base on eviction prevention and housing stability. Potential additional evaluation questions in this area include: * How did the use of fact specific proxies (such as residence in Qualified Census Tract) increase the efficiency of the ERA application and approval process for renters? + Which promising practices were most effective in increasing the speed of assistance to tenants and overall housing stability with minimal impact to program integrity? + Did the effectiveness of landlord engagement strategies vary according to the demographics, size, region, or other landlord characteristics? + What prioritization strategies did grantees leverage or develop for distributing ERA funds to tenants and which strategies were most effective for program administration? * How did the use of promising practices vary over time and to what extent were certain practices more effective at different points in the ERA program lifecycle or for different types of grantees (state, county, city, territory, or Tribe)? 9 ERA grantees are required to prioritize assistance to households with incomes less than 50 percent of area median income or households with one or more individuals that have not been employed for the 90-day period preceding the date of application. Grantees may add additional preferences, provided the grantee does not impose any eligibility criteria that is not consistent with the ERA statute, document the preference system they plan to use and inform all applicants about available preferences. 15 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps It is possible for researchers to examine the effect of program flexibilities by using ERA grantees’ reporting data publicly released by Treasury, information from the National Low Income Housing Coalition about which program flexibilities were adopted by which grantees, and additional data from specific ERA grantees. A potential evaluation could study the effect of grantees’ adoption of different program flexibilities permitted by Treasury on program application rates, speed of distributing ERA funds, or the demographics of who received ERA assistance. In states where a promising practice was adopted after the program began, an evaluation could leverage temporal variation to study whether the adoption of the strategy led to different program outcomes. The Housing Initiative at Penn, New York University Furman Center, and the National Low Income Housing Coalition found that ERA grantees that employed the flexibilities provided by Treasury had better program performance and higher expenditure rates. Specifically, they found that “programs that allow categorical eligibility or use fact-specific proxy for income eligibility spent a higher share of their allocation” as did programs that allowed for self-attestation and direct to tenant assistance. The General Services Administration’s Office of Evaluation Sciences is working with ERA grantees to look at effectiveness of different approaches by grantees to administering ERA using Treasury’s promising practices. Specifically, these studies examine the use of fact-specific proxies using temporal and geographic variation to understand whether streamlining an eligibility requirement broadened and quickened access to assistance. Treasury encourages researchers and ERA grantees to conduct further such studies to examine the effect of promising practices on the distribution of ERA resources to tenants and landlords. 3) What have we learned from ERA about the development of a national eviction prevention and housing stability infrastructure? Background Prior to ERA no national eviction prevention or housing stability infrastructure existed. As such, Treasury faced implementation challenges stemming from the need for over 400 state and local ERA grantees to build an infrastructure able to support the rental assistance for the potential millions of vulnerable renters. The effort to create this new national eviction prevention policy in an equitable way was in many ways a race against time because there was a critical need to ensure ERA relief was widely available when the federal eviction moratorium lifted throughout the country. By balancing speed with program integrity, this effort was largely successful with eviction rates holding below historical levels after the end of the eviction moratorium and millions of payments made to households. However, there remains much to be learned about this experience and how lessons can be applied to future housing crises by exploring questions such as: + What were the most effective strategies to deploy ERA resources quickly and equitably to renters? * How can landlords be effectively engaged in rental assistance and eviction prevention programs? + What was the role of state and local eviction prevention protections (including eviction moratoriums and legal services)? * How did combining ERA funds with additional services (such as case workers or partnerships with tenant organizations) impact the speed and effectiveness of ERA? * To what extent have the housing stability projects permitted under ERA helped tenants in the longer term? U.S. DEPARTMENT OF THE TREASURY 16 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps Awide variety of reports and data are available to examine the development of a national eviction prevention infrastructure through ERA. This includes the data that Treasury has publicly released, which is available for prior periods to understand the historical trajectory of ERA deployment. In addition, external stakeholders have developed eviction prevention resources that can support further research, such as the Local Eviction and Prevention Policy and Program Tool from the National League of Cities and A National Survey of Tenant Protections Under State Landlord Tenant Acts from Freddie Mac. Using this information and newly collected qualitative data, a potential mixed-methods evaluation could explore the barriers and successes for jurisdictions in setting up a rental assistance program from scratch and any best practices that can be applied going forward (including for how state and local governments provide eviction prevention services). Additionally, evaluators could explore the impact of partnerships and outreach strategies, particularly with small organizations that may not have been prepared to receive government funds but were well connected to targeted communities. External stakeholders have already conducted extensive research into the process, effects, and lessons from the development of an eviction prevention and housing stability infrastructure. For example, the Eviction Lab has done research on the impact of eviction moratoria as well as how ERA and related policies avoided millions of evictions. In addition, the National Low Income Housing Coalition and the University of Pennsylvania’s Housing Initiative at Penn released a joint report, Emergency Rental Assistance (ERA) during the Pandemic: Implications for the Design of Permanent ERA Programs. Further, the Terner Center at UC Berkeley found a positive association between the number of nonprofit organizations present in the community and higher ERA take-up rates. Treasury is collaborating with the Department of Housing and Urban Development and external researchers to evaluate the impact of ERA on housing stability. As part of this work, HUD awarded $2 million from a notice of funding that was designed to “inform the development of future approaches to helping families maintain housing stability and avoid eviction.” 4) How are ERA funds changing the housing stability of tenants? Background 17 ERA grantees have made over 12.3 million assistance payments to households as of October 2023 and early research demonstrates that ERA had a positive effect on tenants’ housing stability by contributing to a large reduction in historical eviction rates. Additional and longer-term research could examine how diverse ERA grantees structured rental assistance programs in different ways by using various strategies to meet local needs. Some of these strategies may have been more effective for delivering rental assistance quickly, while others may have been more effective for longer term housing stability. Researchers could also examine other dimensions of rental assistance such as how it impacted other measures of well-being such as mental health and financial health. Researchers might also examine the cost-effectiveness of different approaches to delivering ERA assistance such as direct payments to tenants or pairing funds with legal services related to eviction proceedings and maintaining housing stability. Finally, research could examine how impacts varied across beneficiaries, jurisdiction (state, local, territorial, or Tribal), geography (urban or rural), or demographics (ethnicity, language spoken, and other attributes). Additional research questions in this area could include: + Which strategies or program structures were most effective in providing housing stability to tenants? To what extent did combining ERA funds with additional services or eviction diversion measures impact housing stability of tenants in the short and long term? + Which approaches to rental assistance are most cost-effective for the long-term housing stability of tenants? * How did various approaches to rental assistance affect landlords and the overall rental market in the short and long term? + How have ERA funds changed housing stability on Tribal lands or for Tribal members? * To what extent did tenants’ fiscal stability change after they received ERA funds? U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Researchers could examine the effect of ERA on short and long-term housing stability by taking advantage of the reporting data being publicly released by Treasury, detailed information from an ERA grantee, and complementary information from sources such as Census American Community Survey. This research could use a quasi-experimental design to look at geographic variations in the distribution of ERA and compare that to indicators of housing stability. By using detailed beneficiary information and records from credit bureaus, researchers could also conduct a long-term impact evaluation to examine how receiving ERA impacts credit scores, participation in other public benefits programs, and/or unemployment insurance claims. Work Already Underway The Eviction Lab at Princeton University has already completed an analysis which found that “millions of renters avoided the threat of eviction thanks to expanded legal protections and new social safety net programs, many of them enacted as part of the American Rescue Plan.” In addition, the Joint Center for Housing Studies at Harvard University found that ERA was associated with a lower likelihood of being behind on rent, having difficulty meeting expenses, or reporting poor mental health. Next Steps Treasury is collaborating with the Department of Housing and Urban Development and external researchers to evaluate the impact of ERA on housing stability. Funds are going to researchers from three grantees to “produce policy-relevant evidence on the impact of the ERA program on people and communities, with a focus on housing stability and eviction outcomes.” In addition, Treasury encourages researchers and ERA grantees to conduct studies examining how ERA changed tenants’ housing stability and other measures of well-being such as mental health and financial health. U.S. DEPARTMENT OF THE TREASURY 18 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Homeowner Assistance Fund Learning Agenda The Homeowner Assistance Fund (HAF) is designed to prevent mortgage delinquencies and defaults, foreclosures, loss of utilities or home energy services, and displacement of homeowners experiencing financial hardship. HAF provides $9.961 billion for states, the District of Columbia, U.S. territories, Tribes or Tribal entities, and the Department of Hawaiian Home Lands to provide relief for the country’s most vulnerable homeowners. In the immediate term, a key research priority for the Homeowner Assistance Fund is understanding how its funds are flowing to homeowners most in need of assistance and the effect of the HAF review process on the strategies employed by HAF participants to serve homeowners. In the longer term, the program is interested in assessing the outcomes for homeowners that received assistance through the program. Data On a quarterly basis, HAF is collecting data from states, territories, and Tribes including information on obligations and expenditures; the number of homeowners assisted; the number of applications received and approved; the demographics and income levels of homeowners assisted, and other data. Treasury publicly releases this HAF reporting data on a regular basis. In addition to the data Treasury collects about the homeowners that are receiving assistance through HAF, there is a wide variety of additional information available about homeowners. For example, Black Knight tracks data on foreclosure starts and other items related to housing stability of homeowners. In addition, credit bureau information can provide valuable information about mortgage payments and overall fiscal health of homeowners. Furthermore, individual HAF programs may use their own administrative data to work with researchers, for instance to evaluate specific program strategies. Other research efforts through HAF recipient groups, servicer groups, and advocates may be explored as well. Taken together, and in cooperation with a HAF participant government, this data could allow researchers to examine the aggregate effects of HAF on the housing and fiscal health of homeowners. 1) How did HAF’s structure, flexibilities, and state plan review process encourage states to adopt impactful approaches to assist homeowners? A key difference between HAF and previous efforts to assist homeowners during economic downturns is HAF’s flexibility to use a variety of structures to best meet the needs of homeowners in different communities. For example, HAF not only assists homeowners with expenses related to mortgages but also with expenses related to housing, such as homeowner association fees, homeowner’s insurance, and flood insurance. In addition, HAF allowed participating jurisdictions with a variety of options for how to structure homeowner assistance by allowing for both grants and loans, providing options for loss mitigation, and permitting funds to be used for housing counseling and legal services. To ensure jurisdictions were making full use of HAF’s innovative program features, States, territories, and Tribes were required to submit HAF plans for approval by Treasury. To help Background understand the effects of the of HAF’s program structure and plan review process, researchers could explore detailed evaluation questions such as: « Which HAF program elements (housing counseling, legal services, loss mitigation, utility payments, close coordination with loan servicers) were most effective at providing housing stability? * How did different structures for providing HAF assistance to homeowners (such as loans, grants, or other support) lead to different outcomes? * How did HAF’s review of state plans affect which program elements states employed? * How did HAF guidance, Frequently Asked Questions documents, Promising Practices materials, and other resources help jurisdictions to better serve homeowners? 19 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps A potential evaluation in this area could use publicly available data from the state plans posted on Treasury’s website about the structure of each jurisdiction’s HAF program. For example, researchers could particularly examine the degree to which states used innovative approaches to assisting homeowners such as encouraging loss mitigation, providing housing counseling, supplying legal services, or effectively targeting underserved communities. These program elements could be compared with information on the number and demographics of homeowners served by each state, which is publicly available as part of the regular participant reporting that Treasury releases on its website. This would allow for a descriptive evaluation that could potentially examine the degree to which different program features affected the number and characteristics of homeowners that received assistance. As of October 2023, Treasury has approved HAF plans for all 50 states, almost all U.S. territories, and nearly 250 Tribes. Each approved plan is posted to the HAF website, providing researchers and the public with the details of how each HAF recipient is implementing its HAF program. Treasury is also working with states to identify promising practices and effective program design to help HAF recipients maximize the impact of their HAF award funds for the benefit of homeowners. Treasury is interested in exploring a potential medium-term study to examine how HAF recipients are using the innovative programmatic application flexibilities that Treasury encouraged states to adopt as part of the HAF plan review process (such as loss mitigation and housing counseling) and the effects of these strategies in equitably meeting the needs of homeowners. 2) How equitable is the distribution of HAF funds to homeowners most in need of assistance? Background In comparison to previous programs to assist homeowners, HAF is uniquely focused on equity and meeting the needs of low-income homeowners. Specifically, HAF recipients must use at least 60 percent of their HAF award funds for qualified expenses to assist homeowners with incomes at or below 100 percent of Area Median Income (AMI) or less than 100% of the median income for the U.S.. Additionally, any amount not made available to homeowners that meet the income-targeting requirement must be prioritized for assistance to “socially disadvantaged individuals” with the remaining funds. Taken together with program features that permit payments for wide range of expenses (including utilities and homeowners association fees), allow for a variety of supports for homeowners (legal services and housing counseling), and encourage culturally competent outreach strategies, this focus on assisting the most in need homeowners makes addressing barriers to equitable access a key consideration in the program’s implementation. The following more detailed evaluation questions could allow researchers to explore the various equity elements of HAF: + What program characteristics used by jurisdictions had the biggest impact on improving equity? * Which outreach strategies were most effective in terms of reaching underserved communities (including both urban and rural places)? + What was the overall impact of HAF on racial disparities in housing wealth? * How did allowing more housing types (such as manufactured housing) to receive HAF assistance effect the equity of the program? + What were the impacts of Tribal HAF programs and how do they compare to state HAF programs? * Within four key elements of HAF programs—outreach, the application process, HAF program features, and communication with loan servicers—how has equity been a factor into each of these stages? U.S. DEPARTMENT OF THE TREASURY 20 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps Potential evaluation designs focused on the equitable distribution of HAF funds to homeowners most in need of assistance could utilize of number of complementary data sources. Using HAF reporting data on the demographic distribution of HAF funds can provide a high-level sense of the program’s equity impact. To dive more deeply into how the program is serving low-income and socially disadvantaged homeowners, researchers could partner with a state to use data on the distribution of HAF resources across census tracts to perform a descriptive analysis of the characteristics of areas that had larger numbers of HAF beneficiaries, with a particular focus on examining Qualified Census Tracts. Treasury and the Office of Evaluation Sciences at the General Services Administration collaborated to evaluate outreach strategies for reaching low-income homeowners. Specifically, in cooperation with the Idaho Housing and Finance Association, postcards were mailed to homeowners who were at risk of foreclosure. The study was designed to provide rapid and valuable information because print mailer outreach is costly and labor-intensive, and the extent to which this type of outreach is effective at increasing applications in this context is unknown. The evaluation found that postcards did not increase the number of applications to Idaho’s HAF program. These findings saved IHFA approximately $25,000 for costs related to mailing additional postcards, which was redirected to fund English and Spanish language billboards targeting eligible homeowners in hard-to-reach rural areas with above average shares of Hispanic/Latino populations. Treasury is regularly releasing reporting data from states, which includes demographic data on HAF beneficiaries. This data already shows that HAF is more equitably reaching underserved communities than previous homeowner relief programs. However, Treasury is interested in exploring with external researchers how they could use this data in combination with other data sources such as foreclosure data and credit reporting information to do more in depth examination of the equity impact of the program with the possibility to examine which aspects of the program were most successful for increasing assistance to underserved homeowners. 3) How are HAF funds changing the housing and fiscal stability of homeowners? Background 21 The economic effects of the pandemic posed potentially catastrophic effects for homeowners with 2.15 million loans that were seriously delinquent or more than 90-days past due in December 2020, exceeding pre-pandemic levels by 400 percent. This threatened to undo significant progress in homeownership among women, which increased from 50.9 percent to 61.2 percent in the decade preceding the pandemic and further disadvantage Black and Hispanic households, who, as a result of systemic and structural discrimination, have a homeownership rate of 44.4 percent and 48 percent, respectively, compared to 73.7 percent of White households leading into 2020. In response, HAF was principally designed to reach low-income and socially disadvantaged homeowners. To evaluate the effect of the program on the housing and fiscal stability of homeowners, researchers could explore the following evaluation questions: » What was the short-term impact of HAF on homeowners that were in foreclosure? * What are the longer-term effects of HAF on the financial stability of homeowners? * How does the housing stability of HAF beneficiaries after 5 years compare to the housing stability of similarly situated households that did not receive HAF? Did these effects vary across demographic groups? * How did external factors (like interest rates and forbearance) impact the housing outcomes for homeowners that received HAF assistance? * How did forbearance, mortgage payment assistance, and the other major mortgage assistance options provided through HAF improve outcomes for homeowners, either separately or in combination? U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Potential Evaluation Designs Work Already Underway Next Steps A quasi-experimental evaluation examining the distribution of HAF funds to eligible households could assess the impact of HAF assistance on post-award outcomes by comparing HAF recipients to non-recipients. An evaluation of this type would use detailed data from a state, or set of states, that had geographic, income, or other factors that influenced who could get HAF assistance. By looking at foreclosure or credit data for households that received HAF and similar households that did not receive HAF assistance could provide valuable information about the longer-term impact of HAF on housing and fiscal stability. To date, foreclosure statistics illustrate that HAF and related Administration supports for homeowners have been successful in stemming a potential tide of foreclosures due to the pandemic induced economic recession. Overall foreclosure rates during the pandemic remained low relative to prior economic downturns, especially the Great Recession when the Black and Hispanic foreclosure rates rose to 1.1 and 2.4 percent, respectively, which are more than 20 and 80 times the 2020 rates. In addition, between 2019 and 2022, Black, Hispanic, and white homeownership rates rose 2.9, 1.2, and 1.1 percentage points, respectively. Treasury is interested in collaborating with external researchers to identify the effects of HAF on the housing and fiscal stability on homeowners. As noted above, the most promising approaches are likely quasi-experimental designs as states HAF programs are already underway, limiting the ability to randomize which households receive HAF resources. U.S. DEPARTMENT OF THE TREASURY 22 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA State and Local Fiscal Recovery Funds Learning Agenda The State and Local Fiscal Recovery Funds (SLFRF) program provides $350 billion for state, local, territorial, and Tribal governments to fight the pandemic, support families and businesses struggling with its economic impacts, maintain vital public services despite revenue losses, and make investments that support long-term growth and build a stronger, more equitable economy. In the immediate term, a key research priority for the SLFRF program is understanding how to help recipient governments most effectively and equitably utilize and report on their use of SLFRF funds. In the longer term, the program is interested in understanding the impacts of SLFRF funds in specific priority policy areas and on recipient governments’ overall economic indicators. Data The SLFRF program is regularly collecting a robust set of data from recipient governments. This data includes Project and Expenditure Reports, which require recipients to provide detailed information for every SLFRF-funded project, including amounts obligated and spent; a brief overview of the project; target population(s) for the project; whether the project has capital expenditures; and how the project is spending on evidence-based interventions. This information is regularly released on Treasury’s website via the SLFRF Data Dashboard. In addition, the largest recipients (states, territories, and cities and counties with populations over 250,000 residents) are required to submit an annual Recovery Plan Performance Report, which provides additional details on the jurisdiction’s overall strategy for using SLFRF funds, how these funds are advancing equity, how the recipient has conducted community engagement, and the jurisdiction’s approach to using evidence and evaluation in their SLFRF spending. The Recovery Plan Performance Reports must be submitted to Treasury as well as posted on the jurisdiction’s own website. With the Project and Expenditure Reports and the Recovery Plan Performance Reports, a rich set of public data is available for researchers to conduct evaluations of SLFRF funds at the program, recipient, or project level. In addition, many jurisdictions are also collecting additional information about their SLFRF-funded projects, including information on individual program beneficiaries, which when anonymized could be useful for evaluation purposes. 23 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 1) What strategies and supports can improve recipient capacity and program implementation? Background The SLFRF program is a unique funding structure in comparison to the typical structure of other federal programs because it provided direct aid to over 30,000 governments spanning nearly every local, state, Tribal, and territorial government in the country. Due to the diversity of governments that received funds, ranging from small rural villages with hundreds of residents to large cities with populations over one million people, the capacity of recipient governments varied widely. One of Treasury’s priorities for the SLFRF program is identifying strategies and implementing supports to assist governments in making use of their SLFRF funds in an effective, efficient, and equitable manner. As such, additional evaluation questions for research in this area could include: * How did SLFRF funds create new processes, frameworks, strategies, and tools that allowed for better implementation of federal funds? * How did the tools and strategies that recipients used to implement the SLFRF program (such as demographic data analysis and evidence-based interventions) improve the equitable and effective use of funds to benefit communities over the short and long term? * To what extent did community engagement strategies used by recipients provide greater government accountability, incorporate community concerns, and create more transparency in the implementation of SLFRF? + How did implementation strategies vary by type of government, region, population size, recipient capacity and other characteristics? How are SLFRF recipients sharing best practices for program implementation and how can these practices be applied by other local, state, and federal programs? + How did Treasury supports (such as the Treasury Contact Center, office hours, webinars, and other tools) assist recipients with their implementation and compliance activities? Potential Evaluation Designs A potential evaluation design to do research in this area would be to use public data from Treasury’s SLFRF Data Dashboard to examine how recipient governments used evidence-based interventions in their implementation of the program. Specifically, SLFRF’s Project and Expenditure Reports contain a number of fields where recipients report the amount of funds going toward evidence-based interventions for certain expenditure categories as well as whether an evaluation is being conducted. In addition, Results for America has compiled information about how recipients are using evidence- based models in their SLFRF projects. Utilizing Treasury data, along with complementary information from recipients’ Recovery Plan Performance Reports and outside stakeholders, it could be possible to perform mixed-methods research on the role of evidence in recipients’ spending of SLFRF funds. Work Already Underway Treasury and the General Services Administration’s Office of Evaluation Sciences conducted three A-B tests of email communications to SLFRF recipients to better understand what communication content, frequency, and format most improves compliance and the timeliness of program reporting. One study found that including step-by-step instructions at the beginning of the email resulted in a 13 percent increase in the number of recipient governments taking the desired actions. Treasury applied this finding by inserting step-by-step summaries at the top of new emails to recipients of SLFRF funds and other Treasury programs.. Next Steps In collaboration with Treasury, the General Services Administration’s Office of Evaluation Sciences (OES) performed an evaluation that examined the experience of low-capacity recipients with Treasury’s compliance and reporting requirements. The study, which focused on Tribes and smaller local governments (non-entitlement units of local government or NEUs), found that these recipients limited administrative capacity makes them particularly reliant on Treasury’s assistance to effectively use and report on their fiscal recovery funds. Based on these findings, OES and Treasury are continuing to identify methods to further support Tribes and NEUs in their use and reporting of SLFRF award funds. U.S. DEPARTMENT OF THE TREASURY 24 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 2) How are SLFRF funds being distributed in ways that promote an equitable economic recovery? Background Equity is at the heart of Treasury’s economic recovery work. The focus on equity helps to ensure that resources go to communities and households that need the most assistance and also catalyzes broader economic growth that benefits the entire country. The White House report on Advancing Equity through the American Rescue Plan, issued in May 2022, contains a detailed description of how the SLFRF program was structured in order to advance an equitable economic recovery. In addition, Treasury has developed a wide array of resources—including guides and webinars— to assist recipient governments to incorporate equity into their SLFRF spending. Further research can highlight the equity impact of SLFRF funds and the strategies and tools most likely to encourage an equitable use of funds, including questions such as: + How did the SLFRF program structure of providing funds directly to state, local, territorial, and Tribal governments affect equity, impact, and innovation in communities? + How are recipient governments using procedural equity tools (such as equity budgeting tools, social vulnerability indices, increased public transparency, and community engagement) to increase the distributional equity of their SLFRF funds? * How have SLFRF resources improved individual, household level, and community-wide equity metrics in jurisdictions across the country in the short and long-term? Potential Evaluation Designs Using public data from Treasury’s SLFRF Data Dashboard, a potential research design could be a descriptive equity evaluation that examines how different types of recipient governments distribute funds across expenditure categories and among populations impacted and/or disproportionately impacted by the COVID-19 pandemic, including low-income neighborhoods and communities of color. As part of SLFRF’s Project and Expenditure Reports, recipient governments are required to report on which impacted and/or disproportionally impacted populations, such as low-income households and households in Qualified Census Tracts, are served in certain SLFRF-funded projects. Using this data in combination with jurisdictional data on spending in different SLFRF Expenditure Categories, as well as Census data on household level indicators, could allow for research into the relationship between SLFRF funds and equity indicators for communities and households. Work Already Underway Treasury’s report on Equitable Recovery in the United States has important findings related to the role of SLFRF in catalyzing equity in the country’s economic recovery. In combination with the way that Treasury designed the SLFRF program to focus on an equitable recovery, this makes the program a powerful tool for an equitable economic recovery. Researchers have reached similar conclusions, as demonstrated by the study performed by PolicyLink and the Institute on Race, Power and Political Economy at The New School, which found that 90 percent of the cities they examined described a focus on equity in their SLFRF investments. Next Steps Treasury encourages external stakeholders to analyze the equity focus of spending by individual governments as well as different types of governments (metro cities/counties, non-entitlement units of local government, states) through projects such as the scorecards produced by the California Pan- Ethnic Health Network. 25 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 3) What is the short-term and long-term impact on households, organizations, communities, and governments from specific SLFRF projects in priority policy areas such as affordable housing, workforce, and public safety? Background Treasury has identified a small set of priority areas where SLFRF funds can have the most transformative long-term impact on communities. These priority areas include projects focused on affordable housing, workforce, and public safety. Performing additional evaluations in these areas will allow Treasury and researchers to understand how these funds addressed short-term challenges while also paving the way for longer-term economic growth by examining questions such as: * To what extent have recipients used SLFRF resources to complement and increase the impact of funds from other Treasury housing programs such as the Emergency Rental Assistance program and the Homeowner Assistance Fund? * How have SLFRF-funded community violence interventions—such as violence interrupter programs, 911 mental health response units, and youth employment programs—improved public safety and health in communities? * To what extent have workforce development projects helped disproportionally impacted individuals improve their employment opportunities and earnings? + What innovative approaches to housing stability have jurisdictions employed to meet the housing challenges created by the COVID-19 pandemic and increase long-term housing stability? Potential Evaluation Designs Using public data from Treasury’s SLFRF Data Dashboard in combination with data from the U.S. Census, including the American Community Survey, could facilitate research to assess the impact of SLFRF projects on community-level outcomes. For example, in partnership with local governments, researchers could examine the impact of SLFRF funds on key outcomes, such as housing, employment, and public safety, with one recipient government or a collection of recipient governments. This could focus on community-level outcomes or, by partnering with SLFRF recipient governments, it may be possible to examine outcomes for individual beneficiaries. To facilitate potential evaluation collaborations with jurisdictions, researchers could use SLFRF reporting data, which requires that the largest recipients (states, territories, and cities and counties with a population greater than 250,000 residents) designate projects where they are planning to conduct evaluations. Using this data, researchers could identify potential opportunities for evaluation partnerships on specific SLFRF projects within their area of interest or more broadly examine groups of jurisdictions planning evaluations of similar projects. Work Already Underway Treasury has released a wide variety of resources to aid recipient governments to effectively evaluate their SLFRF projects. These include an Equity and Outcomes Resource Guide as well as a series of webinars, which will facilitate governments performing evaluations of their own SLFRF projects. Several outside stakeholders are also providing in-depth technical assistance to help governments evaluate the results of their SLFRF projects. For example, J-PAL North America and Results for America launched the Leveraging Evaluation and Evidence for Equitable Recovery (LEVER) program to help governments rigorously evaluate their own economic recovery programs. A significant number of recipient governments are already doing evaluations of their SLFRF-funded projects to determine their effects on the community. For example, the State of Connecticut is. conducting quasi-experimental research to examine gains for participants in its CareerconneCT program to train workers impacted by the pandemic for in-demand careers such as advanced manufacturing and clean energy. U.S. DEPARTMENT OF THE TREASURY 26 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Next Steps Treasury and the General Services Administration’s Office of Evaluation Sciences are planning a number of long-term evaluations focused on recipient governments’ SLFRF projects. Through partnerships with county governments, these projects will employ experimental and quasi- experimental evaluation methods in combination with diverse local-level datasets to determine the impact of SLFRF investments on individual-level outcomes, with a particular focus on public safety, housing, and economic stability. In addition, Treasury is interested in exploring potential opportunities to conduct a series of medium and long-term evaluations by collaborating with outside researchers and recipient governments to evaluate the impacts of SLFRF projects on households and communities. 4) What is the impact of SLFRF funds on key economic or fiscal indicators for state and local governments? Background The SLFRF program provided key financial support for recipient governments when their budgets were heavily impacted by the economic downturn associated with the COVID-19 pandemic. For example in December 2020, the National League of Cities reported that the majority of cities were experiencing decreased revenues and increased expenses due to the pandemic. In response, the SLFRF program was designed to avoid a repeat of the Great Recession when state and local governments were a drag on the overall economy for over three years as overall unemployment remained high. To that end, the SLFRF program allows recipient governments to use funds to replace lost public sector revenue. Because the economy recovered rapidly following the passage of the American Rescue Plan, it is valuable to study the impact of SLFRF funds on local, state, Tribal, and territorial governments’ finances as well as local and national economic figures though research question such as: * How did SLFRF funds impact revenue, expenditures, and services during the immediate pandemic as well as the longer term? + What was the effect of SLFRF funds on bond issuance, credit ratings, and other measures of governments fiscal stability? * To what degree did SLFRF spending on projects in particular policy areas have effect on local economic conditions? * How did the availability of the revenue loss eligible use category impact key fiscal indicators for state and local governments? Potential Evaluation Designs There is a variety of data available to study the fiscal health of local governments, including information on state and local governments credit ratings, expenditures, revenues, bond issuance, and levels of indebtedness. For example, the National Association of State Budget Officers produces a report on state fiscal conditions and the U.S. Census has annual survey of state and local government finances. In addition, Treasury’s SLFRF data includes information on the level of revenue loss experienced by recipient governments. With these various datasets, researchers could use a quasi-experimental design to estimate the impact of SLFRF funds on key state and local government fiscal, health, and economic outcomes, including revenue vulnerability, municipal bond rates, budgets, and hiring. Work Already Underway Treasury’s Office of Economic Policy and Office of State and Local Finance have provided technical assistance to develop potential evaluation designs for studying the effect of SLFRF funds on fiscal and economic indicators for recipients. Treasury continues to explore the various data components and evaluation structures that would allow for additional research in this area. Next Steps In the long term, Treasury is interested in exploring the illustrative evaluation above, including through partnerships with outside researchers. U.S. DEPARTMENT OF THE TREASURY 27 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA State Small Business Credit Initiative Learning Agenda The nearly $10 billion State Small Business Credit Initiative (SSBCI) provides funding to states, the District of Columbia, territories, and Tribal governments to expand access to capital for small businesses, build ecosystems of opportunity and entrepreneurship, and create high quality jobs. The program emphasizes expanding access to capital to underserved businesses and building financing ecosystems that support entrepreneurs and small businesses. SSBCI’s Capital Program provides capital to jurisdictions to support their small business financing and investment programs. This is complemented by technical assistance (TA) funding, which includes $200 million in formula technical assistance funds to jurisdictions for programs that provide legal, accounting, and financial advisory services to qualifying very small and undeserved small businesses applying for the SSBCI Capital Program or other federal or other jurisdiction small business programs. In addition, $75 million is available for competitive TA grants to jurisdictions, with a more specific focus on business opportunities aligned with other federal legislation such as infrastructure, manufacturing, and clean energy. Technical assistance funding also includes $125 million for a competitive technical assistance grant program that supports early-stage incubators and emerging-business accelerators by providing guidance on business growth and expansion, operated by the Minority Business Development Administration. Jurisdictions that perform well in reaching underserved small businesses with their initial capital allocation may be eligible for additional incentive funds. Given the program’s long lifecycle, the research priorities for the State Small Business Credit Initiative are principally in the medium and long term when it will be possible to identify the effect of the program on capital access and job creation, including for traditionally underserved businesses. Data SSBCI is collecting a variety of data from participating jurisdictions through quarterly and annual reports on their SSBCI deployment and performance. This data includes information on the overall deployment of capital as well as details on individual business transactions such as the demographics of the business owners (race, ethnicity, gender, sexual orientation, Tribal affiliation, and veteran status), the financial performance of the business, and private capital “crowded in” for each transaction. Jurisdictions will also submit data on small businesses that accessed SSBCl-supported technical assistance, including as described above. Available data, paired with qualitative input from stakeholders, will form a foundation of information available for evaluation and will be complemented by survey or interview data. U.S. DEPARTMENT OF THE TREASURY 28 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 1) How does SSBCI technical assistance impact the ability of underserved businesses to access funds? Background Potential Evaluation Designs Work Already Underway Next Steps 29 Achieving access to capital for underserved businesses can be dependent on a small business’ ability to prepare, apply, and qualify for the right financing. In many cases, small businesses need technical assistance to be ready to receive financing and subsequently unlock the benefits of small business ownership. For example, a company that does not have the right financial documentation, a business plan, accounting capacity, or adequate legal structure is unlikely to receive a loan from the conventional market. Treasury’s SSBCI Technical Assistance program provides funding for jurisdictions’ programs that offer legal, accounting, and financial advisory services to small businesses applying or preparing to apply for SSBCI or other public small business financing programs. To evaluate the impact of this technical assistance, additional research questions in this area include: * To what extent is there a relationship between the type of technical assistance services that an underserved small business receives and its success? » Are some types of technical assistance providers more effective in reaching underserved businesses? * Are technical assistance providers that are embedded in or representative of underserved communities more effective in helping businesses with similar demographics to access funds? * To what extent was technical assistance successful for businesses that took advantage of resources from different SSBCI TA program formats? Using SSBCI reporting data from jurisdictions about business demographics and information from SSBCI technical assistance providers could allow for an evaluation that examines the implementation of technical assistance for underserved businesses. For example, a descriptive comparison of SSBCI technical assistance could examine: a. how businesses that apply for capital support with help from SSBCI technical assistance differ from those that apply without that support; b. whether some types of technical assistance providers connect to more businesses and if providers embedded in or representative of communities are more effective in helping underserved businesses access funds. SSBCI has made available $200 million for grants to states, the District of Columbia, territories, and Tribal governments that are participating in or applied for the SSBCI capital program, to provide technical assistance to qualifying underserved entrepreneurs and very small businesses with fewer than ten employees. In addition, SSBCI has transferred $125 million to the Minority Business Development Agency for its Capital Readiness Program, which will fund a network of 43 providers offering services to small businesses across the country. Eligible service models within the Capital Readiness Program include early-stage technical assistance/incubators and emerging-business technical assistance/accelerators to provide guidance on growth and expansion. In October 2023, Treasury announced a competitive notice of funding opportunity (NOFO) for a further $75 million, open to jurisdictions that are participating in the SSBCI Capital Program. Through this program, Treasury will support jurisdictions’ programs that propose innovative and high-impact models for delivering technical assistance in the areas of legal, accounting, and financial advisory services to underserved businesses and very small businesses. This funding is in addition to technical assistance that will be provided by the $200 million already allocated to SSBCI recipient governments and the $125 million for Minority Business Development Agency technical assistance. Taken together, these technical assistance resources will provide rich data sources for evaluations into various technical assistance models for reaching underserved small businesses. In addition, in collaboration with GSA’s Office of Evaluation Sciences, Treasury is exploring potential evaluations that focus on Tribal governments’ experiences applying for and accessing SSBCI funds, including the role of technical assistance in enabling Tribal governments’ access to funding. U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 2) To what extent does capital from SSBCI create jobs, increase capital access, and provide other measurable community and economic development benefits? Background Potential Evaluation Designs Work Already Underway Next Steps Public small business financing programs often measure jobs created or retained as a standard measure of small business community and economic impact. SSBCI is interested in understanding the program’s impact by not only this measure, but also understanding this impact using a range of measures and metrics such as examining how small business ownership impacts community economic development and household financial stability. As such, additional evaluation questions for research in this area could include: * To what extent has SSBCI funding allowed companies to grow (i.e. expand market, increase product lines, increase revenue, bring on new clients, or other)? + What role does SSBCI play in small business job creation? * To what extent is SSBCI capital “catalytic” in helping to attract private capital? * How has SSBCI funding impacted local or regional economies (i.e. additional vibrancy, provision of crucial goods or services, contributed to development of an industry cluster)? A potential quasi-experimental evaluation of the SSCBI program on business outcomes could determine the direct effects of SSBCI funding on program outcomes of interest, including the number of employees, median wages, revenue, and other economic outcomes. This evaluation design could be implemented in jurisdictions in which there is a policy threshold that determines whether businesses get access to an SSBCI loan. This evaluation would require business outcome data for those above and below the loan threshold. In addition, qualitative analysis could be performed through a survey instrument to a sample of small business recipients on use of funds and any growth by the company. Interviews with state program managers, partner organizations, and participating lenders/investors could provide insights into how jurisdiction-specific SSBCI programming contributed to particular local or regional outcomes. As part of its reporting guidance for SSBCI, Treasury is collecting information about the number of jobs and economic health of businesses that receive funding. In addition, Treasury is exploring other sources of data that could provide valuable information about SSBCI businesses. In order to build on the data it is already receiving through SSBCI reporting, Treasury is interested in collaborating with external researchers to examine the degree to which capital from SSBCI creates jobs, increases capital access, and provides other measurable benefits. This could examine impacts of SSBCI at a state, regional, or national level. U.S. DEPARTMENT OF THE TREASURY 30 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 3) To what extent did SSBCI strengthen the resilience and growth of recipient businesses and in particular, minority-owned, women-owned, and otherwise underserved businesses? Background Potential Evaluation Designs Work Already Underway Next Steps 31 SSBCI’s authorizing statute and program guidance outline a dedicated focus on small businesses in underserved communities - businesses that may have historically faced barriers to engaging in the competitive market for a range of reasons, from geography, veteran status, racial, sexual orientation, or gender identity bias, or other factors. Treasury’s statue also provides for incentive funding for jurisdictions that perform well in reaching underserved businesses, and for technical assistance that supports small businesses in underserved communities applying for SSBCI and other public small business programs. For this reason, Treasury is interested in understanding the extent to which SSBCI funding helped underserved small business owners to stabilize and grow their businesses by examining questions such as: * How many underserved businesses received funds through SSBCI and were there any trends by business type, region, or business owner demographics? * How do the outcomes for underserved businesses that received SSBCI support differ from national trends for underserved small businesses that did not receive an SSBCl-supported investment? + What are the characteristics of underserved business that were most successful following the receipt of SSBCI supported funds? * Of small businesses that self-certify as underserved or are located in a CDFI Investment Area, what were the average outcomes in terms of business growth following the SSBCl-supported transaction? How do these outcomes compare to small businesses that received an SSBCI- supported transaction but do not self-certify as underserved? Using data collected as part of the SSBCI reporting process on the demographics of businesses receiving SSBCI supported funds, as well as information on financial performance of those businesses, it is possible to examine how SSBCI affected the resilience and growth of underserved businesses. A potential research design would be a quasi-experimental evaluation of the SSBC program on business outcomes to determine the direct effects of SSBCI funding on program outcomes of interest, including the number of employees, median wages, revenue, and other economic outcomes. This may guide future small business financing program design for state or federal programs. As part of reporting to Treasury, SSBCI recipients collect information from businesses that receive SSBCl-supported loans, which will help facilitate this kind of study of the relationship between demographic or other characteristics and outcomes. This also includes the collection of data on race, ethnicity, sexual orientation, and gender identity from businesses that receive SSBCI capital. This data will facilitate measuring the impact of SSBCI capital on the communities and businesses that are most in need of resources. Additional information about demographic and equity aspects of the SSBCI is available in the White House report on Advancing Equity through the American Rescue Plan, issued in May 2022. Treasury is potentially interested in partnering with an external organization to conduct a long- term study of the demographic characteristics and outcomes for SSBCl-supported businesses. U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 4) SSBCI has six common program formats, does one particular program format better reach small businesses, including small businesses in underserved communities? Background Potential Evaluation Designs Work Already Underway Next Steps Treasury has outlined six common program formats that jurisdictions use in their portfolio of SSBCI programs: loan guarantee program; loan participation program; collateral support program; capital access program; a direct investment venture capital program; and venture capital program that invests in other venture capital funds. Each of these program formats offer slightly different types of assistance to underserved small businesses. Understanding the value each program format brings to small businesses and how they can play a role in meeting a jurisdiction’s overall SSBCI strategy goals would help to inform SSBCI and future small business programs. Research questions such as the following could help provide additional information about the effects of the six program formats: * Which SSBCI program format has been most effective in reaching underserved small businesses? + What are the implications, if any, for future state or federal small business financing programs based on the impact of each of the six program formats? * How can each of the six program formats play a complementary role in helping jurisdictions best use their SSBCI funds to meet the unique needs of small businesses in their community? SSBCI reporting data provides information on the number, type, demographics, and performance of small businesses that received assistance under each of the six SSBCI program formats. Using this information as part of a quasi-experimental research design would produce findings about which program formats yielded the highest relative value to underserved businesses of different types. In addition, this quasi-experimental design could be complemented with qualitative research focused on SSBCI recipients, jurisdictions, technical assistance providers, and community stakeholders. Treasury is regularly engaging with jurisdictions and reviewing data to understand how various program types are working. Comparing learnings from jurisdictions, complemented by program data, could allow evaluation of how each program format, in aggregate, reached underserved small businesses. Treasury will continue to monitor progress across jurisdictions within each program format and reference learnings for future evaluations, including any program best practices that can inform future state or federal programming for small business support. Treasury would welcome the opportunity to collaborate with an evaluation partner to perform analysis of program impact data, complemented with qualitative data from jurisdictions, to better understand how these programs expanded access to capital to businesses in underserved communities. U.S. DEPARTMENT OF THE TREASURY 32 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 5) How did jurisdictions use SSBCI funds to support emerging venture capital ecosystems, and to what extent were those efforts successful? Background Potential Evaluation Designs Work Already Underway Next Steps 33 Over 39% (up to $3 billion) of all SSBCI transactions by dollar amount are expected to be deployed via venture capital programs. Jurisdictions have opted to run SSBCl-supported equity/venture capital programs for many reasons - including a policy goal of growing a diverse, small, nascent, or emerging venture capital market in their jurisdiction. Within SSBCl-supported venture capital programs, jurisdictions design programs that either make direct investments in small businesses or invest in venture capital funds that then invest SSBCI capital in portfolio companies. Both of these venture capital structures require that 1) private investment accompany the SSBCI funds; and 2) that the jurisdiction demonstrate that SSBCI investment was catalytic in crowding-in private capital. Potential areas for research in this area include: + Was there new venture capital activity in the jurisdiction as a result of SSBCl-supported venture capital programming? * To what extent was SSBCI funding able to support emerging fund managers, increasing equitable representation in this competitive space? Using industry standard data sets, it could be possible to estimate a three-year average of venture capital investment activity prior to the launch of SSBCI venture capital program in target geographies. Comparing this information to the adjusted average data in the ten years following release of SSBCI venture capital programming could help in understanding the role of SSBCI in venture capital. In addition, a qualitative analysis could review the extent to which jurisdictions prioritized working with emerging fund managers in deploying programs. Treasury is partnering with jurisdictions to support program implementation — including in providing dedicated technical assistance to jurisdictions managing venture capital programs in historically anemic markets, and in supporting jurisdictions in working with venture capital funds - including emerging fund managers. Treasury is also monitoring the extent to which jurisdictions are working with emerging fund managers. Treasury will continue to offer technical assistance and support for jurisdictions working to nurture growing and emerging markets, including support for those contracting with emerging fund managers. U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA Appendix: Overarching ORP Learning Agenda ORP has a set of overarching evaluation questions that are complemented by program specific evaluation questions as outlined above. This appendix includes a taxonomy that crosswalks how program specific evaluation questions tie back to the broader ORP-wide evaluation priorities. This illustrates how research in support of these overarching ORP questions would help policymakers understand the broader landscape of the United States economic recovery efforts. 1) How did ORP funds stimulate an equitable economic recovery? 2) How were funds from Treasury programs braided together with each other or with funds from other federal programs to support equitable economic growth? 3) Where did ORP funds go and what are the characteristics of the households, organizations, communities, and governments who received support from ORP programs? Emergency Capital Investment Program #1: After receiving ECIP funds, what are the initial patterns of investment and institutional changes across different types of financial institutions? Capital Projects Fund #3: How are CPF investments in multi-purpose community facilities supporting digital equity? Emergency Rental Assistance #1: How equitable is the distribution of ERA funds to tenants most in need of assistance? Homeowner Assistance Fund #2: How equitable is the distribution of HAF funds to homeowners most in need of assistance? State and Local Fiscal Recovery Funds #2: How are SLFRF funds being distributed in ways that promote an equitable economic recovery? State Small Business Credit Initiative #2: To what extent does capital from SSBCI create jobs, increase capital access, and provide other measurable community and economic development benefits? 4) How equitable was the impact of recovery funds on Americans most in need of assistance? Capital Projects Fund #1: How are recipients using CPF funds to close gaps in broadband access? Capital Projects Fund #2: What approaches are states using to address broadband affordability? Emergency Rental Assistance #4: How did ERA change tenants’ housing stability and other measures of well-being? Homeowner Assistance Fund #3: How are HAF funds changing the housing and fiscal stability of homeowners? State Small Business Credit Initiative #3: To what extent did SSBCI strengthen the resilience and growth of recipient businesses and in particular, minority-owned, women-owned, and otherwise underserved businesses? 5) How did Treasury’s program design choices affect who received support from ORP programs? Emergency Rental Assistance #2: How has the use of promising practices that Treasury encouraged grantees to adopt affected the equitable distribution of ERA funds and the housing stability of tenants? Homeowner Assistance Fund #1: How did HAF’s structure, flexibilities, and state plan review process encourage states to adopt impactful approaches to assist homeowners? State and Local Fiscal Recovery Funds #1: What strategies and supports can improve recipient capacity and program implementation? State Small Business Credit Initiative #4: SSBCI has six common program formats, does one particular program format better reach underserved small businesses? U.S. DEPARTMENT OF THE TREASURY 34 OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA 6) What have we learned from ORP that can be applied to the development of future economic recovery programs? « Emergency Capital Investment Program #2: Over the longer term, how have the Emergency Capital Investment Program, Rapid Response Program, and Equitable Recovery Program changed the capacity of the participating institutions and what broader implications might that have for sector and field level approaches to serving low- and moderate-income communities and populations? « Emergency Rental Assistance #3: What have we learned from ERA about the development of a national eviction prevention and housing stability infrastructure? « State and Local Fiscal Recovery Funds #3: What is the short-term and long-term impact on households, organizations, communities, and governments from specific SLFRF projects in priority policy areas such as affordable housing, workforce, and public safety? « State and Local Fiscal Recovery Funds #4: What is the impact of SLFRF funds on key economic or fiscal indicators for state and local governments? « State Small Business Credit Initiative #1: How does SSBCI technical assistance impact the ability of underserved businesses to access funds? « State Small Business Credit Initiative #5: How did jurisdictions use SSBCI funds to support emerging venture capital ecosystems, and to what extent were those efforts successful? 35 U.S. DEPARTMENT OF THE TREASURY OFFICE OF RECOVERY PROGRAMS: ECONOMIC RECOVERY LEARNING AGENDA
File and source
- File
- TREAS-ORP-Learning-Agenda-Final.pdf
- Size
- 8,049,511 bytes
- SHA-256
- 14e7c93ffd6ffe91f1aaede711321d52274b171d27ca2855e4629d19eda3be19
- Original
- home.treasury.gov