Overview of the Study

The HIV/AIDS Bureau, Special Projects of National Significance (SPNS) Program of the U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA) funded or co-funded 108 national demonstration projects which started between 1991 and 1997 for the purposes of developing, evaluating, and disseminating new service models for HIV/AIDS. In 2003, the cumulative outcomes and impact of these projects were studied. At the time studied, for the initial groups of projects, about nine years had elapsed since the projects ended, while for the more recent grants, two years had elapsed since project completion. The time intervals both during and after the two-, three-, or five-year grant periods were studied. This study represents a cross-cutting evaluation of the outcomes achieved by these programs during their funding periods, the sustainability of the project after the funding period ended, and the larger impact of this national program of HIV/AIDS services demonstration projects.

The following map shows the distribution of the projects throughout the United States. Grantees are geographically diverse and serve clients from a broad range of areas. A total of 39 states and Puerto Rico received grants within at least 1 of the 5 cohorts included in this study. This map is coded to illustrate the number of grants received by agencies within each state for each of the five cohorts. States are shaded to show the number of total SPNS grants received by agencies in the state across the five cohorts; darker states received a more grants than lighter ones. Note that for several grants there were large service sites in more than one state, and in that case all states receiving significant levels of services are shown.

Various sources of data were used to assess program outcomes and impact. Project materials – including final reports from grantees, supplemental progress reports, appendices to grant reports, and other related technical materials – were scanned and converted to electronic text using optical character recognition programs. Publications from projects were identified and abstracts downloaded from several online bibliographic sources. A search of the Internet was conducted to find web sites maintained by the funded project (or its parent agency) and relevant web pages were downloaded. Finally, at least one individual from each project was invited to participate in a semi-structured interview to further identify the outcomes and impact of the project; computer readable transcripts were prepared from tapes of the interviews. Of the total set of 108 grants, it was possible to find a final report or proxy from 100 projects, conduct an interview with at least one key staff member for 90 projects, extract information from a web site for 96 projects, and locate at least one published article from 56 projects. Key outcome information in the form of data from an interview or web site was available for 102 projects. At least one source of data was available for 106 of the 108 grants.

The following chart shows the information available from the alternate sources for the 108 projects.

A semi-automated keyword-based coding system of 41 domains of program characteristics, outcomes, and impact was developed and assays were performed in the text database in order to identify and code paragraphs in which the text appeared. The coding scheme includes the following domains. There were multiple subcategories in most of the domains; the number of subcategories or individual indicators for each domain is indicated in parentheses. A total of 427 automated subcategories were coded for the 108 projects. After the semi-automated keyword-based coding was completed, all codings were inspected and verified to ensure that the coded section matched the intended construct.

	Domain of Outcome	Subcategories
	Medical Patient Services	16
	Psychosocial Client Services	22
	Service Enhancements	12
	Provider Types on Staff or Closely Linked	23
	Referral Network	23
	Outreach Strategies	8
	Training on HIV	13
	Provider Types Trained	23
	Infrastructure Development Activities	9
	Professional Development Activities	4
	Research/Evaluation Capacity	8
	Development of Enhanced Models	6
	Policy for HIV Services	5
	Population Targeted Services	1
	Deal with Diversity	12
	Specific Accomplishments	11
	Other Dissemination	8
	Staff and Students Migrate to Other Providers	5
	Awards and Recognitions	3
	Formal or Informal Linkages	5
	Leveraging of SPNS Designation	17
	Collaborations with Ryan White CARE Act Titles	7
	Changes in Recruitment	2
	Underserved Groups Addressed	28
	Underserved Groups BY Any Barrier Mentioned	28
	Empowerment	1
	Client Connections with Staff and Agency	1
	Types of Models	8
	Evaluation Methods	20
	Lessons Learned	4
	Outcomes	10
	Achievements	5
	Adherence	2
	Managed Care Issues	8
	Legal Advocacy	3
	End of Life Care	4
	Full Continuum of Care	1
	Cost of Care	1
	Problems and Barrier Identified and Addressed	26
	Miscellaneous	11
	Generic Provider Types	23

Over their funding periods, virtually all of the national demonstration projects developed new models for services and provided these enhanced services to a large number of clients. In the coding of 427 attributes of the model programs, it was found that large percentages of the programs had developed integrated care models, targeted historically underserved groups (as defined by ethnical-racial group, gender, child-rearing status, sexual orientation minority status, a history of substance abuse, homelessness, low income, mental illness, or low levels of acculturation as well as many different combinations of these factors), provided culturally-appropriate and culturally-sensitive services, experimented with innovative program elements, and disseminated their results using a number of methods including traditional journal publication, training other providers in their service areas, conducting local and national workshops, and developing web sites.

The programs developed as Special Projects of National Significance continued to provide a services backbone within their service delivery areas after the initial funding periods ended. It was found that 92% of the 108 grantees continued their programs for a significant period of time – defined here as at least a year – beyond the SPNS-funded period, while 87% of the projects were continued in an identifiable and sustained form through the time key staff were interviewed in late 2003. Relatively clear evidence was found that fully 50% of the 108 projects existed in late 2003 with either expanded capacity to serve a larger number of persons living with HIV/AIDS or provided services enhanced beyond those provided during the SPNS-funded period. The overall rates of continuation and expansions of services provided under SPNS funding across all projects studied are summarized in the following chart.

Looking at sustainability issues for each of the grantee cohorts in late 2003, it was found that 78% of the Cohort I grantees (whose programs ended in 1994) were still providing identifiable services similar to those the SPNS Program funded, 78% of the Cohort II grantees ending in 1996 were still active, 100% of the Cohort III grantees ending in 1998 were still active; 93% of the Cohort IV grantees ending in 1999 were still active, and 89% of the Cohort V grantees ending in 2001 were still active. Sustainability figures by grantee cohort for continued and expanded programs are summarized in the next chart.

The next chart summarizes the percentage of grantees studied that had received subsequent funding from HRSA. Through 2003, the SPNS grantees had received funding from a number of HRSA programs (30% from Ryan White CARE Act Title I, 23% from Title II, 19% from Title III, 19% from Title IV, 39% from SPNS in subsequent initiatives, 7% from the AIDS Education and Training Centers (AETC) program, 69% from at least one HRSA Ryan White Care Act funding source).

The SPNS grantees have also received funding from a number of non-HRSA programs (19% from the Substance Abuse and Mental Health Services Administration (SAMHSA), 19% from the Centers for Disease Control and Prevention (CDC), 9% from the National Institutes of Health (NIH), 13% from the Housing Opportunities for Persons with AIDS (HOPWA) program of the Department of Housing and Urban Development, and 51% from other unspecified sources). The percentage of grantees studied with subsequent funding from other non-HRSA sources is shown in the next chart.

Although national service demonstration models are sometimes criticized for developing innovative and enriched models of service delivery without considering whether the services are “practical” and sustainable, it was clearly found in this cross-cutting study that the vast majority of all service providers funded by these public HIV/AIDS service funds continued to provide services to underserved, poor, and disabled populations long after the end of their original HRSA Special Projects of National Significance grants, while at the same time enhancing the efforts of other HIV/AIDS service providers in their local service areas.

There were many examples of activities, outcomes, and lessons learned during and after SPNS funding that were provided by grantees. Examples were reported in the areas of medical services, psychosocial services, service enhancements, outreach strategies, training on HIV, infrastructure developments, professional development, research and evaluation capacity, model development, policy for HIV services, population targeted services, strategies for dealing with diversity, specific accomplishments, dissemination, migration of staff and students to other providers, awards and recognitions, formal and informal linkages, leveraging of the SPNS designation, collaborations with Ryan White CARE Act Titles, changes in recruitment, underserved groups addressed, empowerment, client connections with staff and agency, types of models pursued, evaluation methods, lessons learned, outcomes, achievements, adherence, managed care issues, legal advocacy, end of life care, continuum of care, cost of care, problems and barriers, and provider types.

Click here for a table that shows selected exemplars of the activities and outcomes achieved by these grantees. Most of the examples are taken from interviews conducted with key project stakeholders in late 2003. The examples are slightly modified quotes, edited to translate conventional language to a more formal written form, and to remove information (such as names or unique populations served) that would tend to identify specific projects. All quotes largely retain the intent and flavor of the original language of the person interviewed.

Exemplars of Activities, Outcomes and Lessons Learned During and After the SPNS Funding Period

The following recommendation are derived from the data considered in the study that examined the long-term impacts on HIV/AIDS services among 108 grantees funded between 1991 and 2002 as Special Projects of National Significance by the Health Resources and Services Administration. As described throughout the report, there have been a number of lasting and significant changes to these organizations that can be largely attributed to their experience as HRSA SPNS grantees.

G. J. Huba, Ph.D.
The Measurement Group
5811A Uplander Way
Culver City, CA 90230
ghuba@TheMeasurementGroup.com