CAPUS: Formative Evaluation Provides a Snapshot on Early Progress

Timothy Harrison, Office of HIV/AIDS and Infectious Disease Policy Formative evaluations are evaluations that are generally conducted during the developmental and early implementation phases of large-scale, multiyear projects in order to improve program design or performance. These evaluations can play an invaluable role in HIV programming because they allow us to identify and bolster successful activities and to revise and modify program elements that aren’t working as well as expected.

With that in mind, the HHS Office of HIV/AIDS and Infectious Disease Policy (OHAIDP) contracted with GEARS, Inc. to conduct a formative evaluation of the Care and Prevention of HIV in the United States (CAPUS) project, a 3-year demonstration project to reduce HIV-related morbidity and mortality among racial and ethnic minorities by addressing social, economic, and structural factors influencing HIV health outcomes. Supported by the Secretary’s Minority AIDS Initiative Fund, CAPUS is led by a multi-agency federal partnership. CDC serves as the lead federal agency and works closely with Office of the Assistant Secretary of Health (OHAIDP, Office of Minority Health, and Office on Women’s Health), the Health Resources and Services Administration (both the HIV/AIDS Bureau and the Bureau of Primary Health Care), and the Substance Abuse and Mental Health Services Administration. Eight state health departments bearing disproportionately high burdens of HIV disease among racial and ethnic minority populations were selected as grantees. (Read more in this blog post.)

The formative evaluation examined the first year of CAPUS implementation (FY2013) and sought answers to two key questions: (1) What is the level of Federal inter-agency partnership supporting the development and implementation of the CAPUS Demonstration Project? and (2) What is the capacity of funded grantees to develop, implement, and evaluate the project?

The evaluation report provided important insights on the benefits of inter-agency partnerships and noted that the technical support Federal partners provided to grantees during the review of their initial work plans and during preliminary site visits helped the grantees to improve the coordination of their HIV/AIDS services and ultimately improved the quality of CAPUS-related activities.

However, the assessment also detailed some challenges to optimal federal collaboration around CAPUS, including a lack of shared goals and priorities by all partners and a lack of clearly defined roles and responsibilities. The evaluation also noted that grantees sometimes received mixed messages on how and when to involve community partners in the development of their work plans, and that a long work plan-approval process delayed the release of funding to grantees and may have slowed their hiring processes.

In terms of grantee capacity, the evaluation highlighted several new and promising activities among the grantees, including: collecting and entering HIV testing data in outpatient clinic settings; developing and implementing a resource hub to provide HIV information and resources to the general public, as well as to provide information for primary care providers on how to care for HIV patients; establishing HIV data-sharing agreements and protocols with Medicaid and other external partners; using geo-mapping to identify geographic areas and populations with a high HIV burden; and partnering with a pharmacy chain to provide HIV testing.

The evaluation also noted some grantee challenges, including a lack of state policies to facilitate data-sharing across partners; lack of capacity on the part of some community-based agencies to conduct HIV testing and to link infected clients to HIV care and retain/re-engage them in care; and, in some jurisdictions, limited access to trained technical staff, (e.g., epidemiologists, IT specialists, and database programmers) to support implementation of project activities.

Among the recommendations for moving forward, the GEARS evaluation highlighted four action steps that Federal partners could take to improve program implementation: (1) supporting peer-to-peer/grantee-to-grantee technical assistance; (2) supporting grantees’ data-sharing policies and practices; (3) supporting grantees’ IT infrastructure and data-management capacities; and (4) supporting grantees as they explore ways to implement planned CAPUS activities within the established timeframe.

These findings inform the ongoing implementation of CAPUS activities as we continue to work to improve the health of racial and ethnic minorities living with HIV in disproportionately affected jurisdictions and to identify practical, workable solutions to address social and economic conditions and structural barriers to HIV testing, prevention, treatment, and care.