Development of Logic Model & Performance Measures AHEC GRANTEE PRESENTATION APRIL 14, 2011 HRSA/ Division of Workforce & Performance Management

Steps in BHPr Performance Process Develop Program Logic Models(Feb. 15 – March 31) Develop Cluster Measures and Tables(March 1 – April 29) Program and Grantee Reviews of Measures and Tables(April May 16) Develop Draft Grantee Guidance (May 1 – 15) Review and Revision to Guidance(May 16 – 30) Obtain Input on Guidance from Grantees(May 1– 30) Finalize Guidance(May 30) Prepare Final OMB Package(June 1 – July 15) Obtain Approval from Advisory Groups(November 1 – 30)

OMB Categories Quantity- supply Quality- competency, knowledge gained Diversity – racial/ethnic diversity Distribution – placement in underserved areas

Program Description and Outcomes Program level Activity Level Individual level

Elements of the proposed individual-level data collection PII and contact information – collected only for individuals targeted for longitudinal study and only at graduation/program completion (MD/Residency, PA, APRN(PC NP?), Diversity program participants in structured programs, RN?, faculty development?) Grant-based unique ID (organization-based unique-id; i.e., by university?) SSN Name Current Address Expected Address in one year (if different) Phone numbers (home, cell) , facebook, Name and phone number for someone who will know where you can be contacted

Elements of the proposed individual-level data collection Data that rarely changes – collected on first contact/”enrollment” Grant-based Unique ID Birthdate Gender Race (check all that apply) Ethnicity Parent’s income (broad categories) Address where individual grew up (prior to 21 st birth date) (“rural”)

Elements of the proposed individual- level data collection Annual data – activities – one record for each defined activity Grant-based Unique ID Activity (defined at the program/cluster level and specifics reported at the grant level) Activity-related individual/immediate outcome Levels of activities Year-long training without separate activities – e.g., year 1 in medical or nursing school Structured, time-limited activities – e.g., clinical rotations, CE courses, structured activities in diversity programs, etc.

Elements of the proposed individual- level data collection Activity categories/data Clinical rotations – focus, location/type of facility, length, contact hours, etc. Post-baccalaureate program Saturday Academy Formal mentoring Etc.

Program Name: Area Health Education Centers Program Need(s): There is a shortage of high quality primary health care to meet growing demand in the U.S. Goal(s): Train a greater number of competent health care providers to better meet the growing demand for primary health care. K EY S TRATEGIES O UTPUTS S HORT -T ERM O UTCOMES I NTERMEDIATE O UTCOMES L ONG -T ERM O UTCOMES Provide primary care training to medical and other health professions students. (c)(1)(B) (c)(1)(C) (c)(1)(F) # and type of health professions students placed in community- based primary care rotations. Total # of rotation sites % rotation sites located in: -Rural areas -Federally designated HPSAs and MUAs -Facilities serving a significant proportion of vulnerable patients in non Federally designated areas Increased number of prepared health professions students (those successfully completing community- based primary care rotations). Increased number health professions program graduates who provide primary care within one year after completing training, including in rural/ underserved areas. Increased number and diversity of health training program graduates who provide primary care within five years after completing training, including in rural/ underserved areas.

Provide continuing education (CE) on key primary care topics (determined by national priorities and/or local needs assessments) to health professionals, particularly those providing primary care in underserved areas. (c)(1)(E) (c)(1)(F) # and type of health professionals (by discipline) receiving CE. # of CE courses offered by topic area. % of CE offered in- person and via the Web. Employment location of health professionals participating in CE activities. # of partners/ collaborators for CE program offerings Findings from pre/post knowledge tests. Increased number of prepared health professionals (those successfully earning CE credits). Increased knowledge of health professionals on key CE topics (as determined from pre- post knowledge tests). Increased number of health professionals who integrate CE course content into their clinical practice (within one year after completing training). Increased number of clinical practice changes as a result of health professional’s integration of CE course content into clinical practice (within a five-year period). K EY S TRATEGIES O UTPUTS S HORT -T ERM O UTCOMES I NTERMEDIATE O UTCOMES L ONG -T ERM O UTCOMES

Provide health careers outreach to youth in grades K-12, including careers in public health. Provide health careers outreach to displaced workers or individuals/ adult learners from underrepresented minority populations or from disadvantaged or rural backgrounds. (c)(1)(A) (c)(1)(G) (c)(1)(F) - # and type of activities conducted to raise awareness of health careers among K-12 youth, including public health. - # and type of partner organizations sponsoring health career awareness activities - # of K-12 youth reached by health career awareness activities - % of K-12 youth reached who are URM, disadvantaged, and live in rural areas - # and type of activities conducted to raise awareness of health careers among displaced workers or adult learners - # of partnerships with DOL/WIB entities Increased number and diversity of K-12 youth that are aware of and intend to pursue health careers training programs. Increased number of displaced workers or adult learners receiving health careers outreach and training through partnerships with DOL/WIB entities. Increased number and diversity of K-12 youth that are prepared and qualified to enter health professions training programs, including training programs in public health. Increased number of displaced workers or adult learners who obtain entry level jobs in the health professions. Increased number and diversity of K-12 youth applying to and being accepted at health professions training programs. Increased number of displaced workers or adult learners who establish careers in the health professions. K EY S TRATEGIES O UTPUTS S HORT -T ERM O UTCOMES I NTERMEDIATE O UTCOMES L ONG -T ERM O UTCOMES

Provide interdisciplinary/ interprofessional education and training opportunities to medical and other health professional students, and practicing health professionals. Develop partnerships with community-based, academic, and healthcare workforce entities that promote interdisciplinary approaches to primary care. (c)(1)(D) (c)(1)(F) Total # of rotation or training sites offering interdisciplinary/ interprofessional education. # of disciplines participating in interdisciplinary /interprofessional education or training. % of rotation sites offering IPE located in rural areas. # of interdisciplinary/ interprofessional CE offerings. Increased number of rotation or training sites offering interdisciplinary/ interprofessional education and/or training. Increased number of interdisciplinary/ interprofessional CE offerings. Increased number of community-based, academic, and healthcare workforce partnerships promoting interdisciplinary approaches to primary care. Increased number of health professions program graduates that are trained in interdisciplinary/ interprofessional education. Increased number of practicing health professionals that receive interdisciplinary /interprofessional CE. Increased number of health professionals integrating interdisciplinary/ interprofessional education into clinical practice. Increased number of interdisciplinary teams delivering primary care, including in rural/underserved areas. K EY S TRATEGIES O UTPUTS S HORT -T ERM O UTCOMES I NTERMEDIATE O UTCOMES L ONG -T ERM O UTCOMES

Breakout Groups Best means to describe program activities and outcomes What is missing? Categorize major elements Brief summary report-out

Get to Work!

Proposed Common Quality Measures – The number of program participants demonstrating PC competencies – Proportion of BHPr supported trainees who receive training in medically underserved communities. – Proportion of participants who receive a portion of their clinical training in primary care. – Proportion of participants receiving training in PC focus areas – Proportion of participants with increased knowledge gain at the end of CE as reflected in pre-post testing scores – Proportion of participants receiving multiple modes of PC activities – Overall retention of participants in programs

Proposed Common Quantity Measures – The number and percent of participants in career development / career enhancement/career advancement programs – The number and percent of program participants completing training who indicate their intent to practice as a HP – The number and percent of program participants completing training who indicate their intent to practice in primary care – The number and percent of program participants completing training who indicate their intent to practice in underserved areas. – The number of CE offerings per topic/mode of training – The number of new trainees/slots/units

Proposed Common Diversity Measures – The number/type/proportion of graduates/completers who are URM and/or disadvantaged. – The number/type/percent of URM and disadvantaged participants/faculty – # and % of URM accepted into HP training program – Increased retention rate of URM in BHPR programs – Increased retention rate of URM/faculty in HP school – The number and percent of URM and disadvantaged participants who indicate their intent to work in primary care and/or underserved areas. – The number and percent of URM and disadvantaged participants receiving training in primary care and/or underserved areas.

Other Proposed Common Measures/Reporting Distribution – Proportion of BHPR supported HP who enter practice in underserved areas Infrastructure – # participants completing faculty development training – # PC AAU Progress Report (describe accomplishments) – Describe evaluation activities – Explain how partnerships/leveraging activities have influenced how you conduct training activities (e.g. curriculum, enrollment, placements, etc.)? – Educational innovations (e.g. Innovative curricula) – Best practices – Dissemination of knowledge/strategies