Planning, Using, and Adapting County Data Systems CalMHSA PEI TTACB Work Group March 5, 2014 Facilitated by RAND and SRI Planning, Using, and Adapting County Data Systems CalMHSA PEI TTACB Work Group March 5, 2014 Facilitated by RAND and SRI The California Mental Health Services Authority (CalMHSA) is an organization of county governments working to improve mental health outcomes for individuals, families and communities. Prevention and Early Intervention programs implemented by CalMHSA are funded by counties through the voter-approved Mental Health Services Act (Prop 63). Prop. 63 provides the funding and framework needed to expand mental health services to previously underserved populations and all of Californias diverse communities

© Copyright 2014 RAND Corporation Todays work group is part of RANDs CalMHSA TTACB project Collaboration between CA counties, community-based service providers, CalMHSA, SRI, and RAND Provide support for PEI implementation throughout CA Collaboration between CA counties, community-based service providers, CalMHSA, SRI, and RAND Provide support for PEI implementation throughout CA Today we will focus on electronic data systems being used or planned by counties for capturing PEI process and outcomes information. 2

© Copyright 2014 RAND Corporation Agenda 3 Welcome and introductions Key components of a PEI outcomes data system Presentations from counties using different data systems Los Angeles, Modoc, Monterey & Placer Counties Counties sharing experiences – Session 1 and 2 Break out groups around different platforms Adapting data systems to changing requirements Welcome and introductions Key components of a PEI outcomes data system Presentations from counties using different data systems Los Angeles, Modoc, Monterey & Placer Counties Counties sharing experiences – Session 1 and 2 Break out groups around different platforms Adapting data systems to changing requirements

© Copyright 2014 RAND Corporation Agenda 4 Welcome and introductions Key components of a PEI outcomes data system Presentations from counties using different data systems Los Angeles, Modoc, Monterey & Placer Counties Lunch – catch up with colleagues Counties sharing experiences – Session 1 and 2 Break out groups around different platforms Adapting data systems to changing requirements Welcome and introductions Key components of a PEI outcomes data system Presentations from counties using different data systems Los Angeles, Modoc, Monterey & Placer Counties Lunch – catch up with colleagues Counties sharing experiences – Session 1 and 2 Break out groups around different platforms Adapting data systems to changing requirements

© Copyright 2014 RAND Corporation Functionality required should dictate the components to incorporate Some functions require similar components while others require unique components Sometimes you cannot have both Some functions require real-time data, some do not Some functions may be done well enough already or be too costly to even consider A team of users and experts can best address this question Functionality required should dictate the components to incorporate Some functions require similar components while others require unique components Sometimes you cannot have both Some functions require real-time data, some do not Some functions may be done well enough already or be too costly to even consider A team of users and experts can best address this question What do you want a PEI outcomes data system to do? 5

© Copyright 2014 RAND Corporation Compliance reporting and contract management Clinical supervision and clinician training Client-counselor service planning and progress review Workflow monitoring and billing and case management Performance management Program continuous quality improvement (CQI) Local/statewide evaluation and/or research support Compliance reporting and contract management Clinical supervision and clinician training Client-counselor service planning and progress review Workflow monitoring and billing and case management Performance management Program continuous quality improvement (CQI) Local/statewide evaluation and/or research support Possible functions of outcome data systems 6

© Copyright 2014 RAND Corporation Todays audience includes users of at least 16 different software systems Presentations from four counties and breakout groups will provide opportunities for sharing experiences Vendors and users of other systems in your county can also be consulted Demonstrations are another way to assess options Todays audience includes users of at least 16 different software systems Presentations from four counties and breakout groups will provide opportunities for sharing experiences Vendors and users of other systems in your county can also be consulted Demonstrations are another way to assess options Selecting a system to meet your needs and goals 7

© Copyright 2014 RAND Corporation Consider what it would take to adapt or replicate a system to meet your needs Does it make sense to build your own or purchase? Assess capacity within your county providers and administrators to run and support a system Capacity is more than funding to build and install a system Provider, clinician, agency staff skills e.g., for data entry, data processing, data analysis, interpretation and reporting Technology (hardware and IT support) Data systems already in place Consider what it would take to adapt or replicate a system to meet your needs Does it make sense to build your own or purchase? Assess capacity within your county providers and administrators to run and support a system Capacity is more than funding to build and install a system Provider, clinician, agency staff skills e.g., for data entry, data processing, data analysis, interpretation and reporting Technology (hardware and IT support) Data systems already in place Assess fit and capacity within your county 8

© Copyright 2014 RAND Corporation Key components of a PEI outcomes data system 9

© Copyright 2014 RAND Corporation The infrastructure is the foundation and organizational structure and support needed for data system operation The interfaces are the users who enter and receive the data The content refers to the data elements the system captures The infrastructure is the foundation and organizational structure and support needed for data system operation The interfaces are the users who enter and receive the data The content refers to the data elements the system captures Key components of a data system 10

© Copyright 2014 RAND Corporation Platform selection Integration capacity Security levels System administration –Expertise –Users support Adaptability Platform selection Integration capacity Security levels System administration –Expertise –Users support Adaptability PEI outcomes data system infrastructure 11

© Copyright 2014 RAND Corporation Desktop or web-based platform selection options Security and access are key considerations Counties here today use MS Excel MS Access Anasazi or Avatar EHRs Existing county server based systems Systems custom built/adapted for MHSA/PEI outcomes Desktop or web-based platform selection options Security and access are key considerations Counties here today use MS Excel MS Access Anasazi or Avatar EHRs Existing county server based systems Systems custom built/adapted for MHSA/PEI outcomes Infrastructure – platform selection 12

© Copyright 2014 RAND Corporation Potential integration capacities include: Billing EHR/EMR Other MHSA/BH/county agencies/community providers data systems Contract compliance reports Primary care Potential integration capacities include: Billing EHR/EMR Other MHSA/BH/county agencies/community providers data systems Contract compliance reports Primary care Infrastructure – integration capacity 13

© Copyright 2014 RAND Corporation Protects client privacy and compliance with federal HIPAA regulations and other relevant MediCal and state regulations With individual data access will need to be strictly controlled Levels of access can vary with type of user and limit access to private information MOUs across agencies can articulate data sharing and privacy provisions Need to consider internet, application, and physical security Protects client privacy and compliance with federal HIPAA regulations and other relevant MediCal and state regulations With individual data access will need to be strictly controlled Levels of access can vary with type of user and limit access to private information MOUs across agencies can articulate data sharing and privacy provisions Need to consider internet, application, and physical security Infrastructure – security 14

© Copyright 2014 RAND Corporation System administration requires expertise in: Information technology (IT) Auditing Data processing Data analysis Report interpretation Users support may include: Training and technical assistance Help desks for error resolution and new queries Site visits to review audits Q&A User support groups System administration requires expertise in: Information technology (IT) Auditing Data processing Data analysis Report interpretation Users support may include: Training and technical assistance Help desks for error resolution and new queries Site visits to review audits Q&A User support groups Infrastructure – system administration 15

© Copyright 2014 RAND Corporation Deserves consideration at the start of a system design to understand what changes are possible and what are their costs The addition of a new program can mean the need to add new outcome measures Changing reporting requirements can mean the addition of data elements and data linkages Customization by role promotes user friendliness Screen customization for data entry and reports Data query and ad hoc report customization can facilitate analysis and utility of the data Deserves consideration at the start of a system design to understand what changes are possible and what are their costs The addition of a new program can mean the need to add new outcome measures Changing reporting requirements can mean the addition of data elements and data linkages Customization by role promotes user friendliness Screen customization for data entry and reports Data query and ad hoc report customization can facilitate analysis and utility of the data Infrastructure – adaptability 16

© Copyright 2014 RAND Corporation Key components of a data system 17 Infrastructure Interfaces Content Infrastructure Interfaces Content

© Copyright 2014 RAND Corporation Data Input users Direct data entry (individual and batch) Data import to ease burden of data entry Data quality can be built in (notifications, automated checks, audits) Role-based interfaces are user friendly Front desk Clinician or service provider and supervisors QA Billing IT Management Data Input users Direct data entry (individual and batch) Data import to ease burden of data entry Data quality can be built in (notifications, automated checks, audits) Role-based interfaces are user friendly Front desk Clinician or service provider and supervisors QA Billing IT Management PEI outcomes data system interfaces 18

© Copyright 2014 RAND Corporation Data output users Data exports Standard reports for roles and for audits, e.g., missing data reports Individual participant Aggregate program, provider, site Annual community report Custom reports that users can design and modify easily Data output users Data exports Standard reports for roles and for audits, e.g., missing data reports Individual participant Aggregate program, provider, site Annual community report Custom reports that users can design and modify easily System interfaces – output users 19

© Copyright 2014 RAND Corporation Key components of a data system 20 Infrastructure Interfaces Content Infrastructure Interfaces Content

© Copyright 2014 RAND Corporation Levels Individual Clients, families, clinicians System Providers, programs, counties Community Population and target populations Structure and process information Outcomes information Levels Individual Clients, families, clinicians System Providers, programs, counties Community Population and target populations Structure and process information Outcomes information PEI outcomes data system content 21

© Copyright 2014 RAND Corporation A Logic Model for Understanding the Impact of Prevention and Early Intervention (PEI) Funding PEI Funding Community Planning Process Identified needs Target populations Are there public health benefits? Reduced suicide Mental-health related Prolonged suffering Incarceration Homelessness School drop out Foster care Unemployment Differences across groups Where is it going? (Programs) New and enhanced prevention resources Public awareness campaigns Training and education of gatekeepers Education and counseling programs Community strengthening programs New and enhanced treatment resources Treatment of mild- moderate mental illness Treatment for new onset SMI Increased collaboration and coordination among agencies System change efforts What is it doing? (Program activities) More and better prevention Exposure to social marketing efforts More teachers trained More parents receiving coping skills training More and better early intervention Access/utilization of first-break early intervention programs Use of school-based counseling services Does it make a difference? (Short and long- term outcomes) Changed knowledge, behaviors and attitudes Increased general knowledge about mental illness Increased identification of at-risk clients Increased help-seeking Decreased risk behaviors Improved resilience and emotional well- being Increased social connections Decreased psychological distress Increased family functioning Improved school performance 22

© Copyright 2014 RAND Corporation Information about clients or program participants Unique identifiers for unduplicated counts Individual and family, program, counselor links Client characteristics such as target population status Demographics for reporting requirements, QI and evaluation Client services utilization/attendance/participation Program status and service duration From start to completion or other disposition Collateral services/systems Referrals and transfers to and from other services Information about clients or program participants Unique identifiers for unduplicated counts Individual and family, program, counselor links Client characteristics such as target population status Demographics for reporting requirements, QI and evaluation Client services utilization/attendance/participation Program status and service duration From start to completion or other disposition Collateral services/systems Referrals and transfers to and from other services PEI structure and process information 23

© Copyright 2014 RAND Corporation Benchmark against goals and objectives of treatment plan or program standard Assess symptoms/ dx./ baseline score on standard measure at outset of program (pre) Repeat data collection to measure progress/change over time (post) Core measures (across programs, agencies, counties) Custom measures (specific to a program) Benchmark against goals and objectives of treatment plan or program standard Assess symptoms/ dx./ baseline score on standard measure at outset of program (pre) Repeat data collection to measure progress/change over time (post) Core measures (across programs, agencies, counties) Custom measures (specific to a program) PEI outcomes information 24 Did services received make a difference?

© Copyright 2014 RAND Corporation Thank You! 25