Click here to see the workplan.
Click here for the recommendations.
Click here for the MBOA survey.
Click below for the Establishing Outcomes for Care report.
Establishing Outcomes for Care
Vision
To improve the delivery of services to children and adults with mental illness through the use of a coordinated statewide evaluation system across public and private payors and all providers.
Current Situation
Payers, licensors, levels of government, and others often require mental health service providers and consumers to complete various evaluation, satisfaction, or outcome forms in order to assess progress and quality. The instruments and reporting processes are sometimes duplicative and cumbersome. Training on the administration, use, and interpretation of the instruments is not provided, limiting the ability of providers to use these data to improve their services. Similarly, the data are not compiled, analyzed, or reported at the state level, limiting their usefulness to guide service system enhancements or policy development. Thus, providers use precious time in duplicative efforts that are not used to evaluate direct services or the system as a whole. Consumers cannot access information on the effectiveness of individual providers or the system as a whole. Policy makers cannot make informed decisions.
Solutions
In order to produce useful data, a common evaluation framework should be adopted. A common framework can be used to simplify existing evaluation requirements, identify the strengths and weaknesses of the current service delivery system, guide enhancement of service delivery, inform policy initiatives related to mental health services, and provide increased access to information for consumers and advocates. To achieve this solution, the committee proposes recommendations in the areas of administration of the evaluation system, components to be included in the evaluation design, and strategies for reporting and utilizing the statewide data.
Recommendations for administration
· All evaluation data proposed in this report should be collected and maintained by the Minnesota Department of Human Services, Chemical and Mental Health Services Administration (hereafter referred to as “the State”). The State should work with providers and both public and private sector consumers to create an integrated statewide outcomes database and reporting system for standardized instruments and consumer satisfaction. Once this database is functional, outcomes information could be provided in many domains to many stakeholders and all evaluation data could be sent to the State.
· Financial disincentives for evaluation should be removed by having public and private payors cover the cost of administration (should be addressed by the fiscal committee).
· If outcome tools are not in the public domain, the State should try to negotiate a reduced cost for the tool to be used statewide or payors should provide compensation for the cost of the instruments.
· Training should be available for providers regarding the interpretation and use of required assessments.
· The State should develop a recommended timeframe for the administration of assessments (i.e., at intake and periodically thereafter).
· The State should review the choice of evaluation tools every two years and consider other options as appropriate.
Recommendations for evaluation components
There are four domains in which outcome measurements are needed:
· Access: People should have access to mental health care when and where they need it.
· Appropriateness: People should be treated appropriately with a focus on safety and recovery.
· Effectiveness: People should feel better, exhibit improved symptoms, and function better in the community.
· Efficiency and Equity: People should be treated fairly and funds should not be wasted.
The evaluation design should include measures of treatment outcomes (including consumer’s degree of mental health symptoms and level of functioning), consumer satisfaction of services, and service accessibility. The outcomes in all of these domains need to be measured by the following participants in the mental health delivery system: Consumers/Families, Providers, Payers (Private and Public).
Outcomes: providing information on the effectiveness of services and improved outcomes
· Outcome assessments must measure two key areas, level of functioning and symptomology, and be appropriate for use at key points (such as admission, discharge, and key transition points).
· Outcome assessments would be self-administered where feasible, designed for repeated measures, of modest cost, sensitive to cultural differences, and appropriate for the population (i.e., developmentally appropriate, acceptable reading levels, etc.).
· For both child/youth and adult services, the recommended set of evaluation instruments should be used as a replacement for current requirements of public and private payors, rather than as an addition.
· Separate outcome assessments should be used for children/youth and for adults:
v For adult services: The Behavior and Symptom Identification Scale -32 (BASIS 32: Eisen, 1988) should be used to assess outcomes of services for all consumers except those participating in evidence-based practices for depression and psychosis. Outcomes for these services will be assessed using the Patient Health Questionnaire -9 (PHQ-9: Kroenke, Spitzer, & Williams, 2001) (depression) and the Psychosis Severity Scale (psychosis).
v For child/youth services: the following two instruments are recommended for pilot testing: The Strengths and Difficulties Questionnaire (SDQ: Goodman, 1997) (to measure changes in the degree of symptoms over time) and Child and Adolescent Level of Care Utilization System (CALOCUS: American Academy of Child and Adolescent Psychiatry) (to measure changes in functionality over time). Following the pilot testing period, the State should make recommendations regarding the adoption of these tools.
Current requirements may delay adoption of new evaluation measures. For those required to do so, other measures (namely the Child Behavior Checklist and the Child and Adolescent Functional Assessment Scale) could still be used. A transition plan should be built into Children and Communities Services Act (CCSA) evaluation requirements, including provision of a waiver of existing requirements for providers participating in the pilot testing.
Consumer Satisfaction: providing information on overall satisfaction with the MH system
· The consumer satisfaction survey process currently used by the State should be expanded to include consumers served by private payers.
· For both child/youth and adult services, the recommended satisfaction instruments should replace current satisfaction survey requirements of public and private payors, rather than as an addition. Providers should also be encouraged to use this survey.
· The State should adapt the surveys to ensure that they include questions regarding ease of access into the system, knowledge of where to access services, access to consumer education/support, and access to family support/education.
· The State should review current administration procedures for these surveys and consider strategies for increasing the response rate.
Access to Care: measuring access to services
· The satisfaction surveys administered by the State should include items addressing consumers’ perceptions of service accessibility.
· Because not everyone who needs services successfully enters the mental health system, the following strategies should also be used to measure/evaluate accessibility:
v A “secret shopper” survey should be conducted on an ongoing basis by an independent, consumer-friendly agency under contract with the State. Using standard sample scenarios, individuals should call service providers to obtain quantitative information about the availability of services and waiting times and qualitative information about potential barriers to service. The secret shoppers should explore access to all services covered in the model service set, especially psychiatric services (including psychiatric beds), county case management, acute services, and day treatment services for children/youth.
v Penetration rates should be calculated on an annual basis to estimate the percentage of individuals in the population with mental health issues who have successfully accessed services.
v Periodic surveys should be conducted with representatives of other service systems that may serve as entry points for mental health services (such as criminal justice, primary care, schools, and child welfare) to explore potential accessibility issues.
Recommendations for data management and reporting: providing information on the statewide mental health system of care
An online data management system should be developed that allows providers to enter data directly into the system.
A graduated access system should be used so that individuals have specific levels of access to information in the online system. Providers should have access to the data that they originally entered into the system, others (advocates, consumers, etc.) should only have access to aggregate information.
To ensure confidentiality of consumer information and to promote reliable conclusions, data should be available at the aggregate provider level only when the sample size is statistically sufficient.
In addition to outcome, access, and satisfaction data described above, the online system should be designed to incorporate other data that are currently tracked and monitored by providers and other agencies. These data should include the capacity of the service system (including the number of culturally specific providers), employment rates, criminal justice and juvenile justice system data, homelessness data, education data (such as drop out and truancy rates), key facility data (such as rate of falls, medication errors, and restraint usage), and other mental health service information (such as suicides, readmission to acute psychiatric services within 30 days of discharge, etc.).
To help advocates and consumers draw informed conclusions, reports should be available summarizing data by key variables such as diagnosis, consumer demographics, and geographic region. Strategies should be developed to provide “severity adjustments” to explain variation in results that may be expected due to variation in the nature or severity of the mental health issues addressed.
For those providers demonstrating poorer results (outcomes, access, or satisfaction), technical assistance opportunities should be provided by the State.
On an annual basis, the State should submit a report summarizing all available data to the Commission of Human Services, the State Mental Health Advisory Council, and the Chairs of the Health and Human Services policy and budget committees.
This report should also be made available to advocates and others for use in recommending policy changes and strategies for service enhancements.
Thanks to Committee Members:
This committee met July 22, August 11, September 2, September 23, October 7, October 13, and October 26.
Sue Abderholden
Consumer and family advocate
Cheryl Hosley
Children’s mental health evaluator/researcher
Jim Baxter
County representative
Trisha Beuhring
University of Minnesota
Shelley Brandl
Children’s mental health provider
Louise Brown
Children’s mental health advocate
Amy Dolin
Children’s mental health advocate
Glenace Edwall
Minnesota Department of Human Services
Joel Hetler
County representative
Ruth Knapp
Minnesota Department of Human Services
Gary Mager
Minnesota Department of Human Services
Pat Nygaard
Jerry Pederson
County representative
Wendy Rea
Consumer advocate
Virginia Selleck
Minnesota Department of Human Services
Richard Sethre
BHP, representing several insurance companies
Nancy Houlton
County representative
John Dinsmore
County representative
Tom Steinmetz
Children’s mental health provider
Micheal Trangle, MD
Psychiatrist, Health Partners Medical Group and Regions Hospital