Updated: 30 October 2004 09:24 PM
©BCFPI Inc. 2004
Outcomes / satisfaction / measurement and benchmarking (at discharge and on follow-up) determine how case mental health and functioning changes between the beginning and end of service and then, upon follow-up (6, 12, 24 months). Were clients satisfied with the service? How do the outcomes compare to the best-known, manualized, evidence-based interventions for cases with similar problems (benchmarking)?
This section includes guidelines about conduct and analysis of outcomes projects, a blank template for posting outcomes projects and progress reports on this website, a directory of posted and pending projects, and a link to a forum where these projects can be discussed.
Guidelines for Assessing Service Outcomes
- Identify group(s) of interest; typically cases attending /receiving a specific program or intervention.
- Create this program in your BCFPI data base, register cases attending the program (see the Software Manual - "Entering Program Enrolments", "Program Maintenance").
- Use the discharge form to gather and enter discharge Mental Health and satisfaction data as cases are discharged from the program; encourage these cases to participate in the follow-up process, 6 months hence .
- Mail out 6 month follow-up forms; phone cases which don't return form and attempt phone follow-up interviews.
- Continue this process for 40 - 60 successive discharges,
- Strive to obtain data on 80% of this sample.
- Analyze data and apply results.
Outcomes Data Analysis
The interim reports section of most of our Outcomes Projects (below and pending) provide examples of such a reports, for several programs, showing varying success for different programs and client groups.
- Run a Benchmarking report (BCFPI report, pending as of June 2004) to compare outcomes for a specific problem with outcomes reported in evidence-based citations, for that same problem. This will determine how the effectiveness of the project intervention compared to that of well-established evidence-based interventions, for the selected problems. (Until this report is available, we will assist sites to obtain the same data using the report described above, and then manually cross-referencing this with the Database of Evidence-based interventions.
Agencies are encouraged to use the Blank Outcomes Project Template to post their project for sharing with colleagues and discussion the outcomes forum.
|HWC&AS Brief Intervention Results
||(o1) Parent discharge data analyzed for 101 cases (~36% of referrals) and 6 month Follow-up for 46 cases(16%). Brief service referrals were less severe than referrals for routine clinical services; discharge effect sizes were comparable to evidence-based benchmarks; 6 month averages showed continued or stable average improvement for all domains; cases were very satisfied with Brief Service.
|Lutherwood-KidsLink Intense In-home
||(o2) 2 stage outcomes project regarding Intense In-Home services. Stage 1 (data included) is exploratory, using a convenience sample of referral and discharge profiles, gathered between January 2001 and August 2003. For this stage 1, 164 'Before' profiles are compared to 95 'After' profiles. Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004.
||July 18, 2004
||(o3) 2 stage outcomes project regarding Residential Services for teens. Stage 1 (data included) is exploratory, using a convenience sample of referral and discharge profiles, gathered between January 2001 and August 2003. For this stage 1, 55 'Before' profiles are compared to 11 'After' profiles Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004.
||July 18, 2004
|Lutherwood Day Treatment
||(o4) 2 stage outcomes project regarding Day Treatment for teens. Stage 1 (data included) is exploratory, using a convenience sample of referral and discharge profiles, gathered between December 2000 and June 2003. For this stage 1, 27 'Before' profiles are compared to 13 'After' profiles Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004.
||July 18, 2004
|Lutherwood Open Custody
||(o5) 2 stage outcomes project regarding Open Custody. Stage 1 (data included) is an exploratory, using a convenience sample of referral and discharge profiles, gathered between January 2001 and August 2003. For this stage 1, 21 'Before' profiles are compared to 14 'After' profiles. Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004.
||July 18, 2004
|KidsLINK/NDSA Residential Treatment for Kids from Own Homes
||(o6) 2 stage outcomes project regarding Residential treatment for pre-teens. Stage 1 is an exploratory, using a convenience sample of referral and discharge profiles, gathered between December 2000 and October 2003. Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004. (Residential treatment, pre-teens, kidsLINK, convenience sample, successive referrals, discharge, follow-up, Ontario, Waterloo).
||Oct 2, 2004
|KidsLINK/NDSA Residential Treatment for Permanent Wards
||(o7) 2 stage outcomes project regarding Residential treatment for pre-teens living in long-term foster care. Stage 1 is an exploratory, using a convenience sample of referral and discharge profiles, gathered between September 2001 and July 2003. Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting June 1 2004. (Residential pre-teens, foster, kidsLINK, convenience sample, successive referrals, discharge, follow-up, Ontario, Waterloo).
||Oct 2, 2004
|KidsLINK/NDSA Day Treatment for Pre-teens
||(o8) 2 stage outcomes project regarding Kidslink Day Treatment for pre-teens. Stage 1 is exploratory, using a convenience sample of referral and discharge profiles, gathered between November 2000 and October 2003. Stage 2 will use referral, discharge and 6 month follow-up data for 40 - 60 successive referrals, starting Nov. 2003. Current interim report includes very positive outcomes and large effect sizes on first 11 stage 2 discharges.
||Nov 20, 2004
|Outcome and Satisfaction for 1 - 9 Session Consults as Initial Response to New Referrals
||(o9) Parent reported outcomes and satisfaction for 1 to 9 session eclectic Brief Intervention offered to referrals entering agency's (St. Claire, Sarnia) service system as a first service response. 53 of 69 discharges occurring during 2005 (77% sample), from Brief intervention.
||May 23, 2006
|MST and Therapeutic process
||(o10) Multi-site (Ontario) study seeking to determine the specific therapeutic processes which underlie successful MST interventions. Will involve a total of approximately 100 cases from 5 Ontario CMH providers. Measures of adolescent emotional and behavioural problems will be collected at pre-, mid-, post-treatment and one-year follow-up; improvements on these measures are expected to be related to the quality of the therapeutic alliance. Maternal reports of depression and attitudes about therapy will also be examined.
|Outcomes of Cope program provided to community families
||(o11) This outcome project used the Brief Child and Family Phone Interview to determine: (1) who enrols in universally advertised, community based COPE parenting programs, (2) who successfully completes COPE parenting programs, and (3) what is the outcome of COPE parenting programs for parents of children with highly oppositional behavior (BCFPI Cooperation Scores greater than or equally to 70).
||Feb 16, 2005
|Group and intense in-home intervention to residential treatment
||(o12) Grimes program: Parent (Cope, Barkley) groups and child groups, intense in-home services and crises intervention for high-need children and their families. This pilot alternative to residence is expected to serve up to 100 children per year who would otherwise be candidates for residence. Most of the program resources are re-allocated from a residential program, previously serving 10 children per year. Ottawa; Crossroads CFC.
||Oct 30, 2004
|Uptake, Satisfaction, Outcomes and Productivity for Systematic Triage to Large Groups (COPE), Brief Therapy, Clinic Treatment and Community Supports
(o13) Brant Central Intake systematically triages cases into 1 or more defined service streams based on Intake profiles.
This project examines client uptake of this broad triaging approach; client satisfaction and outcomes at discharge; and outcomes at 6 and 12 month follow-up. Its case-handling capacity will be compared to previous approaches.
Then, satisfaction and outcome results will be grouped by the services to which the cases were referred, to obtain information re the uptake satisfaction and effectiveness of these specific service components.
The hope is that systematic triage, large groups and brief consults will increase system capacity, and that all aspects of the system will achieve high rates of client satisfaction, and outcomes comparable to benchmark outcomes for each of the CMH problems addressed by the system.
|Sep 30, 2005
|Reduced Waiting List, High satisfaction, and Large Effect Sizes Associated with Strategic Triaging
(o13a) This update to the Brant Project shows a sharply decreased waiting list, very high levels of satisfaction, and large effect sizes, as of March 31 2006. Details about Brant's triaging strategy associated with these improvements, are provided in Brant Project o13.
||Jun 14, 2006
|BCFPI Profiles at 3 Successive 12 Month Intervals After Admission to a Swedish CMH Clinic
||(o14) BCFPI profiles at 12 months after admission to an eclectic (Family systems and analytic) Swedish CMH are compared to 'Before' profiles. BCFPI satisfaction data is also tabulated. Current update includes 12 month data on a 90% sample of 56 successive admissions, and shows high effect sizes and mixed satisfaction.
||May 20, 2005
|Symptom Trajectory and Outcomes of Children and Youth Treated re Sexual Misconduct
||(o15) BCFPI follow-up surveys will be administered at the end of each phase of the 'STOP' treatment program, upon discharge from Niagara CYS, and at 6 and 12 month follow up. This will assess symptom course as each stage of treatment is completed; satisfaction at discharge, and will benchmark outcome effect size at discharge, 6 and 12 month follow-up. BCFPI results will be compared with more detailed STOP assessments to explore the cost and utility of each for service management. This will involve 60 successive STOP discharges from each of Niagara CYS's child and youth STOP programs.
||Dec 08, 2004
|Benchmarked Discharge Effect Size Results for Intensive Services
||(o16) Effect sizes for Residence, Day Treatment and Intense-In home services compares favorably to benchmarks in this overview of initial results for successive discharges (further interim results for projects o2, o6, o8 in BCFPI outcomes projects).
||Mar. 28, 2005
|Skills training for caregivers of Selectively Mute children
(o17) Outcome evaluation of a skills-training group program for parents and professionals supporting children with Selective Mutism (SM).
Selective mutism is viewed as an anxiety-based condition characterized by a child's failure to speak in select situations (e.g., to teachers at school), while speaking comfortably in other settings (e.g., to family members at home).
|Sep 30, 2005
|Wrap-Around to Return kids to Own Communities
||(o18) A 50 case pilot before-after study re use of Wrap around to repatriate kids to home community. Wrap-around fidelity, Mental health, Functioning, Parent stress, quality of life and service utilization measures will be used at intake, 3, 6 and 9 months.
||Sep 30, 2005
Pending Outcomes Projects
- 1 session Solution-focused Brief intervention for low severity referrals
- 3 session Brief Interventions for Urgent Referrals
- Urgent Cases presenting at Hospital Emergency room
- Urgent cases presenting at Community Agency
- Uptake and Impact of Assistance Accessing Recreational Services for Needy Families
- Diversion Programs for Early Delinquents
- Teens and pre-teens with sexual misconduct
- Alternative to residence: Intense in-home and groups
See our complete listing of Outcomes Resources
BCFPI users can collaborate on the outcomes forum about running, comparing, and applying the results of such projects.
|^ Top ^