Using Data for Job Retention

I came across an article on “Using Data to Document the Benefit the Social Work Services” that was so excellent I had to share it with you. The article is written by Nic Dibble, an education consultant with the Wisconsin Department of Public Instruction.

In the article, Nic writes:

As school districts’ budgets become tighter, accountability of educational programs and services has become even more imperative. Pupil services professionals have traditionally been able to provide adequate process or formative evaluation data (e.g., how many students were seen, how many evaluations and support groups were conducted, how many home visits were made), but little is generally done to measure the impact of these services. This is in large part due to the difficulty, time, and resources involved in traditional outcome evaluation of prevention and early intervention services.

Newspapers commonly report the results of state-mandated, standardized tests of area school districts, often in ways that facilitate comparison with each other. Consequently, academic achievement and performance on these tests may become a community’s primary “yard stick” for its schools. School boards may in turn place greater value on instruction and services they perceive enhance academic achievement when considering staff reductions or additions.

As a result, school social workers need to document positive outcomes related to the provision of their work. The challenge is to locally design a simple, valid evaluation system that addresses the priorities of the school district while not consuming inordinate amounts of time and resources.

The article outlines the following guidelines for designing an evaluation of school social work services:

1. Identify the priorities of your school district and design an outcome evaluation system that will demonstrate how your services contribute to at least one of those priorities.
2. Identify what state or federal mandates your services meet.
3. Calculate the revenues your services generate for your school district.
4. Determine what data is already collected and available. Use that whenever possible.
5. Design your outcome evaluation system to match the scope of your school social work services.
6. When designing your outcome evaluation system, be sure to involve others in the school-community who will be involved in providing and gathering the data.
7. Make sure the variables you decide to evaluate are the correct one (i.e. the variables will accurately reflect changes in the behaviors, knowledge, and/or skills the school social work service is designed to impact.
8. Whenever possible, gather data on multiple variables from multiple sources.
9. Do not try to evaluate everything you do. Rather, select at least one program or service to evaluate and do it well.
10. Keep the KIS rule in mind: Keep It Simple. Your outcome evaluation system should be a) easy to understand and implement, and b) based upon common sense evident to people outside pupil services and education.
11. Make sure your evaluation does not treat groups differently.
12. Make sure the instrument(s) used, if any, will accurately measure the targeted behaviors, knowledge, and/or skills.
13. Share your evaluation with the administration, the school board, and the community at least annually.
14. Use the evaluation data to improve your school social work services.

Choosing a Variable or Service to Measure

The process that follows is limited to the planning necessary to identify 1) critical data indicators, which are readily available or easily obtainable, and are of value to the important decision-makers in school districts, and 2) how this data can be used to support the effectiveness of school social work services.

The first four narrative steps of this process described below are represented by the corresponding numbers in Figure 1. The intersection of all three circles identifies the data collected by a school district that represents progress on one or more of the school district’s goals, and reflects the positive impact of at least some portion of school social work services.

 

1. Who is your audience? What stakeholders are you trying to influence? To whom will you present the findings of your outcome evaluation? What does your audience value? What established priorities or goals does your audience have?
2. Which one or more of your audience’s established priorities or goals do your school social work services tangibly and significantly impact?
3. What data does your school system presently collect that can help document progress on your audience’s priorities and goals? What data is available to you? What data is easily obtainable?
4. Which of the identified data are appropriate indicators of the success or progress of one or more of your school social work services?
5. Which of the identified data indicators from Step #4 that a) address one or more of your audience’s priorities and b) are significantly and tangibly impacted by one or more of your services, will you use and analyze?

The complete article contains further guidance regarding the previous guidelines and a template to help record the outcome evaluation plans can be access here:
https://sspw.dpi.wi.gov/files/sspw/pdf/sswpgbenefitdata.pdf

After planning an evaluation, you may discover that you need to find an instrument to use to measure the effectiveness of the intervention or service. I’ve compiled a list of Measures for Screening, Assessing, and Evaluating Practice that you may find useful.