Facts In Action

Making it Count is a series of articles designed to help you develop ways to measure outcomes in your program or family child care home. If you would like to receive earlier issues of Making it Count, please contact Erika Argersinger at (617) 695-0700 x271, or by email at eargersinger@associatedearlycareandeducation.org.

Over the last seven installments of Making it Count, we have discussed why

you should consider measuring outcomes in your program and the steps to take to develop an outcome measurement system: assembling a working group, choosing outcomes and indicators, developing a plan for collecting data, and analyzing and reporting on your findings. By following these steps, you will have developed and implemented a plan for collecting outcomes information.

After you have implemented your outcome measurement system, it is important to monitor and review your system periodically to identify what is working well and how the system can be improved. Questions for the working group to consider include: did you get all of the data you needed; did you actually measure what you intended to measure; and does what you measured still seem to represent important outcomes for which your program should be accountable.

To answer these questions, the working group should review:

• Data collection instruments - Talk with your data collectors to determine if there were difficulties or questions with: instructions, wording and content, response categories, layout and format, and length.

• Training of data collectors - Talk with your data collectors to see if training was adequate and what improvements could be made.

• Data collection procedures - Review the data collected to determine whether you have gathered the information that was intended, and if your data collectors had problems providing the information requested.
  
  
• Time and cost of collecting and analyzing data

Based on the working group's review, make whatever changes are necessary to improve both the effectiveness of your system and the ease of implementing it.

Programs change over time in terms of staffing, environment, and priorities. Your outcome measurement system needs to keep pace with these changes. You should plan to build in a periodic formal review of your system to see what processes have improved and what continues to be troublesome.

This article is the last in the series on developing an outcome measurement system. In the next issue of Making it Count, we will review current practices in outcome measurement in nonprofit organizations.

Action Steps

Get a copy of the United Way of America's handbook, Measuring Program Outcomes: A Practical Approach. To order a copy, call (800) 772-0008 and request item number 0989. Get a copy of the United Way of Massachusetts Bay's handbook, Outcome Measurement in Child Care Programs: A Workbook for Practitioners. To order a copy, call (617) 624-8000. Schedule a time to review your outcome measurement system with your working group. Solicit the input of program staff and other stakeholders to determine how the system is working.

Sources:
Measuring Program Outcomes: A Practical Approach, United Way of America, 1996; Outcome Measurement in Child Care Programs: A Workbook for Practitioners, United Way of Massachusetts Bay, 2000.

Facts in Action, August 2001