1. American Evaluation Association
Developing A Method For Classifying Evaluation Activities in a Foundation
Thursday, November 7, 2002
American Evaluation Association
Washington, D.C.

2. Overview 1. Context 2. Key Questions 3. A Methodology
4. Use in Managing Evaluation Activities
5. Summary
6. Future Questions

3. 1. Context ...at the Foundation From January 2000 to December 2001
about 1,800 grants
about $ 900,000,000 granted
Six major program areas
In 2001, evaluation department established

4. 2. Key Questions
What does the portfolio of evaluation activities look like?
How can we classify the various evaluation activities at the Foundation?
What categories are useful for deciding how to manage evaluation activities?

5. 3. Developing A Method Identified Key Questions
Developed Classification Definitions
1. Program Area
2. Budget
3. Evaluation Design: causal or descriptive
4. Report Author: third party or program operator
5. Evaluation Funder: foundation, program operator, or other
6. Deliverable Due Date

6. Additional Definitions
Grant Grouping: “line of work” or individual
Pyramid Level: 1 through 6
Evaluation Purpose: summative, formative, research, monitoring.
Primary Audience
Primary Program Funder
Strategy
Year Program Started
3. Method (continued)

7. Collected Data Grants Database (GIFTS) Grants Summaries
Grant Proposals
Board Docket
Consultant Contracts
Program Staff
3. Method (continued)

8. Reviewed Findings Iteratively with Program Staff
Analyzed Data
Reviewed Findings Iteratively with Program Staff
Definition refinement
Re-collection of data
Re-analysis of data
3. Method (continued)

10. 4. Use in Managing Evaluations
To focus evaluation department efforts.
To guide decisions about what to evaluate.
To develop a reporting schedule to plan discussions and learning opportunities about program performance and improvement.
To improve the design of the grants database (GIFTS).

11. 5. Summary
”Dimensions” of the portfolio of evaluation activities at the Foundation.
Evaluation activities were classified along more than ten dimensions: budget, evaluation design, author, etc.
Categories most useful for managing include program area, budget, design, deliverable due date.

12. 6. Future Questions
What other criteria should be considered when deciding how to classify evaluation activities?
What other criteria should be considered when deciding how to manage an evaluation portfolio? (grant characteristics and/or grant contexts)
How have other grant-making foundations classified their evaluation activities?