1. ASG Cost Model Presentation CCSSO National Conference on Student Assessment

2. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org2 Overview Model description/Structure Model description/Structure Key inputs and outputs Key inputs and outputs Databases Databases Reports Reports Uses Uses Questions Questions

3. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org3 Description/Structure The ASG Assessment Cost Model is a variable input, metric based calculation output model The ASG Assessment Cost Model is a variable input, metric based calculation output model Specific assessment program variables are input to the model and applied against cost factors to derive assessment cost Specific assessment program variables are input to the model and applied against cost factors to derive assessment cost Several hundred variables are contained in the model Several hundred variables are contained in the model Numerous detailed input variables are assembled for each of the functions and elements involved in constructing an assessment (test development, production and manufacturing, logistics, editing and scoring, reporting, psychometrics, program management, quality assurance, information technology, etc.) Numerous detailed input variables are assembled for each of the functions and elements involved in constructing an assessment (test development, production and manufacturing, logistics, editing and scoring, reporting, psychometrics, program management, quality assurance, information technology, etc.) These variables are input into the software and matched to metric based calculations/formulas and/or databases to convert the assessment program variables into dollar costs These variables are input into the software and matched to metric based calculations/formulas and/or databases to convert the assessment program variables into dollar costs

4. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org4 Description/Structure (cont.) The cost for the entire assessment is built from the ground up, element by element, function by function The cost for the entire assessment is built from the ground up, element by element, function by function The model also contains “efficient cost metrics” for each key cost area and function so that a state’s assessment program cost data can be compared to those of an “efficient” assessment The model also contains “efficient cost metrics” for each key cost area and function so that a state’s assessment program cost data can be compared to those of an “efficient” assessment This allows states to see where their assessment program is cost effective and where there are opportunities to improve costs This allows states to see where their assessment program is cost effective and where there are opportunities to improve costs

5. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org5 Description/Structure (cont.) Proprietary functions and calculations have been developed based on our years of experience in the industry Proprietary functions and calculations have been developed based on our years of experience in the industry Model outputs have been tested against real data Model outputs have been tested against real data The model is currently built for ASG use only. We are in the process of developing a web based application that could be shared with states. The model is currently built for ASG use only. We are in the process of developing a web based application that could be shared with states. ASG has partnered with CCSSO and ten states on an Enhanced Assessment Grant to study assessment efficiency in the participating states. The ASG cost model will be the key tool used in the analysis. ASG has partnered with CCSSO and ten states on an Enhanced Assessment Grant to study assessment efficiency in the participating states. The ASG cost model will be the key tool used in the analysis.

6. Cost Model-Main Menu June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org6

7. Administration June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org7 Administrative Center Key Functions: - Update Rate Tables - Define Security Settings - Create Custom Reports

8. Assessment Map June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org8

9. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org9 Key Inputs and Outputs Numerous assessment component inputs are available for each key area, function etc. Numerous assessment component inputs are available for each key area, function etc. Assessment cost can be developed using detailed data or higher level, summary data Assessment cost can be developed using detailed data or higher level, summary data Cost outputs by function, area, cost type, etc. are generated Cost outputs by function, area, cost type, etc. are generated Key metrics are presented to allow comparison to model efficient cost data for that function/area/component Key metrics are presented to allow comparison to model efficient cost data for that function/area/component Canned and custom reports are available at the summary and detailed level Canned and custom reports are available at the summary and detailed level Assessment Cost Model Assessment Cost Model

10. Key Inputs June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org10 This is a sample of the key inputs. There are over 200 different inputs that are captured in the assessment model.

11. Data Input Screens June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org11

12. Data Input Screens - cont’d June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org12

13. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org13 Databases/Tables The Model Contains the Following 70+ Tables to Manage Rates & Data Capture 70+ Tables to Manage Rates & Data Capture Rate Tables for Each Cost Component (examples): Rate Tables for Each Cost Component (examples): Design/Composition Rates Design/Composition Rates Printing Rates Printing Rates Freight/Shipping Rates Freight/Shipping Rates Labor Rates – by State, by Occupation Labor Rates – by State, by Occupation Rates for Delivery of On-Line Assessments Rates for Delivery of On-Line Assessments Open Ended Scoring Reader Rates Open Ended Scoring Reader Rates Item Acquisition/Permission Rates Item Acquisition/Permission Rates

14. Key Databases - Labor June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org14

15. Key Databases - Travel June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org15 Travel Table - Includes Airfare, Lodging and Meal per diem rates for all 50 States and the District of Columbia

16. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org16 Reports Data presented in report and graphical formats Canned reports include: Canned reports include: Assessment Cost Summary Assessment Cost Summary Assessment Cost by Function Assessment Cost by Function Detail Assessment Functional Cost Detail Assessment Functional Cost Fixed and Variable Cost Data Fixed and Variable Cost Data Assessment Price Composition (direct, indirect and margin) Assessment Price Composition (direct, indirect and margin) Key Metrics Summary & Functional Area Metrics Summary Key Metrics Summary & Functional Area Metrics Summary Assessment Costs vs. “efficient costs” Assessment Costs vs. “efficient costs” Assessment Metrics vs. “efficient metrics” Assessment Metrics vs. “efficient metrics” Other reports are currently in development Other reports are currently in development Reports can also be easily customized per customer request Reports can also be easily customized per customer request Assessment Cost Model Assessment Cost Model

17. Reporting June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org17

18. Reporting – cont’d June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org18

19. Reporting – cont’d June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org19

20. Reporting – cont’d June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org20

21. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org21 Uses The model can be used in several ways to improve assessment cost and efficiency: The model can be used in several ways to improve assessment cost and efficiency: Developing a new RFP Developing a new RFP ASG cost sheets key in discerning if vendors understand your program and are proposing apples to apples solutions ASG cost sheets key in discerning if vendors understand your program and are proposing apples to apples solutions Making tradeoffs between various assessment components Making tradeoffs between various assessment components Develop the best assessment for the available dollars Develop the best assessment for the available dollars Review of an existing program to determine areas of inefficiency and opportunity Review of an existing program to determine areas of inefficiency and opportunity Assessment costs tend to be too high because of inefficiencies in the program design Assessment costs tend to be too high because of inefficiencies in the program design Working with the assessment vendor to reduce/eliminate efficiencies and lower costs is a win/win for both the vendor and customer Working with the assessment vendor to reduce/eliminate efficiencies and lower costs is a win/win for both the vendor and customer Devising scope changes Devising scope changes

22. June 22/23, 2009 Assessment Solutions Group www.assessmentgroup.org22 Questions ASG would be happy to answer any questions or provide a state specific demo of the model on request. ASG would be happy to answer any questions or provide a state specific demo of the model on request. Contact information: Contact information: Barry Topol - 210-859-9920 btopol@assessmentgroup.org Barry Topol - 210-859-9920 btopol@assessmentgroup.org Pat Hennon - 210-376-7654 phennon@assessmentgroup.org Pat Hennon - 210-376-7654 phennon@assessmentgroup.org John Olson - 210-859-5593 jolson@assessmentgroup.org John Olson - 210-859-5593 jolson@assessmentgroup.org Ed Roeber - 517-614-4877 eroeber@assessmentgroup.org Ed Roeber - 517-614-4877 eroeber@assessmentgroup.org