1. CBT Management and Process
4/19/2019

2. Courseware Management and Processes Mission Statement
Provide recommendations and guidelines to the Computer-based Training community that identifies the attributes of “Good CBT” processes and product.
4/19/2019

3. Courseware Management and Processes Objectives
Re-establishing the Basics: ISD Model
Establish a High Level CBT Development Process Template
Establish Metric Recommendations
Establish Quality Standard Recommendations
Set Recommendations for ADL/SCORM as it Drives the Process
4/19/2019

4. Why Processes and Metrics?
Streamline Approach to Courseware Development
Collect
Analyze
Improve
4/19/2019

5. METRIC MANAGEMENT SYSTEM
This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation
In Slide Show, click on the right mouse button
Select “Meeting Minder”
Select the “Action Items” tab
Type in action items as they come up
Click OK to dismiss this box
This will automatically create an Action Item slide at the end of your presentation with your points entered.
METRIC MANAGEMENT SYSTEM
The Boeing Company
Integrated Defense Systems
Mario A. Trejo
4/19/2019
V6.1a

6. Metric Management System Objective
Identify the requirements of a metrics system for courseware development and Process Based Management as it relates to:
Quality
Timeliness
Efficiency
Cycle Time
Process Compliance
4/19/2019

7. Metric Management System
Called MMS
Consists of several integrated tracking systems
Tracks
Quality
Performance
Timeliness
Cycle Time
Process Compliance
4/19/2019

8. MMS Project Prototype developed in MS Access
Multi-Phase development project
Each phase is based on the tracking system being integrated
MMS administration tools are upgraded with each tracking system
4/19/2019

9. Integrated Tracking Systems
Internal Discrepancy Tracking
External Discrepancy Tracking
Product Development Tracking
Process Surveillance Tracking
Quality, Timeliness, Efficiency, and Cycle Time Monitor
4/19/2019

10. Internal Discrepancy Tracking System
Internal Discrepancy Tracking System is completed
Used on all projects to track internal deficiencies
Tracks Deficiency Reports
Lesson Element
Checkpoint
Recorder
Recorded Date
Attribute Type deficiency quantities
Notes
4/19/2019

11. Internal Discrepancy Tracking System Reports
Hours of instruction reviewed
Deficiencies per Hour Of Instruction reviewed
Deficiencies by Checkpoint
Deficiencies by Attribute Type per Checkpoint
Deficiencies by Course
4/19/2019

12. External Discrepancy Tracking System
External Discrepancy Tracking System is completed
Used on some projects to track External Discrepancy Reports
Tracks Deficiency Reports
Checkpoint
Lesson Element
Recorder
Recorded Date
Discrepancy, description
Recommendation, for corrective action
Disposition, corrective action
Recorder’s Name, corrective action reviewer
Notes
Attribute Type deficiency quantities
4/19/2019

13. External Discrepancy Tracking System Reports
Hours of instruction reviewed
Deficiencies per Hour Of Instruction reviewed
Deficiencies by Checkpoint
Deficiencies by Attribute Type per Checkpoint
Deficiencies by Course
4/19/2019

14. Product Development Tracking System (1 of 3)
Product Development Tracking System is in development
Used on three different types of projects
Tracks Media Types
Tracks Element Types
Tracks Content Types
Tracks Checkpoints
4/19/2019

15. Product Development Tracking System (2 of 3)
Tracks Project
Scheduled Start and Scheduled Completion Dates
Monthly Budgeted and Actual Hours
Monthly Budgeted and Actual Dollars
Tracks Course and Lesson
Title and Number
Actual Start and Actual Completion Dates
Tracks Lesson Checkpoint
Scheduled Person/Group and Actual Person/Group
Lesson task weights
4/19/2019

16. Product Development Tracking System (3 of 3)
Tracks similar information for Major Topics and Subtopics
Tracks Cost Performance Index
Tracks Scheduled Performance Index
4/19/2019

17. Product Development Tracking System Reports (1 of 3)
Project Timeline
Scheduled versus Elapsed
Project Checkpoint Performance
Scheduled versus Completed
Project Hours
Scheduled versus Used
Course And Lesson Percent Completed
Course Hours
Scheduled and Completed
4/19/2019

18. Product Development Tracking System Reports (2 of 3)
Monthly Hours
Scheduled and Used
Monthly Accumulative Hours
Monthly Index
Cost Performance Index
Scheduled Performance Index
Deficiency Report
Project/Course Percent Completed
Course/Lesson Percent Completed
4/19/2019

19. Product Development Tracking System Reports (3 of 3)
Topic Performance
Scheduled versus Completed
Topic Hours
Scheduled and Completed
4/19/2019

20. Process Surveillance Tracking System
Process Surveillance Tracking System is completed
Used on several projects
Tracks
Management Process
Verify process existence, approval, and usage
Process Requirements
Verify Development Process requirements have been met
Process Tasks
Verify Development Process are being followed
4/19/2019

21. Process Surveillance Tracking System Reports
IS Process Compliance
IS Task Compliance
Total quantities of Process Surveillance Nonconformance results
Monthly quantities of Process Surveillance Nonconformance results
Monthly Risk Level assessments
4/19/2019

22. QTEC Monitor Quality, Timeliness, Efficiency, and Cycle Time Status
Project Level
Program Level
Instructional Systems Level
4/19/2019

23. QTEC Monitor Reports (1 of 2)
Quality, Timeliness, Efficiency, and Cycle Time Reports
Minimum/Maximum Acceptable Levels
Goals
Actual Status
4/19/2019

24. QTEC Monitor Reports (2 of 2)
Instructional Type
Development hours
Total
Monthly
Scheduled
Actuals
Development hours per hour of instruction ratio
Task Type
4/19/2019

25. Two Year Goals Finish PDT and QTEC Reports
Integrate with Employee Timekeeping System
Integrate with Earned Value Management Tracking System
Stabilize user interface
Stabilize database structure
Migrate user interface to ColdFusion
Convert database to Oracle
Require all projects to use MMS
4/19/2019