1. Evaluation
Aims
Process-focused: To monitor and improve the quality of the socio-cognitive engineering process
Product-focused: To assess and improve the quality of the deployed system

2. How to carry out a project
Form a team
Produce general requirements for the project
Decide which methods and tools will be used for each phase of the project
Decide how the process and outcomes will be documented
Decide how the project will be evaluated
Carry out each phase of the project, making sure that the requirements match the design
Carry out a continuous process of documentation and evaluation

3. Process evaluation
Quality control of the socio-cognitive engineering process
Team
Do you have the right team, with appropriate skills?
Requirements
Do you have a process for validating, refining and modifying requirements?
Methods and tools
Do you have a process for selecting methods and tools?
Documentation
Do you document each phase of the process?
Are the outcomes from each phase used to inform the following phases?

4. Process Evaluation Evaluation Project management Documentation
Have you chosen appropriate methods of product-focused evaluation?
Project management
Do you have: a project plan, method for identifying delays and changes to the plan
Documentation
Are all your documents in a consistent and appropriate format?
Are the accessible to everyone who needs to read or modify them?
Do you have a method of making and tracking revisions to documents?

5. Product evaluation Lifecycle approach (Julia Meek) Product in use
Evaluate throughout the project lifecycle: design, implementation, deployment
Product in use
Evaluate not just the product, but how it is used in context
Use outcomes from each evaluation to
Feed forward: e.g. design evaluation can influence how the product is deployed
Feed back: e.g. evaluation of deployed system can be used to improve the design

6. Dimensions of product evaluation
Usability
Is the product usable by the intended users?
Is it easy to learn how to use it?
Usefulness
Is it useful?
Does it improve the quality of work or quality of life?
Desirability
Do people enjoy using it?
Will they want to continue using it?
Cost-effectiveness
Does it increase business income?
Does it reduce costs?
Does it give a return on investment?

7. Sinclair C5 electric car
Useful but not usable
Sinclair C5 electric car

8. Usable but not useful
Cyberpet

9. Usable and useful but no longer desirable
Film camera

10. Usable, useful, and desirable