1. Module 1 Evaluation Overview © Crown Copyright (2000)
This is the opening module of the Evaluator’s training course.
© Crown Copyright (2000)
This module may be reproduced in its entirety provided that this statement is included. Any other reproduction may only be made with the written consent of the Senior Executive of the UK IT Security Evaluation & Certification Scheme.
© Crown Copyright (2000)
M Issue 4.0, July 2000

2. MODULE 1 EVALUATION OVERVIEW
“You Are Here”
MODULE 1 EVALUATION OVERVIEW
MODULE 2 ASSURANCE
MODULE 3 SCHEME RULES & PROCEDURES
This module (M1) is the first of the three modules that form the evaluator training course. It introduces some concepts that are discussed in greater detail in the two other modules.
M Issue 4.0, July 2000

3. Introduction Why is IT Security needed?
What is IT security evaluation?
What is Assurance?
What are the Scheme Rules?
This module provides an overview of IT security evaluation, and thus starts from first principles by examining what “IT Security” is.
Along the way, we introduce and explain key concepts necessary to understanding what IT security is and what evaluation is about.
We then introduce the concept of assurance, which is covered in detail by Module M2.
Finally, we highlight what are the “rules of the game” that influence the quality of the evaluation. This topic is addressed in more detail in Module M3.
M Issue 4.0, July 2000

4. What Protection ? Salaries Database
Group discussion exercise to ‘break the ice’. This objective is for the students to discover, through a meaningful example, the three aspects of IT security . The example could be replaced by another that the trainer is more comfortable with but the objective of the exercise remains the same.
Consider a database containing the salaries of all employees of an organisation - this contains information used to determine, at the end of each month, how much everyone is paid. How should this information be protected? In particular the following questions should be posed:
Who might attack this information? In what way would they do so? And why would they want to?
The aim of this exercise is to identify the threats to this asset. Suggestions by the students should be written down on a flipchart/whiteboard. In so doing, the identified threats should be grouped according to whether it is the Confidentiality, Integrity or Availability of the asset that is potentially compromised. This introduces the three aspects of IT security.
Examples of potential threats under each grouping include:
Confidentiality: unauthorised disclosure of employee salary details to competitors, head-hunters, etc.
Integrity: unauthorised modification of employee salary details by disgruntled employees (e.g. their own salary or that of their boss).
Availability: unauthorised withholding of information at end of month by disgruntled ex-employees - delaying payment of staff salaries.
Salaries Database
M Issue 4.0, July 2000

5. Aspects of IT Security
Confidentiality - protection against unauthorised disclosure of information
Integrity - protection against unauthorised modification of information or loss of accuracy
Availability - protection against unauthorised withholding of information or resources
These are the three aspects of IT Security. Examples include:
C - confidential patient information copied to a PC hard disc accessible by staff and patients
I - bank fraud, by compromising integrity of bank records
A - denial of service attacks or hardware failure
Other examples may be cited from personal experience.
Encourage students to suggest other examples and discuss.
Different assets may have different protection needs. Consider patient information again - integrity is at least as important as confidentiality, e.g. what if a record is modified so that a patient is prescribed the wrong (potentially harmful) medicine ?
Moving on:
What are we trying to protect ?
And from what ?
Next slide Þ
M Issue 4.0, July 2000

6. Assets and Threats Assets Threats valuable organisational resources
disclosure or compromise or loss would be inconvenient or harmful
Threats
a potentially harmful action affecting confidentiality, integrity or availability of assets
Assets can be anything of value to an organisation or individual, for example:
- bank records, patient details, etc.
- information
- software
- physical components
- services
- people
- the good name of the company/group as a credible organisation
In the domain of IT security, however, the assets of interest are normally the information held on an IT system, or the resources provided by the system.
Threats may come from many sources or threat agents, for example:
- hackers
- hostile organisations
- disgruntled employees
- poorly trained employees
- natural disasters
- accidental misuse
M Issue 4.0, July 2000

7. Countermeasures
A measure put in place to counter, or help counter, an identified threat to an asset
Countermeasures can be:
IT, i.e. implemented by hardware, firmware or software; or
non-IT, e.g. physical or procedural measures
Having identified assets that require protection, and the potential threats to those assets, the next step is for the owner of the asset to select appropriate countermeasures.
“Horses for Courses” - It is important that the right countermeasure is chosen to reduce the residual risk to an acceptable level, and counter the threat.
M Issue 4.0, July 2000

8. Types of Countermeasure
Preventative
place restrictions on who can do what
Detective
provide means to detect events which indicate a potential compromise of assets
Corrective
take action in response to undesirable events
Countermeasures can be broadly categorised according to what they do in order to counter an identified threat - both IT and non-IT countermeasures can be categorised in this way.
Note that there is no need to categorise selected countermeasures in this way. Awareness of the three types may help the selection process - by acting as a form of checklist. Once defined, however, it should be clear whether the countermeasure is preventative, detective or corrective.
M Issue 4.0, July 2000

9. Countermeasure Examples
Preventative:
access control (physical or logical)
data encryption
Detective
auditing of security relevant events
data integrity measures, e.g. checksums
Corrective
user account lockout after login failures
suspension of inactive user sessions
Note that “logical access control” refers to access control enforced by an IT system.
Other examples can be cited from personal experience, e.g. ask students how identification and authentication of users would be categorised (preventative).
Countermeasures will typically co-operate to ensure the threats are countered. For example, if an IT system audits security relevant events as a means of detecting their occurrence, there will be a need for related countermeasures to:
ensure they are detected in practice, e.g. analysed by human administrator, possibly with tool support
protect the integrity of the information
take action in the event of discovering actual or potential breaches of security.
M Issue 4.0, July 2000

10. Vulnerabilities and Risks
a security weakness that may allow realisation of a threat to compromise an asset
Risk
likelihood of a threat exploiting a vulnerability to harm an asset and/or cause loss
The selected countermeasures need to be effective at countering the threats to the assets. However, the countermeasures may not be effective because of the presence of vulnerabilities.
Vulnerability Example
- inadequate logical access controls on IT system
- e.g. username but no password
However, the existence of a vulnerability is not necessarily a problem. This leads to consideration of the risk to the assets. For example, given the above vulnerability:
Low risk if system in concrete bunker with armed guards
High risk if in (or accessible via) an Internet Café on the High Street
In this example the risk to the assets stored on the IT system is influenced by the environment and the controls over that environment.
M Issue 4.0, July 2000

11. Castle Example
A further group discussion to further explore the concepts of assets, threats and in particular vulnerabilities. Involve the delegates to derive what the protection mechanisms are within the castle, what they might be trying to protect and from what threat. Get them to suggest weaknesses in the castle’s defences.
Reinforce the idea that the most appropriate countermeasure for the job should be used, rather than the best countermeasure e.g. do you need a moat if your walls are 12 feet tall? Is there sufficient time to build a moat or is it better to post guards on the perimeter ?
Again, write down suggestions from delegates as to what might go wrong with the countermeasures - how might they be defeated. It is helpful if the suggestions are grouped according to the various categories identified in the following slide.
Note that this example can be substituted by alternative (IT-based) examples at the discretion of the trainer. The following should apply:
the example should illustrate that there are various ways in which countermeasures can be defeated (as discussed on the following slide)
all students should be able to contribute to the discussion.
M Issue 4.0, July 2000

12. Sources of Vulnerabilities
Vulnerabilities can arise from:
Inappropriate selection of countermeasures
Errors in their design or implementation
Conflict between countermeasures
Loopholes allowing circumvention of countermeasures
Misuse of countermeasures
The Castle example should lead to identification of a variety of suggestions. These may include the following:
Countermeasures may be inappropriate because they are unsuitable or have insufficient strength:
- (Suitability) drawbridge takes 2 hours to open and close, so need guards to be posted all the time.
- (Strength) what are the walls made of - do the attackers have siege engines?
Other problems in the design or implementation of countermeasures include (noting bypass and tampering are examples of “loopholes”):
- (Refinement Error) are the walls built correctly, i.e. as specified?
- (Bypass) underground delivery tunnel used whilst building is left open for anyone to walk in
- (Tampering) wall foundations could be undermined by tunnel
- (Conflict) water in moat undermines foundations of wall
- (Misuse) leave the drawbridge down until the last one is in - who’s the last one ?
(Misuse may also be an issue if countermeasures are too difficult or time consuming to implement - users will find ways of ‘working around’ them.)
By recognising that vulnerabilities can be categorised according to their source, we can adopt a checklist approach to search for vulnerabilities - giving confidence that none have been overlooked.
M Issue 4.0, July 2000

13. Impact of Vulnerabilities
Vulnerabilities can be:
Exploitable
given sufficient time, resources and expertise an attacker could break through in practice
Non-Exploitable
an attacker will be unable in practice to exploit it to compromise an asset
The existence of a vulnerability does not necessarily mean that the assets are at risk of being compromised. Having found a vulnerability, we need to determine whether it can be exploited in practice by an attacker.
Non-exploitable vulnerabilities are those vulnerabilities that are not exploitable in practice. This could be for various reasons, not just considerations based on strength (e.g. other countermeasures could prevent the attack from taking place, or ensure that it can be detected and stopped before exploitation occurs).
M Issue 4.0, July 2000

14. What is Evaluation ?
An independent assessment of a Target of Evaluation (TOE) involving
analysis
testing
Scope of work is defined in a Security Target
Aimed at establishing a required level of assurance
TheTOE is the software, firmware and/or hardware which is to be evaluated - where IT security evaluation is aimed at determining whether the countermeasures the TOE implements are effective at countering the identified threats to the assets that require protection.
Specifically, the fundamental goal of an IT security evaluation is to establish a required level of assurance in the absence of exploitable vulnerabilities in a TOE. Evaluators should always remember that this is the raison d'être of evaluation.
Such assurance is established through analysis of the TOE, followed by testing using the results of the analysis.
Evaluation is performed in accordance with:
a defined set of evaluation criteria (defining what evidence a developer must provide to satisfy each criteria, and what the evaluator does with the evidence)
a defined evaluation methodology (defining how the evaluation work is to be performed)
M Issue 4.0, July 2000

15. What is Assurance?
A measure of confidence that a TOE meets its security objectives
risk to assets reduced to acceptable level
Assurance is governed by
depth of evaluator analysis
degree of developer and evaluator testing
formality / rigour of developer evidence
Leads to concept of Assurance Levels
Take concept of Confidence and Assurance
- more time is spent on higher assurance levels working in more depth, leading to higher assurance and so more confidence
Depth of evaluator analysis (conferring understanding of the TOE internals):
design documents, source code, formal representations
Degree of testing:
coverage and depth
more assurance with more testing
Formality - greater precision leads to higher assurance through:
fewer design and implementation errors
improved evaluator understanding
The precise assurance requirements are defined by the evaluation criteria. These define a series of assurance levels - ranging from:
a minimum level where the TOE is tested as a ‘black box’ to
high assurance where formal methods are applied
M Issue 4.0, July 2000

16. Scope of Evaluation
Product - an IT package that can be purchased and deployed in a number of different operational environments
System - a specific IT installation with a particular purpose and a known operational environment
Product - such as an operating system, DBMS or firewall
- the product can be used in many different environments, and will counter assumed threats (typically generic) given assumptions about the environment in which it will be used, and the way the product is used (e.g. configured)
System - usually a specific instance in a known environment - where the assets requiring protection, the threats to those assets, and the assertions that can be made about the environment (e.g. for specific non-IT countermeasures) can be articulated.
A system may be composed of one or more products (trainer should give examples based on own experience).
M Issue 4.0, July 2000

17. Security Target This defines:
the assets, threats
assumptions
environment etc.
Everything you need to know about the TOE
including the IT countermeasures or security functions
The Security Target is a document which identifies
the ‘security problem’ to be addressed in terms of the assets to be protected, the risk and threats to those assets
the ‘solution’ to the problem in the form of countermeasures.
The environmental assumptions typically identify or presume the presence of countermeasures not implemented by the TOE - whether non-IT or whether implemented by some other IT system or product.
M Issue 4.0, July 2000

18. Evaluation Criteria European, 1991 World-wide (ISO standard), 1998
Information Technology Security Evaluation Criteria (ITSEC)
World-wide (ISO standard), 1998
Common Criteria (CC)
These are the rules to formalise the analysis of the TOE.
They provide a structured consistent approach defining what has to be done.
ITSEC is the basis of EU mutual recognition where UK, France and Germany recognised as ‘Qualifying Certification Bodies’ under the SOGIS agreement (which involves other EU nations such as Finland, Greece, Italy, the Netherlands, Norway, Portugal, Spain, Sweden).
CC brings mutual recognition involving North America (US, Canada) and Australia - a mutual recognition arrangement already exists.
M Issue 4.0, July 2000

19. Evaluation Methodology
How we do evaluations
defined in ITSEM and CEM
Defines techniques for the various activities:
Refinement Analysis
TOE life-cycle assessment
Vulnerability Analysis
Testing of TOE
Underpinning the evaluation criteria are the documents which define the evaluation methodology - defining HOW to do evaluations.
The evaluation methodology documents are specific to the evaluation criteria. They define mandatory requirements on evaluators, to ensure evaluations are performed to a common technical standard. For example, evaluators must assign verdicts for each item of work (called the work package or work unit), and provide a justification for those verdicts.
The evaluation methodology documents also give guidance on evaluation techniques for the various evaluation activities (analysis and testing) required by the evaluation criteria.
A defined evaluation methodology is needed to help ensure that all evaluations are performed to a common standard. This is an essential prerequisite for mutual recognition of evaluation results between different nations.
M Issue 4.0, July 2000

20. The UK Scheme Scheme rules cover quality and management
security / confidentiality
training
appointment and accreditation
The UK IT Security Evaluation and Certification Scheme is managed overall by a Management Board
Certification Body and UKAS monitor all aspects listed here
On a daily basis the CLEFs are monitored by the CB - from a technical viewpoint, and by UKAS from a quality viewpoint
CLEFs need to conform to rules on physical / personnel security, training, etc.
The CB publishes a series of UK Scheme Publications (the UKSP series) to define procedures and interpretations pertaining to the conduct of evaluations within the UK.
M Issue 4.0, July 2000

21. Evaluation Parties Developer - produces the TOE
Sponsor - pays for the evaluation
Evaluator - performs the evaluation
Certifier - oversees the evaluation and issues certificates where appropriate
Accreditor - relevant to systems only
Developer and Sponsor may be the same organisation.
Accreditor - responsible for ensuring that a system can handle the assets that require protection (usually protectively marked information in the case of government systems). Relies on the recommendations of the Certifier (who in turn makes decisions based on the evaluators’ reports).
BUT Accreditors don’t always follow what the CB says - may accept responsibility for risk where CB recommends otherwise.
e.g. unevaluated COTS product (or product evaluated but certificate not recognised by UK) - Accreditor may accept risk of using COTS product, thus not requiring separate UK evaluation.
M Issue 4.0, July 2000

22. Evaluation Process Developer / Sponsor CLEF Certification Body
Problem Reports
TOE Definition
Deliverables
CLEF
Evaluation Technical Report
Certification Body
Outline of whole process
Developer may be same as sponsor
May not have an Accreditor, just a Sponsor
Certification Report
Accreditor / Sponsor
M Issue 4.0, July 2000

23. Evaluation Conduct Impartiality Repeatability Reproducibility
what interest do you have in the outcome?
Repeatability
could you get the same results?
Reproducibility
would other CLEFs get the same results?
Objectivity
minimise subjective judgement
All of the evaluation is documented in the Evaluation Technical Report.
It is essential that all the details are reported to ensure that Repeatability and Reproducibility are possible.
All evaluation verdicts must be justified, demonstrating the evaluator’s understanding of the TOE.
M Issue 4.0, July 2000

24. Summary - 1
Need IT Security to protect assets from threats using adequate countermeasures
Evaluation allows a Target Of Evaluation to be independently assessed
Assurance gives a level of confidence that a TOE meets its security objectives
We started this module by asking three questions, to which we now have the answers.
Why is IT Security needed?
IT Security stems from the need to protect assets within an IT system. Adequate IT and non-IT countermeasures have to be implemented to counter the identified threats to those assets.
What is IT Security Evaluation?
IT security evaluation is an independent assessment of a Target of Evaluation, consisting of analysis and testing, aimed at establishing assurance that the TOE is free from exploitable vulnerabilities.
What is Assurance?
Assurance is a measure of confidence that the TOE meets its security objectives, and thus contains no exploitable vulnerabilities. The required level of confidence needs to be commensurate with the risk to the assets.
M Issue 4.0, July 2000

25. Summary - 2 Evaluation Criteria - ITSEC and CC
Evaluation Methodology - ITSEM and CEM
Scheme Rules and Interpretations
quality, management, security, training
application of criteria and methodology
Evaluation Conduct
impartial, repeatable, reproducible and objective
The rules of the “evaluation game” are governed by three things:
The evaluation criteria - which define what TOE evidence a developer must provide, and what an evaluator must do with that evidence, for a given level of assurance.
The evaluation methodology - which define how the evaluator performs the tasks required by the criteria.
Scheme Rules governing the conduct of the evaluation work, and the organisation which carries out the work.
M Issue 4.0, July 2000

26. Further Reading UKSP 01 UKSP 04 Part 1 ITSEC, Sections 0 and 1
Common Criteria Part 1, Section 4
UK SP 01: Provides a complete overall description of the UK Scheme.
UK SP 04 Part 1: Gives further information on the roles and responsibilities of sponsor and developer in the evaluation process
ITSEC Section 0 & 1: Provides an overview of IT security and the scope of ITSEC evaluation.
CC Part 1, Section 4: An overview of IT security and the Common Criteria.
M Issue 4.0, July 2000