Software Validation in Accredited Laboratories A Practical Guide Greg Gogates Fasor Inc. gdg@fasor.com 26 Sept 2001 A copy of this paper will be maintained on ftp.fasor.com/pub/iso25/validation/

Preamble ISO guide 25 “Adequate for Use” ISO 17025 “Suitably Validated” Validation - Reasonable Assurance Maintain Design Control Know what's where! Keep it Simple EA/GA(98)95 Draft “EA Guidelines for the use of Computers and Computer Systems in Accredited Laboratories” 26 Sept 2001 NCSL Presentation

Software Classifications COTS - Commercial Off the Shelf MOTS - Modified Off the Shelf CUSTOM - Written from scratch 26 Sept 2001 NCSL Presentation

COTS Use as delivered No modifications/customizations Verify functionality Verify menu contents Verify parameter files Lock down changes Installation checkout 26 Sept 2001 NCSL Presentation

MOTS Needs configuration/customization prior to use. Customizations require validation. Non-modified portion treated as COTS Installation checkout. 26 Sept 2001 NCSL Presentation

CUSTOM Home Brew or Consultant code Functional Requirements Software Design Traceability Matrix Structural / Functional Tests Installation Checkout 26 Sept 2001 NCSL Presentation

Software Life Cycle 26 Sept 2001 NCSL Presentation

Products / Systems 26 Sept 2001 NCSL Presentation

Software Configuration Management What software is on what machine Store originals on Network Share, Tape, or CD-R Maintain all published versions Control Access Ensure user access 26 Sept 2001 NCSL Presentation

Documentation Configuration Management Evidence of validation Evidence of adequate for use Owners / User manuals Software Lifecycle documents Test Evidence 26 Sept 2001 NCSL Presentation

Good Practices - 1 Treat each software product a piece of calibration equipment. Do checks to ensure nothing changes. Place software product masters in a read only directory. Network computers that access a shared program on a server. 26 Sept 2001 NCSL Presentation

Good Practices - 2 Lock spreadsheet cells that contain math. Password protect configuration files or setup screens. Backup, back-up, and backup off site! Plan for hardware/software disaster recovery. Good information maintained @ ftp://ftp.fasor.com/pub/iso25/validation 26 Sept 2001 NCSL Presentation

Software Checklist - 1 Has the Firmware been validated via the calibration by a cal lab who has the capability to thoroughly check the software (i.e. OEM or Authorized partner) (Note: for non-calibratable device Firmware, treat as Software) Have all of the "used" firmware parameters been documented and confirmed that they are correct? Does a Software Requirements document exist for the Software? Does a Software Design document exist detailing either the full design or details of the configuration? Does software testing documents exists describing completed, unique, test cases that exercise the design both +/- and confirms the requirements? Is there a test log showing test failures and corresponding retests/dispositions? Does evidence exist confirming correct software deployment at each target installation? [5.5.11] 26 Sept 2001 NCSL Presentation

Software Checklist - 2 Has configuration control been applied to all of their Software/Firmware to ensure that: [4.12.1.4] The Software source code location is access controlled. [5.4.7.2.b] Firmware/Software formulas & parameters are "locked" to prevent inadvertent changes. [5.5.12] Equipment lists identify Software as a separate line item showing correct version and location. [5.5.5] Does evidence exist showing that personnel involved in Custom Software development have adequate training? Do Databases and spreadsheets include "audit trails" to not allow previously data to be obscured? [4.12.2.3] Do adequate instructions exist for the operation & maintenance of the Software? [5.4.1] Does the accuracy of the Firmware/Software meet or exceed the accuracy required by the test method? [5.5.2] 26 Sept 2001 NCSL Presentation

S U M A R Y 26 Sept 2001 NCSL Presentation

QUESTIONS? 26 Sept 2001 NCSL Presentation http://www.sfgate.com/sf/zippy/ 26 Sept 2001 NCSL Presentation