Test Plans, Test Cases, and Test Reports

Slides:

Advertisements

Similar presentations

1 RXQ Customer Drop Submitted by Supplier Process Flow Diagram Customer Supplier Distribution Company Supplier sends Drop Request to Distribution.

Advertisements

1 Drop Submitted by Supplier Customer Supplier Distribution Company Supplier sends Drop Request to Distribution Company (??) Distribution Company receives.

Software change management

Submitted by Distribution Company Process Flow Diagram

TGDC Meeting, December 2011 Usability and Accessibility (U&A) Research Update Sharon J. Laskowski, Ph.D.

Request for Proposal - Best Value Generally RFP’s are created for large dollar technical purchases, services and consultants.  Consideration in addition.

Regional conference on the 1980 Hague Convention on the Civil Aspects of International Child Abduction, February, Belgrad,Serbia Albania’s Challenges.

United States Election Assistance Commission Pilot Program Testing and Certification Manual & UOCAVA Pilot Program Testing and Certification Manual & UOCAVA.

Voting System Qualification How it happens and why.

TGDC Meeting, July 2011 Overview of July TGDC Meeting Belinda L. Collins, Ph.D. Senior Advisor, Voting Standards, ITL

Page 1Version 2.1 Request for conditional approval Information required to support request for conditional approval to forward a draft standard IEEE 802.

Election Assistance Commission United States VVSG Technical Guidelines Development Committee (TGDC) NIST July 20, 2015 Gaithersburg,

TGDC Meeting, July 2011 Usability and Accessibility Test Methods: Preliminary Findings on Validation Sharon Laskowski, Ph.D. Manager, NIST Visualization.

Testing Summit Sacramento, CA November 28, 2005 Barbara Guttman National Institute of Standards and Technology

TGDC Meeting, Jan 2011 VVSG 2.0 and Beyond: Usability and Accessibility Issues, Gaps, and Performance Tests Sharon Laskowski, PhD National Institute of.

Demystifying the Independent Test Authority (ITA)

Commissioning of Fire Protection and Life Safety Systems Presented by: Charles Kilfoil Bechtel National Waste Treatment Plant Richland WA.

NIST HAVA-Related Work: Status and Plans June 16, 2005 National Institute of Standards and Technology

Making every vote count. United States Election Assistance Commission HAVA 101 TGDC Meeting December 9-10, 2009.

Laboratory Accreditation as a Component of the Help America Vote Act Mary H. Saunders Chief, Standards Services Division.

Accreditation for Voting Equipment Testing Laboratories Gordon Gillerman Standard Services Division Chief

Usability and Accessibility Working Group Report Sharon Laskowski, PhD National Institute of Standards and Technology TGDC Meeting,

Why Proposed TC Procedures? Define how TC reaches “completion” of what OASIS calls “Committee Specifications” TC procedures lead up to the OASIS process:

Briefing for NIST Acting Director James Turner regarding visit from EAC Commissioners March 26, 2008 For internal use only 1.

NIST Voting Program Activities Update February 21, 2007 Mark Skall Chief, Software Diagnostics and Conformance Testing Division.

TGDC Meeting, Jan 2011 Accessibility and Usability Considerations for UOCAVA Remote Electronic Voting Systems Sharon Laskowski, PhD National Institute.

Election Assistance Commission 1 Technical Guidelines Development Committee Meeting Certification Updates July 20, United States.

12/9-10/2009 TGDC Meeting Usability and Accessibility Progress and Challenges Sharon Laskowski, PhD National Institute of Standards and Technology

July 2004 Jay Bain, Fearn Consulting doc.: IEEE /0379r0 Submission Project: IEEE P Working Group for Wireless Personal Area Networks (WPANs)

Making every vote count. United States Election Assistance Commission EAC Voting System Certification TGDC Meeting December 9-10, 2009.

Understanding the Green Deal Arrangements Agreement (GDAA) Green Deal Provider Guidance Webinar Series.

NIST Voting Program Page 1 NIST Voting Program Lynne Rosenthal National Institute of Standards and Technology

TGDC Meeting, December 2011 Overview of December TGDC Meeting Belinda L. Collins, Ph.D. Senior Advisor, Voting Standards

NIST Voting Program Barbara Guttman 12/6/07

TGDC Meeting, July 2011 Voluntary Voting System Guidelines Roadmap Nelson Hastings, Ph.D. Technical Project Leader for Voting Standards, ITL

TGDC Meeting, Jan 2011 Help America Vote Act (HAVA) Roadmap Nelson Hastings National Institute of Standards and Technology

TGDC Meeting, July 2010 Report on Other Resolutions from Dec 2009 TGDC Meeting John Wack National Institute of Standards and Technology

TDTWG Update to RMS Wednesday January 14. TDTWG Update to RMS Scope Texas Data Transport Working Group (TDTWG) is responsible for creating and maintaining.

TGDC Meeting, Jan 2011 Review of UOCAVA Roadmap Nelson Hastings National Institute of Standards and Technology

NIST Voting Program Activities Update January 4, 2007 Mark Skall Chief, Software Diagnostics and Conformance Testing Division.

1 DECEMBER 9-10, 2009 Gaithersburg, Maryland TECHNICAL GUIDELINES DEVELOPMENT COMMITTEE Commissioner Donetta Davidson.

EAC Voting System Certification Process (Test Plan) - 3/21/05

The VVSG Version 1.1 Overview Matthew Masterson Election Assistance Commission

Doc.: IEEE /0828r0 Submission May 2007 Harry Worstell, AT&TSlide 1 Procedural Clarification Notice: This document has been prepared to assist.

TGDC Meeting, July 2010 Overview of NIST Activities and TGDC Meeting Agenda Martin Herman, PhD National Institute of Standards and Technology

COMET Working Group Progress Report. Contents of Report Provide COMET Recommendations, with May 6 revisions (Vote) –Competitive Meter Approval Process.

TGDC Meeting, Jan 2011 Development of High Level Guidelines for UOCAVA voting systems Andrew Regenscheid National Institute of Standards and Technology.

1 Addendum FALL 2004 RFP Detailed Instructions for Bidder Registration and Proposal Submission ENTERGY SERVICES, INC. November 2004.

Briefing for the EAC Public Meeting Boston, Massachusetts April 26, 2005 Dr. Hratch Semerjian, Acting Director National Institute of Standards and Technology.

Next VVSG Training Security: Testing Requirements October 15-17, 2007 Nelson Hastings Alicia Clay Jones National Institute of Standards and Technology.

Supplier Performance Development Process

Doc.: IEEE /152r0 Submission May 2000 Tom Siep, Texas InstrumentsSlide 1 Project: IEEE P Working Group for Wireless Personal Area Networks.

P1622 Voting Systems Electronic Data Interchange Arthur Keller, chair (self employed at Minerva Consulting, Affiliated with Univ. Calif., Santa Cruz) 1IEEE.

TGDC Meeting, Jan 2011 VVSG 2.0 and Beyond: Usability and Accessibility Issues, Gaps, and Performance Tests Sharon Laskowski, PhD National Institute of.

TGDC Meeting, July 2011 VVSG 1.1 Test Suite Status Mary Brady Manager, NIST Information Systems Group, Software and Systems Division, ITL

The VVSG 2005 Revision Overview EAC Standards Board Meeting February 26-27, 2009 John P. Wack NIST Voting Program National Institute.

COMET Working Group Progress Report 11/13/03

Agenda Item 2: Catch up session 15 May 2017

National Institute of Standards and Technology

PERL 2 – Procedural Changes

Request for Proposal - Best Value

Partnering with the Defense Acquisition University Equivalent Provider Application (EPA) Process Phase I - Details END NO YES Provider contacts an approved.

Request for Proposal - Best Value

Registration Process for new messages

December 18 Project: IEEE P Working Group for Wireless Personal Area Networks (WPANs) Submission Title: TG4i conditional approval package Date Submitted:

February 19 November 2008 Project: IEEE P Working Group for Wireless Personal Area Networks (WPANs) Submission Title: SG-NAN Closing Report for.

Registration Process for new message variants

August 19 November 2008 Project: IEEE P Working Group for Wireless Personal Area Networks (WPANs) Submission Title: SG-NAN Closing Report for Dallas,

Commission Direction on Locational Disaggregation

Submission Title: TG9ma Closing Report for September Meeting

Presentation transcript:

Test Plans, Test Cases, and Test Reports Making every vote count. United States Election Assistance Commission The Testing Process: Test Plans, Test Cases, and Test Reports TGDC Meeting December 9-10, 2009

The Testing Process from Beginning to End (in an ideal world): Making every vote count. United States Election Assistance Commission The Testing Process from Beginning to End (in an ideal world): Application for Testing Creation, submission, and review of Test Plan Creation, submission and review of test cases Testing of the Voting System Creation, submission and review of the test report (This is how things have actually gone)

The Test Plan Package United States Election Assistance Commission Making every vote count. United States Election Assistance Commission The Test Plan Package Test Plan Contains high level test methods for the testing engagement. Contains standard lab test methods, system specific test methods, and pass/fail criteria. Contains EAC Requirements Matrix. EAC Technical Reviewers review test plan for completeness and to ensure system is tested to the standards. EAC sends comments to test plan to VSTL for review and response. VSTL submits new version of test plan for further review.

Test Case Submission United States Election Assistance Commission Making every vote count. United States Election Assistance Commission Test Case Submission After approval of Test Plan VSTL creates system specific test cases. Test cases are the step by step instructions given to test personnel to test the system. VSTL is to notify EAC in writing when test cases are completed. EAC may then request the test cases to be submitted to them for review. Test cases are not approved or rejected by EAC. EAC does spot check of various test cases in order to ensure test plan is being followed and system is being tested to the standards.

Testing of the Voting System Making every vote count. United States Election Assistance Commission Testing of the Voting System Test cases used to test the voting system to the VSS or VVSG. VSTLs are responsible for the quality and level of testing. This includes review and approval of 3rd party lab testing. Maintaining the independence of the testing process. Documentation and resolution of all discrepancies and engineering change orders.

Test Report Submission and Review Making every vote count. United States Election Assistance Commission Test Report Submission and Review Test Report Package represents the documentation of the testing performed and the results of that testing. VSTL submits as part of the package: Requirements Matrix Same matrix as before but with the test report information complete Test Report Documents the testing process and results Includes all discrepancies, resolutions, and engineering change orders. VSTL attestation VSTL makes recommendation and asserts all testing was done per the approved test plan.