The Voluntary Voting System Guidelines (VVSG) were developed in 2005 by the Technical Guidelines Development Committee (TGDC), working in conjunction with the National Institute of Standards and Technology (NIST) and the Election Assistance Commission (EAC). The VVSG is a detailed specification/standard consisting of hundreds of requirements that must be implemented by voting systems in order to help ensure that the voting system is accurate, secure usable, and accessible. The standard  is used by the EAC to certify voting systems.  EAC certification is accomplished via Voting System Test Laboratories (VSTLs), who test each manufacturer’s voting system to determine whether the voting system meets the requirements in the VVSG. At the conclusion of testing, the VSTL issues a report to the EAC detailing the results of the testing and a recommendation of whether the voting system  should be certified.

Currently, each VSTL develops its own set of cases to test all the requirements of the VVSG.  However, since these test cases are proprietary, the public has no way to determine whether the test cases correctly and completely test for all VVSG requirements.  Also, since these test cases are developed individually by each VSTL, there is no way to ensure consistency in testing among all of the VSTLs.  Furthermore, many of the requirements in the VVSG are at a high level, vague, and open to interpretation.

In order for the VSTLs and the EAC to correctly do their job, they need to be able to break down each and every requirement in the VVSG into unambiguous, specific and testable conditions.  The development and promulgation of test assertions is the best way to accomplish this.  These test assertions are conditions that must be tested to determine conformance to specific requirements within the VVSG.  The goal is to eventually have one or more test assertions for each and every requirement in the VVSG.

We are currently leading a project to develop these test assertions for the Voluntary Voting System Guidelines (VVSG).  The initial set of test assertions targets the usability and accessibility sections of the VVSG.  Future assertions will target other sections of the VVSG, eventually resulting in a complete set of assertions for the entire standard.  The assertions, which are developed for each requirement within the VVSG, break down higher level requirements into discrete and specific testable conditions.  The complete set of test assertions will be distributed to Voting System Test Laboratories (VSTLs) for their use during certification testing.  The assertions ensure that the VSTLs test each requirement in the VVSG correctly and comprehensively.  Furthermore, since these test assertions will be mandated for use by all VSTLs, the assertions help ensure that the testing is uniform and consistent among all of the Laboratories, thus ensuring the same pass/fail parameters regardless of which Laboratory is used to test a specific voting system. 

Contributed by:  Mark Skall