3 Verification failed, blocked or not run
The requirement links a verification, but its latest result is not a clean pass: it failed, was blocked, has not run, was inconclusive or deferred, passed with a deviation, or has no row in the results. Until a passing result is recorded against the current version, the chain has no proof at its end.
- the test
- a linked verification whose latest result is failed, blocked, not run, inconclusive, deferred or passed with a deviation, or has no result row
- the question
- What is the analysis of the result, is a re-run or a problem report open, and when will a passing result be recorded against this version?
- asked of
- the verification lead
- columns it reads
- Verified By and the results: Result and Date
With no results pasted and no result column, whether a verification ran is not read: the page says so once, and never calls a requirement not run on a blank.
The clauses behind it, by standard
each shown when you tick its standard- ISO 26262: Part 8, 9.4.3, Part 5, 10 (hardware requirements), Part 6, 10.4 (software requirements and high-level requirements) and Part 6, 9.4 (software unit or detailed designs and low-level requirements)
- Automotive SPICE: SYS.5.BP3 (system level) and SWE.6.BP3 (software requirements)
- IEC 62304: 5.5.5 (software unit or detailed designs and low-level requirements), 5.6.7 and 5.8.1
- ISO 13485: 7.3.6
- FDA QMSR: 820.10
- DO-178C: A-7.2
- AS9100D: 8.3.4
The clause text
Verification is executed per the specification, the results evaluated against the expected results and criteria, deviations and anomalies analysed, and the verification report produced. Requirement text not held.
Held text: ISO 26262:2018, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Verification reports with evaluation of every deviation
where it usually breaks Failed tests closed without analysis
The hardware is integrated and verified against the hardware safety requirements using the methods of the part's tables for deriving test cases and for test methods by ASIL (part 11 Tables 27 and 28 interpret Tables 10 and 11 for semiconductors), including safety mechanism verification, environmental and stress tests and fault injection where applicable. Requirement text not held.
Held text: ISO 26262:2018, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Hardware integration and verification plan and results by ASIL-selected methods
where it usually breaks Hardware verified functionally with the safety mechanisms never exercised
The software elements are integrated per the architectural design and the integration verified against it using the methods the standard recommends by ASIL (requirements-based test, interface test, fault injection, resource usage evaluation, back-to-back test), with structural coverage at the function and call level and the verification of freedom from interference and timing. Requirement text not held.
Held text: ISO 26262:2018, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Software integration verification results with function and call coverage
where it usually breaks Integration verified with unit tests re-run
Software units are verified against the unit design and the software safety requirements using the methods of Table 7 (walk-through, pair programming, inspection, semi-formal and formal verification, control flow analysis, data flow analysis, static code analysis, static analyses based on abstract interpretation, requirements-based test, interface test, fault injection test, resource usage evaluation, back-to-back comparison test between model and code), with test cases derived per Table 8 (analysis of requirements, equivalence classes, boundary values, error guessing) and structural coverage measured per Table 9 (statement, branch, MC/DC by ASIL). Requirement text not held.
Held text: ISO 26262:2018, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Unit verification specification with methods by ASIL; Static analysis results against the coding guidelines; Structural coverage measurements with the target per ASIL
where it usually breaks Coverage measured at statement level for an ASIL D unit; Static analysis run without MISRA or an equivalent rule set
Software unit verification is performed and the results documented. Applies to classes B and C.
Applies to software safety class B and C: a line of another class does not carry it.
Held text: IEC 62304:2015, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Unit verification results.
where it usually breaks Unit test results discarded after a passing run.
The integration test record documents the result (pass or fail with a list of anomalies), retains enough to repeat the test (test case specifications with actions and expected results, equipment, test environment including tools), and identifies the tester. Applies to classes B and C.
Applies to software safety class B and C: a line of another class does not carry it.
Held text: IEC 62304:2015, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Integration test records with the three contents.
where it usually breaks Test records that cannot say which environment produced them.
All software verification activities are completed and their results evaluated before the software is released. Applies to classes A, B and C.
Applies to software safety class A, B and C: a line of another class does not carry it.
Held text: IEC 62304:2015, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Verification completion evidence at release.
where it usually breaks Release with open verification tasks deferred to 'post-release'.
Verification of design and development follows planned, documented arrangements and confirms that the outputs satisfy the input requirements. The organization documents verification plans setting out the methods, the acceptance criteria and, where appropriate, statistical techniques together with the reasoning behind the sample size. If the intended use means the device must connect or interface with other devices, verification must show the outputs meet the inputs in that connected state. Records are kept of verification results, conclusions and any actions needed.
Held text: ISO 13485:2016, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Verification plans with methods, acceptance criteria and sample-size rationale; Verification reports and records of actions; Interface or connected-use verification where applicable; Traceability showing every input verified
where it usually breaks Sample sizes chosen without a documented rationale; Verification of interfaces with other devices omitted; Inputs with no corresponding verification evidence
Section 820.10 establishes the substantive QMS requirements by incorporating ISO 13485:2016 SECTIONS 4 THROUGH 8 in full. Manufacturers must implement: ISO 13485:2016 SECTION 4 General QMS requirements + documentation (Quality Manual + Medical Device File + control of documents + records); SECTION 5 Management responsibility + customer focus + quality policy + planning + responsibility / authority / communication + management review; SECTION 6 Resource management (provision of resources + human resources + infrastructure + work environment + contamination control); SECTION 7 Product realization (planning + customer-related + design and development - the medical-device 'design controls' parallel + purchasing + production and service provision + control of monitoring + measuring equipment); SECTION 8 Measurement + analysis + improvement (monitoring + measurement + control of nonconforming product + analysis of data + improvement - including CAPA Sections 8.5.2 + 8.5.3). NB: ยง820.10 also clarifies that ISO 13485:2016 internal-audit + management-review requirements (Sections 8.2.4 + 5.6) apply to the QMSR.
Held text: 21 CFR Part 820 (QMSR), our statement of the clause, not the instrument verbatim.
what an assessor asks to see ISO 13485 Section 4-8 compliance file; Internal audit programme + records; Management review records + frequency; Cross-reference matrix between QMSR + ISO 13485 + EU MDR/IVDR + FDA Part 11
where it usually breaks ISO 13485 implemented in parts without full Section 4-8 coverage; Internal audit infrequent or scope-limited; Management review absent + meeting minutes incomplete
Test results are checked for correctness and each discrepancy between expected and actual results is explained. Table A-7 objective 2, described in 6.4.5.c; applies at levels A, B, C, independence required at level A; activities 6.4.5; outputs Software Verification Results.
Applies at software level A, B, C: a line at another level does not carry it.
Held text: DO-178C / ED-12C, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Review of test results with every discrepancy explained and dispositioned
where it usually breaks Failures left unexplained or re-run without analysis
The design and development process is controlled so that: the intended results are set out; reviews judge whether results can meet requirements; verification confirms that outputs match inputs; validation confirms that the resulting products satisfy the requirements of their intended use; problems are acted on; records are kept; and, under 9100, authorization is given before moving to the next stage, with people from the functions involved in the stage under review taking part in design reviews. 9100 adds that tests needed for verification and validation are planned, controlled, reviewed and documented, so that: test plans name the item tested, the resources, the aims, the conditions, the parameters to be recorded and the acceptance criteria; test procedures explain how the test is run and performed and how results are recorded; the right configuration is presented for test; the plan and procedures are followed; and the acceptance criteria are satisfied. Equipment used to measure tests is controlled under 7.1.5. When design is complete, reports, calculations and test results show that the design satisfies the specification under every operating condition identified.
Held text: AS9100D, our statement of the clause, not the instrument verbatim.
what an assessor asks to see Design review, verification and validation records with stage authorization.; Test plans and procedures meeting a) to e) with configuration of the test item recorded.; Completion evidence covering all identified operational conditions.
where it usually breaks Design reviews without representatives of manufacturing or quality.; Tests run on an undocumented configuration.; Next stage started before authorization.
Verify the integrated system using the chosen measures and record each result as pass or fail together with the measure data; results that deviate are handled under SUP.9.
Held text: Automotive SPICE v4.1, our statement of the clause, not the instrument verbatim.
what an assessor asks to see System verification results (15-52) with pass or fail per measure, the system build tested and the test date
where it usually breaks System verification run on a configuration differing from the delivered system; Failed system tests not raised as problem reports under SUP.9
Verify the integrated software with the selected measures and record the results with pass/fail status and the measure data; deviating results go to SUP.9.
Held text: Automotive SPICE v4.1, our statement of the clause, not the instrument verbatim.
what an assessor asks to see 15-52 Verification Results; 08-58 Verification Measure Selection Set; 03-50 Verification Measure Data; 08-60 Verification Measure
where it usually breaks Qualification tests run on a software build other than the one released to the customer; Failed qualification tests marked as known issues without a problem report or customer agreement