Configuration-aware evidence validity: when verified stops meaning verified

A verification matrix that says a requirement is verified is really saying: on a particular date, on a particular configuration, under a particular procedure, a result met the requirement as it was then written.

Programmes change continuously. Firmware is updated, boards are revised, units are swapped after a fault, procedures are corrected and requirements are tightened. Each change is approved through change control, and each is reasonable on its own. What is rarely re-checked is the evidence that was produced before it.

The result is a matrix that is green and wrong. It is usually discovered late: during a configuration audit, at a pre-ship review, or after an anomaly in test or in orbit.

A change does not automatically invalidate evidence. Engineering judgement often shows it is unaffected, for example a connector change on a board whose evidence concerns software timing. The point is that the judgement is made, recorded and reviewable, instead of assumed.

1. 01Change request approved through change control
2. 02Evidence produced on the earlier revision identified
3. 03Affected requirements and reviews identified
4. 04Each item dispositioned by an engineer
5. 05Retain with rationale, delta test, or full reverification
6. 06Requirement closure updated and re-approved
7. 07Affected review gates informed

In manufacturing quality the same logic drives a partial or delta First Article Inspection after a design or process change; DAQIP™ applies it at the characteristic level. AEAIP applies it at the system level, across requirements, verification evidence and review gates.

Configuration-aware validity rests on ordinary, well-established configuration management. Configuration management standards and guidance, such as SAE EIA-649C, ISO 10007 and ECSS-M-ST-40, describe the same core functions:

• Planning — how configuration will be managed on the programme.
• Identification — every configuration item, its documentation and its revision.
• Change control — proposing, evaluating, approving and implementing changes.
• Status accounting — the current, recorded state of every item and change.
• Verification and audit — confirming the product matches its documentation, for example through functional and physical configuration audits before acceptance.

It helps to keep four configurations distinct: as-designed, as-built, as-tested and as-flown. Evidence is produced on the as-tested configuration and relied on for the as-flown one. Every difference between them needs a rationale.

Spacecraft programmes test several physical models: engineering models for development, qualification models or protoflight units for design proof, and flight models for acceptance. Each has its own build history, nonconformances and repairs.

A test article's pedigree records that history and how the article differs from flight. It is what decides whether a result on one unit is evidence for another. In the demo, an imager MTF result on the engineering model is treated as an evidence gap for the flight model because the focal-plane mount differs.

For software, configuration means more than a version number: source revision, build configuration, compiler and library versions, configuration tables and the target hardware. A result is reproducible only if the build is.

The same applies to the tools that produce evidence: analysis software, simulators, test scripts and measurement instruments with their calibration. These are part of the evidence's configuration, and AEAIP records them as such.