Utility description: Classifier Version Control
Classifier Version Control Checker — ML Classifier Version Control
ℹ️ Utility performs strict verification of all ML classifier components before clinical production launch according to FDA 21 CFR Part 11, EU AI Act and CAP/CLIA:
• Model Weights: Version AND SHA256 hash reconciliation to guarantee binary file identity.
• Configuration: Verification of inference config files (thresholds, preprocessing parameters).
• Dependencies: Version control of all libraries (PyTorch, TensorFlow, NumPy, Python).
• Containers: Docker/Singularity image digest verification.
⚠️ IMPORTANT:
• Medical ML classifier = medical device. Any change = new version requiring revalidation.
• NumPy update from 1.24 to 1.26 may change float operation results and model predictions.
Usage:
ClassifierVersionControlChecker.exe → demo mode (console output)
ClassifierVersionControlChecker.exe input.csv output.json → evaluate your data
Input format:
RunID,ClassifierName,ComponentType,ComponentName,ActualVersion,ValidatedVersion,ActualHash,ValidatedHash
Example:
RUN-001,VarPath_v3,ModelWeights,model.pt,3.2.1,3.2.1,a1b2c3,a1b2c3
📍 Scope of Application (Usage Where):
• Clinical NGS Laboratories: Gatekeeper before each classifier run.
• MLOps/CI-CD: Automatic check before production deployment.
• Regulatory Inspections: Proof of using only validated versions.
• Change Management: Documentation of configuration drift.
— WHY IS THIS NEEDED?
Unlike traditional software, ML models are sensitive to minute environment changes.
"Silent" dependency update via pip may invisibly change clinical result.
Weight hash verification excludes model substitution, file corruption, or unvalidated hotfix usage.
This is fundamental reproducibility requirement for SaMD.
⚠️ CRITICAL:
• Exact Match Only: No "compatibility" — only exact version and hash match.
• Hash Verification: Model version may match but file replaced — hash is mandatory.
• Transitive Dependencies: Check ALL dependencies including transitive ones.
• Container Digest: "latest" or "v3.2" tag is insufficient — sha256 digest required.
Key features:
• Dual verification: version + hash for critical components
• Support for 4 component types (Weights, Config, Dependency, Container)
• Instant blocking upon any mismatch
• Detailed discrepancy report for each component
• Compliance with GMLP Principle 5 (Software Configuration Management)
Critical parameters:
• Model Weights: Version + SHA256 exact match
• Config Files: Version + hash match
• Dependencies: Exact version string match
• Container: SHA256 digest match
💡 Usage tips:
1. Manifest: Maintain YAML/JSON manifest of validated environment in repository.
2. Lock Files: Use pip-tools/poetry/conda-lock to pin transitive dependencies.
3. Artifact Registry: Store model weights in artifact registry (MLflow, DVC) with hashes.
4. Immutable Containers: Build containers deterministically, pin digest.
5. Integration: Embed checker as first step in Nextflow/Snakemake/Airflow pipeline.
⚠️ Note: This utility is a Software Configuration Management tool. It does not replace functional model validation but guarantees that exactly the validated configuration is being used.input.csv
RunID,ClassifierName,ComponentType,ComponentName,ActualVersion,ValidatedVersion,ActualHash,ValidatedHash
NGS-RUN-2026-001,VariantPathogenicity_v3,ModelWeights,model_v3.pt,3.2.1,3.2.1,a1b2c3d4e5f6,a1b2c3d4e5f6
NGS-RUN-2026-001,VariantPathogenicity_v3,Config,inference_config.yaml,3.2.1,3.2.1,f6e5d4c3b2a1,f6e5d4c3b2a1
NGS-RUN-2026-001,VariantPathogenicity_v3,Dependency,torch,2.1.0+cu118,2.1.0+cu118,,
NGS-RUN-2026-002,VariantPathogenicity_v3,ModelWeights,model_v3.pt,3.2.1-hotfix,3.2.1,x9y8z7w6v5u4,a1b2c3d4e5f6
NGS-RUN-2026-002,VariantPathogenicity_v3,Dependency,numpy,1.26.0,1.24.3,,
NGS-RUN-2026-002,VariantPathogenicity_v3,Container,classifier-docker,sha256:def456,sha256:abc123,def456,abc123
URS & FS — User Requirements and Functional Specification
This document describes the controlled interface and behaviour of ClassifierVersionControlChecker for Classifier Version Control Checker.
Domain limits and critical parameters
Key fragments from the source description are shown below. Before production use, limits must be verified against the approved specification, registration dossier and local SOPs.
- ⚠️ CRITICAL:
- • Dual verification: version + hash for critical components
- Critical parameters:
URS — User Requirements Specification
| ID | Requirement | Criticality | Acceptance criterion |
|---|
| URS-001 | The utility shall accept an input.csv file for Classifier Version Control Checker with headers defined in the data contract. | High | The file is processed without manual header editing. |
| URS-002 | The utility shall perform deterministic QC evaluation without machine learning and without probabilistic conformance decisions. | High | Identical input data, rule version and configuration produce reproducible results. |
| URS-003 | The utility shall validate mandatory fields, data types, ranges, units and domain plausibility. | High | Schema, conversion and range errors are explicitly reported. |
| URS-004 | The utility shall apply domain limits and rules from the description, approved specification, registration dossier and local SOPs. | High | Each check has PASS/WARNING/FAIL and a clear message. |
| URS-005 | The utility shall generate output.json with machine-readable results, source values, warnings, failures and critical findings. | High | JSON is suitable for LIMS/ELN/MES integration and QA/QC review. |
| URS-006 | The utility shall preserve traceability between batch/sample, input file, applied rules and final status. | High | Output contains identifiers, checked parameters and audit metadata. |
| URS-007 | The documentation shall support IQ/OQ/PQ, CSV/CSA and review by internal QA or inspectors. | Medium | URS, FS, input/output contract and test scenarios are supplied with the utility. |
| URS-008 | The utility shall be used as a QC decision-support tool and not as a substitute for approved specifications and QA/QP release decision. | Medium | Documentation states change control and limit-verification expectations. |
input.csv contract
| # | Field | Type | Sample | Purpose |
|---|
| 1 | RunID | string / controlled vocabulary | NGS-RUN-2026-001 | Controlled input parameter for deterministic QC rules. |
| 2 | ClassifierName | string / controlled vocabulary | VariantPathogenicity_v3 | Biological/molecular component controlled as a CQA. |
| 3 | ComponentType | string / controlled vocabulary | ModelWeights | Controlled input parameter for deterministic QC rules. |
| 4 | ComponentName | string / controlled vocabulary | model_v3.pt | Controlled input parameter for deterministic QC rules. |
| 5 | ActualVersion | string / controlled vocabulary | 3.2.1 | Controlled input parameter for deterministic QC rules. |
| 6 | ValidatedVersion | string / controlled vocabulary | 3.2.1 | Controlled input parameter for deterministic QC rules. |
| 7 | ActualHash | string / controlled vocabulary | a1b2c3d4e5f6 | Controlled input parameter for deterministic QC rules. |
| 8 | ValidatedHash | string / controlled vocabulary | a1b2c3d4e5f6 | Controlled input parameter for deterministic QC rules. |
RunID,ClassifierName,ComponentType,ComponentName,ActualVersion,ValidatedVersion,ActualHash,ValidatedHash
NGS-RUN-2026-001,VariantPathogenicity_v3,ModelWeights,model_v3.pt,3.2.1,3.2.1,a1b2c3d4e5f6,a1b2c3d4e5f6
NGS-RUN-2026-001,VariantPathogenicity_v3,Config,inference_config.yaml,3.2.1,3.2.1,f6e5d4c3b2a1,f6e5d4c3b2a1
NGS-RUN-2026-001,VariantPathogenicity_v3,Dependency,torch,2.1.0+cu118,2.1.0+cu118,,
Input validation rules
| ID | Field | Rule | Criticality |
|---|
| VR-001 | RunID | The field shall match an approved dictionary or accepted string representation. | High |
| VR-002 | ClassifierName | The field shall match an approved dictionary or accepted string representation. | High |
| VR-003 | ComponentType | The field shall match an approved dictionary or accepted string representation. | High |
| VR-004 | ComponentName | The field shall match an approved dictionary or accepted string representation. | Medium |
| VR-005 | ActualVersion | The field shall match an approved dictionary or accepted string representation. | Medium |
| VR-006 | ValidatedVersion | The field shall match an approved dictionary or accepted string representation. | Medium |
| VR-007 | ActualHash | The field shall match an approved dictionary or accepted string representation. | Medium |
| VR-008 | ValidatedHash | The field shall match an approved dictionary or accepted string representation. | Medium |
FS — Functional Specification
| ID | Function | Implementation |
|---|
| FS-001 | CLI execution | Support execution modes: demo mode without arguments and production mode input.csv output.json. |
| FS-002 | CSV import | Read input.csv in UTF-8/CSV-compatible format and validate header and expected columns. |
| FS-003 | Schema validation | Check mandatory fields, column count, unknown key fields and empty mandatory values. |
| FS-004 | Type conversion | Convert numeric, flag and text values; invalid format is recorded as a row-level error. |
| FS-005 | Domain rule engine | Apply rules for Classifier Version Control Checker, including critical limits from the description and approved specification. |
| FS-006 | Status aggregation | Produce final status: FAIL for critical failure, WARNING for non-critical deviation, PASS for conformance. |
| FS-007 | JSON export | Write output.json with detailed checks, source values, warnings, failures and critical findings. |
| FS-008 | Audit support | Keep result structure suitable for review, deviation investigation and calculation reproduction. |
| FS-009 | Integration contract | Support the scenario LIMS/ELN/MES → input.csv → utility → output.json → portal/admin review. |
| FS-010 | Error handling | Return explicit messages for missing file, empty CSV, invalid schema, output write failure and invalid format. |
Example output.json
{
"utilityId": "classifierversioncontrolchecker",
"utilityFolder": "ClassifierVersionControlChecker",
"package": "LiquidBiopsy",
"overallStatus": "PASS|WARNING|FAIL",
"sourceFile": "input.csv",
"processedAtUtc": "2026-06-10T00:00:00Z",
"checks": [
{
"parameter": "RunID",
"value": "NGS-RUN-2026-001",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-001"
},
{
"parameter": "ClassifierName",
"value": "VariantPathogenicity_v3",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-002"
},
{
"parameter": "ComponentType",
"value": "ModelWeights",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-003"
},
{
"parameter": "ComponentName",
"value": "model_v3.pt",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-004"
},
{
"parameter": "ActualVersion",
"value": "3.2.1",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-005"
},
{
"parameter": "ValidatedVersion",
"value": "3.2.1",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-006"
},
{
"parameter": "ActualHash",
"value": "a1b2c3d4e5f6",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-007"
},
{
"parameter": "ValidatedHash",
"value": "a1b2c3d4e5f6",
"status": "PASS|WARNING|FAIL",
"message": "Deterministic rule-based check result",
"ruleReference": "FS-RULE-008"
}
],
"criticalFindings": [],
"warnings": [],
"audit": {
"inputHash": "sha256:<calculated at runtime>",
"rulesVersion": "<utility executable version>",
"documentation": "ClassifierVersionControlChecker.documentation.html"
}
}
Traceability matrix
| URS | FS | Test | Evidence |
|---|
| URS-001 | FS-001, FS-002 | OQ-001 | Verify execution and import of valid input.csv. |
| URS-002 | FS-005, FS-006 | OQ-004 | Repeat the same dataset and compare output.json. |
| URS-003 | FS-003, FS-004, FS-010 | OQ-002, OQ-003 | Verify missing columns and invalid types. |
| URS-004 | FS-005, FS-006 | OQ-004, PQ-001 | Verify critical deviations on real/boundary data. |
| URS-005 | FS-007, FS-009 | OQ-005 | Verify JSON schema and downstream-system suitability. |
| URS-006 | FS-008 | OQ-006 | Verify identifiers and audit metadata. |
| URS-007 | FS-008, FS-010 | IQ-001, OQ-007 | Verify documentation completeness and control evidence. |
| URS-008 | FS-005, FS-008 | PQ-002 | Verify review workflow and no replacement of QA decision. |
IQ/OQ/PQ test scenarios
| ID | Scenario | Expected result |
|---|
| IQ-001 | Verify executable, input.csv, documentation and checksum availability. | Delivery set is complete; version is recorded. |
| OQ-001 | Valid sample row from input.csv. | PASS or acceptable WARNING according to rules. |
| OQ-002 | Remove a mandatory CSV column. | Schema error or FAIL with missing-column reference. |
| OQ-003 | Place a non-numeric value into a numeric field. | Type-conversion error with row/field reference. |
| OQ-004 | Set a critical parameter outside the limit. | FAIL and critical finding. |
| OQ-005 | Verify output.json structure. | All mandatory sections are present and JSON is valid. |
| OQ-006 | Verify batch/sample traceability. | Input and result identifiers match. |
| PQ-001 | Verify 3–5 real user batches/samples. | Result is confirmed by QC/QA review. |
| PQ-002 | Verify deviation workflow and manual QA decision. | Utility supports review but does not replace approved decision. |
QA/QC and change control
- Do not rename columns without updating validator, documentation and test set.
- Retain
input.csv, output.json, executable version and checksum. - Before production use, perform IQ/OQ/PQ or equivalent CSV/CSA verification.
- Critical limits shall be verified against the approved specification, registration dossier and local SOPs.
- The utility provides structured QC decision support; final release decision remains with QA/QP and approved procedures.