Purpose

Define user requirements for a standalone FUZKK utility that accepts laboratory CSV data, evaluates the records using limits embedded in code, and produces LabWare-compatible JSON.

Scope

The utility is intended for preliminary QC/QA review, integration testing, LIMS/LabWare flow and evidence-trail preparation. Final release decisions remain under the laboratory's validated procedure and responsible personnel.

Users

QC analyst, QA reviewer, CSV/validation engineer, LIMS/LabWare integration engineer, responsible laboratory specialist.

User requirements

1. The utility shall run without arguments and print its self-description, a built-in input.csv example from GetDemoData(), and demo evaluation for the embedded records.
2. The utility shall run with two arguments: input.csv output.json.
3. The utility shall not read input.csv and shall not write output.json when started without arguments.
4. CSV numeric values shall be parsed using CultureInfo.InvariantCulture.
5. Output shall be generated as LabWare-compatible JSON with Header, Samples, Results, Status, StatusCode, ErrorMessage, Description and DescriptionEN.
6. For PASS records, ErrorMessage shall be an empty string.
7. Embedded limits shall follow this priority: Ph. Eur. → British Pharmacopoeia / UK implementation → EAEU / regional requirements → EMA/ICH/EU guidance → USP fallback.
8. If an exact monograph is not known, strict standard API limits are used where applicable: assay 98–102%, total impurities ≤1.0%, individual impurity ≤0.5%.
9. For biologics and mAb-like products, aggregation, sterility and endotoxin checks shall be included where relevant to the utility purpose.
10. If a parameter may arrive in different units, the unit shall be represented as a separate input field or explicitly reflected in the input.csv field name.

Input CSV

BatchNumber,ProductName,Marker_1_Conc,Marker_2_Conc,Marker_3_Conc,Marker_4_Conc,Ref_Marker_1_Mean,Ref_Marker_1_StDev,Ref_Marker_2_Mean,Ref_Marker_2_StDev,Mahalanobis_Distance,Cluster_ID
TCM-CLUS-2026-001,Panax Notoginseng,25.0,30.0,10.0,80.0,24.0,1.0,29.0,1.5,1.2,1.0
TCM-CLUS-2026-002,Panax Notoginseng,24.5,29.5,10.2,79.0,24.0,1.0,29.0,1.5,0.8,1.0
TCM-CLUS-2026-003,Panax Notoginseng,20.0,25.0,8.0,70.0,24.0,1.0,29.0,1.5,5.5,2.0

input.csv fields

Utility description

Batch Similarity Clustering Utility — TCM Batch Similarity and Clustering Utility

Batch Similarity Clustering Utility — TCM Batch Similarity and Clustering Utility

ℹ️ Utility performs multivariate statistical analysis (PCA/HCA):
• Mahalanobis Distance (≤3.0)
• Main cluster assignment
• Key marker Z-Scores (±2σ)

⚠️ CRITICAL: Natural raw materials are variable!
Statistical methods help distinguish normal variation from defects.

Usage:
BatchSimilarityClusteringUtility.exe → demo mode (console output)
BatchSimilarityClusteringUtility.exe input.csv output.json → evaluate your data

Input format:
BatchNumber,ProductName,Marker_1_Conc,Marker_2_Conc,Marker_3_Conc,Marker_4_Conc,Ref_Marker_1_Mean,Ref_Marker_1_StDev,Ref_Marker_2_Mean,Ref_Marker_2_StDev,Mahalanobis_Distance,Cluster_ID

Example:
TCM-CLUS-2026-001,Panax Notoginseng,25.0,30.0,10.0,80.0,24.0,1.0,29.0,1.5,1.2,1.0

— WHY IS THIS NEEDED?
Statistical similarity assessment is critical for TCM standardization (Yunnan Baiyao, Tong Ren Tang):
• Herbal chemical composition depends on soil, climate, and harvest time
• Single limits do not always reflect overall profile quality
• PCA (Principal Component Analysis) and HCA (Hierarchical Cluster Analysis) allow visualization of batch grouping
• Outlier detection helps prevent release of atypical raw materials

⚠️ CRITICAL:
• Mahalanobis Distance ≤3.0 — batch is within confidence ellipse of "good" batches
• Cluster 1 assignment — matches main historical profile
• Marker Z-Scores within ±2σ — no anomalies in key substances
• Clustering stability when adding new data

Key features:
• Multivariate analysis instead of univariate control
• Statistical distance calculation to distribution center
• Automatic cluster assignment
• Support for historical data to build reference model

Critical parameters:
• Mahalanobis Distance: ≤3.0
• Cluster ID: 1 (Main)
• Z-Score: ±2.0 σ

💡 Usage tips:
1. Use at least 20-30 historical batches to build reliable model
2. Update reference means and standard deviations when supplier changes
3. Combine clustering with chromatographic fingerprinting
4. Investigate reasons for batch falling into different cluster (geography, weather)
5. Visualize data on PCA Score Plot for clarity

⚠️ Note: Unlike synthetic drugs where each batch must be identical, herbs allow natural variability. The utility helps define boundaries of this "normal" variability using statistical limits rather than rigid specifications.

Traceability and limitations

• The URS is used as the source document for functional specification, CSV review and later validation work.
• This document does not replace an approved pharmacopoeial monograph, validated analytical method or internal product specification.
• For product-specific limits, the approved customer specification takes priority.

The functional specification describes the behaviour of the standalone FUZKK console utility, input-data format, evaluation algorithm and output JSON structure.

Functional flow

1. Main() checks the number of arguments.
2. If no arguments are provided: PrintHello() prints the description and built-in input.csv example, then RunDemoEvaluation() executes Evaluate() over GetDemoData() and prints demo JSON.
3. If two arguments are provided: RunWithFiles(input.csv, output.json) reads CSV, evaluates each record and writes LabWare-compatible JSON.
4. LoadData() uses CultureInfo.InvariantCulture and shall not be called in no-arguments mode.
5. Evaluate() returns a named tuple with BatchNumber, ProductName, Parameters, CriticalFailCount, WarningCount, Recommendation and RecommendationEN.
6. GetIssues() builds messages for ErrorMessage in WARNING/FAIL cases.

Evaluation rules

• PASS: CriticalFailCount = 0 and WarningCount = 0.
• WARNING: CriticalFailCount = 0 and WarningCount > 0.
• FAIL: CriticalFailCount > 0.
• ERROR: exception during reading or processing.
• ErrorMessage remains empty for PASS.
• Limits are embedded in Program.cs; no external limit configuration is required.

Input and fields

BatchNumber,ProductName,Marker_1_Conc,Marker_2_Conc,Marker_3_Conc,Marker_4_Conc,Ref_Marker_1_Mean,Ref_Marker_1_StDev,Ref_Marker_2_Mean,Ref_Marker_2_StDev,Mahalanobis_Distance,Cluster_ID
TCM-CLUS-2026-001,Panax Notoginseng,25.0,30.0,10.0,80.0,24.0,1.0,29.0,1.5,1.2,1.0
TCM-CLUS-2026-002,Panax Notoginseng,24.5,29.5,10.2,79.0,24.0,1.0,29.0,1.5,0.8,1.0
TCM-CLUS-2026-003,Panax Notoginseng,20.0,25.0,8.0,70.0,24.0,1.0,29.0,1.5,5.5,2.0

Output JSON

{
  "Header": {
    "UtilityName": "Batch_Similarity_Clustering_Utility",
    "Version": "1.0.0",
    "Timestamp": "UTC",
    "InstrumentID": "FUZKK-QC-WORKSTATION",
    "OperatorID": "Admin"
  },
  "Samples": [
    {
      "SampleID": "from BatchNumber",
      "BatchNumber": "from CSV",
      "ProductName": "from CSV",
      "TestName": "utility-specific test",
      "AnalysisCode": "utility-specific code",
      "Status": "PASS | WARNING | FAIL | ERROR",
      "StatusCode": "1 | 2 | 0 | -1",
      "ErrorMessage": "",
      "Description": "Russian recommendation",
      "DescriptionEN": "English recommendation",
      "Results": [
        {
          "ParameterName": "parameter",
          "ResultValue": 0.0,
          "UnitOfMeasure": "unit",
          "SpecificationLimit": "limit",
          "IsWithinSpec": true
        }
      ]
    }
  ]
}