FDA Software-Validation Statistical Analysis

A statistical validation pipeline that proved diagnostic-algorithm safety for FDA clearance and cut the validation cycle from six months to three weeks.

English한국어

Role: Technical Lead / Data Scientist — 16-person cross-functional team (Data Scientist 3, Data Engineer 2, biologists 8, patent 3)  ·  Period: 2023.05 – 2023.12  ·  Stack: R, Matlab, Apache Airflow, Quarto, statistical testing

To take a diagnostic product into the North American market, the algorithm’s safety had to be proven statistically — FDA clearance demanded advanced testing beyond the prior software-engineering process. As statistical-analysis lead I:

  • built the statistical validation pipeline end-to-end;
  • developed the in-house Switch Model ablation methodology;
  • led FDA-regulation training that raised the BT and IT teams’ regulatory literacy.
FDA Software-Validation Statistical Analysis — project poster

Highlights

  • Validation cycle 6 months → 3 weeks (87.5% reduction), proving DSP-algorithm safety at 99.2% statistical confidence.
  • Switch Model — an in-house ablation framework (10 scenarios × 8 core modules on/off) that isolates each module’s contribution to Ct and positive/negative calls, quantifying the impact of the four signal-risk-management modules.
  • Two-track testing — structural (code-based) testing plus statistical testing (χ² goodness-of-fit / association for qualitative, repeated-measures ANOVA for quantitative), aligned to SGS guidance (EN 62304) + the FDA General Principles of Software Validation.
  • C++-port statistical tests implemented at a low level — 2-way repeated-measures ANOVA, McNemar, Breslow-Day, Cochran-Mantel-Haenszel.
  • 200-page V&V report semi-automated from an Airflow (ETL) → R + Quarto pipeline, establishing the company’s first integrated reagent + algorithm performance-evaluation system.

Approach

A statistical validation system proves algorithm safety by combining structural and statistical testing over standardized, QC’d inputs, with report generation automated end-to-end.

flowchart TB
    IN[Device raw data +<br/>experiment design] --> QC[5-stage data QC<br/>+ cross-check]
    QC --> ST[Structural testing<br/>code-based]
    QC --> SS[Statistical testing<br/>ANOVA / McNemar / CMH]
    ST --> SW[Switch Model<br/>10 scenarios x 8 modules]
    SS --> SW
    SW --> REP[200-page V&V report<br/>Airflow to R + Quarto]
    REP --> OUT[6 months to 3 weeks<br/>99.2% confidence]