VALIDATION RECORD · 13 NOVEMBER 2026
What we checked.
Software checks and scientific benchmarks answer different questions. This record keeps their evidence and limits separate.
Software and hosted workflow checks
- 361 backend tests passed, with no skips and 23 dependency/deprecation warnings.
- 83 disposable full-stack browser tests passed. The live-email test is deliberately excluded from that offline suite.
- Three public-site browser workflows passed: project/experiment/chat persistence and archive/restore; benchmark evidence and mobile layout; measured XRD upload, worker, plot and PDF.
- One fresh hosted email-onboarding browser test passed: link redemption, persisted Neon profile, reload, replay rejection and session revocation on logout.
- The actual Claude provider completed project chat, scoped hypotheses, a peak-evidence tool call and a structured next-experiment proposal. A deliberately inconclusive software-test log produced a cited draft report with PDF, Markdown and JSON exports.
- Twenty jobs for each of six fixed task inputs completed on the deployed worker: 120 successful executions, with stable ranking/outcome fingerprints. These are six distinct inputs, not 120 independent accuracy cases.
- Both pinned original NMR checkpoints loaded strictly. Twenty reference regenerations per nucleus produced 960 structure-only inferences, reproducing prepared shifts with zero delta in this environment. This adds no experimental accuracy cases.
Offline language responses are labelled software fixtures; the separate hosted checks use actual Resend, Neon, scientific workers and Claude. Provider delivery events do not prove inbox placement or that a message was opened. Software automation is not scientist approval.
Scientific model comparisons
Sixteen model/task rows ran twenty complete repetitions, totalling 6,720 sample evaluations. The dashboard preserves per-case expected answers, predictions, abstentions, source hashes, comparisons and timing scope.
| Task | Measured inputs | Observed result |
|---|---|---|
| NMR ¹H | 24 measured shift lists | Top-1 18/24 (75%) |
| NMR ¹³C | 24 lists, sharing molecular groups | Top-1 18/24; coverage 23/24 |
| IR components | 9 mixtures | Micro F1 0.9565; exact set 7/9 |
| Raman original set | 24 measured traces | Top-1 16/24; coverage 18/24 |
| Raman additional development set | 20 parents, six mineral identities | Selected variant 20/20; previous variant 19/20 |
| XRD precursor compounds | 40 scans, 20 paired recipe groups | Micro F1 0.9071; exact set 24/40 |
| General oxide XRD boundary | 12 IUCr measurements | All abstained; zero coverage |
| Extra FTIR calibration research comparison | 14 test scans from 74 | UltraIR MAE 0.0006754, R² 0.9632; not an app task |
Scroll horizontally to see every result →
Limits that matter
The additional Raman set excludes earlier query/reference parents and duplicate traces, but it was used for model selection and is development evidence. It contains six mineral identities, not twenty new classes. NMR and UltraIR upstream training overlap is not independently ruled out. UltraIR uses the source target scale, not a validated physical concentration unit.
The ten-compound precursor XRD task does not fix the general XRD boundary failure or establish arbitrary phase identification. Finite public datasets do not establish transfer to arbitrary laboratories, calibrated probabilities or measured scientist time savings.
A complete checksum-verified opXRD archive inventory counted 92,552 measured patterns, 2,179 nonempty potentially partial phase annotations and 912 records with reported lattice/basis data. This is inventory with zero scored cases, not worldwide model validation.
Inspect the source evidence
Open the full benchmark dashboard after email sign-in, then download its JSON evidence. Read the technical guide for file formats, storage and workflow limits.