# Synthetic calibration software rerun

Run `python reproduce.py` with numpy and matplotlib. Standalone reproduction using a bundled numeric baseline, without importing legacy files. Forty seed/scenario metric rows and four summaries; original two-panel figure PNG/SVG/PDF; runtime and generator/baseline/source SHA256 metadata. All180 original scalar metric comparisons match exactly in the recorded runtime. The rerun itself does not modify legacy files. After its numeric snapshot was captured, root corrected only legacy baseline labels; numeric values are unchanged (see the project correction history).

Ten seeds41001–41010 each generate5,000 independent training and5,000 test rows with six normally distributed features. Binary labels follow a known logistic model with coefficients[-0.25,0.9,-0.6,0.45,0.25,-0.3,0.2], intercept first. Candidate logistic regression deliberately matches that family: Newton updates, slope penalty0.01, unpenalized intercept. The negative control permutes training labels. RNG order matches the original.

The stress RNG uses seed+100000, fresh training rows, shifted test features with mean0.75 and scale1.25 and intercept reduced by0.8. A separate version removes15% of feature entries independently at random in training and test; imputation uses training feature means only. The labels remain the same generated labels. These are chosen failure examples, not population models.

Every baseline is the constant TRAINING-set prevalence, evaluated on the corresponding test labels. This corrects a misleading stored summary-field name suggesting test prevalence without changing the numbers. Brier score is mean squared probability error; Brier skill=1-candidate Brier/baseline Brier. Positive skill means improvement over this baseline. Log loss uses probabilities clipped to[1e-9,1-1e-9]. Calibration discrepancy is the sample-weighted absolute difference between average prediction and observed frequency in ten equal-width probability bins. Bin endpoints follow the source convention: lower inclusive, upper exclusive except the last bin includes1. This finite bin-dependent metric is not proof of calibration everywhere.

Matched median Brier skill0.2305824 and discrepancy0.0156152; shifted0.2287901 and0.1300481; shifted/missing0.1827967 and0.1104681. The negative-control medians are-0.0010482 and0.0073664. A near-constant prediction can show small binned calibration discrepancy while being uninformative: low calibration discrepancy alone does not establish useful discrimination. The figure shows all ten seed points and median bars, not confidence intervals.

This reruns a software check on deliberately synthetic data. It does not evaluate learning, adaptive instruction, real people, fairness, compatibility or educational benefit. Better Brier score during transfer does not guarantee calibrated probabilities. The stored matched-case pass thresholds are not new success criteria for shifted cases. No real-data validation or actionable decision threshold is supplied.