A region-by-region audit of how a regularized atmospheric-profile residual model compares with the exact physical baseline on fixed, unseen-station holdouts—plus the uncertainty, weak slices, failed regions, licensing limits and reasons it remains off.
Candidate
32-feature ridge
Validation
Station + storm CV
Deployment
Disabled
Fixed held-out evaluation
Every card compares the exact MySnow Kuchera baseline with the profile-aware full-feature ridge on one untouched holdout. Positive MAE and RMSE reductions are good; bias is signed and judged by absolute magnitude.
Relative improvement from original Kuchera to the full profile-aware ridge.
All four fixed holdouts improve in MAE and RMSE.
This is not deployment evidence. Small regional holdouts, unresolved U.S. label authorization, and profile-source licensing keep the candidate research-only.
Synchronized regional explorer
Tabs update every chart, table and slice below. The selected region is encoded in the URL for sharing.
Continental mountain snow
Three entirely unseen stations in a WY2025 holdout after WY2024 training.
Mean absolute error
43.5% lower
Root mean square error
36.5% lower
Signed bias
improves |bias| by 4.393
Lower is better · exact holdout values
MAE improves from 5.180 to 2.928. RMSE improves from 6.084 to 3.866.
Centered on zero · negative means underprediction
Cluster-aware sampling uncertainty
Intervals show MAE(full ridge) − MAE(Kuchera). Negative values favor the profile-aware model; zero means no difference. Each interval uses 2,000 deterministic bootstrap replicates.
Every interval remains left of zero under both cluster definitions. These intervals describe sampling variation within the observed station or storm clusters—not universal model certainty.
They resample observed stations or storms as clusters and describe sampling variation inside these fixed holdouts. They are not confidence intervals for worldwide performance, do not establish exchangeability, and are not formal conformal guarantees.
U.S. Rockies · slice audit
Counts summarize where full-ridge MAE is strictly lower than Kuchera. Expand each table for canonical event counts and method-by-method metrics.
Elevation band
Held-out station
Complete storm
Sorted tabular alternative to the aggregate chart.
| Slice | Events | Kuchera MAE | Full-ridge MAE | Change | Kuchera bias | Full-ridge bias |
|---|---|---|---|---|---|---|
| 2000 To 2499 M | 3 | 4.392 | 1.474 | −66.4% | +4.392 | -1.311 |
| 2500 M And Above | 17 | 4.946 | 2.788 | −43.6% | +4.560 | +0.249 |
| Below 2000 M | 1 | 11.531 | 9.672 | −16.1% | +11.531 | +9.672 |
Sorted tabular alternative to the aggregate chart.
| Slice | Events | Kuchera MAE | Full-ridge MAE | Change | Kuchera bias | Full-ridge bias |
|---|---|---|---|---|---|---|
| US1WYAB0005 | 3 | 4.392 | 1.474 | −66.4% | +4.392 | -1.311 |
| US1COBO0233 | 17 | 4.946 | 2.788 | −43.6% | +4.560 | +0.249 |
| US1WYPK0007 | 1 | 11.531 | 9.672 | −16.1% | +11.531 | +9.672 |
Sorted tabular alternative to the aggregate chart.
| Slice | Events | Kuchera MAE | Full-ridge MAE | Change | Kuchera bias | Full-ridge bias |
|---|---|---|---|---|---|---|
| Rockies-2024-11-04 | 1 | 3.776 | 0.686 | −81.8% | +3.776 | -0.686 |
| Rockies-2024-11-25 | 2 | 5.451 | 1.151 | −78.9% | +5.451 | +1.151 |
| Rockies-2025-02-21 | 1 | 5.411 | 1.170 | −78.4% | +5.411 | +1.170 |
| Rockies-2025-05-06 | 1 | 5.659 | 1.298 | −77.1% | +5.659 | +1.298 |
| Rockies-2025-03-04 | 1 | 3.559 | 1.244 | −65% | +3.559 | +1.244 |
| Rockies-2025-02-14 | 2 | 3.424 | 1.357 | −60.4% | +3.424 | -1.357 |
| Rockies-2024-11-06 | 3 | 5.397 | 2.297 | −57.4% | +5.397 | +0.454 |
| Rockies-2024-12-26 | 1 | 3.889 | 1.680 | −56.8% | +3.889 | +1.680 |
| Rockies-2025-04-18 | 2 | 10.501 | 5.230 | −50.2% | +10.501 | +5.230 |
| Rockies-2025-01-07 | 1 | 3.942 | 1.996 | −49.4% | +3.942 | -1.996 |
| Rockies-2025-02-03 | 1 | 11.531 | 9.672 | −16.1% | +11.531 | +9.672 |
| Rockies-2025-01-25 | 2 | 6.166 | 6.236 | +1.1% | +4.324 | -2.454 |
| Rockies-2025-04-14 | 1 | 0.988 | 1.862 | +88.6% | -0.988 | -1.862 |
| Rockies-2024-12-10 | 1 | 2.306 | 4.844 | +110% | +2.306 | -4.844 |
| Rockies-2025-03-29 | 1 | 0.448 | 2.204 | +392.3% | -0.448 | -2.204 |
Ablation and selection
Shared settings were selected only on Rockies, Sierra and Swiss training partitions using unseen-station and complete-storm grouped cross-validation. Northeast was excluded from every selection decision.
| Method | What it adds | Rockies MAE | Sierra MAE | Swiss Alps MAE | Northeast MAE |
|---|---|---|---|---|---|
Original MySnow Kuchera | The exact physical SLR baseline currently used for this comparison. | 5.180 | 3.767 | 5.186 | 4.913 |
Shrunk residual intercept | A station-balanced regional log-residual correction with strong shrinkage. | 3.491 | 3.742 | 4.887 | 4.931 |
Thermal-profile ridge | Regularized residual model using surface and vertical thermal structure. | 2.851 | 3.939 | 4.422 | 4.487 |
Profile-aware full ridge | Thermal structure plus precipitation, snowfall, elevation, baseline SLR, SWE proxy and profile completeness. | 2.928 | 2.694 | 3.291 | 4.147 |
Leave-one-region-out prior | A station-balanced pooled offset learned from the other development regions; rejected as a global fallback. | 4.368 | 3.567 | 4.039 | 5.081 |
Selected by grouped CV
Selected by grouped CV
Selected by grouped CV
deterministic leave-one-station-out folds
Tests spatial transfer by excluding every event from one station. It prevents local station behavior from leaking into its own evaluation.
up to five deterministic contiguous chronological blocks of complete storm groups; no storm crosses folds
Holds out complete chronological storm blocks so correlated observations from one storm do not straddle train and validation.
The failure that taught us the most
A single broad intercept could not represent a holdout whose aggregate residual changed sign and whose stations disagreed with one another.
What the old hierarchy actually supplied to the 80 held-out events.
Most events received only broad global and climate offsets. The profile-aware model can vary its correction with each event’s thermal structure, precipitation, elevation, baseline SLR and profile completeness.
Training mean log(obs/base) was -0.018; the untouched test mean was +0.098.
Held-out station means ranged from -0.088 to +0.284.
Full-ridge MAE improved 4.913 → 4.147 across all 4 stations, 21/27 storms, 4/4 geographic slices and 2/2 elevation bands.
Empirical 80% interval heuristic
Station CV
71.3%
Storm CV
71.3%
Mean widths 11.023 / 11.086.
Empirical 80% interval heuristic
Station CV
75%
Storm CV
77.5%
Mean widths 10.343 / 11.050.
Empirical 80% interval heuristic
Station CV
81.3%
Storm CV
78.8%
Mean widths 11.955 / 11.519.
Pooled event residuals from grouped cross-fits are used only as an empirical interval heuristic. They do not establish event exchangeability, cluster-level exchangeability, group-conditional coverage, or a finite-sample conformal guarantee.
Rejected fallback
The leave-one-region-out prior improves all three development holdouts, then makes untouched Northeast worse. That single failure is enough to reject a global fallback.
MAE · pooled offset from other regions
MAE · pooled offset from other regions
MAE · pooled offset from other regions
MAE · pooled offset from other regions
Northeast MAE worsens from 4.913 to 5.081 after multiplying Kuchera by 0.923890, a pooled offset fit on Rockies, Sierra and Swiss. Unsupported regions continue to use unchanged physical Kuchera/Cobb behavior.
Coverage and evidence gates
Only Northeast was a genuine modeling-transfer failure. The other searches stopped earlier because physical-label, authorization, spatial-support or reproducibility gates failed.
Audited operational networks did not establish independent fresh-depth plus fresh-SWE measurements over one interval. CoCoRaHS was the only plausible direct source, but reuse and commercial-training authorization was not positively established.
data/pnw-slr-research/pnw-blocker-report.jsonFour ordinary ECCC stations were entirely exact 10:1 and a fifth was partly exact 10:1. One potentially non-circular station cannot support unseen-station validation.
data/eccc/canadian-rockies/reports/decision.mdNIED Shinjo has genuine paired daily fresh-snow depth and SWE, but only at one fixed station. JMA positive snow-depth change remains an invalid fresh-snow label.
The old intercept hierarchy moved MAE from 4.913 to 4.946 on 80 unseen-station events because training and test residuals changed sign and held-out stations were heterogeneous.
The alternate mask did not produce independent, same-window fresh-depth and fresh-SWE labels, so modeling never began.
data/northern-appalachians-slr-research/northern-appalachians-blocker-report.jsonEvery candidate paired WESF row had source flag N / CoCoRaHS. PRCP and total-pack WESD are not substitutes for independently measured fresh-snow SWE.
data/great-lakes-slr-research/great-lakes-blocker-report.jsonAll 2,770 threshold-and-field-pass events retained source N on WESF; zero events had proven separate-core provenance.
data/alaska-slr-research/alaska-blocker-report.jsonThe exploratory audit could not be reproduced reliably enough to satisfy the acceptance gate. No artifact was retained.
The 239-station audit left 5,205 source-N near misses and 323 exact-10:1 rejections. Broader geometry cannot repair missing provenance.
Model and data methodology
The diagnostic is intentionally CPU-light and auditable. It tests increasingly expressive residual models without manufacturing predictors or changing the physical baseline.
Kuchera physics
Exact MySnow Kuchera SLR remains the comparison anchor and live physical baseline. No fixed-10:1 or Open-Meteo comparison mode is calibrated.
32 retained features
Learns log(observed SLR / baseline SLR) from physically present surface, vertical-profile and event fields using regularized ridge regression.
Stations + storms
Shared hyperparameters freeze after training-only grouped CV. Each regional holdout is evaluated once; Northeast stays entirely outside selection.
Only retained temperature2mC, freezingLevelHeightM, pressure-level temperature/geopotential, precipitationMm, snowfallCm, elevation, baseline SLR and SWE proxy fields are used; no humidity or wind is synthesized.
baseline_slrelevation_mfreezing_level_m_maxfreezing_level_m_meanfreezing_level_m_mingeopotential_1000hpa_m_meangeopotential_500hpa_m_meangeopotential_600hpa_m_meangeopotential_700hpa_m_meangeopotential_800hpa_m_meangeopotential_850hpa_m_meangeopotential_900hpa_m_meangeopotential_925hpa_m_meangeopotential_950hpa_m_meangeopotential_975hpa_m_meanprecipitation_mm_sumprofile_hour_countsnowfall_cm_sumswe_proxy_mmtemperature2m_c_maxtemperature2m_c_meantemperature2m_c_mintemperature_1000hpa_c_meantemperature_500hpa_c_meantemperature_600hpa_c_meantemperature_700hpa_c_meantemperature_800hpa_c_meantemperature_850hpa_c_meantemperature_900hpa_c_meantemperature_925hpa_c_meantemperature_950hpa_c_meantemperature_975hpa_c_meanAll schemes are normalized to sum to the event count.
Data rights and scientific caveats
Strong retrospective metrics do not override provenance, licensing, support and calibration requirements.
All selected U.S. labels are source-N/CoCoRaHS: Rockies 148/148 N; Sierra 197 SNOW:N/WESF:N plus 48 SNOW:7/WESF:N; Northeast 179/179 N. NOAA documentation defines source N as CoCoRaHS. The later strict broad-U.S. audit excludes source N because underlying reuse and commercial-training authorization was not positively established. This does not assert illegality; authorization is unresolved. U.S. results therefore remain research-only even where physical pairs pass non-circular quality control.
Open-Meteo Historical Forecast remains research-only because hosted commercial-use rights have not been cleared.
Verified literature
References are reproduced from the committed research note. Each card separates the source finding from the implication for MySnow.
Weather and Forecasting 20, 729–744
DOI 10.1175/WAF856.1MySnow implication: SLR varies geographically and through in-cloud, subcloud and surface processes; strict provenance is needed because exact-10:1 reports, settling, mixed precipitation and gauge undercatch contaminate labels.
Roebber, Bruening, Schultz & Cortinas · 2003
Weather and Forecasting 18, 264–287
DOI 10.1175/1520-0434(2003)018<0264:ISFBDS>2.0.CO;2MySnow implication: Vertical temperature and humidity, surface wind and liquid amount motivate a profile-aware residual model. Missing humidity and wind are limitations, not values to synthesize.
Water 9, 461
DOI 10.3390/w9070461MySnow implication: Gauge undercatch and roughly 7% variation in paired manual measurements show why cluster-aware uncertainty can matter more than small point-estimate gains.
Weather and Forecasting
DOI 10.1175/WAF-D-24-0233.1MySnow implication: Multi-site mountain models improve with atmospheric covariates, while pack-depth differences and wind-driven gauge undercatch remain unsuitable foundations for clean labels.
Weather and Forecasting 41, 317–338
MySnow implication: Profile-aware machine learning can beat legacy methods, but its CoCoRaHS labels do not resolve MySnow’s independent authorization requirement or extreme-SLR uncertainty.