Files
pdf_signature_extraction/signature_analysis
gbanyan 92f1db831a Add script 37: K=3 LOOO check (P2_PARTIAL — v4.0 is salvageable with K=3)
Follow-up to Script 36's K=2 UNSTABLE finding. Tests whether K=3's
C1 hand-leaning component (~14% weight, cos~0.946, dh~9.17 from
Script 35) is firm-mass driven or a real cross-firm sub-population.

Result: C1 component shape IS stable across LOOO folds.

  Fold       C1 cos    C1 dh    C1 weight
  baseline   0.9457    9.1715   0.143
  -FirmA     0.9425   10.1263   0.145
  -KPMG      0.9441    9.1591   0.127
  -PwC       0.9504    8.4068   0.126
  -EY        0.9439    9.2897   0.120

  Max drift vs baseline: cos 0.0047, dh 0.955, weight 0.023
  -- all within heuristic stability bars (0.01, 1.0, 0.10).

Held-out prediction divergence vs Script 35 baseline:

  Firm A     predicted  4.68%  vs baseline  0.0%   (+4.68 pp)
  KPMG       predicted  7.14%  vs baseline  8.9%   (-1.76 pp)
  PwC        predicted 36.27%  vs baseline 23.5%   (+12.77 pp)
  EY         predicted 17.31%  vs baseline 11.5%   (+5.81 pp)

Verdict: P2_PARTIAL.

Methodological insight: K=3 disentangles the firm-mass/mechanism
confound that broke K=2. C3 (cos~0.983, dh~2.4) absorbs Firm A's
templated mass; C1 (cos~0.946, dh~9.17) captures cross-firm
hand-leaning. Membership boundary shifts slightly (±5-13 pp)
across folds, reflecting honest calibration uncertainty rather
than collapse.

Implication: v4.0 can pivot to a "characterized cluster structure
with bounded reproducibility" framing instead of the original
"clean natural threshold" pitch. Honest, defensible, but a
different paper than v3.20.0 was building.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:57:40 +08:00
..