gbanyan/pdf_signature_extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
54 Commits 4 Branches 1 Tag
62a22ceb8382fa9f6ed2b38d91942307342935fe
Commit Graph

22 Commits

Author SHA1 Message Date
gbanyan 338737d9a1 Add script 40: pixel-identity FAR (0% across all v4 classifiers) 
Phase 1.8 follow-up. Validates the v4.0 classifier family against
the only hard ground truth in the corpus: pixel_identical_to_closest=1
(byte-identical to nearest same-CPA neighbor; mathematically impossible
under independent hand-signing).

n = 262 pixel-identical Big-4 signatures.

  Firm A   145
  KPMG       8
  PwC      107
  EY         2

FAR (lower better; Wilson 95% CI for the misclassification rate):

  PaperA box rule           0.00%  [0.00%, 1.45%]
  K=3 per-CPA hard label    0.00%  [0.00%, 1.45%]
  Reverse-anchor (calibr.)  0.00%  [0.00%, 1.45%]

Per-firm: 0% misclass on every firm.

Reverse-anchor cut chosen by prevalence calibration (overall
replicated rate matches Paper A's 49.58%). Documented v4.0
limitation: no signature-level ground truth for hand-leaning
class, so cannot ROC-optimize the cut directly.

PwC's 107 pixel-identical signatures despite being the most
hand-leaning firm overall (Script 38 per-CPA P_C1=0.31)
illustrates the within-firm heterogeneity that v4.0's K=3
mixture captures: a PwC CPA can be hand-leaning on average
while still occasionally reusing template signatures.

Implication: at the only hard ground truth available in the
corpus, all three v4.0 classifiers achieve perfect detection.
This satisfies REQ-001 acceptance for pixel-identity FAR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 15:10:03 +08:00
gbanyan 39575cef49 Add script 39: signature-level convergence (SIG_CONVERGENCE_MODERATE) 
Phase 1.7 follow-up to Script 38's per-CPA convergence. Tests
whether the convergence holds at signature granularity, preempting
"per-CPA aggregation washes out signal" reviewer attacks.

Three signature-level labels per Big-4 signature (n=150,442):
  L1 PaperA      non_hand iff cos > 0.95 AND dh <= 5
  L2 K=3 perCPA  hard assignment under per-CPA-fit components
  L3 K=3 perSig  hard assignment under fresh signature-level fit

Component comparison (per-CPA vs per-signature K=3):

  Component        Per-CPA cos/dh/wt     Per-Sig cos/dh/wt
  C1 hand-leaning  0.9457/9.17/0.143     0.9280/9.75/0.146
  C2 mixed         0.9558/6.66/0.536     0.9625/6.04/0.582
  C3 replicated    0.9826/2.41/0.321     0.9890/1.27/0.272

  Component drift modest: max |dcos| = 0.018, max |ddh| = 1.15.

Cohen kappa (binary, 1 = replicated):

  PaperA vs K=3 perCPA       kappa = 0.6616  substantial
  PaperA vs K=3 perSig       kappa = 0.5586  moderate
  K=3 perCPA vs K=3 perSig   kappa = 0.8701  almost perfect

Per-firm binary agreement PaperA vs K=3 perCPA:

  Firm A 86.13%, KPMG 77.46%, PwC 82.64%, EY 85.01%.

Verdict: SIG_CONVERGENCE_MODERATE (all kappas >= 0.40; per-CPA
aggregation captures most signature-level structure).

Implication for v4.0: per-CPA K=3 is robust to aggregation level
(kappa = 0.87 vs per-signature fit). The modest disagreement
between K=3 and Paper A's box rule (kappa 0.56-0.66) reflects
different decision geometries -- K=3 posterior soft boundary vs
Paper A rectangle box -- not a fundamental signal disagreement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 15:07:48 +08:00
gbanyan bc36dcc2b6 Add script 38: v4.0 convergence (CONVERGENCE_STRONG, three lenses agree) 
Phase 1.6 (G2 path) script. Tests whether three INDEPENDENT
statistical approaches converge on the same Big-4 CPA ranking:

  1. K=3 GMM cluster posterior P_C1 (hand-leaning)
     -- from full Big-4 K=3 fit (Script 37 baseline).
  2. Reverse-anchor directional score
     -- non-Big-4 (n=249, mid/small firms only) as the
        reference Gaussian; -cos_left_tail_pct as score.
     -- Strict separation: no Big-4 CPA in the reference.
  3. Paper A v3.x operational rule per-CPA hand_frac
     -- (cos > 0.95 AND dh <= 5) failure rate per CPA.

Pairwise Spearman correlations:

  p_c1 vs paperA_hand_frac           rho = +0.9627  (p < 1e-248)
  reverse_anchor vs paperA_hand_frac rho = +0.8890  (p < 1e-149)
  p_c1 vs reverse_anchor             rho = +0.8794  (p < 1e-142)

Verdict: CONVERGENCE_STRONG (all 3 |rho| >= 0.7).

Per-firm consistency across lenses:

  Firm    n     C1%      C3%      E[P_C1]  E[rev]   E[hand]
  FirmA  171   0.00%   82.46%    0.007   -0.973    0.193
  KPMG   112   8.93%    0.00%    0.141   -0.820    0.696
  PwC    102  23.53%    0.98%    0.311   -0.767    0.790
  EY      52  11.54%    1.92%    0.241   -0.713    0.761

Same monotone ordering by all three metrics:
  Firm A < KPMG < EY ~= PwC on hand-leaning.

Implication for v4.0: methodology paper now has THREE
independent lines of evidence converging on the same population
structure -- a much harder thing for a reviewer to dismiss
than any single lens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 15:03:55 +08:00
gbanyan 92f1db831a Add script 37: K=3 LOOO check (P2_PARTIAL — v4.0 is salvageable with K=3) 
Follow-up to Script 36's K=2 UNSTABLE finding. Tests whether K=3's
C1 hand-leaning component (~14% weight, cos~0.946, dh~9.17 from
Script 35) is firm-mass driven or a real cross-firm sub-population.

Result: C1 component shape IS stable across LOOO folds.

  Fold       C1 cos    C1 dh    C1 weight
  baseline   0.9457    9.1715   0.143
  -FirmA     0.9425   10.1263   0.145
  -KPMG      0.9441    9.1591   0.127
  -PwC       0.9504    8.4068   0.126
  -EY        0.9439    9.2897   0.120

  Max drift vs baseline: cos 0.0047, dh 0.955, weight 0.023
  -- all within heuristic stability bars (0.01, 1.0, 0.10).

Held-out prediction divergence vs Script 35 baseline:

  Firm A     predicted  4.68%  vs baseline  0.0%   (+4.68 pp)
  KPMG       predicted  7.14%  vs baseline  8.9%   (-1.76 pp)
  PwC        predicted 36.27%  vs baseline 23.5%   (+12.77 pp)
  EY         predicted 17.31%  vs baseline 11.5%   (+5.81 pp)

Verdict: P2_PARTIAL.

Methodological insight: K=3 disentangles the firm-mass/mechanism
confound that broke K=2. C3 (cos~0.983, dh~2.4) absorbs Firm A's
templated mass; C1 (cos~0.946, dh~9.17) captures cross-firm
hand-leaning. Membership boundary shifts slightly (±5-13 pp)
across folds, reflecting honest calibration uncertainty rather
than collapse.

Implication: v4.0 can pivot to a "characterized cluster structure
with bounded reproducibility" framing instead of the original
"clean natural threshold" pitch. Honest, defensible, but a
different paper than v3.20.0 was building.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 14:57:40 +08:00
gbanyan ccd9f23635 Add script 36: v4.0 calibration + LOOO validation (UNSTABLE verdict) 
Phase 1 foundation script for Paper A v4.0 Big-4 reframe.

Sections:
  A. Big-4 calibration recap (replicates Script 34: K=2 marginal
     crossings cos=0.9755, dh=3.7549; bootstrap 95% CI tight;
     dip-test cos p<0.0001, dh p<0.0001).
  B. Leave-one-firm-out (LOOO) cross-validation: refit K=2 on the
     other 3 firms, predict the held-out firm's CPAs.
  C. Cross-fold stability verdict.

Result: UNSTABLE.

  Held-out firm   Fold rule                       Replicated rate
  Firm A          cos>0.9380 AND dh<=8.7902       171/171 = 100%
  KPMG            cos>0.9744 AND dh<=3.9783       0/112 = 0%
  PwC             cos>0.9752 AND dh<=3.7470       0/102 = 0%
  EY              cos>0.9756 AND dh<=3.7409       0/52 = 0%

  Max |dev_cos| from fold-mean = 0.028 (5.6x over 0.005 stability bar).

Methodological implication:

  The Big-4 K=2 bimodality that Script 34 celebrated (dip
  p<0.0001) is firm-mass driven, not mechanism driven. K=2
  separates Firm A from the other three Big-4, then mis-applies
  to held-out non-Firm-A firms (everyone falls below the cosine
  cut).  Same conceptual problem as Paper A v3.x's between-firm
  threshold, just at smaller scope.

  v4.0 narrative as currently planned does not survive a reviewer
  who runs LOOO.

  Forward options under discussion: P1 firm-templatedness reframe,
  P2 K=3 primary (next: Script 37 = K=3 LOOO), P3 rollback to
  v3.20.0, P4 reverse-anchor as v4.0 core.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 14:54:54 +08:00
gbanyan 55f9f94d9a Add scripts 34 + 35: Big-4-only calibration foundation 
Scripts 34 and 35 produced the empirical foundation that triggers the
Paper A v4.0 Big-4 reframe.

Script 34 (Big-4-only pooled calibration):
  Pool Firm A + KPMG + PwC + EY (437 CPAs); first time the
  three-method framework yields dip-test multimodal results
  (p<0.0001 on both cos and dh axes) anywhere in the analysis
  family.  2D-GMM K=2 marginal crossings with bootstrap 95% CI
  (n=500): cos = 0.9755 [0.974, 0.977], dh = 3.755 [3.48, 3.97].
  Crossing offsets from Paper A v3.20.0 baseline (0.945, 8.10):
  +0.030 (cos), -4.345 (dh) -- mid/small-firm tail had
  substantially shifted the published threshold.

Script 35 (Big-4 K=3 cluster membership):
  Hard-assigns each Big-4 CPA to one of the K=3 components.
  Findings:
    * Firm A (Deloitte): 0% in C1 (hand-sign-leaning),
      17.5% in C2 (mixed), 82.5% in C3 (replicated).
    * PwC has the strongest hand-sign tradition (24/102 = 23.5%
      in C1), followed by EY (11.5%) and KPMG (8.9%).
    * 40 CPAs total in C1 across KPMG/PwC/EY.

Implications confirmed by these scripts:
  * Big-4-only scope is the methodologically defensible primary
    analysis; the published 0.945/8.10 reflects between-firm
    structure rather than within-pool mechanism boundary.
  * Firm A's role pivots from "calibration anchor" to
    "case study of templated end of Big-4."
  * Paper A is being reframed as v4.0 on sub-branch
    paper-a-v4-big4, per Partner Jimmy's earlier direction
    suggestion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 14:35:37 +08:00
gbanyan 8ac09888ae Add script 33: reverse-anchor spike (PAPER_C_STRONG verdict) 
Follow-up to Script 32 verdict C. Tests whether using the non-Firm-A
population (515 CPAs) as a "fully-replicated reference" recovers the
Paper A hand-signed signal through deviation analysis on Firm A.

Methodology:
  * Robust 2D Gaussian fit (MCD, support_fraction=0.85) on
    (cos_mean, dh_mean) of all_non_A CPAs.  Reference center =
    (cos=0.946, dh=8.29).
  * Score Firm A CPAs by symmetric Mahalanobis distance, log-
    likelihood, and directional cosine left-tail percentile.
  * Cross-validate against Paper A's per-CPA hand_frac proxy
    (signatures with cos<=0.95 OR dh>5).

Key findings:
  * Directional metric (-cos_left_tail_pct) vs Paper A hand_frac:
    Spearman rho = +0.744 (p < 1e-30) -- PAPER_C_STRONG.
  * Symmetric Mahalanobis vs hand_frac: rho = -0.927 (p < 1e-73).
    The negative sign is a feature, not a bug: Firm A bifurcates
    into two anomaly directions from the non-Firm-A reference --
    (a) ultra-replicated CPAs (cos>=0.985, dh~1) sitting beyond
    the reference's high-cos tail, and (b) hand-signed CPAs
    (cos~0.95, dh~6-7) sitting near or below the reference
    center.  Symmetric distance lumps both into a positive
    magnitude; directional metrics distinguish them.

Implication: a "Paper C" reframing is statistically supported.
Use non-Firm-A as the replication reference, not Firm A as the
hand-signed anchor.  This removes the "why is Firm A ground
truth?" reviewer attack and reveals the bifurcation structure
that Paper A's symmetric framing obscures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 12:09:36 +08:00
gbanyan e1d81e3732 Add script 32: non-Firm-A calibration spike (verdict C with twist) 
Spike for the from-outside-of-firmA branch. Runs the three-method
threshold framework (KDE+dip, BD/McCrary, Beta mixture / logit-GMM,
2D-GMM) on three subsets:

  Subset I  big4_non_A   KPMG+PwC+EY pooled (266 CPAs, 89.9k sigs)
  Subset II all_non_A    every firm except Firm A (515 CPAs, 108k sigs)
  Subset III firm_A      reference baseline (171 CPAs, 60.4k sigs)

Plus pre_2018 / post_2020 time-stratified secondary on subsets I and II.

Result: verdict C -- every subset is unimodal at the dip-test level
(dip p > 0.76 across the board), including Firm A itself.  Time
stratification does not recover bimodality.

Cross-subset Beta-2 cosine crossings: Firm A 0.977, big4_non_A 0.930,
all_non_A 0.938; Paper A's published 0.945 sits between the two mass
centers, indicating the published "natural threshold" is effectively
a between-firm separator rather than a within-pool mechanism boundary.
This finding motivates a follow-up reverse-anchor spike (script 33).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 12:05:18 +08:00
gbanyan c0ed9aa5dc Add script 27: within-auditor-year uniformity empirical check (A2 test) 
Empirical verification of the A2 within-year label-uniformity
assumption flagged by Opus round-12. Result falsified A2 and led to
its removal in Paper A v3.14; script retained as due-diligence
evidence in the repo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 11:34:17 +08:00
gbanyan 53125d11d9 Paper A v3.20.0: partner Jimmy 2026-04-27 review + DOCX rendering overhaul 
Substantive content (addresses partner Jimmy's 2026-04-27 review of v3.19.1):

Must-fix items (6/6):
- §III-F SSIM/pixel rejection rewritten from first principles (design-level
  argument from luminance/contrast/structure local-window product, not the
  prior empirical 0.70 result)
- Table VI restructured by population × method; added missing Firm A
  logit-Gaussian-2 0.999 row; KDE marked undefined (unimodal), BD/McCrary
  marked bin-unstable (Appendix A)
- Tables IX / XI / §IV-F.3 dHash 5/8/15 inconsistency resolved: ≤8 demoted
  from "operational dual" to "calibration-fold-adjacent reference"; the
  actual classifier rule cos>0.95 AND dH≤15 = 92.46% added throughout
- New Fig. 4 (yearly per-firm best-match cosine, 5 lines, 2013-2023, Firm A
  on top); script 30_yearly_big4_comparison.py
- Tables XIV / XV extended with top-20% (94.8%) and top-30% (81.3%) brackets
- §III-K reframed P7.5 from "round-number lower-tail boundary" to operating
  point; new Table XII-B (cosine-FAR-capture tradeoff at 5 thresholds:
  0.9407 / 0.945 / 0.95 / 0.977 / 0.985)

Nice-to-have items (3/3):
- Table XII expanded to 6-cut classifier sensitivity grid (0.940-0.985)
- Defensive parentheticals (84,386 vs 85,042; 30,226 vs 30,222) moved to
  table notes; cut "invite reviewer skepticism" and "non-load-bearing"

Codex 3-pass verification cleanup:
- Stale 0.973/0.977/0.979 references unified on canonical 0.977 (Firm A
  Beta-2 forced-fit crossing from beta_mixture_results.json)
- dHash≤8 wording corrected to P95-adjacent (P95 = 9, ≤8 is the integer
  immediately below) instead of misleading "rounded down"
- Table XII-B prose corrected: per-segment qualification of "non-Firm-A
  capture falls faster" (true on 0.95→0.977 segment but contracts on
  0.977→0.985 segment); arithmetic now from exact counts

Within-year analyses removed:
- Within-year ranking robustness check (Class A) was added in nice-to-have
  pass but contradicts v3.14 A2-removal stance; removed from §IV-G.2 + the
  Appendix B provenance row
- Within-CPA future-work disclosures (Class B) removed from Discussion
  limitation #5 and Conclusion future-work paragraph; subsequent limitations
  renumbered Sixth → Fifth, Seventh → Sixth

DOCX rendering pipeline overhaul (paper/export_v3.py):

Critical fix - every v3 DOCX since v3.0 was shipping WITHOUT TABLES:
strip_comments() was wholesale-deleting HTML comments, but every numerical
table is wrapped in <!-- TABLE X: ... -->, so the table body was deleted
alongside the wrapper. Now unwraps TABLE comments (emit synthetic
__TABLE_CAPTION__: marker + table body) while still stripping non-TABLE
editorial comments. Result: 19 tables now render in the DOCX.

Other rendering fixes:
- LaTeX → Unicode conversion (50+ token replacements: Greek alphabet, ≤≥,
  ×·≈, →↔⇒, etc.); \frac/\sqrt linearisation; TeX brace tricks ({=}, {,})
- Math-context-scoped sub/superscript via PUA sentinels (/):
  no more underscore-eating in identifiers like signature_analysis
- Display equations rendered via matplotlib mathtext to PNG (3 equations:
  cosine sim, mixture crossing, BD/McCrary Z statistic), embedded as
  numbered equation blocks (1), (2), (3); content-addressed cache at
  paper/equations/ (gitignored, regenerable)
- Manual numbered/bulleted list rendering with hanging indent (replaces
  python-docx style="List Number" which silently drops the number prefix
  when no numbering definition is bound)
- Markdown blockquote (> ...) defensively stripped
- Pandoc footnote ([^name]) markers no longer leak (inlined at source)
- Heading text cleaned of LaTeX residue + PUA sentinels
- File paths in body text (signature_analysis/X.py, reports/Y.json)
  trimmed to "(reproduction artifact in Appendix B)" pointers

New leak linter: paper/lint_paper_v3.py - two-pass markdown source +
rendered DOCX leak detector; auto-runs at end of export_v3.py.

Script changes:
- 21_expanded_validation.py: added 0.9407, 0.977, 0.985 to canonical FAR
  threshold list so Table XII-B is reproducible from persisted JSON
- 30_yearly_big4_comparison.py: NEW; generates Fig. 4 + per-firm yearly
  data (writes to reports/figures/ and reports/firm_yearly_comparison/)
- 31_within_year_ranking_robustness.py: NEW; supports the within-year
  robustness check (no longer cited in paper but kept as repo-internal
  due-diligence artifact)

Partner handoff DOCX shipped to
~/Downloads/Paper_A_IEEE_Access_Draft_v3.20.0_20260505.docx (536 KB:
19 tables + 4 figures + 3 equation images).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 13:44:49 +08:00
gbanyan af08391a68 Paper A v3.19.0: address Gemini 3.1 Pro round-19 Major Revision findings 
Gemini 3.1 Pro round-19 (paper/gemini_review_v3_18_4.md) caught FOUR
serious issues that all 18 prior AI review rounds missed, including
fabricated rationalizations and a real statistical flaw. All four
verified by direct DB / script inspection. Verdict: Major Revision; this
commit closes every flagged item.

Fabricated rationalization corrections (text only, numbers unchanged):

- Section IV-H "656 documents excluded" rewritten. Previous text claimed
  the exclusion was because "single-signature documents have no same-CPA
  pairwise comparison" -- a fabricated explanation that contradicts the
  paper's cross-document matching methodology. The truth, verified
  against signature_analysis/09_pdf_signature_verdict.py L44 (WHERE
  s.is_valid = 1 AND s.assigned_accountant IS NOT NULL): the 656
  documents are excluded because none of their detected signatures could
  be matched to a registered CPA name (assigned_accountant IS NULL).
- Section IV-F.2 "two CPAs excluded for disambiguation ties" rewritten.
  No disambiguation logic exists in script 24; the 178 vs 180 difference
  comes from two registered Firm A partners being singletons in the
  corpus (one signature each, so per-signature best-match cosine is
  undefined and they do not appear in the matched-signature table that
  feeds the 70/30 split).
- Appendix B Table XIII provenance corrected. The previous attribution
  to 13_deloitte_distribution_analysis.py / accountant_similarity_analysis.json
  was wrong: neither artifact has year_month grouping. New script
  29_firm_a_yearly_distribution.py reproduces Table XIII exactly from
  the database via accountants.firm + signatures.year_month grouping.

Statistical flaw corrections (numbers updated):

- Inter-CPA negative anchor rewritten in 21_expanded_validation.py. The
  prior implementation drew 50,000 random cross-CPA pairs from a
  LIMIT-3000 random subsample, reusing each signature ~33 times and
  artificially tightening Wilson FAR confidence intervals on Table X.
  The corrected implementation samples 50,000 i.i.d. pairs uniformly
  across the full 168,755-signature matched corpus.
- Re-run script 21. Table X numbers are close to v3.18.4 but no longer
  rest on the inflated-precision artifact:
    cos > 0.837: FAR 0.2101 (was 0.2062), CI [0.2066, 0.2137]
    cos > 0.900: FAR 0.0250 (was 0.0233), CI [0.0237, 0.0264]
    cos > 0.945: FAR 0.0008 (unchanged at this resolution)
    cos > 0.950: FAR 0.0005 (was 0.0007), CI [0.0003, 0.0007]
    cos > 0.973: FAR 0.0002 (was 0.0003), CI [0.0001, 0.0004]
    cos > 0.979: FAR 0.0001 (was 0.0002), CI [0.0001, 0.0003]
- Inter-CPA cosine summary stats also updated:
    mean 0.763 (was 0.762)
    P95 0.886 (was 0.884)
    P99 0.915 (was 0.913)
    max 0.992 (was 0.988)
- Manuscript IV-F.1 prose updated to reflect the i.i.d. full-corpus
  sampling.

Rebuild Paper_A_IEEE_Access_Draft_v3.docx.

Note: this is v3.19.0 because v3.19 closes both fabrication and a
genuine statistical flaw, not just provenance polish.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 21:40:42 +08:00
gbanyan 6b64eabbfb Paper A v3.18.4: address codex GPT-5.5 round-18 self-comparing review findings 
Codex round-18 (paper/codex_review_gpt55_v3_18_3.md) caught a falsified
provenance claim I introduced in v3.18.3 plus four cleaner narrative items
that survived the prior 17 rounds. Verdict was Minor Revision; this
commit closes all 5 actionable items.

- Harmonize signature_analysis/28_byte_identity_decomposition.py to use
  accountants.firm (joined on signatures.assigned_accountant) for Firm A
  membership, matching the convention in 24_validation_recalibration.py.
  Regenerated reports/byte_identity_decomp/byte_identity_decomposition.json.
  Cross-firm convergence now reports Firm A 49,389 / 55,922 = 88.32% and
  Non-Firm-A 27,595 / 65,514 = 42.12% (percentages unchanged at two
  decimal places; counts now match Table IX exactly).
- Replace the Section IV-H.2 reconciliation note. The previous note
  speculated that the one-record discrepancy was a snapshot/floating-point
  artifact, which codex round-18 falsified by direct DB queries: the real
  cause was that script 28 used signatures.excel_firm while Table IX uses
  accountants.firm. With script 28 now harmonized, Table IX and the
  cross-firm artifact agree exactly at 55,922; the new note documents the
  Firm A grouping convention plus the dHash-non-null filter.
- Replace residual "known-majority-positive" wording with
  "replication-dominated" in Introduction (contributions 4 and 6) and
  Methodology III-I (anchor-rationale paragraph).
- Correct Methodology III-G's auditor-year description: the per-signature
  best-match cosine that feeds each auditor-year mean is computed against
  the full same-CPA cross-year pool, not within-year only. The aggregation
  unit is within-year, but the underlying similarity statistic is not.
- Add the 145 / 50 / 180 / 35 Firm A byte-decomposition sentence to
  Results IV-F.1 with explicit pointer to script 28 and the JSON artifact;
  this resolves the round-18 finding that several manuscript locations
  cited IV-F.1 for a decomposition that was not actually reported there.
- Rebuild Paper_A_IEEE_Access_Draft_v3.docx.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 20:59:07 +08:00
gbanyan 4bb7aa9189 Paper A v3.18.2: address codex GPT-5.5 round-16 Minor-Revision findings 
Codex independent peer review (paper/codex_review_gpt55_v3_18_1.md) audited
empirical claims against scripts/JSON reports rather than rubber-stamping
prior Accept verdicts. Verdict: Minor Revision. This commit addresses every
flagged item.

- Soften mechanism-identification language (Results IV-D.1, Discussion B):
  per-signature cosine "fails to reject unimodality" rather than "reflects a
  single dominant generative mechanism"; framing tied to joint evidence.
- Replace overabsolute "single stored image" with multi-template phrasing
  in Introduction and Methodology III-A.
- Reframe Methodology III-H so practitioner knowledge is non-load-bearing;
  evidentiary basis is the paper's own image evidence.
- Fix stale section cross-references after the v3.18 retitling: IV-F.* ->
  IV-G.* in 11 locations across methodology and results.
- Fix 0.941 / 0.945 / 0.9407 wording in Methodology III-K to use the
  calibration-fold P5 = 0.9407 and the rounded sensitivity cut 0.945.
- Soften "sharp discontinuity" in Results IV-G.3 to "23-28 percentage-point
  gap consistent with firm-wide non-hand-signing practice".
- Soften Conclusion's "directly generalizable" with explicit conditions on
  analogous anchors and artifact-generation physics.
- Add Appendix B: table-to-script provenance map (15 manuscript tables
  mapped to generating scripts and JSON report artifacts).
- New script signature_analysis/28_byte_identity_decomposition.py produces
  reproducible artifacts for two previously-unverified claims:
  (a) 145 / 50 / 180 / 35 Firm A byte-identity decomposition (verified);
  (b) cross-firm dual-descriptor convergence -- corrected from the previous
      manuscript text "non-Firm-A 11.3% vs Firm A 58.7% (5x)" to the
      database-verified "non-Firm-A 42.12% vs Firm A 88.32% (~2.1x)".
- Clarify scripts 19 / 21 docstrings: legacy EER / FRR / Precision / F1
  helpers are retained for diagnostic use only and are NOT cited as
  biometric performance in the paper. Remove "interview evidence" wording.
- Rebuild Paper_A_IEEE_Access_Draft_v3.docx.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 20:23:08 +08:00
gbanyan 552b6b80d4 Paper A v3.7: demote BD/McCrary to density-smoothness diagnostic; add Appendix A 
Implements codex gpt-5.4 recommendation (paper/codex_bd_mccrary_opinion.md,
"option (c) hybrid"): demote BD/McCrary in the main text from a co-equal
threshold estimator to a density-smoothness diagnostic, and add a
bin-width sensitivity appendix as an audit trail.

Why: the bin-width sweep (Script 25) confirms that at the signature
level the BD transition drifts monotonically with bin width (Firm A
cosine: 0.987 -> 0.985 -> 0.980 -> 0.975 as bin width widens 0.003 ->
0.015; full-sample dHash transitions drift from 2 to 10 to 9 across
bin widths 1 / 2 / 3) and Z statistics inflate superlinearly with bin
width, both characteristic of a histogram-resolution artifact. At the
accountant level the BD null is robust across the sweep. The paper's
earlier "three methodologically distinct estimators" framing therefore
could not be defended to an IEEE Access reviewer once the sweep was
run.

Added
- signature_analysis/25_bd_mccrary_sensitivity.py: bin-width sweep
  across 6 variants (Firm A / full-sample / accountant-level, each
  cosine + dHash_indep) and 3-4 bin widths per variant. Reports
  Z_below, Z_above, p-values, and number of significant transitions
  per cell. Writes reports/bd_sensitivity/bd_sensitivity.{json,md}.
- paper/paper_a_appendix_v3.md: new "Appendix A. BD/McCrary Bin-Width
  Sensitivity" with Table A.I (all 20 sensitivity cells) and
  interpretation linking the empirical pattern to the main-text
  framing decision.
- export_v3.py: appendix inserted into SECTIONS between conclusion
  and references.
- paper/codex_bd_mccrary_opinion.md: codex gpt-5.4 recommendation
  captured verbatim for audit trail.

Main-text reframing
- Abstract: "three methodologically distinct estimators" ->
  "two estimators plus a Burgstahler-Dichev/McCrary density-
  smoothness diagnostic". Trimmed to 243 words.
- Introduction: related-work summary, pipeline step 5, accountant-
  level convergence sentence, contribution 4, and section-outline
  line all updated. Contribution 4 renamed to "Convergent threshold
  framework with a smoothness diagnostic".
- Methodology III-I: section renamed to "Convergent Threshold
  Determination with a Density-Smoothness Diagnostic". "Method 2:
  BD/McCrary Discontinuity" converted to "Density-Smoothness
  Diagnostic" in a new subsection; Method 3 (Beta mixture) renumbered
  to Method 2. Subsections 4 and 5 updated to refer to "two threshold
  estimators" with BD as diagnostic.
- Methodology III-A pipeline overview: "three methodologically
  distinct statistical methods" -> "two methodologically distinct
  threshold estimators complemented by a density-smoothness
  diagnostic".
- Methodology III-L: "three-method analysis" -> "accountant-level
  threshold analysis (KDE antimode, Beta-2 crossing, logit-Gaussian
  robustness crossing)".
- Results IV-D.1 heading: "BD/McCrary Discontinuity" ->
  "BD/McCrary Density-Smoothness Diagnostic". Prose now notes the
  Appendix-A bin-width instability explicitly.
- Results IV-E: Table VIII restructured to label BD rows
  "(diagnostic only; bin-unstable)" and "(diagnostic; null across
  Appendix A)". Summary sentence rewritten to frame BD null as
  evidence for clustered-but-smoothly-mixed rather than as a
  convergence failure. Table cosine P5 row corrected from 0.941 to
  0.9407 to match III-K.
- Results IV-G.3 and IV-I.2: "three-method convergence/thresholds"
  -> "accountant-level convergent thresholds" (clarifies the 3
  converging estimates are KDE antimode, Beta-2, logit-Gaussian,
  not KDE/BD/Beta).
- Discussion V-B: "three-method framework" -> "convergent threshold
  framework".
- Conclusion: "three methodologically distinct methods" -> "two
  threshold estimators and a density-smoothness diagnostic";
  contribution 3 restated; future-work sentence updated.
- Impact Statement (archived): "three methodologically distinct
  threshold-selection methods" -> "two methodologically distinct
  threshold estimators plus a density-smoothness diagnostic" so the
  archived text is internally consistent if reused.

Discussion V-B / V-G already framed BD as a diagnostic in v3.5
(unchanged in this commit). The reframing therefore brings Abstract /
Introduction / Methodology / Results / Conclusion into alignment with
the Discussion framing that codex had already endorsed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 14:32:50 +08:00
gbanyan 12f716ddf1 Paper A v3.5: resolve codex round-4 residual issues 
Fully addresses the partial-resolution / unfixed items from codex
gpt-5.4 round-4 review (codex_review_gpt54_v3_4.md):

Critical
- Table XI z/p columns now reproduce from displayed counts. Earlier
  table had 1-4-unit transcription errors in k values and a fabricated
  cos > 0.9407 calibration row; both fixed by rerunning Script 24
  with cos = 0.9407 added to COS_RULES and copying exact values from
  the JSON output.
- Section III-L classifier now defined entirely in terms of the
  independent-minimum dHash statistic that the deployed code (Scripts
  21, 23, 24) actually uses; the legacy "cosine-conditional dHash"
  language is removed. Tables IX, XI, XII, XVI are now arithmetically
  consistent with the III-L classifier definition.
- "0.95 not calibrated to Firm A" inconsistency reconciled: Section
  III-H now correctly says 0.95 is the whole-sample Firm A P95 of the
  per-signature cosine distribution, matching III-L and IV-F.

Major
- Abstract trimmed to 246 words (from 367) to meet IEEE Access 250-word
  limit. Removed "we break the circularity" overclaim; replaced with
  "report capture rates on both folds with Wilson 95% intervals to
  make fold-level variance visible".
- Conclusion mirrors the Abstract reframe: 70/30 split documents
  within-firm sampling variance, not external generalization.
- Introduction no longer promises precision / F1 / EER metrics that
  Methods/Results don't deliver; replaced with anchor-based capture /
  FAR + Wilson CI language.
- Section III-G within-auditor-year empirical-check wording corrected:
  intra-report consistency (IV-H.3) is a different test (two co-signers
  on the same report, firm-level homogeneity) and is not a within-CPA
  year-level mixing check; the assumption is maintained as a bounded
  identification convention.
- Section III-H "two analyses fully threshold-free" corrected to "only
  the partner-level ranking is threshold-free"; longitudinal-stability
  uses 0.95 cutoff, intra-report uses the operational classifier.

Minor
- Impact Statement removed from export_v3.py SECTIONS list (IEEE Access
  Regular Papers do not have a standalone Impact Statement). The file
  itself is retained as an archived non-paper note for cover-letter /
  grant-report reuse, with a clear archive header.
- All 7 previously unused references ([27] dHash, [31][32] partner-
  signature mandates, [33] Taiwan partner rotation, [34] YOLO original,
  [35] VLM survey, [36] Mann-Whitney) are now cited in-text:
    [27] in Methodology III-E (dHash definition)
    [31][32][33] in Introduction (audit-quality regulation context)
    [34][35] in Methodology III-C/III-D
    [36] in Results IV-C (Mann-Whitney result)

Updated Script 24 to include cos = 0.9407 in COS_RULES so Table XI's
calibration-fold P5 row is computed from the same data file as the
other rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 12:23:03 +08:00
gbanyan 0ff1845b22 Paper A v3.4: resolve codex round-3 major-revision blockers 
Three blockers from codex gpt-5.4 round-3 review (codex_review_gpt54_v3_3.md):

B1 Classifier vs three-method threshold mismatch
  - Methodology III-L rewritten to make explicit that the per-signature
    classifier and the accountant-level three-method convergence operate
    at different units (signature vs accountant) and are complementary
    rather than substitutable.
  - Add Results IV-G.3 + Table XII operational-threshold sensitivity:
    cos>0.95 vs cos>0.945 shifts dual-rule capture by 1.19 pp on whole
    Firm A; ~5% of signatures flip at the Uncertain/Moderate boundary.

B2 Held-out validation false "within Wilson CI" claim
  - Script 24 recomputes both calibration-fold and held-out-fold rates
    with Wilson 95% CIs and a two-proportion z-test on each rule.
  - Table XI replaced with the proper fold-vs-fold comparison; prose
    in Results IV-G.2 and Discussion V-C corrected: extreme rules agree
    across folds (p>0.7); operational rules in the 85-95% band differ
    by 1-5 pp due to within-Firm-A heterogeneity (random 30% sample
    contained more high-replication C1 accountants), not generalization
    failure.

B3 Interview evidence reframed as practitioner knowledge
  - The Firm A "interviews" referenced throughout v3.3 are private,
    informal professional conversations, not structured research
    interviews. Reframed accordingly: all "interview*" references in
    abstract / intro / methodology / results / discussion / conclusion
    are replaced with "domain knowledge / industry-practice knowledge".
  - This avoids overclaiming methodological formality and removes the
    human-subjects research framing that triggered the ethics-statement
    requirement.
  - Section III-H four-pillar Firm A validation now stands on visual
    inspection, signature-level statistics, accountant-level GMM, and
    the three Section IV-H analyses, with practitioner knowledge as
    background context only.
  - New Section III-M ("Data Source and Firm Anonymization") covers
    MOPS public-data provenance, Firm A/B/C/D pseudonymization, and
    conflict-of-interest declaration.

Add signature_analysis/24_validation_recalibration.py for the recomputed
calib-vs-held-out z-tests and the classifier sensitivity analysis;
output in reports/validation_recalibration/.

Pending (not in this commit): abstract length (368 -> 250 words),
Impact Statement removal, BD/McCrary sensitivity reporting, full
reproducibility appendix, references cleanup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 11:45:24 +08:00
gbanyan 51d15b32a5 Paper A v3.2: partner v4 feedback integration (threshold-independent benchmark validation) 
Partner v4 (signature_paper_draft_v4) proposed 3 substantive improvements;
partner confirmed the 2013-2019 restriction was an error (sample stays
2013-2023). The remaining suggestions are adopted with our own data.

## New scripts
- Script 22 (partner ranking): ranks all Big-4 auditor-years by mean
  max-cosine. Firm A occupies 95.9% of top-10% (base 27.8%), 3.5x
  concentration ratio. Stable across 2013-2023 (88-100% per year).
- Script 23 (intra-report consistency): for each 2-signer report,
  classify both signatures and check agreement. Firm A agrees 89.9%
  vs 62-67% at other Big-4. 87.5% Firm A reports have BOTH signers
  non-hand-signed; only 4 reports (0.01%) both hand-signed.

## New methodology additions
- III-G: explicit within-auditor-year no-mixing identification
  assumption (supported by Firm A interview evidence).
- III-H: 4th Firm A validation line: threshold-independent evidence
  from partner ranking + intra-report consistency.

## New results section IV-H (threshold-independent validation)
- IV-H.1: Firm A year-by-year cosine<0.95 rate. 2013-2019 mean=8.26%,
  2020-2023 mean=6.96%, 2023 lowest (3.75%). Stability contradicts
  partner's hypothesis that 2020+ electronic systems increase
  heterogeneity -- data shows opposite (electronic systems more
  consistent than physical stamping).
- IV-H.2: partner ranking top-K tables (pooled + year-by-year).
- IV-H.3: intra-report consistency per-firm table.

## Renumbering
- Section H (was Classification Results) -> I
- Section I (was Ablation) -> J
- Tables XIII-XVI new (yearly stability, top-K pooled, top-10% per-year,
  intra-report), XVII = classification (was XII), XVIII = ablation
  (was XIII).

These threshold-independent analyses address the codex review concern
about circular validation by providing benchmark evidence that does not
depend on any threshold calibrated to Firm A itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 01:59:49 +08:00
gbanyan 9d19ca5a31 Paper A v3.1: apply codex peer-review fixes + add Scripts 20/21 
Major fixes per codex (gpt-5.4) review:

## Structural fixes
- Fixed three-method convergence overclaim: added Script 20 to run KDE
  antimode, BD/McCrary, and Beta mixture EM on accountant-level means.
  Accountant-level 1D convergence: KDE antimode=0.973, Beta-2=0.979,
  LogGMM-2=0.976 (within ~0.006). BD/McCrary finds no transition at
  accountant level (consistent with smooth clustering, not sharp
  discontinuity).
- Disambiguated Method 1: KDE crossover (between two labeled distributions,
  used at signature all-pairs level) vs KDE antimode (single-distribution
  local minimum, used at accountant level).
- Addressed Firm A circular validation: Script 21 adds CPA-level 70/30
  held-out fold. Calibration thresholds derived from 70% only; heldout
  rates reported with Wilson 95% CIs (e.g. cos>0.95 heldout=93.61%
  [93.21%-93.98%]).
- Fixed 139+32 vs 180: the split is 139/32 of 171 Firm A CPAs with >=10
  signatures (9 CPAs excluded for insufficient sample). Reconciled across
  intro, results, discussion, conclusion.
- Added document-level classification aggregation rule (worst-case signature
  label determines document label).

## Pixel-identity validation strengthened
- Script 21: built ~50,000-pair inter-CPA random negative anchor (replaces
  the original n=35 same-CPA low-similarity negative which had untenable
  Wilson CIs).
- Added Wilson 95% CI for every FAR in Table X.
- Proper EER interpolation (FAR=FRR point) in Table X.
- Softened "conservative recall" claim to "non-generalizable subset"
  language per codex feedback (byte-identical positives are a subset, not
  a representative positive class).
- Added inter-CPA stats: mean=0.762, P95=0.884, P99=0.913.

## Terminology & sentence-level fixes
- "statistically independent methods" -> "methodologically distinct methods"
  throughout (three diagnostics on the same sample are not independent).
- "formal bimodality check" -> "unimodality test" (dip test tests H0 of
  unimodality; rejection is consistent with but not a direct test of
  bimodality).
- "Firm A near-universally non-hand-signed" -> already corrected to
  "replication-dominated" in prior commit; this commit strengthens that
  framing with explicit held-out validation.
- "discrete-behavior regimes" -> "clustered accountant-level heterogeneity"
  (BD/McCrary non-transition at accountant level rules out sharp discrete
  boundaries; the defensible claim is clustered-but-smooth).
- Softened White 1982 quasi-MLE claim (no longer framed as a guarantee).
- Fixed VLM 1.2% FP overclaim (now acknowledges the 1.2% could be VLM FP
  or YOLO FN).
- Unified "310 byte-identical signatures" language across Abstract,
  Results, Discussion (previously alternated between pairs/signatures).
- Defined min_dhash_independent explicitly in Section III-G.
- Fixed table numbering (Table XI heldout added, classification moved to
  XII, ablation to XIII).
- Explained 84,386 vs 85,042 gap (656 docs have only one signature, no
  pairwise stat).
- Made Table IX explicitly a "consistency check" not "validation"; paired
  it with Table XI held-out rates as the genuine external check.
- Defined 0.941 threshold (calibration-fold Firm A cosine P5).
- Computed 0.945 Firm A rate exactly (94.52%) instead of interpolated.
- Fixed Ref [24] Qwen2.5-VL to full IEEE format (arXiv:2502.13923).

## New artifacts
- Script 20: accountant-level three-method threshold analysis
- Script 21: expanded validation (inter-CPA anchor, held-out Firm A 70/30)
- paper/codex_review_gpt54_v3.md: preserved review feedback

Output: Paper_A_IEEE_Access_Draft_v3.docx (391 KB, rebuilt from v3.1
markdown sources).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 01:11:51 +08:00
gbanyan 68689c9f9b Correct Firm A framing: replication-dominated, not pure 
Interview evidence from multiple Firm A accountants confirms that MOST
use replication (stamping / firm-level e-signing) but a MINORITY may
still hand-sign. Firm A is therefore a "replication-dominated" population,
not a "pure" one. This framing is consistent with:

- 92.5% of Firm A signatures exceed cosine 0.95 (majority replication)
- The long left tail (~7%) captures the minority hand-signers, not scan
  noise or preprocessing artifacts
- Hartigan dip test: Firm A cosine unimodal long-tail (p=0.17)
- Accountant-level GMM: of 180 Firm A accountants, 139 cluster in C1
  (high-replication) and 32 in C2 (middle band = minority hand-signers)

Updates docstrings and report text in Scripts 15, 16, 18, 19 to match.
Partner v3's "near-universal non-hand-signing" language corrected.

Script 19 regenerated with the updated text.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 21:57:16 +08:00
gbanyan fbfab1fa68 Add three-convergent-method threshold scripts + pixel-identity validation 
Implements Partner v3's statistical rigor requirements at the level of
signature vs. accountant analysis units:

- Script 15 (Hartigan dip test): formal unimodality test via `diptest`.
  Result: Firm A cosine UNIMODAL (p=0.17, pure non-hand-signed population);
  full-sample cosine MULTIMODAL (p<0.001, mix of two regimes);
  accountant-level aggregates MULTIMODAL on both cos and dHash.

- Script 16 (Burgstahler-Dichev / McCrary): discretised Z-score transition
  detection. Firm A and full-sample cosine transitions at 0.985; dHash
  at 2.0.

- Script 17 (Beta mixture EM + logit-GMM): 2/3-component Beta via EM
  with MoM M-step, plus parallel Gaussian mixture on logit transform
  as White (1982) robustness check. Beta-3 BIC < Beta-2 BIC at signature
  level confirms 2-component is a forced fit -- supporting the pivot
  to accountant-level mixture.

- Script 18 (Accountant-level GMM): rebuilds the 2026-04-16 analysis
  that was done inline and not saved. BIC-best K=3 with components
  matching prior memory almost exactly: C1 (cos=0.983, dh=2.41, 20%,
  Deloitte 139/141), C2 (0.954, 6.99, 51%, KPMG/PwC/EY), C3 (0.928,
  11.17, 28%, small firms). 2-component natural thresholds:
  cos=0.9450, dh=8.10.

- Script 19 (Pixel-identity validation): no human annotation needed.
  Uses pixel_identical_to_closest (310 sigs) as gold positive and
  Firm A as anchor positive. Confirms Firm A cosine>0.95 = 92.51%
  (matches prior 2026-04-08 finding of 92.5%), dual rule
  cos>0.95 AND dhash_indep<=8 captures 89.95% of Firm A.

Python deps added: diptest, scikit-learn (installed into venv).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 21:51:41 +08:00
gbanyan a261a22bd2 Add Deloitte distribution & independent dHash analysis scripts 
- Script 13: Firm A normality/multimodality analysis (Shapiro-Wilk, Anderson-Darling, KDE, per-accountant ANOVA, Beta/Gamma fitting)
- Script 14: Independent min-dHash computation across all pairs per accountant (not just cosine-nearest pair)
- THRESHOLD_VALIDATION_OPTIONS: 2026-01 discussion doc on threshold validation approaches
- .gitignore: exclude model weights, node artifacts, and xlsx data

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 21:34:24 +08:00
gbanyan 939a348da4 Add Paper A (IEEE TAI) complete draft with Firm A-calibrated dual-method classification 
Paper draft includes all sections (Abstract through Conclusion), 36 references,
and supporting scripts. Key methodology: Cosine similarity + dHash dual-method
verification with thresholds calibrated against known-replication firm (Firm A).

Includes:
- 8 section markdown files (paper_a_*.md)
- Ablation study script (ResNet-50 vs VGG-16 vs EfficientNet-B0)
- Recalibrated classification script (84,386 PDFs, 5-tier system)
- Figure generation and Word export scripts
- Citation renumbering script ([1]-[36])
- Signature analysis pipeline (12 steps)
- YOLO extraction scripts

Three rounds of AI review completed (GPT-5.4, Claude Opus 4.6, Gemini 3 Pro).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 23:05:33 +08:00