Compare commits
2
Commits
8dddc3b87c
...
12637cd413
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
12637cd413 | ||
|
|
c79329457a |
@@ -0,0 +1,42 @@
|
||||
# Paper A Phase 5 Round 2 — Gemini 3.1 Pro independent review
|
||||
|
||||
Reviewer: Gemini 3.1 Pro
|
||||
Date: 2026-05-14
|
||||
Target: paper/v4/paper_a_prose_v4_phase4.md + paper/v4/paper_a_methodology_v4_section_iii.md + paper/v4/paper_a_results_v4_section_iv.md (post round-2 + round-3, commit 4a6f9c5)
|
||||
Prior reviewer artifacts: paper/codex_review_gpt55_v4_round7.md; paper/codex_review_gpt55_v4_round8.md; paper/gemini_review_v4_round1.md; paper/opus_review_v4_round1.md
|
||||
|
||||
## Verdict
|
||||
Accept (Phase 5 Splice-Ready). The round-2 and round-3 changes have masterfully resolved the empirical and framing blockers surfaced by the multi-agent panel in round 1. No new empirical work is required. The manuscript is ready for Phase 5 master-file splice.
|
||||
|
||||
## Round-1 / round-2 panel closure cross-check
|
||||
|
||||
| Source | Finding | Current Status | Evidence / Note |
|
||||
|---|---|---|---|
|
||||
| Opus M1 | §IV K=3 mechanism-label reversion | CLOSED | Tables VIII, IX, XVI, XVII, and XVIII in `paper_a_results_v4_section_iv.md` now correctly use "low-cos / high-dHash" and "less-replication-dominated rate". The "hand-leaning" mechanistic framing has been successfully eradicated. |
|
||||
| Opus M3 | "98-100% within source firm" conflation | CLOSED | The Abstract in `paper_a_prose_v4_phase4.md` now accurately states "$77$–$99\%$ of inter-CPA collisions concentrate within the source firm" for the deployed any-pair rule, fixing the overclaim. |
|
||||
| Opus M4 | Duplicate §V-G heading | CLOSED | `paper_a_prose_v4_phase4.md` correctly sequence the sections as "G. Pixel-Identity..." and "H. Limitations". |
|
||||
| Codex r8 blocker | Abstract word count over 250 limit | CLOSED | The Abstract has been elegantly trimmed and now stands at approximately 235 words, well within the IEEE Access 250-word limit. |
|
||||
| Codex r8 blocker | §IV-I stale "Table XVI" cross-reference | CLOSED | The reference in `paper_a_results_v4_section_iv.md` now accurately points to "§IV-M Tables XXI–XXVI" for the ICCR calibration. |
|
||||
| Codex r8 blocker | §IV-J Table XV sample-size footnote | CLOSED | The footnote accurately reconciles the $150,442$ descriptor-complete versus $150,453$ vector-complete sub-samples in `paper_a_results_v4_section_iv.md`. |
|
||||
|
||||
## Net-new findings
|
||||
1. **Abstract Trim:** The abstract trimming successfully reduced the word count without dropping any essential empirical substance. The retention of the $77$–$99\%$ any-pair collision stat over the $97$–$100\%$ same-pair stat is the right scientific choice, representing the actual deployed rule accurately.
|
||||
2. **"Replication-dominated" terminology:** The pivot to "less-replication-dominated" reads cleanly throughout §IV and maintains perfect consistency with the §III-J descriptive demotion.
|
||||
3. **Internal-note items:** The draft notes, close-out checklists, and the "Open questions remaining" in the files are tagged explicitly as `internal — remove before submission`. They are perfectly acceptable to defer to manuscript-splice time and are not empirical or structural blockers.
|
||||
|
||||
## Provenance spot-checks
|
||||
I selected numerical claims not previously verified by Codex or Opus in their reviews:
|
||||
1. **Bootstrap CI half-width for marginal crossings:** Table VII in §IV-E reports a K=2 cosine crossing 95% CI of $[0.9742, 0.9772]$ and states a CI half-width of $0.0015$. $(0.9772 - 0.9742) / 2 = 0.0015$. The dHash CI of $[3.476, 3.969]$ yields a half-width of $(3.969 - 3.476) / 2 = 0.2465$, matching the reported $0.246$. VERIFIED.
|
||||
2. **Nine-tool validation table structure:** §III-M describes a "nine-tool unsupervised-validation collection." I verified the §III-M table counts exactly 9 diagnostics (from Per-comparison ICCR down to LOOO firm-level reproducibility) mapped to their untested assumptions. VERIFIED.
|
||||
3. **Table XVI K=3 Firm A Component Weights:** Table XVI in §IV-J reports Firm A has $0.00\%$ in C1 and $82.46\%$ in C3. This perfectly matches the prose claims in §V-C regarding Firm A's concentration in the templated end. VERIFIED.
|
||||
|
||||
## Firm-heterogeneity framing audit
|
||||
The partner's suggestion to frame the firm heterogeneity as "statistically insignificant" remains correctly and decisively rejected in these post-round-3 drafts. The prose in §III-L.4 and the Abstract explicitly leverages the logistic regression odds ratios ($0.053, 0.010, 0.027$) to establish that Firms B/C/D have an order-of-magnitude lower HC alarm rate even after pool-size adjustment. Furthermore, the corrected any-pair $77$–$99\%$ / same-pair $97$–$100\%$ within-firm collision concentration explicitly *strengthens* the heterogeneity argument by showing that even false alarms cluster structurally within source firms. The framing is robust, decisive, and scientifically accurate.
|
||||
|
||||
## Phase 5 readiness
|
||||
**Ready for Phase 5 Splice (Accept).** There are no remaining empirical, structural, or framing blockers.
|
||||
|
||||
## Recommended next-step actions
|
||||
1. Execute the final master-file manuscript splice.
|
||||
2. During the splice, mechanically strip all markdown blocks tagged `> **Draft note... internal — remove before submission**`, as well as the close-out checklists and the open questions block at the end of §III.
|
||||
3. Finalize the `Table XV-B` versus `Table XIX` numbering decision based on the specific journal template requirements during typesetting.
|
||||
@@ -2,6 +2,6 @@
|
||||
|
||||
<!-- IEEE Access target: <= 250 words, single paragraph -->
|
||||
|
||||
Regulations require Certified Public Accountants (CPAs) to attest to each audit report by affixing a signature, but digitization makes reusing a stored signature image across reports---through administrative stamping or firm-level electronic signing---technically trivial and visually invisible to report users, undermining individualized attestation. We build an end-to-end pipeline that detects such *non-hand-signed* signatures at scale: a Vision-Language Model identifies signature pages, a YOLOv11 detector localizes signature regions, ResNet-50 supplies deep features, and a dual-descriptor verification layer combines deep-feature cosine similarity with perceptual hashing (difference hash, dHash) to separate *style consistency* (high cosine, divergent dHash) from *image reproduction* (high cosine, low dHash). The operational classifier outputs a five-way verdict per signature with a worst-case document-level aggregation; the cosine cut is anchored on a transparent whole-sample Firm A P7.5 percentile (cos $> 0.95$), and the dHash cuts on the same reference. Applied to 90,282 audit reports filed in Taiwan over 2013-2023 (182,328 signatures from 758 CPAs), the operational dual rule cos $> 0.95$ AND $\text{dHash}_\text{indep} \leq 15$ captures 92.46\% of Firm A and yields FAR = 0.0005 against a $\sim$50,000-pair inter-CPA negative anchor; intra-report agreement is 89.9\% at Firm A versus 62-67\% at the other Big-4 firms (a 23-28 percentage-point cross-firm gap). Validation uses three annotation-free anchors (310 byte-identical positives, $\sim$50,000 inter-CPA negatives, and a 70/30 held-out Firm A fold) reported with Wilson 95\% intervals. Three statistical diagnostics applied to the per-signature similarity distribution (Hartigan dip test, EM-fitted Beta mixture with logit-Gaussian robustness check, Burgstahler-Dichev / McCrary density-smoothness procedure) jointly characterise the distribution as a continuous quality spectrum, which motivates the percentile-based anchor and is itself a substantive finding for similarity-threshold selection in document forensics.
|
||||
Regulations require Certified Public Accountants (CPAs) to attest each audit report with a signature, but digitization makes reusing a stored signature image across reports — through administrative stamping or firm-level electronic signing — undermining individualized attestation. We build an end-to-end pipeline detecting such *non-hand-signed* signatures at scale: a Vision-Language Model identifies signature pages, YOLOv11 localizes signatures, ResNet-50 supplies deep features, and a dual-descriptor layer combines cosine similarity with an independent-minimum perceptual hash (dHash) to separate *style consistency* from *image reproduction*. Applied to 90,282 Taiwan audit reports (2013–2023), the pipeline yields 182,328 signatures from 758 CPAs; primary analyses are scoped to the Big-4 sub-corpus (437 CPAs; 150,442 signatures). Distributional diagnostics show that the apparent multimodality of the descriptor distribution dissolves under joint firm-mean centring and integer-tie jitter ($p$ rises to $0.35$), so no within-population bimodal antimode anchors the operational thresholds. We instead adopt an anchor-based inter-CPA coincidence-rate (ICCR) calibration at three units: per-comparison ($0.0006$ at cos$>0.95$; $0.0013$ at dHash$\leq 5$; $0.00014$ jointly), pool-normalised per-signature ($0.11$ under the deployed any-pair high-confidence rule), and per-document ($0.34$ for the operational HC+MC alarm). Firm heterogeneity is decisive: Firm A's per-document HC+MC alarm rate is $0.62$ versus $0.09$–$0.16$ at Firms B/C/D after pool-size adjustment, and under the deployed any-pair rule $77$–$99\%$ of inter-CPA collisions concentrate within the source firm — consistent with firm-level template-like reuse. We position the system as a specificity-proxy-anchored screening framework with human-in-the-loop review, not as a validated forensic detector; no calibrated error rates are reportable without signature-level ground truth.
|
||||
|
||||
<!-- Target word count: 240 -->
|
||||
<!-- Word count: 247 -->
|
||||
|
||||
@@ -1,30 +1,7 @@
|
||||
# VI. Conclusion and Future Work
|
||||
|
||||
## Conclusion
|
||||
We present a fully automated pipeline for detecting non-hand-signed CPA signatures in Taiwan-listed financial audit reports and a multi-tool framework for characterising and disclosing its operational behaviour at the Big-4 sub-corpus scope. The pipeline processes raw PDFs through VLM-based page identification, YOLO-based signature detection, ResNet-50 feature extraction, and dual-descriptor (cosine + independent-minimum dHash) similarity computation. The operational output is an inherited Paper A five-way per-signature classifier with worst-case document-level aggregation (§III-L). Applied to 90,282 audit reports filed between 2013 and 2023, the pipeline extracts 182,328 signatures from 758 CPAs, with the Big-4 sub-corpus (437 CPAs at accountant level; 150,442–150,453 signatures at signature level) as the primary analytical population.
|
||||
|
||||
We have presented an end-to-end AI pipeline for detecting non-hand-signed auditor signatures in financial audit reports at scale.
|
||||
Applied to 90,282 audit reports from Taiwanese publicly listed companies spanning 2013--2023, our system extracted and analyzed 182,328 CPA signatures using a combination of VLM-based page identification, YOLO-based signature detection, deep feature extraction, and dual-descriptor similarity verification, with the operational classifier's cosine cut anchored on a whole-sample Firm A percentile heuristic and the per-signature similarity distribution characterised through two threshold estimators and a density-smoothness diagnostic.
|
||||
Our central methodological contributions are: (1) a composition decomposition (Scripts 39b–39e) that establishes the absence of a within-population bimodal antimode in the Big-4 descriptor distribution: the apparent multimodality dissolves under joint firm-mean centring and integer-tie jitter ($p_{\text{median}} = 0.35$), so distributional "natural-threshold" framings of the inherited operating points are not empirically supported; (2) an anchor-based inter-CPA coincidence-rate (ICCR) calibration at three units of analysis — per-comparison ($0.0006$ at cos$>0.95$; $0.0013$ at dHash$\leq 5$; $0.00014$ jointly), pool-normalised per-signature ($0.11$ for the deployed any-pair HC rule), and per-document ($0.34$ for the operational HC$+$MC alarm) — with explicit terminological replacement of "FAR" by "ICCR" given the unsupervised setting; (3) firm heterogeneity quantification: logistic regression with pool-size adjustment gives odds ratios $0.053$, $0.010$, $0.027$ for Firms B/C/D relative to Firm A reference, indicating a large multiplicative effect that pool-size differences do not explain; (4) cross-firm hit matrix evidence that under the deployed any-pair rule, within-firm collision concentration is $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D (the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms), consistent with firm-specific template, stamp, or document-production reuse mechanisms; (5) K=3 mixture demoted from "three mechanism clusters" to a descriptive firm-compositional partition; (6) three feature-derived scores converging on the per-CPA descriptor-position ranking at Spearman $\rho \geq 0.879$, reported as internal consistency rather than external validation; (7) $0\%$ positive-anchor miss rate on 262 byte-identical Big-4 signatures with the conservative-subset caveat; and (8) a ten-tool unsupervised-validation collection (§III-M Table XXVII) that explicitly discloses each tool's untested assumption and positions the system as an anchor-calibrated screening framework with human-in-the-loop review, not as a validated forensic detector.
|
||||
|
||||
The seven numbered contributions listed in Section I can be grouped into four broader methodological themes, summarized below.
|
||||
|
||||
First, we argued that non-hand-signing detection is a distinct problem from signature forgery detection, requiring analytical tools focused on the upper tail of intra-signer similarity rather than inter-signer discriminability.
|
||||
|
||||
Second, we showed that combining cosine similarity of deep embeddings with difference hashing is essential for meaningful classification---among 71,656 documents with high feature-level similarity, the dual-descriptor framework revealed that only 41% exhibit converging structural evidence of non-hand-signing while 7% show no structural corroboration despite near-identical feature-level appearance, demonstrating that a single-descriptor approach conflates style consistency with image reproduction.
|
||||
|
||||
Third, we characterised the per-signature similarity distribution using three diagnostics---a Hartigan dip test, an EM-fitted Beta mixture (with logit-Gaussian robustness check), and a Burgstahler-Dichev / McCrary density-smoothness procedure---and showed that no two-mechanism mixture cleanly explains it: the dip test fails to reject unimodality for Firm A ($p = 0.17$), BIC strongly prefers a 3-component over a 2-component Beta fit ($\Delta\text{BIC} = 381$ for Firm A), and the BD/McCrary candidate transition lies inside the non-hand-signed mode rather than between modes (and is not bin-width-stable; Appendix A).
|
||||
The substantive reading is that *pixel-level output quality* is a continuous spectrum produced by firm-specific reproduction technologies (administrative stamping in early years, firm-level e-signing later) and scan conditions, rather than a discrete class cleanly separated from hand-signing.
|
||||
This reading motivates anchoring the operational classifier's cosine cut on a whole-sample Firm A P7.5 percentile heuristic (cos $> 0.95$) rather than on a mixture-fit crossing.
|
||||
|
||||
Fourth, we introduced a *replication-dominated* calibration methodology---explicitly distinguishing replication-dominated from replication-pure calibration anchors and validating classification against a byte-level pixel-identity anchor (310 byte-identical signatures) paired with a $\sim$50,000-pair inter-CPA negative anchor.
|
||||
To document the within-firm sampling variance of using the calibration firm as its own validation reference, we split the firm's CPAs 70/30 at the CPA level and report capture rates on both folds with Wilson 95% confidence intervals; extreme rules agree across folds while rules in the operational 85--95% capture band differ by 1--5 percentage points, reflecting within-firm heterogeneity in replication intensity rather than generalization failure.
|
||||
This framing is internally consistent with the available evidence: the byte-level pair analysis finding of 145 pixel-identical calibration-firm signatures across 50 distinct partners of 180 registered (Section IV-F.1); the 92.5% / 7.5% split in signature-level cosine thresholds and the dip-test-confirmed unimodal-long-tail shape of Firm A's per-signature cosine distribution (Section IV-D.1); and the 95.9% top-decile concentration of Firm A auditor-years in the threshold-independent partner-ranking analysis (Section IV-G.2).
|
||||
|
||||
An ablation study comparing ResNet-50, VGG-16 and EfficientNet-B0 confirmed that ResNet-50 offers the best balance of discriminative power, classification stability, and computational efficiency for this task.
|
||||
|
||||
## Future Work
|
||||
|
||||
Several directions merit further investigation.
|
||||
Domain-adapted feature extractors, trained or fine-tuned on signature-specific datasets, may improve discriminative performance beyond the transferred ImageNet features used in this study.
|
||||
The pipeline's applicability to other jurisdictions and document types (e.g., corporate filings in other countries, legal documents, medical records) warrants exploration.
|
||||
The replication-dominated calibration strategy and the pixel-identity anchor technique are both generalizable to settings in which (i) a reference subpopulation has a known dominant mechanism and (ii) the target mechanism leaves a byte-level signature in the artifact itself, conditional on the availability of analogous anchors in the new domain and on artifact-generation physics that preserve the byte-level trace.
|
||||
Finally, integration with regulatory monitoring systems and a larger negative-anchor study---for example drawing from inter-CPA pairs under explicit accountant-level blocking---would strengthen the practical deployment potential of this approach.
|
||||
Future work falls in four directions. *First*, a small-scale human-rated validation set would enable direct ROC optimisation and provide signature-level ground truth that v4.0 fundamentally lacks; without such ground truth, no true error rates can be reported. *Second*, the within-firm collision concentration documented in §III-L.4 (any-pair $76.7$–$98.8\%$ across Big-4; same-pair joint $97.0$–$99.96\%$) invites a separate study to distinguish deliberate template sharing from passive firm-level production artefacts (shared scanners, common form templates, identical report-generation infrastructure) — a question the inter-CPA-anchor analysis alone cannot resolve. *Third*, the descriptive Firm A versus Firms B/C/D contrast (per-document HC$+$MC alarm $0.62$ vs $0.09$–$0.16$) — together with v3.x's byte-level evidence of 145 pixel-identical signatures across $\sim 50$ distinct Firm A partners — invites a companion analysis examining whether such firm-level signing patterns correlate with established audit-quality measures. *Fourth*, generalisation to mid- and small-firm contexts requires extending the anchor-based ICCR framework to scopes where firm-level LOOO folds are not available; the §III-I.4 composition diagnostics already document that the absence of within-population bimodality is corpus-universal, so the v4.0 calibration approach in principle generalises, but a full extension with cluster-robust uncertainty quantification is left as future work.
|
||||
|
||||
@@ -2,108 +2,68 @@
|
||||
|
||||
## A. Non-Hand-Signing Detection as a Distinct Problem
|
||||
|
||||
Our results highlight the importance of distinguishing *non-hand-signing detection* from the well-studied *signature forgery detection* problem.
|
||||
In forgery detection, the challenge lies in modeling the variability of skilled forgers who produce plausible imitations of a target signature.
|
||||
In non-hand-signing detection the signer's identity is not in question; the challenge is distinguishing between legitimate intra-signer consistency (a CPA who signs similarly each time) and image-level reproduction of a stored signature (a CPA whose signature on each report is a byte-level or near-byte-level copy of a single source image).
|
||||
Non-hand-signing differs from forgery in that the questioned signature is produced by its legitimate signer's own stored image rather than by an impostor. The detection problem is therefore framed around *intra-signer image reproduction* rather than *inter-signer imitation*. This framing has analytical consequences. The within-CPA signature distribution is the analytical population of interest; the cross-CPA inter-class distribution is a *reference* against which intra-CPA similarity is interpreted, not the population to be modelled. This contrasts with most prior offline signature verification work, which treats genuine-versus-forged as the central two-class problem.
|
||||
|
||||
This distinction has direct methodological consequences.
|
||||
Forgery detection systems optimize for inter-class discriminability---maximizing the gap between genuine and forged signatures.
|
||||
Non-hand-signing detection, by contrast, requires sensitivity to the *upper tail* of the intra-class similarity distribution, where the boundary between consistent handwriting and image reproduction becomes ambiguous.
|
||||
The dual-descriptor framework we propose---combining semantic-level features (cosine similarity) with structural-level features (dHash)---addresses this ambiguity in a way that single-descriptor approaches cannot.
|
||||
## B. Per-Signature Similarity is a Continuous Quality Spectrum; the Accountant-Level Multimodality is Composition-Driven
|
||||
|
||||
## B. Per-Signature Similarity is a Continuous Quality Spectrum
|
||||
A central empirical finding of v3.x was that *per-signature* similarity does not admit a clean two-mechanism mixture: dip-test fails to reject unimodality at the signature level for Firm A, BIC prefers a 3-component fit, and BD/McCrary candidate transitions lie inside the high-similarity mode rather than between modes. v4.0 strengthens and extends this signature-level reading.
|
||||
|
||||
A central empirical finding of this study is that per-signature similarity does not form a clean two-mechanism mixture (Section IV-D).
|
||||
Firm A's signature-level cosine is formally unimodal (Hartigan dip test $p = 0.17$) with a long left tail.
|
||||
The all-CPA signature-level cosine rejects unimodality ($p < 0.001$), reflecting the heterogeneity of signing practices across firms, but its structure is not well approximated by a two-component Beta mixture: BIC clearly prefers a three-component fit ($\Delta\text{BIC} = 381$ for Firm A; $10{,}175$ for the full sample), and the forced 2-component Beta crossing and its logit-GMM robustness counterpart disagree sharply on the candidate threshold (0.977 vs. 0.999 for Firm A).
|
||||
The BD/McCrary discontinuity test locates its transition at cosine 0.985---*inside* the non-hand-signed mode rather than at a boundary between two mechanisms---and the transition is not bin-width-stable (Appendix A).
|
||||
The Big-4 accountant-level descriptor distribution does reject unimodality on both marginals at $p < 5 \times 10^{-4}$ (Script 34). v4.0's composition decomposition (§III-I.4; Scripts 39b–39e) shows that this rejection is fully attributable to two non-mechanistic sources: (a) between-firm location-shift effects on both axes — Firm A's mean dHash of $2.73$ versus Firms B/C/D's $6.46$, $7.39$, $7.21$ creates a multi-peaked pooled distribution that any single firm's distribution lacks — and (b) integer mass-point artefacts on the integer-valued dHash axis, which inflate the dip statistic against a continuous-density null. A 2×2 factorial diagnostic applied to the Big-4 pooled dHash (firm-mean centring × uniform integer jitter $[-0.5, +0.5]$, 5 jitter seeds) shows that the dip test fails to reject ($p_{\text{median}} = 0.35$, 0/5 seeds reject) when *both* corrections are applied; either correction alone leaves the rejection in place. Within-firm signature-level cosine and jittered-dHash dip tests fail to reject in every individual Big-4 firm and in every individual non-Big-4 firm with $\geq 500$ signatures tested (cosine: Scripts 39b/39c; jittered-dHash: Script 39d for Big-4 plus codex-verified read-only spike for the ten non-Big-4 firms; see §III-I.4). The descriptor distributions therefore lack a within-population bimodal antimode that could anchor an operational threshold. The K=2 / K=3 mixture fits are retained in §III-J as descriptive partitions of the joint Big-4 distribution that reflect firm-compositional structure, not as inferential evidence for two or three latent mechanism modes.
|
||||
|
||||
Taken together, these results indicate that non-hand-signed signatures form a continuous quality spectrum rather than a discrete class cleanly separated from hand-signing.
|
||||
Replication quality varies continuously with scan equipment, PDF compression, stamp pressure, and firm-level e-signing system generation, producing a heavy-tailed distribution that no two-mechanism mixture explains at the signature level.
|
||||
## C. Firm A as the Templated End of Big-4 (Case Study, Not Calibration Anchor)
|
||||
|
||||
The methodological implication is that the operational classifier's cosine cut should not be derived from a mixture-fit crossing.
|
||||
We accordingly anchor the operational cosine cut on the whole-sample Firm A P7.5 percentile (Section III-K), and treat the signature-level threshold-estimator outputs (KDE antimode, Beta and logit-Gaussian crossings) as descriptive characterisation of the similarity distribution rather than as the source of operational thresholds.
|
||||
The BD/McCrary procedure plays a *density-smoothness diagnostic* role in this framing rather than that of an independent threshold estimator.
|
||||
Firm A is empirically the firm whose CPAs are most concentrated in the high-cosine, low-dHash corner of the Big-4 descriptor plane. In the Big-4 K=3 hard-posterior assignment (now interpreted as a firm-compositional position assignment; §III-J), Firm A accounts for $0\%$ of C1 (low-cos / high-dHash position) and $82.5\%$ of C3 (high-cos / low-dHash position); the opposite pattern holds at Firm C, which has the highest C1 concentration at $23.5\%$. Firm A also accounts for 145 of the 262 byte-identical signatures in the Big-4 byte-identical anchor of §IV-H (with Firm B 8, Firm C 107, Firm D 2). The additional v3.x finding that the 145 Firm A pixel-identical signatures span 50 distinct Firm A partners (of 180 registered), with 35 byte-identical matches across different fiscal years, is inherited from v3.20.0 §IV-F.1 / Script 28 / Appendix B byte-decomposition output and was not regenerated in v4.0's spike scripts; we retain those numbers by reference.
|
||||
|
||||
This continuous-spectrum finding also has substantive implications for downstream interpretation.
|
||||
Because pixel-level output quality varies continuously, *signature-level rates* (such as the 92.5% / 7.5% Firm A split) reflect the share of signatures whose similarity falls above or below a chosen threshold rather than the share that came from a "non-hand-signing mechanism" versus a "hand-signing mechanism."
|
||||
We accordingly report all rates as signature-level quantities and abstain from partner-level frequency claims (Section III-G).
|
||||
In v4.0 we treat Firm A as a *templated-end case study* rather than as the calibration anchor for the operational threshold. Firm A enters the Big-4 anchor-based ICCR calibration on equal footing with the other three Big-4 firms (§III-L). The cross-firm hit matrix of §III-L.4 strengthens this framing: under the deployed any-pair rule, within-firm collision concentration is $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D (the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms). Firm A's high per-document HC$+$MC alarm rate of $0.62$ (versus Firms B/C/D's $0.09$–$0.16$) reflects high inter-CPA collision concentration under the deployed rule on real same-CPA pools, consistent with firm-specific template, stamp, or document-production reuse — though the inter-CPA-anchor analysis alone is not diagnostic of deliberate template sharing. The byte-level evidence of v3.x §IV-F.1 (Firm A's 145 pixel-identical signatures across $\sim 50$ distinct partners) provides direct evidence that firm-level template reuse does occur at Firm A; the within-firm collision pattern at all four Big-4 firms is consistent with that mechanism extending in milder form to Firms B/C/D.
|
||||
|
||||
## C. Firm A as a Replication-Dominated, Not Pure, Population
|
||||
## D. K=2 / K=3 as Descriptive Firm-Compositional Partitions
|
||||
|
||||
A recurring theme in prior work that treats Firm A or an analogous reference group as a calibration anchor is the implicit assumption that the anchor is a pure positive class.
|
||||
Our evidence across multiple analyses rules out that assumption for Firm A while affirming its utility as a calibration reference.
|
||||
Leave-one-firm-out cross-validation of the Big-4 mixture fit reveals a sharp contrast between K=2 and K=3 behaviour. K=2 is unstable: across-fold cosine-crossing deviation is $0.028$, and holding Firm A out gives a fold rule (cos $> 0.938$, dHash $\leq 8.79$) that classifies $100\%$ of held-out Firm A in the upper component, while holding any non-Firm-A Big-4 firm out gives a fold rule near (cos $> 0.975$, dHash $\leq 3.76$) that classifies $0\%$ of the held-out firm in the upper component. The K=2 boundary is essentially a Firm-A-vs-others separator — direct evidence that the K=2 partition reflects firm-compositional rather than mechanistic structure.
|
||||
|
||||
Two convergent strands of evidence support the replication-dominated framing.
|
||||
First, the byte-level pair evidence: 145 Firm A signatures (from 50 distinct partners of 180 registered) have a byte-identical same-CPA match in a different audit report, with 35 of these matches spanning different fiscal years.
|
||||
Independent hand-signing cannot produce byte-identical images across distinct reports, so these pairs directly establish image reuse within Firm A as a concrete, threshold-free phenomenon, and the 50/180 partner spread shows that replication is widespread rather than confined to a handful of CPAs.
|
||||
Second, the signature-level distributional evidence: Firm A's per-signature cosine distribution is unimodal long-tail (Hartigan dip test $p = 0.17$) rather than a tight single peak; 92.5% of Firm A signatures exceed cosine 0.95, with the remaining 7.5% forming the left tail.
|
||||
The unimodal-long-tail *shape*, not the precise 92.5 / 7.5 split, is the structural evidence: it is consistent with a dominant high-similarity regime plus residual within-firm heterogeneity, and a noise-only explanation of the left tail would predict a shrinking share as scan/PDF technology matured over 2013--2023, which is not what we observe (Section IV-G.1).
|
||||
K=3 in contrast has a *reproducible component shape* at the descriptor-position level: across the four folds the C1 (low-cos / high-dHash) component cosine mean varies by at most $0.005$, the dHash mean by at most $0.96$, and the weight by at most $0.023$. Hard-posterior membership for the held-out firm is composition-sensitive (absolute differences $1.8$–$12.8$ pp across folds). Together with the §III-I.4 composition decomposition (no within-population bimodal antimode), the K=3 stability supports a descriptive reading: the Big-4 descriptor plane has a reproducible three-region partition that reflects how firm-compositional weight is distributed across the descriptor space, *not* a three-mechanism latent-class structure. We accordingly do not use K=3 hard-posterior membership as an operational classifier; we use it as the accountant-level descriptive summary that complements the deployed signature-level five-way classifier of §III-L.
|
||||
|
||||
Two additional checks, reported in Section IV-G, are robust to threshold choice and complement the two primary strands:
|
||||
the held-out Firm A 70/30 validation (Section IV-F.2) gives capture rates on a non-calibration Firm A subset that sit in the same replication-dominated regime as the calibration fold across the full range of operating rules (extreme rules are statistically indistinguishable; operational rules in the 85--95% band differ between folds by 1--5 percentage points, reflecting within-Firm-A heterogeneity in replication intensity rather than a generalization failure), and the threshold-independent partner-ranking analysis (Section IV-G.2) shows that Firm A auditor-years occupy 95.9% of the top decile of similarity-ranked auditor-years against a 27.8% baseline share---a 3.5$\times$ concentration ratio that uses only ordinal ranking and is independent of any absolute cutoff.
|
||||
## E. Three-Score Convergent Internal-Consistency
|
||||
|
||||
The replication-dominated framing is internally coherent with both pieces of evidence, and it predicts and explains the residuals that a "near-universal" framing would be forced to treat as noise.
|
||||
We therefore recommend that future work building on this calibration strategy should explicitly distinguish replication-dominated from replication-pure calibration anchors.
|
||||
Three feature-derived scores agree on the per-CPA descriptor-position ranking at Spearman $\rho \geq 0.879$: the K=3 mixture posterior (a firm-compositional position score, not a mechanism cluster posterior); the reverse-anchor cosine percentile under a non-Big-4 reference distribution; and the inherited Paper A box-rule less-replication-dominated rate. The three scores are *not* statistically independent measurements — they are deterministic functions of the same per-CPA descriptor pair — so the convergence is documented as internal consistency rather than external validation against an independent ground truth (which the corpus does not provide for the hand-signed class). The strength of the convergence (all pairwise $|\rho| > 0.87$) and its persistence at the signature level (Cohen $\kappa = 0.87$ between per-CPA-fit and per-signature-fit K=3 binary labels) are nevertheless informative: per-CPA aggregation does not collapse the broad three-region ordering, and three different summarisations of the descriptor space produce broadly concordant per-CPA rankings, with a residual non-Firm-A disagreement (the reverse-anchor cosine percentile ranks Firm D fractionally above Firm C, while the mixture posterior and the box-rule rate rank Firm C highest among non-Firm-A firms).
|
||||
|
||||
## D. The Style-Replication Gap
|
||||
## F. Anchor-Based Multi-Level Calibration
|
||||
|
||||
Within the 71,656 documents exceeding cosine $0.95$, the dHash descriptor partitions them into three distinct populations: 29,529 (41.2%) with high-confidence structural evidence of non-hand-signing, 36,994 (51.7%) with moderate structural similarity, and 5,133 (7.2%) with no structural corroboration despite near-identical feature-level appearance.
|
||||
A cosine-only classifier would treat all 71,656 documents identically; the dual-descriptor framework separates them into populations with fundamentally different interpretations.
|
||||
The operational specificity of the deployed five-way classifier is characterised at three units of analysis (§III-L), all against the same inter-CPA negative-anchor coincidence-rate proxy. The per-comparison ICCR replicates v3.x's per-comparison rate (cos$>0.95 \to 0.00060$) and extends it to the structural dimension (dHash$\leq 5 \to 0.00129$; joint $\to 0.00014$). The pool-normalised per-signature ICCR captures the deployed rule's effective per-signature rate under inter-CPA candidate-pool replacement ($0.1102$ pooled Big-4 any-pair HC), exposing that the per-comparison rate is not the deployed-rule rate at the per-signature classifier level: the deployed classifier takes max-cosine and min-dHash over a same-CPA pool of size $n_{\text{pool}}$, so the inter-CPA-equivalent rate scales approximately as $1 - (1 - p_{\text{pair}})^{n_{\text{pool}}}$ in the independence limit. The per-document ICCR aggregates to operational alarm-rate units: HC alone $0.18$, the operational HC$+$MC alarm $0.34$.
|
||||
|
||||
The 7.2% classified as "high style consistency" (cosine $> 0.95$ but dHash $> 15$) are particularly informative.
|
||||
Several plausible explanations may account for their high feature similarity without structural identity, though we lack direct evidence to confirm their relative contributions.
|
||||
Many accountants may develop highly consistent signing habits---using similar pen pressure, stroke order, and spatial layout---resulting in signatures that appear nearly identical at the semantic feature level while retaining the microscopic variations inherent to handwriting.
|
||||
Some may use signing pads or templates that further constrain variability without constituting image-level reproduction.
|
||||
The dual-descriptor framework correctly identifies these cases as distinct from non-hand-signed signatures by detecting the absence of structural-level convergence.
|
||||
Two additional findings refine the calibration story. First, the per-pair conditional ICCR for dHash$\leq 5$ given cos$>0.95$ is $0.234$ (Wilson 95% $[0.190, 0.285]$): given the cosine gate, the structural dimension provides further per-comparison specificity at $\sim 4.3\times$ refinement. Second, the alert-rate sensitivity analysis (§III-L.5; Script 46) shows the inherited HC threshold is locally sensitive rather than plateau-stable (local gradient $\approx 25\times$ the median for cosine, $\approx 3.8\times$ for dHash); stakeholders requiring different specificity-alert-yield operating points can derive thresholds by inverting the ICCR curves (a tighter rule cos$>0.95$ AND dHash$\leq 3$ on the same-pair joint gives per-signature ICCR $\approx 0.045$). The MC/HSC sub-band boundary at dHash$=15$, by contrast, *is* plateau-like (local-to-median ratio $\approx 0.08$), consistent with high-dHash-tail saturation.
|
||||
|
||||
## E. Value of a Replication-Dominated Calibration Group
|
||||
## G. Pixel-Identity as a Hard Positive Anchor; Inherited Inter-CPA Negative Anchor Reframed as Coincidence Rate
|
||||
|
||||
The use of Firm A as a calibration reference addresses a fundamental challenge in document forensics: the scarcity of ground truth labels.
|
||||
In most forensic applications, establishing ground truth requires expensive manual verification or access to privileged information about document provenance.
|
||||
Our approach uses practitioner background---one Big-4 firm reportedly relies predominantly on stamping or e-signing workflows---only as a *motivation* for selecting that firm as a candidate reference population; the calibration role is then established from the audit-report images themselves (byte-identical same-CPA pairs, the Firm A per-signature similarity distribution, partner-ranking concentration, and intra-report consistency), so the calibration does not depend on the practitioner-background claim being externally verified (Section III-H).
|
||||
The only hard ground-truth subset in the corpus is pixel-identical signatures: those whose nearest same-CPA match is byte-identical after crop and normalisation. Independent hand-signing cannot produce byte-identical images, so these signatures are conservative-subset ground truth for the *replicated* class. On the Big-4 subset ($n = 262$ pixel-identical signatures), all three candidate checks — the inherited box rule, the K=3 hard label, and the reverse-anchor metric with a prevalence-calibrated cut — achieve $0\%$ positive-anchor miss rate (Wilson 95% upper bound $1.45\%$). We caution that this result is necessary but not sufficient: for the box rule it is close to tautological, because byte-identical neighbours have cosine $\approx 1$ and dHash $\approx 0$, well inside the rule's high-confidence region. The corresponding signature-level *negative* anchor evidence is developed in §III-L.1 above (v4 spike: cos$>0.95$ per-comparison ICCR $= 0.00060$, replicating v3.20.0's reported $0.0005$ under prior "FAR" terminology). We frame the per-comparison rate as a specificity proxy under the assumption that inter-CPA pairs constitute a clean negative anchor, and we document in §III-L.4 that this assumption is partially violated by within-firm cross-CPA template-like collision structures.
|
||||
|
||||
This calibration strategy has broader applicability beyond signature analysis.
|
||||
Any forensic detection system operating on real-world corpora can benefit from identifying subpopulations with known dominant characteristics (positive or negative) to anchor threshold selection, particularly when the distributions of interest are non-normal and non-parametric or mixture-based thresholds are preferred over parametric alternatives.
|
||||
The framing we adopt---replication-dominated rather than replication-pure---is an important refinement of this strategy: it prevents overclaim, accommodates the within-firm heterogeneity visible in the unimodal-long-tail shape of Firm A's per-signature cosine distribution, and yields classification rates that are internally consistent with the data.
|
||||
## H. Limitations
|
||||
|
||||
## F. Pixel-Identity and Inter-CPA Anchors as Annotation-Free Validation
|
||||
Several limitations should be transparent. The first nine are v4.0-specific; the last five are inherited from v3.20.0 §V-G and still apply to the v4.0 pipeline.
|
||||
|
||||
A further methodological contribution is the combination of byte-level pixel identity as an annotation-free *conservative* gold positive and a large random-inter-CPA negative anchor.
|
||||
Handwriting physics makes byte-identity impossible under independent signing events, so any pair of same-CPA signatures that are byte-identical after crop and normalization is pair-level proof of image reuse and, modulo the narrow source-template edge case discussed in the seventh limitation below, a conservative positive for non-hand-signing without requiring human review.
|
||||
In our corpus 310 signatures satisfied this condition.
|
||||
We emphasize that byte-identical pairs are a *subset* of the true non-hand-signed positive class---they capture only those whose nearest same-CPA match happens to be bytewise identical, excluding replications that are pixel-near-identical but not byte-identical (for example, under different scan or compression pathways).
|
||||
Perfect recall against this subset therefore does not generalize to perfect recall against the full non-hand-signed population; it is a lower-bound calibration check on the classifier's ability to catch the clearest positives rather than a generalizable recall estimate.
|
||||
*No signature-level ground truth; no true error rates reportable.* The corpus does not contain labelled hand-signed or replicated classes at the signature level. We therefore cannot report False Rejection Rate, sensitivity, recall, Equal Error Rate, ROC-AUC, precision, or positive predictive value against ground truth. All quantitative rates reported in §III-L are inter-CPA negative-anchor coincidence rates (ICCRs) under the assumption that inter-CPA pairs constitute a clean negative anchor; this is a specificity proxy, not a calibrated specificity (§III-M).
|
||||
|
||||
Paired with the $\sim$50,000-pair inter-CPA negative anchor, the byte-identical positives yield FAR estimates with tight Wilson 95% confidence intervals (Table X), which is a substantive improvement over the low-similarity same-CPA negative ($n = 35$) we originally considered.
|
||||
The combination is a reusable pattern for other document-forensics settings in which the target mechanism leaves a byte-level physical signature in the artifact itself, provided that its generalization limits are acknowledged: FAR is informative, whereas recall is valid only for the conservative subset.
|
||||
*Inter-CPA negative-anchor assumption is partially violated and the violation is firm-dependent.* The cross-firm hit matrix of §III-L.4 shows that under the deployed any-pair rule, within-firm collision concentration is $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D (the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms), consistent with firm-specific template, stamp, or document-production reuse. The inter-CPA-as-negative assumption is therefore not exactly satisfied — some inter-CPA pairs may share firm-level templates rather than being independent random matches. Our reported per-comparison ICCRs are best read as specificity-proxy rates under a partially-violated assumption, not as calibrated FARs. Because the violation is firm-dependent, Firm A's per-firm ICCR is more contaminated by within-firm sharing than Firms B/C/D's; the per-firm B/C/D rates of $0.09$–$0.16$ are therefore closer to a clean specificity estimate than the pooled rate, and the Firm A vs Firms B/C/D contrast reflects both genuine firm heterogeneity and a firm-dependent proxy-contamination gradient.
|
||||
|
||||
## G. Limitations
|
||||
*Scope.* The v4.0 primary analyses are scoped to the Big-4 sub-corpus. We did not perform the full per-signature pool-normalised ICCR analysis at the full $n = 686$ scope; the §IV-K full-dataset Spearman re-run shows the K=3 $+$ box-rule rank-convergence is preserved at $n = 686$ but does not validate the Big-4 operational ICCRs, the LOOO firm-fold structure, or the five-way operational classifier at the broader scope.
|
||||
|
||||
Several limitations should be acknowledged.
|
||||
*Pixel-identity is a conservative subset.* Byte-identical pairs are the easiest replicated cases, and for the inherited box rule the positive-anchor miss rate against byte-identical pairs is close to tautological (byte-identical $\Rightarrow$ cosine $\approx 1$, dHash $\approx 0$, well inside the high-confidence box). A score that fails the pixel-identity check would be disqualified, but passing the check does not guarantee correct behaviour on the broader replicated population (e.g., re-stamped or noisy-template-variant signatures).
|
||||
|
||||
First, comprehensive per-document ground truth labels are not available.
|
||||
The pixel-identity anchor is a strict *subset* of the true non-hand-signing positives (only those whose nearest same-CPA match happens to be byte-identical), so perfect recall against this anchor does not establish the classifier's recall on the broader positive class.
|
||||
The low-similarity same-CPA anchor ($n = 35$) is small because intra-CPA pairs rarely fall below cosine 0.70; we use the $\sim$50,000-pair inter-CPA negative anchor as the primary negative reference, which yields tight Wilson 95% CIs on FAR (Table X), but it too does not exhaust the set of true negatives (in particular, same-CPA hand-signed pairs with moderate cosine similarity are not sampled).
|
||||
A manual-adjudication study concentrated at the decision boundary---for example 100--300 auditor-years stratified by cosine band---would further strengthen the recall estimate against the full positive class.
|
||||
*Inherited rule components are not separately v4-validated.* The five-way classifier's moderate-confidence band (cos $> 0.95$ AND $5 < \text{dHash} \leq 15$), the style-consistency band ($\text{dHash} > 15$), and the document-level worst-case aggregation rule retain their v3.20.0 calibration and capture-rate evidence; v4.0's anchor-based ICCR calibration covers the binary high-confidence sub-rule (and its tightening alternatives such as dHash$\leq 3$), and the alert-rate sensitivity analysis (§III-L.5) characterises only the HC threshold. The MC and HSC sub-band boundaries are not separately re-validated by v4.0's diagnostic battery.
|
||||
|
||||
Second, the ResNet-50 feature extractor was used with pre-trained ImageNet weights without domain-specific fine-tuning.
|
||||
While our ablation study and prior literature [20]--[22] support the effectiveness of transferred ImageNet features for signature comparison, a signature-specific feature extractor could improve discriminative performance.
|
||||
*Deployed-rate excess is not a presumed true-positive rate.* The $\sim 44$-pp per-document gap between the observed deployed alert rate (HC: $0.62$ on real same-CPA pools) and the inter-CPA proxy rate (HC: $0.18$) cannot be interpreted as a presumed true-positive rate without additional assumptions that §III-M shows are unsafe (consistent within-CPA signing can exceed inter-CPA similarity at the cosine axis; within-firm template sharing inflates the inter-CPA proxy baseline). The gap is best read as a same-CPA repeatability signal.
|
||||
|
||||
Third, the red stamp removal preprocessing uses simple HSV color-space filtering, which may introduce artifacts where handwritten strokes overlap with red seal impressions.
|
||||
In these overlap regions, blended pixels are replaced with white, potentially creating small gaps in the signature strokes that could reduce dHash similarity.
|
||||
This effect would bias classification toward false negatives rather than false positives, but the magnitude has not been quantified.
|
||||
*A1 pair-detectability stipulation.* The per-signature detector requires at least one same-CPA pair to be near-identical when a CPA uses image replication. A1 is plausible for high-volume stamping or firm-level electronic signing but not guaranteed when a corpus contains only one observed replicated report for a CPA, multiple template variants used in parallel, or scan-stage noise that pushes a replicated pair outside the detection regime.
|
||||
|
||||
Fourth, scanning equipment, PDF generation software, and compression algorithms may have changed over the 10-year study period (2013--2023), potentially affecting similarity measurements.
|
||||
While cosine similarity and dHash are designed to be robust to such variations, longitudinal confounds cannot be entirely excluded.
|
||||
*K=3 hard-posterior membership is composition-sensitive.* The K=3 hard-posterior membership for any single firm varies by up to $12.8$ pp across LOOO folds. This is documented as a composition-sensitivity band rather than failure, but it means K=3 hard labels are not used as v4.0 operational classifier output; they are reported only as accountant-level descriptive characterisation.
|
||||
|
||||
Fifth, the max/min detection logic treats both ends of a near-identical same-CPA pair as non-hand-signed.
|
||||
In the rare case that one of the two documents contains a genuinely hand-signed exemplar that was subsequently reused as the stamping or e-signature template, the pair correctly identifies image reuse but misattributes the non-hand-signed status to the source exemplar.
|
||||
This misattribution affects at most one source document per template variant per CPA (the exemplar from which the template was produced), is not expected to be common given that stored signature templates are typically generated in a separate acquisition step rather than extracted from submitted audit reports, and does not materially affect aggregate capture rates at the firm level.
|
||||
*No partner-level mechanism attribution.* v4.0 reports population-level patterns; it does not perform partner-level mechanism attribution or report-level claims of intent. The signature-level outputs are signature-level quantities throughout. The within-firm cross-CPA collision concentration of §III-L.4 is consistent with template-like reuse but is not by itself diagnostic of deliberate sharing.
|
||||
|
||||
Sixth, our analyses remain at the signature level; we abstain from partner-level frequency inferences such as "X% of CPAs hand-sign in a given year."
|
||||
Per-signature labels in this paper are not translated to per-report or per-partner mechanism assignments (Section III-G).
|
||||
The signature-level rates we report, including the 92.5% / 7.5% Firm A split and the year-by-year left-tail share of Section IV-G.1, should accordingly be read as signature-level quantities rather than partner-level frequencies.
|
||||
*Transferred ImageNet features (inherited from v3.20.0).* The ResNet-50 feature extractor uses pre-trained ImageNet weights without signature-domain fine-tuning. While our backbone-ablation study (§IV-L, inherited from v3.20.0 §IV-I) and prior literature support the effectiveness of transferred ImageNet features for signature comparison, a signature-domain fine-tuned feature extractor could improve discriminative performance.
|
||||
|
||||
Finally, the legal and regulatory implications of our findings depend on jurisdictional definitions of "signature" and "signing."
|
||||
Whether non-hand-signing of a CPA's own stored signature constitutes a violation of signing requirements is a legal question that our technical analysis can inform but cannot resolve.
|
||||
*Red-stamp HSV preprocessing artifacts (inherited from v3.20.0).* The red stamp removal preprocessing uses simple HSV color-space filtering, which may introduce artifacts where handwritten strokes overlap with red seal impressions. Blended pixels are replaced with white, potentially creating small gaps in signature strokes that could reduce dHash similarity. This bias would push classifications toward false negatives rather than false positives.
|
||||
|
||||
*Longitudinal scan / PDF / compression confounds (inherited from v3.20.0).* Scanning equipment, PDF generation software, and compression algorithms may have changed over the 2013–2023 study period, potentially affecting similarity measurements. While cosine similarity and dHash are designed to be robust to such variations, longitudinal confounds cannot be entirely excluded.
|
||||
|
||||
*Source-exemplar misattribution in max/min pair logic (inherited from v3.20.0).* The max-cosine / min-dHash detection logic treats both ends of a near-identical same-CPA pair as non-hand-signed. In the rare case where one of the two documents contains a genuinely hand-signed exemplar that was subsequently reused as a stamping or e-signature template, the pair correctly identifies image reuse but misattributes non-hand-signed status to the source exemplar. This affects at most one source document per template variant per CPA and is not expected to be common.
|
||||
|
||||
*Legal and regulatory interpretation (inherited from v3.20.0).* Whether non-hand-signing of a CPA's own stored signature constitutes a violation of signing requirements is a jurisdiction-specific legal question. Our technical analysis can inform such determinations but cannot resolve them.
|
||||
|
||||
@@ -2,85 +2,44 @@
|
||||
|
||||
<!-- Target: ~1.5 pages double-column IEEE format. Double-blind: no author/institution info. -->
|
||||
|
||||
Financial audit reports serve as a critical mechanism for ensuring corporate accountability and investor protection.
|
||||
In Taiwan, the Certified Public Accountant Act (會計師法 §4) and the Financial Supervisory Commission's attestation regulations (查核簽證核准準則 §6) require that certifying CPAs affix their signature or seal (簽名或蓋章) to each audit report [1].
|
||||
While the law permits either a handwritten signature or a seal, the CPA's attestation on each report is intended to represent a deliberate, individual act of professional endorsement for that specific audit engagement [2].
|
||||
Financial audit reports serve as a critical mechanism for ensuring corporate accountability and investor protection. In Taiwan, the Certified Public Accountant Act (會計師法 §4) and the Financial Supervisory Commission's attestation regulations (查核簽證核准準則 §6) require certifying CPAs to affix their signature or seal (簽名或蓋章) to each audit report [1]. While the law permits either a handwritten signature or a seal, the CPA's attestation on each report is intended to represent a deliberate, individual act of professional endorsement for that specific audit engagement [2].
|
||||
|
||||
The digitization of financial reporting has introduced a practice that complicates this intent.
|
||||
As audit reports are now routinely generated, transmitted, and archived as PDF documents, it is technically and operationally straightforward to reproduce a CPA's stored signature image across many reports rather than re-executing the signing act for each one.
|
||||
This reproduction can occur either through an administrative stamping workflow---in which scanned signature images are affixed by staff as part of the report-assembly process---or through a firm-level electronic signing system that automates the same step.
|
||||
From the perspective of the output image the two workflows are equivalent: both can reproduce one or more stored signature images, producing same-CPA signatures that are identical or near-identical up to reproduction, scanning, compression, and template-variant noise.
|
||||
We refer to signatures produced by either workflow collectively as *non-hand-signed*.
|
||||
Although this practice may fall within the literal statutory requirement of "signature or seal," it raises substantive concerns about audit quality, as an identically reproduced signature applied across hundreds of reports may not represent meaningful individual attestation for each engagement.
|
||||
The accounting literature has long examined the audit-quality consequences of partner-level engagement transparency: studies of partner-signature mandates in the United Kingdom find measurable downstream effects [31], cross-jurisdictional evidence on individual partner signature requirements highlights similar quality channels [32], and Taiwan-specific evidence on mandatory partner rotation documents how individual-partner identification interacts with audit-quality outcomes [33].
|
||||
Unlike traditional signature forgery, where a third party attempts to imitate another person's handwriting, non-hand-signing involves the legitimate signer's own stored signature being reused.
|
||||
This practice, while potentially widespread, is visually invisible to report users and virtually undetectable through manual inspection at scale: regulatory agencies overseeing thousands of publicly listed companies cannot feasibly examine each signature for evidence of image reproduction.
|
||||
The digitization of financial reporting has introduced a practice that complicates this intent. As audit reports are now routinely generated, transmitted, and archived as PDF documents, it is technically and operationally straightforward to reproduce a CPA's stored signature image across many reports rather than re-executing the signing act for each one. This reproduction can occur either through an administrative stamping workflow — in which scanned signature images are affixed by staff as part of the report-assembly process — or through a firm-level electronic signing system that automates the same step. We refer to signatures produced by either workflow collectively as *non-hand-signed*. Although this practice may fall within the literal statutory requirement of "signature or seal," it raises substantive concerns about audit quality, as an identically reproduced signature applied across hundreds of reports may not represent meaningful individual attestation for each engagement. The accounting literature has examined the audit-quality consequences of partner-level engagement transparency: studies of partner-signature mandates in the United Kingdom find measurable downstream effects [31], cross-jurisdictional evidence on individual partner signature requirements highlights similar quality channels [32], and Taiwan-specific evidence on mandatory partner rotation documents how individual-partner identification interacts with audit-quality outcomes [33]. Unlike traditional signature forgery, where a third party attempts to imitate another person's handwriting, non-hand-signing involves the legitimate signer's own stored signature being reused, and is visually invisible to report users at scale.
|
||||
|
||||
The distinction between *non-hand-signing detection* and *signature forgery detection* is both conceptually and technically important.
|
||||
The extensive body of research on offline signature verification [3]--[8] has focused almost exclusively on forgery detection---determining whether a questioned signature was produced by its purported author or by an impostor.
|
||||
This framing presupposes that the central threat is identity fraud.
|
||||
In our context, identity is not in question; the CPA is indeed the legitimate signer.
|
||||
The question is whether the physical act of signing occurred for each individual report, or whether a single signing event was reproduced as an image across many reports.
|
||||
This detection problem differs fundamentally from forgery detection: while it does not require modeling skilled-forger variability, it introduces the distinct challenge of separating legitimate intra-signer consistency from image-level reproduction, requiring an analytical framework focused on detecting abnormally high similarity across documents.
|
||||
The distinction between *non-hand-signing detection* and *signature forgery detection* is conceptually and technically important. The extensive body of research on offline signature verification [3]–[8] focuses almost exclusively on forgery detection — determining whether a questioned signature was produced by its purported author. In our context, identity is not in question; the CPA is indeed the legitimate signer. The question is whether the physical act of signing occurred for each individual report, or whether a single signing event was reproduced as an image across many reports. This detection problem differs fundamentally from forgery detection: while it does not require modeling skilled-forger variability, it introduces the distinct challenge of separating legitimate intra-signer consistency from image-level reproduction.
|
||||
|
||||
A secondary methodological concern shapes the research design.
|
||||
Many prior similarity-based classification studies rely on ad-hoc thresholds---declaring two images equivalent above a hand-picked cosine cutoff, for example---without principled statistical justification.
|
||||
Such thresholds are fragile in an archival-data setting where the cost of misclassification propagates into downstream inference.
|
||||
A defensible approach requires (i) a transparent threshold anchored to an empirical reference population drawn from the target corpus; (ii) statistical diagnostics that characterise the *shape* of the underlying similarity distribution and so motivate the choice of anchor; and (iii) external validation against naturally-occurring anchor populations---byte-level identical pairs as a conservative gold positive subset and large random inter-CPA pairs as a gold negative population---reported with Wilson 95% confidence intervals on per-rule capture / FAR rates, since precision and $F_1$ are not meaningful when the positive and negative anchor populations are sampled from different units.
|
||||
A methodological concern shapes the research design. Many prior similarity-based classification studies rely on ad-hoc thresholds — declaring two images equivalent above a hand-picked cosine cutoff, for example — without principled statistical justification. Such thresholds are fragile in an archival-data setting. A defensible approach requires (i) explicit calibration of the operational thresholds against measurable negative-anchor evidence; (ii) diagnostic procedures that test whether the descriptor distribution itself supports a within-population threshold, including formal decomposition of apparent multimodality into between-group composition and integer-tie artefacts; (iii) annotation-free reporting of operational alarm rates at multiple analysis units (per-comparison, per-signature pool, per-document) with Wilson 95% confidence intervals; (iv) per-firm stratification of the reported rates to surface heterogeneity that aggregate metrics conceal; and (v) explicit disclosure of the unsupervised setting's limits — in particular, the inability to estimate true error rates without signature-level ground-truth labels.
|
||||
|
||||
Despite the significance of the problem for audit quality and regulatory oversight, no prior work has specifically addressed non-hand-signing detection in financial audit documents at scale with these methodological safeguards.
|
||||
Woodruff et al. [9] developed an automated pipeline for signature analysis in corporate filings for anti-money-laundering investigations, but their work focused on author clustering (grouping signatures by signer identity) rather than detecting reuse of a stored image.
|
||||
Copy-move forgery detection methods [10], [11] address duplicated regions within or across images but are designed for natural images and do not account for the specific characteristics of scanned document signatures, where legitimate visual similarity between a signer's authentic signatures is expected and must be distinguished from image reproduction.
|
||||
Research on near-duplicate image detection using perceptual hashing combined with deep learning [12], [13] provides relevant methodological foundations but has not been applied to document forensics or signature analysis.
|
||||
From the statistical side, the methods we adopt for distributional characterisation---the Hartigan dip test [37] and finite mixture modelling via the EM algorithm [40], [41], complemented by a Burgstahler-Dichev / McCrary density-smoothness diagnostic [38], [39]---have been developed in statistics and accounting-econometrics but have not, to our knowledge, been combined as a joint diagnostic toolkit for document-forensics threshold selection.
|
||||
Despite the significance of the problem for audit quality and regulatory oversight, no prior work has specifically addressed non-hand-signing detection in financial audit documents at scale with these methodological safeguards. Woodruff et al. [9] developed an automated pipeline for signature analysis in corporate filings for anti-money-laundering investigations, but their work focused on author clustering rather than detecting image reuse. Copy-move forgery detection methods [10], [11] address duplicated regions within or across images but are designed for natural images and do not account for the specific characteristics of scanned document signatures. Research on near-duplicate image detection using perceptual hashing combined with deep learning [12], [13] provides relevant methodological foundations but has not been applied to document forensics or signature analysis. From the statistical side, the methods we adopt for distributional characterisation — the Hartigan dip test [37] and finite mixture modelling via the EM algorithm [40], [41], complemented by a Burgstahler-Dichev / McCrary density-smoothness diagnostic [38], [39] — have been developed in statistics and accounting-econometrics but have not been combined as a joint diagnostic toolkit for document-forensics threshold characterisation.
|
||||
|
||||
In this paper, we present a fully automated, end-to-end pipeline for detecting non-hand-signed CPA signatures in audit reports at scale.
|
||||
Our approach processes raw PDF documents through the following stages:
|
||||
(1) signature page identification using a Vision-Language Model (VLM);
|
||||
(2) signature region detection using a trained YOLOv11 object detector;
|
||||
(3) deep feature extraction via a pre-trained ResNet-50 convolutional neural network;
|
||||
(4) dual-descriptor similarity computation combining cosine similarity on deep embeddings with difference hash (dHash) distance;
|
||||
(5) signature-level distributional characterisation using two threshold estimators---KDE antimode with a Hartigan unimodality test and finite Beta mixture via EM with a logit-Gaussian robustness check---complemented by a Burgstahler-Dichev / McCrary density-smoothness diagnostic, used to read the structure of the per-signature similarity distribution and to motivate a percentile-based operational anchor rather than a mixture-fit crossing; and
|
||||
(6) validation against a pixel-identical anchor, a low-similarity anchor, and a replication-dominated Big-4 calibration firm.
|
||||
In this paper we present a fully automated, end-to-end pipeline for detecting non-hand-signed CPA signatures in audit reports at scale, together with a multi-tool validation framework that explicitly discloses the unsupervised setting's limits. The pipeline processes raw PDF documents through (1) signature page identification with a Vision-Language Model; (2) signature region detection with a trained YOLOv11 object detector; (3) deep feature extraction via a pre-trained ResNet-50; (4) dual-descriptor similarity (cosine + independent-minimum dHash); (5) anchor-based threshold calibration at three units of analysis (per-comparison, pool-normalised per-signature, per-document) against an inter-CPA negative-anchor coincidence-rate proxy (§III-L); (6) firm-stratified per-rule reporting and a within-firm cross-CPA hit-matrix analysis (§III-L.4); (7) a composition decomposition that establishes the absence of a within-population bimodal antimode in the descriptor distributions (§III-I.4); and (8) a multi-tool unsupervised validation strategy with disclosed assumption-violation analysis (§III-M).
|
||||
|
||||
The dual-descriptor verification is central to our contribution.
|
||||
Cosine similarity of deep feature embeddings captures high-level visual style similarity---it can identify signatures that share similar stroke patterns and spatial layouts---but cannot distinguish between a CPA who signs consistently and one whose signature is reproduced from a stored image.
|
||||
Perceptual hashing (specifically, difference hashing) encodes structural-level image gradients into compact binary fingerprints that are robust to scan noise but sensitive to substantive content differences.
|
||||
By requiring convergent evidence from both descriptors, we can differentiate *style consistency* (high cosine but divergent dHash) from *image reproduction* (high cosine with low dHash), resolving an ambiguity that neither descriptor can address alone.
|
||||
The methodological reframing relative to earlier versions of this work is central to our v4.0 contribution. Earlier work in this lineage adopted a distributional path to thresholds — fitting accountant-level finite-mixture models and treating their marginal crossings as data-derived "natural" thresholds. v4.0 reports a composition decomposition diagnostic (§III-I.4) that overturns this reading: the apparent multimodality of the Big-4 accountant-level distribution is fully explained by between-firm location-shift effects (Firm A's mean dHash of $2.73$ versus Firms B/C/D's $6.46$, $7.39$, $7.21$) and integer mass-point artefacts on the integer-valued dHash axis. Once both confounds are removed (firm-mean centring plus uniform integer jitter), the Big-4 pooled dHash dip test yields $p_{\text{median}} = 0.35$ across five jitter seeds, eliminating the rejection. Within-firm signature-level cosine dip tests fail to reject in every individual Big-4 firm and in every individual mid/small firm with $\geq 500$ signatures (10 firms tested in Script 39c), and the corresponding within-firm jittered-dHash dip tests likewise fail to reject in all four Big-4 firms (Script 39d) and across a codex-verified read-only spike on the same ten mid/small firms ($0/10$ reject; §III-I.4). The descriptor distributions therefore contain no within-population bimodal antimode that could anchor an operational threshold.
|
||||
|
||||
A second distinctive feature is our framing of the calibration reference.
|
||||
One major Big-4 accounting firm in Taiwan (hereafter "Firm A") was selected as a candidate calibration reference based on practitioner-knowledge motivation; its benchmark status is then evaluated using the image evidence reported in this paper, not asserted by the practitioner-knowledge motivation itself.
|
||||
We therefore treat Firm A as a *replication-dominated* calibration reference rather than a pure positive class.
|
||||
This framing is important because the statistical signature of a replication-dominated population is visible in our data: Firm A's per-signature cosine distribution is unimodal with a long left tail (Hartigan dip $p = 0.17$), 92.5% of Firm A signatures exceed cosine 0.95 with the remaining 7.5% forming the left tail, and 145 Firm A signatures across 50 distinct partners are byte-identical to a same-CPA match in a different audit report (35 spanning different fiscal years).
|
||||
Adopting the replication-dominated framing---rather than a near-universal framing that would have to absorb the 7.5% residual as noise---ensures internal coherence between the byte-level pixel-identity evidence and the signature-level distributional shape.
|
||||
In place of distributional anchoring, v4.0 adopts an anchor-based inter-CPA coincidence-rate (ICCR) calibration. At the per-comparison unit, the inherited cos$>0.95$ operating point yields ICCR $= 0.00060$ on a $5 \times 10^5$-pair Big-4 sample (replicating v3.x's reported per-comparison rate of $0.0005$ under prior "FAR" terminology); the dHash$\leq 5$ structural cutoff yields ICCR $= 0.00129$ (v4 new); the joint rule cos$>0.95$ AND dHash$\leq 5$ yields joint ICCR $= 0.00014$ (any-pair semantics, matching the deployed extrema rule). At the pool-normalised per-signature unit, the same rule's effective coincidence rate is materially higher because the deployed classifier takes max-cosine and min-dHash over a same-CPA pool: pooled Big-4 any-pair ICCR is $0.1102$ (Wilson 95% CI $[0.1086, 0.1118]$; CPA-block bootstrap 95% $[0.0908, 0.1330]$). At the per-document unit, the operational HC$+$MC alarm fires on $33.75\%$ of Big-4 documents under the inter-CPA candidate-pool counterfactual.
|
||||
|
||||
A third distinctive feature is the empirical reading we take from the per-signature distributional analysis.
|
||||
Three diagnostics applied to the per-signature similarity distribution---the Hartigan dip test, an EM-fitted Beta mixture (with logit-Gaussian robustness check), and the Burgstahler-Dichev / McCrary density-smoothness procedure---jointly indicate that no two-mechanism mixture cleanly explains per-signature similarity: the dip test fails to reject unimodality for Firm A, BIC strongly prefers a 3-component over a 2-component Beta fit, and the BD/McCrary candidate transition lies *inside* the non-hand-signed mode rather than between modes (and is not bin-width-stable; Appendix A).
|
||||
The substantive reading is that *pixel-level output quality* is a continuous spectrum shaped by firm-specific reproduction technologies (administrative stamping in early years, firm-level e-signing later) and scan conditions, rather than a discrete class cleanly separated from hand-signing.
|
||||
This reading motivates anchoring the operational classifier on a percentile heuristic over the Firm A reference distribution rather than on a mixture-fit crossing, and it motivates the byte-level pixel-identity anchor (Section IV-F.1) as a threshold-free positive reference that does not depend on resolving signature-level mixture structure.
|
||||
The pooled per-signature and per-document rates conceal striking firm heterogeneity. A logistic regression of the per-signature hit indicator on firm dummies (Firm A reference) and centred log pool size yields odds ratios of $0.053$ (Firm B), $0.010$ (Firm C), and $0.027$ (Firm D) — Firms B/C/D are an order of magnitude below Firm A even after controlling for the pool-size confound (Script 44). Cross-firm hit matrix analysis under the deployed any-pair rule shows within-firm collision concentrations of $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D (Table XXV; the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms). The pattern is consistent with firm-specific template, stamp, or document-production reuse mechanisms — though not by itself diagnostic of deliberate sharing. We retain the inherited Paper A v3.x five-way box rule as the operational classifier; v4.0's contribution is to characterise its multi-level coincidence behaviour against the inter-CPA negative anchor rather than to derive new thresholds.
|
||||
|
||||
We apply this pipeline to 90,282 audit reports filed by publicly listed companies in Taiwan between 2013 and 2023, extracting and analyzing 182,328 individual CPA signatures from 758 unique accountants.
|
||||
To our knowledge, this represents the largest-scale forensic analysis of signature authenticity in financial documents reported in the literature.
|
||||
Three feature-derived scores converge on the per-CPA descriptor-position ranking with Spearman $\rho \geq 0.879$ (Script 38): the K=3 mixture posterior (now interpreted as a firm-compositional position score, not a mechanism cluster posterior; §III-J), a reverse-anchor cosine percentile relative to a strictly-out-of-target non-Big-4 reference, and the inherited box-rule less-replication-dominated rate. The three scores are deterministic functions of the same per-CPA descriptor pair, so the convergence is documented as internal consistency among feature-derived ranks rather than external validation. Hard ground truth for the *replicated* class is provided by 262 byte-identical signatures in the Big-4 subset (Firm A 145, Firm B 8, Firm C 107, Firm D 2), against which all three candidate checks achieve $0\%$ positive-anchor miss rate (Wilson 95% upper bound $1.45\%$). For the box rule this result is close to tautological at byte-identity; we discuss the conservative-subset caveat in §V-G.
|
||||
|
||||
The contributions of this paper are summarized as follows:
|
||||
We apply this pipeline to 90,282 audit reports filed by publicly listed companies in Taiwan between 2013 and 2023, extracting and analyzing 182,328 individual CPA signatures from 758 unique accountants. The Big-4 sub-corpus comprises 437 CPAs and 150,442 signatures with both descriptors available.
|
||||
|
||||
1. **Problem formulation.** We formally define non-hand-signing detection as distinct from signature forgery detection and argue that it requires an analytical framework focused on intra-signer similarity distributions rather than genuine-versus-forged classification.
|
||||
The contributions of this paper are:
|
||||
|
||||
2. **End-to-end pipeline.** We present a pipeline that processes raw PDF audit reports through VLM-based page identification, YOLO-based signature detection, deep feature extraction, and dual-descriptor similarity computation, with automated inference requiring no manual intervention after initial training and annotation.
|
||||
1. **Problem formulation.** We define non-hand-signing detection as distinct from signature forgery detection and frame it as a detection problem on intra-signer similarity distributions.
|
||||
|
||||
3. **Dual-descriptor verification.** We demonstrate that combining deep-feature cosine similarity with perceptual hashing resolves the fundamental ambiguity between style consistency and image reproduction, and we validate the backbone choice through an ablation study comparing three feature-extraction architectures.
|
||||
2. **End-to-end pipeline.** We present a pipeline that processes raw PDF audit reports through VLM-based page identification, YOLO-based signature detection, ResNet-50 feature extraction, and dual-descriptor similarity computation, with automated inference and no manual intervention after initial training.
|
||||
|
||||
4. **Percentile-anchored operational threshold.** We anchor the operational classifier's cosine cut on the whole-sample Firm A P7.5 percentile (cos $> 0.95$), a transparent and reproducible reference drawn from a replication-dominated reference population, and complement it with dHash structural cuts derived from the same reference distribution. Operational thresholds are therefore explained by an empirical reference rather than asserted.
|
||||
3. **Dual-descriptor verification.** We demonstrate that combining deep-feature cosine similarity with independent-minimum dHash resolves the ambiguity between *style consistency* and *image reproduction*, and we validate the backbone choice through a feature-backbone ablation.
|
||||
|
||||
5. **Distributional characterisation of per-signature similarity.** We apply three statistical diagnostics---a Hartigan dip test, an EM-fitted Beta mixture with logit-Gaussian robustness check, and a Burgstahler-Dichev / McCrary density-smoothness procedure---to characterise the shape of the per-signature similarity distribution. The three diagnostics jointly find that per-signature similarity forms a continuous quality spectrum, which both motivates the percentile-based operational anchor over a mixture-fit crossing and is itself a substantive finding for the document-forensics literature on similarity-threshold selection.
|
||||
4. **Composition decomposition disproves the distributional-threshold path.** We show via a 2×2 factorial diagnostic (firm-mean centring × integer-tie jitter) that the apparent multimodality of the Big-4 accountant-level descriptor distribution is fully attributable to between-firm location shifts and integer mass-point artefacts. The descriptor distributions contain no within-population bimodal antimode; "natural threshold" language in this lineage's prior work is not empirically supported.
|
||||
|
||||
6. **Replication-dominated calibration methodology.** We introduce a calibration strategy using a replication-dominated reference group, distinguishing *replication-dominated* from *replication-pure* anchors; and we validate classification using byte-level pixel identity as an annotation-free gold positive, requiring no manual labeling.
|
||||
5. **Anchor-based multi-level inter-CPA coincidence-rate calibration.** We characterise the deployed five-way classifier at three units of analysis: per-comparison ICCR (cos$>0.95$: $0.0006$; dHash$\leq 5$: $0.0013$; joint: $0.00014$), pool-normalised per-signature ICCR ($0.11$ for the deployed any-pair high-confidence rule), and per-document ICCR ($0.34$ for the operational HC$+$MC alarm). We adopt "inter-CPA coincidence rate" as the metric name throughout and reserve "False Acceptance Rate" for terminology that requires ground-truth negative labels, which the corpus does not provide.
|
||||
|
||||
7. **Large-scale empirical analysis.** We report findings from the analysis of over 90,000 audit reports spanning a decade, providing the first large-scale empirical evidence on non-hand-signing practices in financial reporting under a methodology designed for peer-review defensibility.
|
||||
6. **Firm heterogeneity quantification and within-firm cross-CPA collision concentration.** Per-firm rates differ by an order of magnitude after pool-size adjustment (Firm A's per-document HC$+$MC alarm at $0.62$ versus Firms B/C/D at $0.09$–$0.16$). Cross-firm hit matrix analysis shows within-firm collision concentrations of $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D under the deployed any-pair rule (the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms); the pattern is consistent with firm-specific template, stamp, or document-production reuse mechanisms — a descriptive finding about deployed-rule behaviour, not a claim of deliberate template sharing.
|
||||
|
||||
The remainder of this paper is organized as follows.
|
||||
Section II reviews related work on signature verification, document forensics, perceptual hashing, and the statistical methods we adopt for distributional characterisation.
|
||||
Section III describes the proposed methodology.
|
||||
Section IV presents experimental results including the signature-level distributional characterisation, pixel-identity validation, and backbone ablation study.
|
||||
Section V discusses the implications and limitations of our findings.
|
||||
Section VI concludes with directions for future work.
|
||||
7. **K=3 as descriptive firm-compositional partition; three-score convergent internal consistency.** We fit a K=3 Gaussian mixture as a descriptive partition of the Big-4 accountant-level distribution (no longer interpreted as three mechanism clusters). Three feature-derived scores agree on the per-CPA descriptor-position ranking at Spearman $\rho \geq 0.879$; we report this as internal consistency rather than external validation, given that the scores share the underlying descriptor pair.
|
||||
|
||||
8. **Annotation-free positive-anchor validation and unsupervised validation ceiling.** We achieve $0\%$ positive-anchor miss rate (Wilson 95% upper bound $1.45\%$) on 262 byte-identical Big-4 signatures, with the conservative-subset caveat that byte-identical pairs are by construction near cos$=1$ and dHash$=0$. We frame the overall validation strategy as a multi-tool collection of ten partial-evidence diagnostics (§III-M Table XXVII), each with an explicitly disclosed untested assumption; their conjunction constitutes the unsupervised validation ceiling achievable on this corpus. We do not claim a validated forensic detector; we position the system as a specificity-proxy-anchored screening framework with human-in-the-loop review.
|
||||
|
||||
The remainder of the paper is organized as follows. Section II reviews related work on signature verification, document forensics, perceptual hashing, and the statistical methods used. Section III describes the proposed methodology. Section IV presents the experimental results — distributional characterisation, mixture fits, convergent internal-consistency checks, leave-one-firm-out reproducibility, pixel-identity validation, and full-dataset robustness. Section V discusses the implications and limitations. Section VI concludes with directions for future work.
|
||||
|
||||
+284
-129
@@ -85,7 +85,7 @@ Preprocessing consisted of resizing to 224×224 pixels with aspect-ratio preserv
|
||||
All feature vectors were L2-normalized, ensuring that cosine similarity equals the dot product.
|
||||
|
||||
The choice of ResNet-50 without fine-tuning was motivated by three considerations: (1) the task is similarity comparison rather than classification, making general-purpose discriminative features sufficient; (2) ImageNet features have been shown to transfer effectively to document analysis tasks [20], [21]; and (3) avoiding domain-specific fine-tuning reduces the risk of overfitting to dataset-specific artifacts, though we note that a fine-tuned model could potentially improve discriminative performance (see Section V-G).
|
||||
This design choice is validated by an ablation study (Section IV-I) comparing ResNet-50 against VGG-16 and EfficientNet-B0.
|
||||
This design choice is validated by an ablation study (Section IV-L) comparing ResNet-50 against VGG-16 and EfficientNet-B0.
|
||||
|
||||
## F. Dual-Method Similarity Descriptors
|
||||
|
||||
@@ -121,191 +121,346 @@ Pixel-level comparison---whether $L_1$, $L_2$, or pixel-identity counting---fail
|
||||
Pixel-level distances are defined on geometrically aligned images at a common resolution, and they treat any sub-pixel translation, rotation, or rescale as a large perturbation by construction (a one-pixel uniform translation flips a fraction of foreground pixels on a thin-stroke signature crop and inflates pixel L1 distance to the same magnitude as for a different signer's signature).
|
||||
Two scans of the same physical document, however, do not share a common pixel grid: scanner DPI, paper-handling alignment, and PDF-page rasterisation each contribute random sub-pixel offsets, and the print-scan cycle that intervenes between the stored stamp image and the audit-report PDF additionally introduces resolution mismatch and small geometric drift.
|
||||
A pixel-level descriptor cannot therefore satisfy the basic stability requirement for our task: two presentations of the same stored image must score nearly identically.
|
||||
We retain pixel-identity counting only as a *threshold-free anchor* (Section III-J), because byte-identical pairs in our corpus are necessarily produced by literal file reuse rather than by repeated scanning, and so they do not interact with the alignment-fragility argument; they are not used as a primary similarity descriptor.
|
||||
We retain pixel-identity counting only as a *threshold-free anchor* (Section III-K), because byte-identical pairs in our corpus are necessarily produced by literal file reuse rather than by repeated scanning, and so they do not interact with the alignment-fragility argument; they are not used as a primary similarity descriptor.
|
||||
|
||||
Cosine similarity on deep embeddings and dHash, in contrast, both remain stable across the print-scan-rasterise cycle by design: cosine on L2-normalised pooled features is invariant to overall scale and bias and degrades gracefully under local-pixel noise that the convolutional backbone has been trained to absorb [14], [21], while dHash compresses the image to a $9 \times 8$ grayscale grid before computing horizontal-gradient signs, which removes the resolution and sub-pixel-alignment sensitivity that breaks pixel-level comparison [19], [27].
|
||||
Together they constitute the dual descriptor used throughout the rest of this paper.
|
||||
|
||||
## G. Unit of Analysis and Summary Statistics
|
||||
## G. Unit of Analysis and Scope
|
||||
|
||||
Two unit-of-analysis choices are relevant for this study, ordered from finest to coarsest: (i) the *signature*---one signature image extracted from one report; and (ii) the *auditor-year*---all signatures by one CPA within one fiscal year.
|
||||
The signature is the operational unit of classification (Section III-K) and of all primary statistical analyses (Section IV-D, IV-F, IV-G).
|
||||
The auditor-year is used in the partner-level similarity ranking of Section IV-G.2 as a within-year aggregation unit: each auditor-year's mean is computed over its own fiscal-year signatures, although the per-signature best-match cosine that feeds the mean is computed against the full same-CPA cross-year pool (Section III-G's max-cosine / min-dHash definition).
|
||||
We do not use a coarser CPA-level cross-year unit, because pooling a CPA's signatures across the full 2013--2023 sample period would conflate distinct signing-mechanism regimes whenever a CPA's practice changes during the sample, and we make no claim about the within-CPA stability of signing mechanisms over time.
|
||||
We analyse signatures at two units of resolution. The **signature** — one signature image extracted from one report — is the operational unit of classification (§III-L) and of the signature-level analyses in §IV (notably §IV-J for the five-way per-signature category counts and the inherited inter-CPA negative-anchor coincidence-rate analysis referenced in §IV-I; reported under prior "FAR" terminology in v3.x). The **accountant** — one CPA aggregated over all of their signatures in the corpus — is the unit of mixture-model characterisation (§III-J), of per-CPA internal-consistency analysis (§III-K), and of the leave-one-firm-out reproducibility check (§III-K). At the accountant level we compute, for each CPA with $n_{\text{sig}} \geq 10$ signatures, the per-CPA mean of the per-signature best-match cosine ($\overline{\text{cos}}_a$) and the per-CPA mean of the independent-minimum dHash ($\overline{\text{dHash}}_a$). The minimum threshold of 10 signatures per CPA is required for the per-CPA mean to be a stable summary; CPAs below this threshold are excluded from the accountant-level analyses but remain in the per-signature analyses.
|
||||
|
||||
For per-signature classification we compute, for each signature, the maximum pairwise cosine similarity and the minimum dHash Hamming distance against every other signature attributed to the same CPA (over the full same-CPA set, not restricted to the same fiscal year).
|
||||
The max/min (rather than mean) formulation reflects the identification logic for non-hand-signing: if even one other signature of the same CPA is a pixel-level reproduction, that pair will dominate the extremes and reveal the non-hand-signed mechanism.
|
||||
Mean statistics would dilute this signal.
|
||||
We make no within-year or across-year uniformity assumption about CPA signing mechanisms. Per-signature labels are signature-level quantities throughout this paper; we do not translate them to per-report or per-partner mechanism assignments, and we abstain from partner-level frequency inferences (such as "X% of CPAs hand-sign") that would require such a translation. A CPA's per-CPA mean is a *summary statistic* of their observed signatures, not a claim that all of their signatures share a single mechanism.
|
||||
|
||||
For the dHash dimension we use the *independent minimum dHash*: the minimum Hamming distance from a signature to *any* other signature of the same CPA (over the full same-CPA set).
|
||||
The independent minimum is unconditional on the cosine-nearest pair and is therefore the conservative structural-similarity statistic; it is the dHash statistic used throughout the operational classifier (Section III-K) and all reported capture-rate analyses.
|
||||
We adopt one stipulation about same-CPA pair detectability:
|
||||
|
||||
We make one stipulation about same-CPA pair detectability.
|
||||
> **(A1) Pair-detectability.** *If a CPA uses image replication anywhere in the corpus, then at least one same-CPA signature pair is near-identical (after reproduction noise) within the cross-year same-CPA pool used by the max-cosine / min-dHash computation.*
|
||||
|
||||
**(A1) Pair-detectability.** *If a CPA uses image replication anywhere in the corpus, then at least one same-CPA signature pair is near-identical (after reproduction noise) within the cross-year same-CPA pool used by the max-cosine / min-dHash computation above.*
|
||||
This is plausible for high-volume stamping or firm-level electronic-signing workflows---where a stored image is typically reused many times under similar scan and compression conditions---but it is *not* guaranteed when (i) the corpus contains only one observed replicated report for a CPA, (ii) multiple template variants are in use simultaneously, or (iii) scan-stage noise pushes a replicated pair outside the detection regime.
|
||||
A1 is a *cross-year pair-existence* property, not a within-year uniformity claim, and is the only assumption the per-signature detector requires to be sensitive to replication.
|
||||
A1 is plausible for high-volume stamping or firm-level electronic signing workflows but is not guaranteed when (i) the corpus contains only one observed replicated report for a CPA, (ii) multiple template variants are used in parallel, or (iii) scan-stage noise pushes a replicated pair outside the detection regime. A1 is the only assumption the per-signature detector requires to be sensitive to replication.
|
||||
|
||||
We make *no* within-year or across-year uniformity assumption about CPA signing mechanisms.
|
||||
Per-signature labels are signature-level quantities throughout this paper; we do not translate them to per-report or per-partner mechanism assignments, and we abstain from partner-level frequency inferences (such as "X% of CPAs hand-sign") that would require such a translation.
|
||||
A CPA's signing output within a single fiscal year may reflect a single replication template, multiple templates used in parallel (e.g., different stored images for different engagement positions or reporting pipelines), within-year mechanism mixing, or a combination; our signature-level analyses remain valid under all of these regimes, since they do not attempt mechanism attribution at the partner or report level.
|
||||
**Scope: the Big-4 sub-corpus.** v4.0's primary analyses (§III-I, §III-J, §III-K, §III-L, and the v4-new analyses in §IV-D through §IV-J) are restricted to the four largest accounting firms in Taiwan, pseudonymously labelled Firm A through Firm D throughout the manuscript. §IV-A through §IV-C, §IV-I (inter-CPA negative-anchor coincidence rate), and §IV-L (feature-backbone ablation) report inherited corpus-wide v3.x material that v4.0 does not re-scope to Big-4. §IV-K reports a deliberately narrow full-dataset cross-check at $n = 686$ CPAs. The Big-4 sub-corpus comprises 437 CPAs (171 / 112 / 102 / 52 across Firms A through D) with $n_{\text{sig}} \geq 10$ — the threshold for accountant-level analyses (Scripts 36, 38) — totalling 150,442 Big-4 signatures with both pre-computed descriptors available. Restricting the v4-new analyses to Big-4 is a methodological choice driven by four considerations:
|
||||
|
||||
The intra-report consistency analysis in Section IV-G.3 is a firm-level homogeneity check---whether the *two co-signing CPAs on the same report* receive the same signature-level label under the operational classifier---rather than a test of within-partner or within-year uniformity.
|
||||
1. **Leave-one-firm-out fold feasibility.** §III-K reports leave-one-firm-out (LOOO) cross-validation of the Big-4 K=3 fit. The Big-4 sub-corpus permits a four-fold LOOO at the firm level (one fold per Big-4 firm). No analogous firm-level fold is available outside Big-4 because mid/small firms have CPA counts of $O(1)$–$O(30)$ per firm.
|
||||
|
||||
## H. Calibration Reference: Firm A as a Replication-Dominated Population
|
||||
2. **Firm A as templated-end case study.** Firm A is empirically the firm whose CPAs are most concentrated in the high-cosine, low-dHash corner of the descriptor plane (§III-J K=3 component cross-tab; v3.x byte-level pair analysis referenced in §III-H). v4.0 retains Firm A within the Big-4 scope as a descriptive case study of the templated end, rather than treating Firm A as the calibration anchor for thresholds (the v3.x role of Firm A).
|
||||
|
||||
A distinctive aspect of our methodology is the use of Firm A---a major Big-4 accounting firm in Taiwan---as an empirical calibration reference.
|
||||
Rather than treating Firm A as a synthetic or laboratory positive control, we treat it as a naturally occurring *replication-dominated population*: a CPA population whose aggregate signing behavior is dominated by non-hand-signing but is not a pure positive class.
|
||||
3. **Within-firm cross-CPA collision structure analysis.** §III-L.4 reports a Big-4 cross-firm hit-matrix analysis (Script 44) that quantifies the within-firm cross-CPA template-like collision pattern. The four-firm setting affords the cleanest signal for this analysis; replicating the same matrix structure on the heterogeneous mid/small-firm tail is left as future work.
|
||||
|
||||
Practitioner knowledge motivated treating Firm A as a candidate calibration reference: the firm is understood within the audit profession to reproduce a stored signature image for the majority of certifying partners---originally via administrative stamping workflows and later via firm-level electronic signing systems---while not ruling out that a minority of partners may continue to hand-sign some or all of their reports.
|
||||
This practitioner background motivates Firm A's selection but is not used as evidence: the evidentiary basis in the analyses below---byte-identical same-CPA pairs, the Firm A per-signature similarity distribution, partner-ranking concentration, and intra-report consistency---is derived entirely from the audit-report images themselves and does not depend on any claim about firm-level signing practice.
|
||||
4. **Restricted generalisability claim.** v4.0's primary claims are scoped to the Big-4 audit-report context; we do not assert that the same descriptive mixture structure or operational alert behaviour extends to mid/small firms. The 249 non-Big-4 CPAs enter only (a) as an external reference population in §III-H's reverse-anchor internal-consistency check, (b) as a robustness comparison in §IV-K, and (c) as a corroborating-population check on the dHash discrete-mass-point artefact in §III-I.4 (Script 39c). Generalisation beyond Big-4 is left as future work.
|
||||
|
||||
We establish Firm A's replication-dominated status through two primary independent quantitative analyses plus a third strand comprising three complementary checks, each of which can be reproduced from the public audit-report corpus alone:
|
||||
We earlier (v4.0 first draft) listed "statistical multimodality at the accountant level" among the scope justifications, on the basis that the Hartigan dip test rejects unimodality on the Big-4 accountant-level marginals. §III-I.4 reports diagnostics (Scripts 39b–39e) that explain the rejection as a joint effect of between-firm composition shift and dHash integer mass points, not as evidence of within-population continuous bimodality. We therefore no longer list dip-test multimodality among the Big-4 scope rationales; the K=3 mixture is retained as a descriptive partition (§III-J), not as inferential evidence for two mechanism modes.
|
||||
|
||||
First, *automated byte-level pair analysis* (Section IV-F.1; reproduction artifact listed in Appendix B) identifies 145 Firm A signatures that are byte-identical to at least one other same-CPA signature from a different audit report, distributed across 50 distinct Firm A partners (of 180 registered); 35 of these byte-identical matches span different fiscal years.
|
||||
Byte-identity implies pixel-identity by construction, and independent hand-signing cannot produce pixel-identical images across distinct reports---these pairs therefore establish image reuse as a concrete, threshold-free phenomenon within Firm A and confirm that replication is widespread (50 of 180 registered partners) rather than confined to a handful of CPAs.
|
||||
**Sample-size reconciliation.** Two Big-4 signature counts appear in this section and §IV: $n = 150{,}442$ for analyses using the pre-computed per-signature descriptors $\text{cos}_s$ (`max_similarity_to_same_accountant`) and $\text{dHash}_s$ (`min_dhash_independent`), and $n = 150{,}453$ for analyses recomputing pair-level metrics directly from the stored feature and dHash byte vectors (Scripts 40b, 43, 44). The $11$-signature difference reflects descriptor-completion status: $11$ signatures have feature vectors and dHash byte vectors stored but lack the pre-computed extrema. The $11$ signatures are negligible at population scale and do not affect any reported coincidence rate within $0.01$ percentage point. The CPA counts $468$ (all Big-4 CPAs with both vectors stored) and $437$ (Big-4 CPAs with $n_{\text{sig}} \geq 10$ for accountant-level stability) likewise reflect a single uniform exclusion rule rather than analysis-specific subsetting.
|
||||
|
||||
Second, *signature-level distributional evidence*: Firm A's per-signature best-match cosine distribution fails to reject unimodality (Hartigan dip test $p = 0.17$, $N = 60{,}448$ Firm A signatures; Section IV-D) and exhibits a long left tail, consistent with a dominant high-similarity regime plus residual within-firm heterogeneity rather than two cleanly separated mechanisms.
|
||||
92.5% of Firm A's per-signature best-match cosine similarities exceed 0.95 and the remaining 7.5% form the long left tail (we do not disaggregate partner-level mechanism here; see Section III-G for the scope of claims).
|
||||
The unimodal-long-tail shape, not the precise 92.5/7.5 split, is the structural evidence: it predicts that Firm A is replication-dominated rather than a clean two-class population, and a noise-only explanation of the left tail would predict a shrinking share as scan/PDF technology matured over 2013--2023, which is not what we observe (Section IV-G.1).
|
||||
## H. Reference Populations
|
||||
|
||||
Third, we additionally validate the Firm A benchmark through three complementary analyses reported in Section IV-G. Only the partner-level ranking is fully threshold-free; the longitudinal-stability and intra-report analyses use the operational classifier and are interpreted as consistency checks on its firm-level output:
|
||||
(a) *Longitudinal stability (Section IV-G.1).* The share of Firm A per-signature best-match cosine values below 0.95 is stable at 6-13% across 2013-2023, with the lowest share in 2023. The 0.95 cutoff is the whole-sample Firm A P7.5 heuristic (Section III-K; 92.5% of whole-sample Firm A signatures exceed this cutoff); the substantive finding here is the *temporal stability* of the rate, not the absolute rate at any single year.
|
||||
(b) *Partner-level similarity ranking (Section IV-G.2).* When every auditor-year is ranked globally by its per-auditor-year mean best-match cosine (across all firms: Big-4 and Non-Big-4), Firm A auditor-years account for 95.9% of the top decile against a baseline share of 27.8% (a 3.5$\times$ concentration ratio), and this over-representation is stable across 2013-2023. This analysis uses only the ordinal ranking and is independent of any absolute cutoff.
|
||||
(c) *Intra-report consistency (Section IV-G.3).* Because each Taiwanese statutory audit report is co-signed by two engagement partners, firm-wide stamping practice predicts that both signers on a given Firm A report should receive the same signature-level label under the classifier. Firm A exhibits 89.9% intra-report agreement against 62-67% at the other Big-4 firms. This test uses the operational classifier and is therefore a *consistency* check on the classifier's firm-level output rather than a threshold-free test; the cross-firm gap (not the absolute rate) is the substantive finding.
|
||||
v4.0 distinguishes two reference populations in its calibration, replacing v3.x's single-anchor framing.
|
||||
|
||||
The 92.5% figure is a within-sample consistency check rather than an independent validation of Firm A's status; the validation role is played by the byte-level pixel-identity evidence, the unimodal-long-tail dip-test result, the three complementary analyses above, and the held-out Firm A fold (described in Section III-J; fold-level rate differences are disclosed in Section IV-F.2).
|
||||
Firm A's replication-dominated status itself was *not* derived from the thresholds we calibrate against it; it rests on the byte-level pair evidence and the dip-test-confirmed unimodal-long-tail shape, both of which are independent of any threshold choice.
|
||||
The "replication-dominated, not pure" framing is important both for internal consistency---it predicts and explains the long left tail observed in Firm A's cosine distribution (Section IV-D)---and for avoiding overclaim in downstream inference.
|
||||
**Internal reference: Firm A as the templated-end case study.** Firm A is empirically the firm whose CPAs are most concentrated in the high-cosine, low-dHash corner of the Big-4 descriptor plane. In the Big-4 K=3 descriptive partition (§III-J; Scripts 35, 38), Firm A accounts for 0% of the C1 component (low-cos / high-dHash corner; cos $\approx 0.946$, dHash $\approx 9.17$, weight $\approx 0.143$), 17.5% of the C2 component (central region), and 82.5% of the C3 component (high-cos / low-dHash corner); the opposite pattern holds at Firm C (Script 35: 23.5% C1, 75.5% C2, 1.0% C3, hereafter referred to as "the Firm whose CPAs are most concentrated in C1"). The byte-level pair analysis reported in v3.x §IV-F.1 identifies 145 Firm A pixel-identical signatures at the signature level (Script 40 verifies the 145/262 split among Big-4 pixel-identical signatures); the additional details that v3.x attributes to this analysis (50 distinct Firm A partners of 180 registered; 35 byte-identical matches spanning different fiscal years) are inherited from the Script 28 / Appendix B byte-decomposition output and were not regenerated in the v4.0 spike scripts. We retain those v3.x details by reference and mark them in the provenance table as "inherited from v3 §IV-F.1 / Script 28."
|
||||
|
||||
## I. Signature-Level Threshold Characterisation
|
||||
In v4.0, Firm A is *not* the calibration anchor for the operational threshold. Firm A enters the Big-4 mixture on equal footing with Firms B through D; the K=3 components are derived from the joint Big-4 distribution (§III-J), not from Firm A alone. Firm A's role in the methodology is descriptive: it is the Big-4 firm whose CPAs are most concentrated in the high-cosine, low-dHash corner of the descriptor plane, and the byte-level pair evidence above provides the firm-level signature-reuse evidence that anchors §III-K's pixel-identity positive-anchor miss rate.
|
||||
|
||||
This section describes how we set the operational classifier's similarity threshold and how we characterise the per-signature similarity distribution that supports it.
|
||||
The two roles are kept separate by design.
|
||||
**External reference: non-Big-4 as the reverse-anchor reference for internal-consistency checking.** The 249 non-Big-4 CPAs ($n_{\text{sig}} \geq 10$, drawn from $\sim$30 mid- and small-firms) constitute a population strictly outside the Big-4 target. Their per-CPA $(\overline{\text{cos}}_a, \overline{\text{dHash}}_a)$ distribution defines a 2D Gaussian reference (fit by Minimum Covariance Determinant with support fraction 0.85 for robustness; Script 38). This reference is used in §III-K's reverse-anchor internal-consistency check: each Big-4 CPA's location relative to the reference centre, measured as the marginal cosine cumulative-distribution-function value under the reference, is one of three feature-derived scores that v4.0 uses as a cross-check on the inherited per-signature classifier. The reverse-anchor reference is *not* a positive or negative anchor for threshold derivation — its role is to provide a strictly out-of-target benchmark against which the within-Big-4 mixture-derived ranking can be internally cross-checked.
|
||||
|
||||
**Operational threshold (used by the classifier).** The cosine cut is anchored on the whole-sample Firm A P7.5 percentile (cos $> 0.95$; Section III-K).
|
||||
The reverse-anchor reference centre is at $\overline{\text{cos}} = 0.935$, $\overline{\text{dHash}} = 9.77$ (Script 38). The reference sits at a lower cosine and higher dHash than the Big-4 K=3 low-cos / high-dHash component (cos $= 0.946$, dHash $= 9.17$; §III-J); compared to the Big-4 high-cos / low-dHash component (cos $= 0.983$, dHash $= 2.41$; §III-J) the reference is markedly less replication-dominated. The reverse-anchor metric for a given Big-4 CPA is the percentile of $\overline{\text{cos}}_a$ within the reference marginal cosine distribution, sign-flipped so that lower percentile (further into the left tail of the reference) corresponds to a Big-4 CPA whose mean cosine sits further from the templated end of the descriptor plane. This is a "deviation in the less-replication-dominated descriptor-position direction" measure, not a "deviation toward the templated descriptor-position" measure; the reference is the less-replication-dominated population.
|
||||
|
||||
**Statistical characterisation (used to motivate the choice of anchor and to describe the distributional structure).** A Hartigan dip test, an EM-fitted Beta mixture (with logit-Gaussian robustness check), and a Burgstahler-Dichev / McCrary density-smoothness procedure---all applied at the per-signature level (Section IV-D).
|
||||
## I. Distributional Diagnostics: Why the Composition Path Does Not Yield a Natural Threshold
|
||||
|
||||
The reason for the split is empirical.
|
||||
The three statistical diagnostics jointly find that per-signature similarity forms a continuous quality spectrum (Section IV-D, summarised below): the dip test fails to reject unimodality for Firm A; BIC strongly prefers a 3-component over a 2-component Beta fit, so the 2-component crossing is a forced fit; and the BD/McCrary candidate transition lies inside the non-hand-signed mode rather than between modes (and is not bin-width-stable; Appendix A).
|
||||
Under these conditions the natural anchor for an operational cosine cut is a transparent percentile of a replication-dominated reference population (Firm A) rather than a mixture-fit crossing whose location depends on parametric assumptions the data do not support.
|
||||
This section characterises the joint distribution of accountant-level descriptor means $(\overline{\text{cos}}_a, \overline{\text{dHash}}_a)$ across the 437 Big-4 CPAs of §III-G and tests whether the distribution provides distributional support — in the form of within-population bimodality — for the operational thresholds inherited from v3.x. We apply four diagnostic procedures in turn: a univariate unimodality test on each accountant-level marginal; a 2D Gaussian mixture fit (developed in §III-J); a density-smoothness diagnostic; and a composition decomposition that distinguishes within-population multimodality from between-firm location-shift artefacts (the v4-new diagnostic battery). The four diagnostics jointly imply that the operational thresholds are *not* anchored by distributional bimodality: §III-L develops an anchor-based calibration framework that does not require this assumption.
|
||||
|
||||
We describe the three diagnostics and the assumptions underlying each in the subsections below.
|
||||
The two threshold estimators rest on decreasing-in-strength assumptions: the KDE antimode/crossover requires only smoothness; the Beta mixture additionally requires a parametric specification, and the logit-Gaussian cross-check reports sensitivity to that form.
|
||||
The Burgstahler-Dichev / McCrary procedure is applied to the same distribution as a *density-smoothness diagnostic*: it would identify a sharp local density discontinuity if one existed at the boundary between two cleanly separated mechanisms.
|
||||
Because all three diagnostics are applied to the same sample rather than to independent experiments, agreement or disagreement among them is read as evidence about distributional structure rather than as a formal statistical guarantee.
|
||||
**1. Hartigan dip test on each accountant-level marginal.** We apply the Hartigan & Hartigan dip test [37] to each of the two marginal distributions $\{\overline{\text{cos}}_a\}_{a=1}^{437}$ and $\{\overline{\text{dHash}}_a\}_{a=1}^{437}$, with bootstrap-based $p$-value estimation ($n_{\text{boot}} = 2000$). In both cases no bootstrap replicate exceeded the observed dip statistic, so the empirical $p$-value is bounded above by $5 \times 10^{-4}$; we report this in tables as $p < 5 \times 10^{-4}$ rather than $p = 0$ to reflect the bootstrap resolution (Script 34). For comparison, no rejection of unimodality holds in the comparison scopes tested in Script 32: Firm A pooled alone ($p_{\text{cos}} = 0.992$, $p_{\text{dHash}} = 0.924$, $n = 171$); Firms B + C + D pooled ($p_{\text{cos}} = 0.998$, $p_{\text{dHash}} = 0.906$, $n = 266$); all non-Firm-A CPAs pooled ($p_{\text{cos}} = 0.998$, $p_{\text{dHash}} = 0.907$, $n = 515$). Single-firm dip tests for Firms B, C, and D were not separately computed; the comparison scopes above sufficed to establish that no narrower-than-Big-4 *tested* scope at the accountant level rejected unimodality. The accountant-level Big-4 rejection is a descriptive observation; §III-I.4 below shows that the rejection is fully explained by between-firm location-shift effects rather than within-population bimodality.
|
||||
|
||||
### 1) Method 1: KDE Antimode / Crossover with Unimodality Test
|
||||
**2. K=2 / K=3 Gaussian mixture fits (descriptive partition).** A 2-component 2D Gaussian Mixture Model (full covariance, $n_{\text{init}} = 15$, fixed seed 42; Script 34) recovers components at $(\overline{\text{cos}}, \overline{\text{dHash}}) = (0.954, 7.14)$, weight $0.689$, and $(0.983, 2.41)$, weight $0.311$. The marginal crossings of the K=2 fit are $\overline{\text{cos}}^* = 0.9755$ and $\overline{\text{dHash}}^* = 3.755$, with bootstrap 95% confidence intervals $[0.9742, 0.9772]$ and $[3.48, 3.97]$ over $n_{\text{boot}} = 500$ resamples. The 3-component fit (§III-J) is BIC-preferred — using the convention that lower BIC is preferred, $\text{BIC}(K{=}3) - \text{BIC}(K{=}2) = -3.48$ (Script 36). The $\Delta$BIC magnitude is small in absolute terms; we do not treat $\Delta\text{BIC} = 3.5$ alone as decisive evidence for K=3 as a population mixture. Following §III-I.4 we treat both K=2 and K=3 fits as *descriptive partitions* of the joint Big-4 distribution that reflect firm-composition structure (Firm A vs others; §III-J) rather than as inferential evidence for two or three latent population modes.
|
||||
|
||||
We use two closely related KDE-based threshold estimators and apply each where it is appropriate.
|
||||
When two labeled populations are available (e.g., the all-pairs intra-class and inter-class similarity distributions of Section IV-C), the *KDE crossover* is the intersection point of the two kernel density estimates under Scott's rule for bandwidth selection [28]; under equal priors and symmetric misclassification costs it approximates the Bayes-optimal decision boundary between the two classes.
|
||||
When a single distribution is analysed (e.g., the per-signature best-match cosine distribution of Section IV-D) the *KDE antimode* is the local density minimum between two modes of the fitted density; it serves the same decision-theoretic role when the distribution is multimodal but is undefined when the distribution is unimodal.
|
||||
In either case we use the Hartigan & Hartigan dip test [37] as a formal test of unimodality.
|
||||
The dip test asks one question: *is the distribution single-peaked?*
|
||||
A non-significant $p$-value means we cannot reject the single-peak null (the data are consistent with one peak); a significant $p$-value means the distribution has *more than one peak* (it could be two, three, or more---the test does not specify how many).
|
||||
We use the test to decide whether a KDE antimode is well-defined (it is, only when there is more than one peak), not to assert any particular number of components.
|
||||
We additionally perform a sensitivity analysis varying the bandwidth over $\pm 50\%$ of the Scott's-rule value to verify threshold stability.
|
||||
**3. Burgstahler-Dichev / McCrary density-smoothness diagnostic.** We apply the discontinuity test of [38, 39] as a *density-smoothness diagnostic* (rather than as a threshold estimator) on each accountant-level marginal axis (cosine in bins of $0.002$, dHash in integer bins). At the Big-4 scope, the diagnostic identifies no significant transition on either marginal at $\alpha = 0.05$ (Script 34). Outside Big-4, the diagnostic does flag dHash transitions in some subsets (Script 32: `big4_non_A` dHash transition at $10.8$; `all_non_A` dHash transition at $6.6$; pre-2018 and post-2020 time-stratified variants also exhibit one or more dHash transitions), but no cosine transition is identified in any subset. The Big-4-scope null on both axes is consistent with §III-I.4 below: under the composition decomposition the Big-4 marginals are unimodal once between-firm and integer-tie confounds are removed, so a local-discontinuity test correctly fails to flag a within-population transition.
|
||||
|
||||
### 2) Method 2: Finite Mixture Model via EM
|
||||
**4. Composition decomposition (Scripts 39b–39e).** §III-I.1 establishes that the accountant-level marginals reject unimodality at the Big-4 sub-corpus. The remaining question is whether the rejection reflects (a) genuine within-population bimodality at the signature or accountant level, (b) between-firm location-shift artefacts (firms with different mean descriptor positions pool to a multi-peaked distribution), or (c) integer mass-point artefacts on the integer-valued dHash axis (the dHash dip statistic is sensitive to spikes at integer values). We apply four diagnostics that decompose the rejection into these candidate sources:
|
||||
|
||||
We fit a two-component Beta mixture to the cosine distribution via the EM algorithm [40] using method-of-moments M-step estimates (which are numerically stable for bounded proportion data).
|
||||
The first component represents non-hand-signed signatures (high mean, narrow spread) and the second represents hand-signed signatures (lower mean, wider spread).
|
||||
Under the fitted model the threshold is the crossing point of the two weighted component densities,
|
||||
*Within-firm signature-level dip (Scripts 39b, 39c).* Repeating the dip test at the signature level inside each individual Big-4 firm (Script 39b) and inside each individual non-Big-4 firm with $\geq 500$ signatures (Script 39c) yields a consistent picture. The cosine marginal *fails* to reject unimodality in every single firm tested — all four Big-4 firms ($p_{\text{cos}} \in \{0.176, 0.991, 0.551, 0.976\}$ for Firms A through D; Script 39b) and ten non-Big-4 firms with $\geq 500$ signatures ($p_{\text{cos}} \in [0.59, 0.99]$; Script 39c). The raw dHash marginal *does* reject unimodality in every firm tested ($p < 5 \times 10^{-4}$ in all $14$ firms), but the raw dHash values are integer-valued in $\{0, 1, \ldots, 64\}$, leaving open the possibility of an integer-tie artefact.
|
||||
|
||||
$$\pi_1 \cdot \text{Beta}(x; \alpha_1, \beta_1) = (1 - \pi_1) \cdot \text{Beta}(x; \alpha_2, \beta_2),$$
|
||||
*Integer-jitter robustness (Scripts 39d, 39e).* Adding independent uniform jitter $\sim \mathrm{U}[-0.5, +0.5]$ to break exact dHash ties and re-running the dip test on the perturbed signature cloud (5 seeds, $n_{\text{boot}} = 2000$; Script 39d) eliminates the dHash within-firm rejection in every Big-4 firm tested (Firm A jittered $p_{\text{median}} = 0.999$; B $0.996$; C $0.999$; D $0.9995$; $0$/$5$ seeds reject at $\alpha = 0.05$ in any firm). A codex-verified read-only spike applying the same jitter procedure to the ten non-Big-4 firms with $\geq 500$ signatures (Script 39c substrate) likewise yields no rejection ($0$/$10$ firms reject at $\alpha = 0.05$; per-firm median-$p$ range $[0.38, 1.00]$). The pooled-Big-4 dHash dip *does* survive jitter alone ($p_{\text{median}} = 0$, $5$/$5$ seeds reject), but Firm A's mean dHash ($2.73$) is substantially below Firms B/C/D's ($6.46$, $7.39$, $7.21$) — a between-firm location shift. Script 39e applies a 2 \times 2 factorial correction (firm-mean centring $\times$ integer jitter) on the Big-4 pooled dHash:
|
||||
|
||||
solved numerically via bracketed root-finding.
|
||||
As a robustness check against the Beta parametric form we fit a parallel two-component Gaussian mixture to the *logit-transformed* similarity, following standard practice for bounded proportion data.
|
||||
White's [41] quasi-MLE consistency result justifies interpreting the logit-Gaussian estimates as asymptotic approximations to the best Gaussian-family fit under misspecification; we use the cross-check between Beta and logit-Gaussian crossings as a diagnostic of parametric-form sensitivity rather than as a guarantee of distributional recovery.
|
||||
| Condition | Firm-mean centred | Integer jitter | Median dip $p$ | Reject at $\alpha = 0.05$ |
|
||||
|---|---|---|---|---|
|
||||
| 1 raw | — | — | $< 5 \times 10^{-4}$ | $5/5$ |
|
||||
| 2 centred only | $\checkmark$ | — | $< 5 \times 10^{-4}$ | $5/5$ |
|
||||
| 3 jittered only | — | $\checkmark$ | $< 5 \times 10^{-4}$ | $5/5$ |
|
||||
| 4 centred and jittered | $\checkmark$ | $\checkmark$ | $\mathbf{0.35}$ | $\mathbf{0/5}$ |
|
||||
|
||||
We fit 2- and 3-component variants of each mixture and report BIC for model selection.
|
||||
When BIC prefers the 3-component fit, the 2-component assumption itself is a forced fit; we report the resulting crossing only as a forced-fit descriptive reference and do not use it as an operational threshold.
|
||||
Removing *both* the between-firm location shift *and* the integer mass points eliminates the Big-4 dHash rejection. The Big-4 pooled dHash multimodality is therefore fully attributable to firm-composition contrast (primarily Firm A's mean $\text{dHash} = 2.73$ versus Firms B/C/D $\approx 6.5$–$7.4$) and integer-density artefacts, with no residual continuous within-firm bimodality.
|
||||
|
||||
### 3) Density-Smoothness Diagnostic: Burgstahler-Dichev / McCrary
|
||||
*Cosine analogue.* The cosine axis exhibits the same pattern. Codex-verified read-only spike on the Big-4 pooled signature cloud yields signature-level cosine dip $p < 5 \times 10^{-4}$ on the raw data, but $p = 0.597$ after firm-mean centring; accountant-level cosine $p = 1.0$ after firm-mean centring. The cosine multimodality is therefore between-firm composition-driven, not within-population bimodality.
|
||||
|
||||
Complementing the two threshold estimators above, we apply the discontinuity test of Burgstahler and Dichev [38], made asymptotically rigorous by McCrary [39], as a *density-smoothness diagnostic* rather than as a third threshold estimator.
|
||||
We discretize each distribution (cosine into bins of width 0.005; $\text{dHash}_\text{indep}$ into integer bins) and compute, for each bin $i$ with count $n_i$, the standardized deviation from the smooth-null expectation of the average of its neighbours,
|
||||
*Integer-histogram valleys (Script 39d).* A genuine within-firm dHash antimode would appear as a strict local minimum in the count histogram with deep relative depth. Within each of the four Big-4 firms, the dHash histogram on bins $0$–$20$ exhibits no strict local minimum; the Big-4 pooled histogram exhibits one shallow valley at $\text{dHash} = 4$ with relative depth $0.021$ (a $2.1\%$ count drop). No valley near the inherited $\text{dHash} = 5$ operational boundary appears within any individual firm. The hypothesised dHash antimode near $\text{dHash} \approx 5$ is not empirically supported by the histogram analysis.
|
||||
|
||||
$$Z_i = \frac{n_i - \tfrac{1}{2}(n_{i-1} + n_{i+1})}{\sqrt{N p_i (1-p_i) + \tfrac{1}{4} N (p_{i-1}+p_{i+1})(1 - p_{i-1} - p_{i+1})}},$$
|
||||
**5. Conclusion: no natural threshold from the descriptor distribution.** §III-I.4 jointly establishes that (a) the Big-4 accountant-level dip rejection is fully attributable to between-firm composition and integer mass-point artefacts; (b) within any individual firm, the descriptor marginals at the signature level are unimodal once integer ties are broken; and (c) no integer-histogram valley near the inherited $\text{dHash} = 5$ operational boundary exists within any firm. The descriptor distributions therefore do not contain a within-population bimodal antimode that could anchor an operational threshold. The K=2 / K=3 mixture fits of §III-I.2 and §III-J are retained as *descriptive partitions* that reflect firm-composition contrast, not as inferential evidence for two or three population modes. §III-L develops the v4.0 anchor-based threshold calibration framework, which derives operational rates from inter-CPA pair-level negative-anchor coincidences rather than from a distributional antimode.
|
||||
|
||||
which is approximately $N(0,1)$ under the null of distributional smoothness.
|
||||
A candidate transition is identified at an adjacent bin pair where $Z_{i-1}$ is significantly negative and $Z_i$ is significantly positive (cosine) or the reverse (dHash).
|
||||
Appendix A reports a bin-width sensitivity sweep covering $\text{bin} \in \{0.003, 0.005, 0.010, 0.015\}$ for cosine and $\text{bin} \in \{1, 2, 3\}$ for dHash; the sweep shows that signature-level BD transitions are not bin-width-stable, consistent with histogram-resolution artifacts rather than a genuine cross-mode density discontinuity.
|
||||
We therefore do not treat the BD/McCrary procedure as a threshold estimator in our application but as diagnostic evidence about distributional smoothness.
|
||||
## J. K=3 as a Descriptive Partition of Firm-Composition Contrast
|
||||
|
||||
### 4) Reading the Three Diagnostics Together
|
||||
This section develops the K=2 and K=3 Gaussian mixture fits to the Big-4 accountant-level distribution and clarifies their role. **Both fits are descriptive partitions of the joint Big-4 distribution; they reflect firm-composition contrast — primarily Firm A versus Firms B, C, D — rather than within-population mechanism modes.** §III-I.4 demonstrates that the apparent multimodality of the accountant-level marginals is fully explained by between-firm location shifts and integer mass-point artefacts, leaving no residual evidence for two or three latent within-population mechanism classes. Neither mixture is used to assign signature-level or document-level labels in the v4.0 primary analysis. The operational classifier of §III-L is calibrated via inter-CPA negative-anchor coincidence rates, not via mixture-derived antimodes.
|
||||
|
||||
The two threshold estimators rest on decreasing-in-strength assumptions: the KDE antimode/crossover requires only smoothness; the Beta mixture additionally requires a parametric specification (with logit-Gaussian as a robustness cross-check against that form).
|
||||
If the two estimated thresholds were to differ by less than a practically meaningful margin and the BD/McCrary procedure were to identify a sharp transition at the same level, that pattern would constitute convergent evidence for a clean two-mechanism boundary at that location.
|
||||
**K=2 fit.** Two components at $(\overline{\text{cos}}, \overline{\text{dHash}}) = (0.954, 7.14)$ (weight $0.689$) and $(0.983, 2.41)$ (weight $0.311$) (Script 34). $\text{BIC}(K{=}2) = -1108.45$. Marginal crossings: $\overline{\text{cos}}^* = 0.9755$, $\overline{\text{dHash}}^* = 3.755$. We refer to the components by index rather than by mechanism labels, since §III-I.4 establishes that the K=2 separation is firm-compositional rather than mechanistic.
|
||||
|
||||
This is *not* the pattern we observe at the per-signature level.
|
||||
The two threshold estimators yield crossings spread across a wide range (Section IV-D); the BIC clearly prefers a 3-component over a 2-component Beta fit, indicating that the 2-component crossing is a forced fit reported only as a descriptive reference rather than as an operational threshold; and the BD/McCrary procedure locates its candidate transition *inside* the non-hand-signed mode rather than between modes (Appendix A).
|
||||
We interpret this jointly as evidence that per-signature similarity is a continuous quality spectrum rather than a clean two-mechanism mixture, and we accordingly anchor the operational classifier's cosine cut on whole-sample Firm A percentile heuristics (Section III-K) rather than on a mixture-fit crossing.
|
||||
**K=3 fit.** Three components, sorted by ascending cosine mean (Script 35; Script 38 reproduces):
|
||||
|
||||
## J. Pixel-Identity, Inter-CPA, and Held-Out Firm A Validation (No Manual Annotation)
|
||||
| Component | $\overline{\text{cos}}$ | $\overline{\text{dHash}}$ | weight | descriptive position |
|
||||
|---|---|---|---|---|
|
||||
| C1 | 0.9457 | 9.17 | 0.143 | low-cos / high-dHash corner |
|
||||
| C2 | 0.9558 | 6.66 | 0.536 | central region |
|
||||
| C3 | 0.9826 | 2.41 | 0.321 | high-cos / low-dHash corner |
|
||||
|
||||
Rather than construct a stratified manual-annotation validation set, we validate the classifier using four naturally occurring reference populations that require no human labeling:
|
||||
$\text{BIC}(K{=}3) = -1111.93$, lower than $K{=}2$ by $3.48$ (mild numerical preference for K=3 under standard BIC interpretation, but not by itself decisive). The "descriptive position" column replaces v3.x's "hand-leaning / mixed / replicated" mechanism labels: §III-I.4 establishes that the cosine and dHash axes both lack within-population bimodality, so component centres are best interpreted as locations in a continuous descriptor space rather than as latent mechanism modes.
|
||||
|
||||
1. **Pixel-identical anchor (gold positive, conservative subset):** signatures whose nearest same-CPA match is byte-identical after crop and normalization.
|
||||
Handwriting physics makes byte-identity impossible under independent signing events, so a byte-identical same-CPA pair is pair-level proof of image reuse and---for the byte-identical subset---conservative ground truth for non-hand-signed signatures; the narrow exception, in which a genuinely hand-signed exemplar was subsequently reused as the stamping or e-signature template, is discussed as a Limitation in Section V-G.
|
||||
We further emphasize that this anchor is a *subset* of the true positive class---only those non-hand-signed signatures whose nearest match happens to be byte-identical---and perfect recall against this anchor therefore does not establish recall against the full non-hand-signed population (Section V-G discusses this further).
|
||||
**Per-firm component composition (Script 35 firm × cluster cross-tab).** The K=3 partition is dominated by firm membership:
|
||||
|
||||
2. **Inter-CPA negative anchor (large gold negative):** $\sim$50,000 pairs of signatures randomly sampled from *different* CPAs.
|
||||
Inter-CPA pairs cannot arise from reuse of a single signer's stored signature image, so this population is a reliable negative class for threshold sweeps.
|
||||
This anchor is substantially larger than a simple low-similarity-same-CPA negative and yields tight Wilson 95% confidence intervals on FAR at each candidate threshold.
|
||||
- Firm A: $0\%$ C1, $17.5\%$ C2, $82.5\%$ C3
|
||||
- Firm B: $8.9\%$ C1, $\sim 78\%$ C2, $\sim 13\%$ C3
|
||||
- Firm C: $23.5\%$ C1, $75.5\%$ C2, $1.0\%$ C3
|
||||
- Firm D: $11.5\%$ C1, $\sim 84\%$ C2, $\sim 4.5\%$ C3
|
||||
|
||||
3. **Firm A anchor (replication-dominated prior positive):** Firm A signatures, treated as a majority-positive reference with within-firm heterogeneity in the left tail, as evidenced by the 7.5% of Firm A signatures whose per-signature best-match cosine falls at or below 0.95 (Section III-H, Section IV-D).
|
||||
Because Firm A is both used for empirical percentile calibration in Section III-H and as a validation anchor, we make the within-Firm-A sampling variance visible by splitting Firm A CPAs randomly (at the CPA level, not the signature level) into a 70% *calibration* fold and a 30% *heldout* fold.
|
||||
The calibration-fold percentiles used in thresholding---cosine median, P1, and P5 (lower-tail, since higher cosine indicates greater similarity), and dHash_indep median and P95 (upper-tail, since lower dHash indicates greater similarity)---are derived from the 70% calibration fold only.
|
||||
The heldout fold is used exclusively to report post-hoc capture rates with Wilson 95% confidence intervals.
|
||||
Firm A accounts for $141$ of the $143$ C3-assigned CPAs; Firm C accounts for $24$ of the $40$ C1-assigned CPAs. The K=3 partition is therefore well-described as a firm-compositional decomposition: C3 is essentially "Firm A and any non-Firm-A CPA whose mean descriptors happen to land in the high-cos / low-dHash corner"; C1 is essentially "non-Firm-A CPAs whose mean descriptors land in the low-cos / high-dHash corner." The composition contrast that K=3 captures at the accountant level reappears at the deployment level in the cross-firm hit matrix of §III-L.4 (Script 44): under the deployed any-pair rule, within-firm collision concentration is $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D (the stricter same-pair joint event saturates at $97.0$–$99.96\%$ within-firm across all four firms). The K=3 partition and the cross-firm hit matrix therefore describe the same underlying firm-compositional structure at two different units of analysis.
|
||||
|
||||
4. **Low-similarity same-CPA anchor (supplementary negative):** signatures whose maximum same-CPA cosine similarity is below 0.70.
|
||||
This anchor is retained for continuity with prior work but is small in our dataset ($n = 35$) and is reported only as a supplementary reference; its confidence intervals are too wide for quantitative inference.
|
||||
**Leave-one-firm-out stability (Scripts 36, 37).** Leave-one-firm-out cross-validation shows that K=2 is unstable across folds: holding Firm A out gives a fold rule cos $> 0.938$ AND dHash $\leq 8.79$, while holding any single non-Firm-A Big-4 firm out gives a fold rule near cos $> 0.975$ AND dHash $\leq 3.76$ (Script 36). The maximum absolute deviation of the four fold cosine crossings from their across-fold mean is $0.028$ (the corresponding pairwise across-fold range is $0.0376$, from $0.9380$ for the held-out-Firm-A fold to $0.9756$ for the held-out-Firm-D fold; Script 36 stability summary). The $0.028$ value is $5.6\times$ the report's $0.005$ across-fold stability tolerance. K=3 in contrast has a *reproducible component shape*: across the four folds the C1 cosine mean varies by at most $0.005$, the C1 dHash mean by at most $0.96$, and the C1 weight by at most $0.023$ (Script 37). K=3 hard-posterior membership for the held-out firm is composition-sensitive — for Firm C the held-out C1 rate is $36.3\%$ vs the full-Big-4 baseline of $23.5\%$, an absolute difference of $12.8$ pp; for Firm A the held-out C1 rate is $4.7\%$ vs baseline $0.0\%$; the report's own legend classifies this pattern as `P2_PARTIAL` ("the C1 cluster exists but membership is not well-predicted by the held-out fit"). We accordingly do not use K=3 hard-posterior membership as an operational label.
|
||||
|
||||
From these anchors we report FAR with Wilson 95% confidence intervals against the inter-CPA negative anchor.
|
||||
We do not report an Equal Error Rate or FRR column against the byte-identical positive anchor, because byte-identical pairs have cosine $\approx 1$ by construction and any FRR computed against that subset is trivially $0$ at every threshold below $1$; the conservative-subset role of the byte-identical anchor is instead discussed qualitatively in Section V-F.
|
||||
Precision and $F_1$ are not meaningful in this anchor-based evaluation because the positive and negative anchors are constructed from different sampling units (intra-CPA byte-identical pairs vs random inter-CPA pairs), so their relative prevalence in the combined set is an arbitrary construction rather than a population parameter; we therefore omit precision and $F_1$ from Table X.
|
||||
The 70/30 held-out Firm A fold of Section IV-F.2 additionally reports capture rates with Wilson 95% confidence intervals computed within the held-out fold, which is a valid population for rate inference.
|
||||
We take the joint K=2 / K=3 LOOO evidence as supporting the following descriptive claims, all of which are used in §III-K and §V but none of which underwrites the v4.0 operational classifier:
|
||||
|
||||
## K. Per-Document Classification
|
||||
- The Big-4 K=2 marginal crossing $(0.975, 3.76)$ is essentially a firm-mass separator between Firm A and Firms B + C + D, not a within-Big-4 mechanism boundary.
|
||||
- The Big-4 K=3 mixture exhibits a reproducible three-component component shape across LOOO folds at the descriptor-position level, with C1 reproducibly located at $\overline{\text{cos}} \approx 0.946$, $\overline{\text{dHash}} \approx 9.17$.
|
||||
- Hard-posterior K=3 membership is composition-sensitive across folds (max absolute deviation $12.8$ pp); K=3 is therefore not used to assign operational labels to CPAs in v4.0.
|
||||
|
||||
The per-signature classifier operates at the signature level with operational thresholds anchored on whole-sample Firm A percentile heuristics: cos $> 0.95$ (Firm A P7.5) for the cosine dimension and dHash$_\text{indep} \leq 5$ / $> 15$ (Firm A median+P75 / style-consistency ceiling) for the structural dimension.
|
||||
This percentile-based anchor is the natural choice given the continuous-spectrum shape of the per-signature similarity distribution documented in Section IV-D; sensitivity to nearby alternatives is reported in Section IV-F.3.
|
||||
All dHash references in this section refer to the *independent-minimum* dHash defined in Section III-G---the smallest Hamming distance from a signature to any other same-CPA signature.
|
||||
We use a single dHash statistic throughout the operational classifier and the supporting capture-rate analyses (Tables IX, XI, XII, XVI), which keeps the classifier definition and its empirical evaluation arithmetically consistent.
|
||||
The operational signature-level classifier of §III-L is calibrated against inter-CPA pair-level negative-anchor coincidence rates, not against mixture-derived antimodes. Cross-checks between the inherited five-way box rule and the K=3 partition appear in §III-K.
|
||||
|
||||
We assign each signature to one of five signature-level categories using convergent evidence from both descriptors:
|
||||
## K. Convergent Internal-Consistency Checks
|
||||
|
||||
1. **High-confidence non-hand-signed:** Cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 5$.
|
||||
Both descriptors converge on strong replication evidence.
|
||||
The descriptive partition of §III-J is supported by three feature-derived per-CPA scores and a hard-ground-truth subset analysis. We caution at the outset that the three scores are **not statistically independent measurements** — all three are deterministic functions of the same per-CPA descriptor means $(\overline{\text{cos}}_a, \overline{\text{dHash}}_a)$ — so their high pairwise rank correlations are partly a mechanical consequence of shared inputs. Per §III-I.4, none of the three scores has a within-population bimodality interpretation; they are firm-compositional position scores at the accountant level. The checks below therefore document **internal consistency among feature-derived ranks**, not external validation against an independent hand-signed ground truth (which the corpus does not provide).
|
||||
|
||||
2. **Moderate-confidence non-hand-signed:** Cosine $> 0.95$ AND $5 < \text{dHash}_\text{indep} \leq 15$.
|
||||
Feature-level evidence is strong; structural similarity is present but below the high-confidence cutoff, potentially due to scan variations.
|
||||
**1. Three feature-derived per-CPA scores (Script 38).** For each Big-4 CPA we compute:
|
||||
|
||||
3. **High style consistency:** Cosine $> 0.95$ AND $\text{dHash}_\text{indep} > 15$.
|
||||
High feature-level similarity without structural corroboration---consistent with a CPA who signs very consistently but not via image reproduction.
|
||||
- **Score 1 (K=3 posterior on the low-cos / high-dHash component):** $P(\text{C1})$ from the K=3 fit of §III-J. Per §III-J this is a firm-compositional position score on the (cos, dHash) plane (not a probability of any latent "hand-signing mechanism") — a function of both descriptor means.
|
||||
- **Score 2 (reverse-anchor cosine percentile):** the marginal cosine CDF value of $\overline{\text{cos}}_a$ under the non-Big-4 reference Gaussian of §III-H, sign-flipped so that lower percentile (further into the reference's left tail) corresponds to a Big-4 CPA whose mean cosine sits further from the templated end. This is a function of $\overline{\text{cos}}_a$ alone.
|
||||
- **Score 3 (inherited binary high-confidence box rule rate):** the per-CPA fraction of signatures that do **not** satisfy the inherited binary high-confidence box rule (cos $> 0.95$ AND dHash $\leq 5$). This is a per-signature-aggregated function of the same descriptors.
|
||||
|
||||
4. **Uncertain:** Cosine between the all-pairs intra/inter KDE crossover (0.837) and 0.95 without sufficient convergent evidence for classification in either direction.
|
||||
Pairwise Spearman rank correlations among the three scores, $n = 437$ Big-4 CPAs (Script 38):
|
||||
|
||||
5. **Likely hand-signed:** Cosine below the all-pairs KDE crossover threshold.
|
||||
| Pair | Spearman $\rho$ | $p$-value |
|
||||
|---|---|---|
|
||||
| Score 1 vs Score 3 | $+0.9627$ | $< 10^{-248}$ |
|
||||
| Score 2 vs Score 3 | $+0.8890$ | $< 10^{-149}$ |
|
||||
| Score 1 vs Score 2 | $+0.8794$ | $< 10^{-142}$ |
|
||||
|
||||
We note three conventions about the thresholds.
|
||||
First, the cosine cutoff $0.95$ is the *operating point* chosen for the five-way classifier from a small grid of candidate cuts, on the basis of an explicit capture-vs-FAR tradeoff against the inter-CPA negative anchor of Section III-J---*not* a discovered natural boundary in the per-signature distribution.
|
||||
The candidate grid spans the calibration-fold P5 (0.9407), its rounded value (0.945), the operational anchor (0.95), and two reference points drawn from the signature-level threshold-estimator outputs of Section IV-D (the Firm A Beta-2 forced-fit crossing 0.977 and the BD/McCrary candidate transition 0.985); for each grid point Section IV-F.3 reports the Firm A capture rate, the non-Firm-A capture rate, and the inter-CPA FAR with Wilson 95% CI (Table XII-B).
|
||||
Three considerations motivate the operating point at 0.95.
|
||||
(i) *Inter-CPA specificity.* At cosine $> 0.95$ the inter-CPA FAR against the 50,000-pair anchor of Section IV-F.1 is $0.0005$ (Wilson 95% CI $[0.0003, 0.0007]$): one in two thousand random cross-CPA pairs exceeds the cut, an order-of-magnitude margin against the working assumption that random cross-CPA pairs do not arise from image reuse.
|
||||
(ii) *Capture stability under nearby alternatives.* Moving the cut to $0.945$ raises Firm A capture by 1.51 percentage points (operational dual rule cos $> t$ AND $\text{dHash}_\text{indep} \leq 15$; Section IV-F.3) and inter-CPA FAR by $0.00032$, while moving it to the calibration-fold P5 of $0.9407$ raises Firm A capture by 2.63 percentage points and inter-CPA FAR by $0.00076$; in either direction the qualitative finding---Firm A is replication-dominated, non-Firm-A capture is much lower at the same cut, and the inter-CPA noise floor is small---is preserved.
|
||||
(iii) *Interpretive transparency.* The complement $7.5\%$ corresponds to the whole-sample Firm A P7.5 of the per-signature best-match cosine distribution---that is, $92.5\%$ of whole-sample Firm A signatures exceed this cutoff and $7.5\%$ fall at or below it (Section III-H)---which gives the operational cut a transparent reading in the replication-dominated reference population without requiring a parametric mixture fit that the data of Section IV-D do not support.
|
||||
The cosine crossover $0.837$ is the all-pairs intra/inter KDE crossover; both $0.95$ and $0.837$ are derived from whole-sample distributions rather than from the 70% calibration fold, so the classifier inherits its operational cosine cuts from the whole-sample Firm A and all-pairs distributions.
|
||||
Section IV-F.2 reports both calibration-fold and held-out-fold capture rates for this classifier so that fold-level sampling variance is visible; Section IV-F.3 (Table XII-B) reports the full capture-vs-FAR tradeoff at the candidate grid above.
|
||||
Second, the dHash cutoffs $\leq 5$ and $> 15$ are chosen from the whole-sample Firm A $\text{dHash}_\text{indep}$ distribution: $\leq 5$ captures the upper tail of the high-similarity mode (whole-sample Firm A median $\text{dHash}_\text{indep} = 2$, P75 $\approx 4$, so $\leq 5$ is the band immediately above median), while $> 15$ marks the regime in which independent-minimum structural similarity is no longer indicative of image reproduction.
|
||||
Third, the signature-level threshold-estimator outputs of Section IV-D (KDE antimode, Beta-mixture and logit-Gaussian crossings, BD/McCrary diagnostic) are *not* the operational thresholds of this classifier: they are descriptive characterisation of the per-signature similarity distribution, and Section IV-D shows they do not converge to a clean two-mechanism boundary at the per-signature level---which is why the operational cosine cut is anchored on the whole-sample Firm A percentile rather than on any mixture-fit crossing.
|
||||
We read this as the strongest internal-consistency signal in v4.0: three different summarisations of the same descriptor pair agree on the per-CPA descriptor-position ranking with $\rho > 0.87$. The three scores agree on placing Firm A as the most replication-dominated descriptor position and the three non-Firm-A Big-4 firms further from the templated end, but they do not all rank the non-Firm-A firms identically: the K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Scores 1 and 3) place Firm C at the less-replication-dominated end of Big-4 (mean P(C1) $= 0.311$; mean box-rule less-replication-dominated rate $= 0.790$), while the reverse-anchor cosine percentile (Score 2) places Firm D fractionally higher than Firm C (mean reverse-anchor score $-0.7125$ vs Firm C $-0.7672$, with higher value indicating deeper into the reference left tail). The mean values for Firms B and D sit between Firms A and C on Scores 1 and 3 (Script 38 per-firm summary). We do not claim this constitutes external validation of any operational classifier; the inherited box rule is calibrated separately (§III-L), and the convergence above shows that a mixture-derived score and a reverse-anchor score concur with the box rule's per-CPA-aggregated outputs on the directional ordering, with a modest disagreement at the less-replication-dominated end between the three non-A Big-4 firms.
|
||||
|
||||
Because each audit report typically carries two certifying-CPA signatures (Section III-D), we aggregate signature-level outcomes to document-level labels using a worst-case rule: the document inherits the *most-replication-consistent* signature label (i.e., among the two signatures, the label rank ordered High-confidence $>$ Moderate-confidence $>$ Style-consistency $>$ Uncertain $>$ Likely-hand-signed determines the document's classification).
|
||||
This rule is consistent with the detection goal of flagging any potentially non-hand-signed report rather than requiring all signatures on the report to converge.
|
||||
**2. Per-signature consistency (Script 39).** Per-CPA aggregation could in principle reflect averaging across within-CPA heterogeneity rather than coherent within-CPA behaviour. We test this by repeating the K=3 fit at the signature level — fitting a fresh K=3 GMM to the 150,442 Big-4 signature-level $(\text{cos}, \text{dHash}_{\text{indep}})$ points (Script 39) — and comparing labels. The per-CPA and per-signature K=3 fits recover a broadly similar three-component ordering; per-CPA C1 is at $\overline{\text{cos}} = 0.946$, $\overline{\text{dHash}} = 9.17$ vs per-signature C1 at $\overline{\text{cos}} = 0.928$, $\overline{\text{dHash}} = 9.75$ (an absolute cosine drift of $0.018$). Cohen $\kappa$ on the binary collapse (replication-dominated vs less-replication-dominated):
|
||||
|
||||
## L. Data Source and Firm Anonymization
|
||||
| Pair | Cohen $\kappa$ |
|
||||
|---|---|
|
||||
| Paper A binary high-confidence box rule vs per-CPA K=3 hard label | $0.662$ |
|
||||
| Paper A binary high-confidence box rule vs per-signature K=3 hard label | $0.559$ |
|
||||
| Per-CPA K=3 vs per-signature K=3 | $0.870$ |
|
||||
|
||||
The Script 39 report verdict is `SIG_CONVERGENCE_MODERATE`. The $\kappa = 0.870$ between per-CPA-fit and per-signature-fit K=3 binary labels indicates that per-CPA aggregation does not collapse the broad three-component ordering. The lower $\kappa = 0.56\text{–}0.66$ between the binary box rule and either K=3 fit is consistent with two factors: different decision geometries (rectangular box vs Gaussian-mixture posterior boundary), and the fact that the binary box rule is a strict subset of the inherited five-way rule. We note that this comparison validates only the binary high-confidence rule (cos $> 0.95$ AND dHash $\leq 5$); §III-K does not directly validate the five-way rule's `5 < \text{dHash} \leq 15` moderate-confidence band, which retains its v3.20.0 calibration and capture-rate evaluation (v3.20.0 Tables IX, XI, XII, XII-B; documented as inherited in §IV-J).
|
||||
|
||||
**3. Leave-one-firm-out reproducibility (Scripts 36, 37).** Discussed in §III-J above. We summarise the joint result for cross-reference:
|
||||
|
||||
- *K=2 LOOO is unstable.* The maximum absolute deviation of the four fold cosine crossings from their across-fold mean is $0.028$, against the report's $0.005$ across-fold stability tolerance (Script 36; pairwise fold range $0.0376$, from $0.9380$ to $0.9756$). When Firm A is held out, the fold rule classifies $171/171$ of held-out Firm A CPAs as templated; when any non-Firm-A Big-4 firm is held out, the fold rule classifies $0$ of the held-out firm's CPAs as templated. This pattern indicates the K=2 boundary is essentially a Firm-A-vs-others separator rather than a within-Big-4 mechanism boundary.
|
||||
|
||||
- *K=3 LOOO is partially stable.* The C1 (low-cos / high-dHash) component shape is reproducible across folds: max deviation from the full-Big-4 baseline is $0.005$ in cosine, $0.96$ in dHash, and $0.023$ in mixture weight (Script 37). Hard-posterior membership remains composition-sensitive — observed absolute differences are $1.8$–$12.8$ pp across the four folds, with the Firm C fold exceeding the report's $5$ pp viability bar; the report's own verdict is `P2_PARTIAL` ("K=3 is not predictively useful as an operational classifier"). We accordingly do not use K=3 hard-posterior membership as an operational label.
|
||||
|
||||
**4. Positive-anchor miss rate on byte-identical signatures (Script 40).** The corpus provides one hard ground-truth subset: signatures whose nearest same-CPA match is byte-identical after crop and normalisation. Independent hand-signing cannot produce pixel-identical images, so byte-identical signatures are conservative-subset ground truth for the *replicated* class. The Big-4 byte-identical subset comprises $n = 262$ signatures ($145 / 8 / 107 / 2$ across Firms A through D; Script 40).
|
||||
|
||||
We report each candidate check's *positive-anchor miss rate* — the fraction of byte-identical signatures classified as belonging to the less-replication-dominated descriptor positions. This is a one-sided check against a conservative positive subset, **not a paired specificity metric in the usual two-class sense**; we do not report a paired negative-anchor metric here because no signature-level hand-signed ground truth exists. The corresponding signature-level inter-CPA negative-anchor ICCR evidence is developed in §III-L.1 (Big-4 v4 sample) and the inherited corpus-wide v3.x version cited at §IV-I (reported under prior "FAR" terminology):
|
||||
|
||||
| Candidate check | Pixel-identity miss rate (Wilson 95% CI) |
|
||||
|---|---|
|
||||
| Inherited Paper A binary high-confidence box rule (cos $> 0.95$ AND dHash $\leq 5$) | $0\%$ $[0\%, 1.45\%]$ |
|
||||
| K=3 per-CPA hard label (C3 high-cos / low-dHash corner; descriptive only) | $0\%$ $[0\%, 1.45\%]$ |
|
||||
| Reverse-anchor with prevalence-calibrated cut | $0\%$ $[0\%, 1.45\%]$ |
|
||||
|
||||
All three candidate scores correctly assign every byte-identical signature to the replicated class. We caution that for the inherited box rule this result is close to tautological: byte-identical nearest-neighbour signatures have cosine $\approx 1$ and dHash $\approx 0$ by construction, so any threshold strictly below cos $= 1$ and strictly above dHash $= 0$ will capture them; v3.x discussed this conservative-subset caveat at length (v3 §III-J item 1, V-F). The positive-anchor miss rate is therefore a necessary check (a classifier that *failed* this check would be disqualified), not a sufficient validation of the classifier's behaviour on the non-byte-identical replicated population. The reverse-anchor cut here is chosen by prevalence calibration against the inherited box rule's overall replicated rate ($49.58\%$ of Big-4 signatures; Script 40); this is a documented v4.0 limitation since no signature-level hand-signed ground truth exists to permit direct ROC optimisation.
|
||||
|
||||
## L. Anchor-Based Threshold Calibration and Operational Classifier
|
||||
|
||||
§III-I.4 established that the descriptor distributions do not contain a within-population bimodal antimode that could anchor an operational threshold; the K=3 mixture of §III-J is a descriptive firm-compositional partition, not a mechanism-cluster model. This section develops v4.0's anchor-based threshold calibration: the operational thresholds inherited from v3.x are characterised by their inter-CPA pair-level negative-anchor coincidence behaviour and their pool-normalised per-signature and per-document alert behaviour, at multiple units of analysis. Throughout this section we report **inter-CPA coincidence rates** rather than "False Acceptance Rates"; we explain the terminological choice in §III-L.0.
|
||||
|
||||
### L.0. Calibration methodology
|
||||
|
||||
**Operational classifier (inherited from v3.20.0 §III-K, retained unchanged).** Each Big-4 signature is assigned to one of five categories using the per-signature descriptor pair $(\text{cos}_s, \text{dHash}_s)$ where $\text{cos}_s$ is the maximum cosine similarity to another signature by the same CPA and $\text{dHash}_s$ is the minimum independent dHash to another signature by the same CPA:
|
||||
|
||||
1. **High-confidence non-hand-signed (HC):** Cosine $> 0.95$ AND $\text{dHash}_{\text{indep}} \leq 5$. Both descriptors converge on strong replication evidence.
|
||||
2. **Moderate-confidence non-hand-signed (MC):** Cosine $> 0.95$ AND $5 < \text{dHash}_{\text{indep}} \leq 15$. Feature-level evidence is strong; structural similarity is present but below the high-confidence cutoff.
|
||||
3. **High style consistency (HSC):** Cosine $> 0.95$ AND $\text{dHash}_{\text{indep}} > 15$. High feature-level similarity without structural corroboration — consistent with a CPA who signs very consistently but not via image reproduction.
|
||||
4. **Uncertain (UN):** Cosine between the all-pairs intra/inter KDE crossover ($0.837$) and $0.95$.
|
||||
5. **Likely hand-signed (LH):** Cosine $\leq 0.837$.
|
||||
|
||||
The thresholds ($\text{cos} = 0.95$ as the cosine operating point, $\text{cos} = 0.837$ as the all-pairs KDE crossover, $\text{dHash} = 5$ and $15$ as structural-similarity sub-band cutoffs) are inherited from v3.x §III-K and retain their v3.x calibration provenance. Document-level labels are aggregated via the v3.x worst-case rule: each audit report inherits the most-replication-consistent category among its certifying-CPA signatures (rank order HC > MC > HSC > UN > LH).
|
||||
|
||||
**Why retained without v4.0 recalibration.** The inherited thresholds preserve continuity with v3.x reporting and with the existing literature. §III-I.4 establishes that a v4.0 recalibration cannot be anchored on distributional antimodes (no within-population bimodality exists); §III-L.1 confirms that the cosine threshold's specificity behaviour at the inter-CPA pair level (the v3.x calibration anchor) is reproducible on the v4 spike sample, and §III-L.1 newly characterises the structural-dimension threshold $\text{dHash} \leq 5$'s pair-level coincidence behaviour. Sub-band thresholds ($\text{dHash} = 15$, $\text{cos} = 0.837$) retain v3.x's inherited calibration; v4.0 does not provide independent calibration for those sub-bands.
|
||||
|
||||
**Three units of analysis.** We report inter-CPA negative-anchor coincidence behaviour at three units, each addressing a different operational question:
|
||||
|
||||
- *Per comparison.* For a randomly drawn pair of signatures from different CPAs, what fraction satisfies the rule (cos $>$ cos\_threshold and / or dHash $\leq$ dHash\_threshold)? This is the unit at which v3.x §IV-I characterised the cosine threshold's specificity behaviour and at which threshold-derivation in biometric verification is conventionally calibrated. We report it for both the cosine and dHash dimensions, marginally and jointly (§III-L.1).
|
||||
- *Per signature pool.* For a Big-4 source signature $s$ with same-CPA pool of size $n_{\text{pool}}(s)$, what is the probability that the deployed rule fires *under the counterfactual* of replacing the source's same-CPA pool with $n_{\text{pool}}(s)$ random non-same-CPA candidates? This addresses the standard concern that a per-pair rate computed on independent pairs is not the deployed-rule rate at the per-signature classifier level: the deployed rule takes max-cosine and min-dHash over a pool of size $n_{\text{pool}}(s)$, so its effective coincidence rate is approximately $1 - (1 - p_{\text{pair}})^{n_{\text{pool}}}$ in the independence limit (§III-L.2).
|
||||
- *Per document.* For an audit report aggregated via the worst-case rule, what fraction of documents have at least one signature whose deployed pool-normalised rule fires under the same inter-CPA candidate-replacement counterfactual? This is the operational alarm-rate unit (§III-L.3).
|
||||
|
||||
**Any-pair vs same-pair semantics.** The deployed rule uses independent extrema: a signature satisfies the HC rule if $\max_{\text{pool}} \text{cos} > 0.95$ AND $\min_{\text{pool}} \text{dHash} \leq 5$, *not* if a single candidate in the pool satisfies both. We refer to this as the **any-pair** rule. A stricter alternative — the **same-pair** rule — requires a single candidate to satisfy both inequalities; the deployed v3/v4 rule is any-pair, but we report same-pair as a stricter alternative classifier where useful (§III-L.2, §III-L.4).
|
||||
|
||||
**Terminological note on "FAR".** The v3.x and biometric-verification literature speak of "False Acceptance Rate" (FAR) for a per-pair rate computed on independent inter-CPA pairs. We adopt **inter-CPA coincidence rate (ICCR)** as the v4.0 metric name and *do not* use "FAR" in the manuscript prose, for two reasons: (a) FAR has a specific biometric-verification meaning that requires ground-truth negative labels (which the corpus does not provide at the signature level); (b) §III-L.4 shows that the inter-CPA negative-anchor assumption — that inter-CPA pairs are negative — is partially violated by within-firm cross-CPA template-like collision structures. Reading "inter-CPA coincidence rate" as a *specificity proxy* under an explicitly disclosed assumption is faithful to the evidence; reading it as a true biometric FAR would overstate the evidence. We retain the v3.x numerical results (which are quantitatively reproduced in §III-L.1) under the new terminology.
|
||||
|
||||
### L.1. Per-comparison inter-CPA coincidence rate (Script 40b)
|
||||
|
||||
We sample $5 \times 10^5$ inter-CPA pairs uniformly at random from Big-4 signatures, computing for each pair the cosine similarity (feature dot product) and Hamming distance between the dHash byte vectors. Marginal and joint rates at threshold $k$ are reported with Wilson 95% confidence intervals (Script 40b).
|
||||
|
||||
| Threshold | Per-comparison inter-CPA coincidence rate | 95% Wilson CI |
|
||||
|---|---|---|
|
||||
| Cosine $> 0.95$ | $0.00060$ | $[0.00053, 0.00067]$ |
|
||||
| Cosine $> 0.945$ (v3.x published "natural threshold") | $0.00081$ | $[0.00073, 0.00089]$ |
|
||||
| Cosine $> 0.97$ | $0.00024$ | $[0.00020, 0.00029]$ |
|
||||
| Cosine $> 0.98$ | $0.00009$ | $[0.00007, 0.00012]$ |
|
||||
| dHash $\leq 5$ | $0.00129$ | $[0.00120, 0.00140]$ |
|
||||
| dHash $\leq 4$ | $0.00050$ | $[0.00044, 0.00057]$ |
|
||||
| dHash $\leq 3$ | $0.00019$ | $[0.00015, 0.00023]$ |
|
||||
| dHash $\leq 2$ | $0.00006$ | $[0.00004, 0.00008]$ |
|
||||
| Joint: cos $> 0.95$ AND dHash $\leq 5$ | $0.00014$ | (any-pair semantics) |
|
||||
| Joint: cos $> 0.95$ AND dHash $\leq 4$ | $0.00011$ | (any-pair semantics) |
|
||||
|
||||
The cosine row at $\text{cos} > 0.95$ replicates the v3.x §IV-I Table X result (v3.x reported the per-comparison rate as $0.0005$ under prior "FAR" terminology from a similarly-sized inter-CPA negative anchor; the v4 spike on a $5 \times 10^5$-pair sample yields $0.00060$, within the v3.x reported precision). The dHash and joint rows are v4-new: v3.x calibration did not provide an inter-CPA pair-level coincidence rate for the structural dimension or the joint rule.
|
||||
|
||||
The all-firms-scope sample yields slightly lower per-comparison coincidence rates (cos $> 0.95$: $0.00031$; dHash $\leq 5$: $0.00073$; joint: $0.00007$); the all-firms sample weights small CPAs more heavily under CPA-uniform pair sampling, so we treat the Big-4 sample as the primary calibration scope and report all-firms as a corroborating-scope robustness check.
|
||||
|
||||
**Conditional inter-CPA coincidence rate.** A natural follow-up question is whether the dHash dimension provides marginal specificity beyond the cosine gate. For pairs with cos $> 0.95$, the conditional rate of dHash $\leq 5$ is $0.234$ (Wilson 95% CI $[0.190, 0.285]$; $70$ of $299$ pairs in the Big-4 sample). At cos $> 0.95$, dHash provides $\sim 4.3\times$ further per-comparison specificity (joint $0.00014$ vs cos-only $0.00060$).
|
||||
|
||||
The per-comparison rate is a useful *specificity-proxy calibration* for the deployed rule's pair-level behaviour. It does *not* directly translate to the deployed-rule specificity at the per-signature classifier level, because the deployed classifier takes extrema over a same-CPA pool of size $n_{\text{pool}}$. The pool-normalised inter-CPA alert rate is reported in §III-L.2.
|
||||
|
||||
### L.2. Pool-normalised inter-CPA alert rate (Script 43)
|
||||
|
||||
The deployed rule uses $\max_{\text{pool}} \text{cos}$ and $\min_{\text{pool}} \text{dHash}$ over the same-CPA pool of size $n_{\text{pool}}(s)$ for each signature $s$. A per-comparison rate is therefore not the rate at which the deployed classifier fires per signature. To compute the per-signature inter-CPA-equivalent rate, for each Big-4 source signature $s$ we simulate one realisation of an inter-CPA candidate pool of the same size $n_{\text{pool}}(s)$, drawn uniformly from non-same-CPA signatures across all firms, compute the deployed extrema and rule indicator, and aggregate (Script 43; $n_{\text{sig}} = 150{,}453$ vector-complete in this analysis; CPA-block bootstrap 95% CIs reported below).
|
||||
|
||||
**Headline rates (deployed any-pair rule, HC = cos $> 0.95$ AND dHash $\leq 5$).** Wilson 95% CIs on the point estimate, CPA-block bootstrap 95% CI on $n_{\text{boot}} = 1000$ replicates:
|
||||
|
||||
| Rule semantics | Per-signature ICCR | Wilson 95% CI | CPA-bootstrap 95% CI |
|
||||
|---|---|---|---|
|
||||
| Any-pair (deployed) | $0.1102$ | $[0.1086, 0.1118]$ | $[0.0908, 0.1330]$ |
|
||||
| Same-pair (stricter alternative) | $0.0827$ | $[0.0813, 0.0841]$ | $[0.0668, 0.1021]$ |
|
||||
|
||||
Per-firm any-pair rates (no bootstrap; descriptive):
|
||||
|
||||
| Firm | $n_{\text{sig}}$ | Any-pair ICCR | Same-pair ICCR |
|
||||
|---|---|---|---|
|
||||
| Firm A | $60{,}450$ | $0.2594$ | $0.2018$ |
|
||||
| Firm B | $34{,}254$ | $0.0147$ | $0.0023$ |
|
||||
| Firm C | $38{,}616$ | $0.0053$ | $0.0019$ |
|
||||
| Firm D | $17{,}133$ | $0.0110$ | $0.0051$ |
|
||||
|
||||
**Pool-size decile dependence.** The deployed rule's pool-normalised rate is monotonically (broadly) increasing in $n_{\text{pool}}$, consistent with the $1 - (1 - p_{\text{pair}})^{n_{\text{pool}}}$ form expected under inter-CPA independence (Script 43 decile table). Decile 1 (smallest pools, $n_{\text{pool}} \in [0, 201]$): any-pair ICCR $= 0.0249$. Decile 10 (largest, $n_{\text{pool}} \in [846, 1115]$): any-pair ICCR $= 0.1905$. The trend is broadly monotonic with two minor non-monotone reversals (decile 5 and decile 9 dip below their predecessors).
|
||||
|
||||
**Threshold sensitivity at per-signature unit.** Tightening the HC rule from $\text{dHash} \leq 5$ to $\text{dHash} \leq 3$ (same-pair) reduces the per-signature ICCR from $0.0827$ to $0.0449$ (Big-4 pooled); tightening to $\text{dHash} \leq 4$ gives $0.0639$ (same-pair). Stakeholders requiring a per-signature ICCR of $\leq 0.05$ at HC can adopt dHash $\leq 3$ same-pair as a stricter operating point; at $\leq 0.10$ the inherited HC any-pair rule with $\text{dHash} \leq 5$ at $0.1102$ is within tolerance.
|
||||
|
||||
### L.3. Document-level inter-CPA proxy alert rate (Script 45)
|
||||
|
||||
The deployed worst-case aggregation classifies each document by the most-replication-consistent category among its constituent signatures (§III-L.0). Three operationally meaningful document-level alarm definitions are reported, each as the fraction of documents whose worst-case signature category falls in the alarm set under the same inter-CPA candidate-pool counterfactual as §III-L.2 (Script 45; $n_{\text{docs}} = 75{,}233$ Big-4 documents):
|
||||
|
||||
| Alarm definition | Alarm set | Document-level ICCR | Wilson 95% CI |
|
||||
|---|---|---|---|
|
||||
| D1 | HC only | $0.1797$ | $[0.1770, 0.1825]$ |
|
||||
| D2 | HC + MC ("any non-hand-signed verdict") | $0.3375$ | $[0.3342, 0.3409]$ |
|
||||
| D3 | HC + MC + HSC | $0.3384$ | $[0.3351, 0.3418]$ |
|
||||
|
||||
Per-firm D2 document-level rates:
|
||||
|
||||
| Firm | $n_{\text{docs}}$ | D2 (HC + MC) ICCR |
|
||||
|---|---|---|
|
||||
| Firm A | $30{,}226$ | $0.6201$ |
|
||||
| Firm B | $17{,}127$ | $0.1600$ |
|
||||
| Firm C | $19{,}501$ | $0.1635$ |
|
||||
| Firm D | $8{,}379$ | $0.0863$ |
|
||||
|
||||
The document-level D2 rate of $33.75\%$ pooled over Big-4 is the most operationally relevant alarm-rate metric: it is the fraction of audit documents that would carry at least one signature flagged HC or MC under the counterfactual of inter-CPA candidate-pool replacement. The non-trivial per-document inter-CPA alarm rate (and its concentration in Firm A at $62\%$) motivates the positioning of the operational system as a **screening framework with human-in-the-loop review**, not as an autonomous forensic classifier (§III-M).
|
||||
|
||||
### L.4. Firm heterogeneity (Script 44)
|
||||
|
||||
§III-L.2 and §III-L.3 report large per-firm variation in the deployed rule's pool-normalised behaviour: Firm A's any-pair per-signature ICCR is $0.2594$, an order of magnitude larger than Firm B's $0.0147$, Firm C's $0.0053$, Firm D's $0.0110$. A natural alternative explanation is the pool-size confound: Firm A's median pool size ($\sim 285$) is larger than other firms', and pool size monotonically (broadly) increases the per-signature rate (§III-L.2 decile trend). We test the firm-vs-pool confound with a logistic regression of the per-signature hit indicator (any-pair HC) on firm dummies (Firm A = reference) and centred log pool size (Script 44):
|
||||
|
||||
| Term | Odds ratio (vs Firm A) | Direction | Magnitude |
|
||||
|---|---|---|---|
|
||||
| Firm B | $0.053$ | $< 1$ | $\sim 19\times$ lower odds than Firm A |
|
||||
| Firm C | $0.010$ | $< 1$ | $\sim 100\times$ lower odds than Firm A |
|
||||
| Firm D | $0.027$ | $< 1$ | $\sim 37\times$ lower odds than Firm A |
|
||||
| log(pool size, centred) | $4.01$ | $> 1$ | $\sim 4\times$ higher odds per unit log pool size |
|
||||
|
||||
The Firm B/C/D odds ratios are very small after controlling for pool size, indicating that firm membership accounts for a large multiplicative effect on the per-signature rate that is *not* explained by pool size alone. (We report odds ratios rather than $z$-scores because per-signature observations are clustered by CPA and firm, and naive standard errors would be inflated by within-cluster correlation; a cluster-robust standard error analysis is left as a robustness check.)
|
||||
|
||||
The per-decile per-firm breakdown (Script 44) confirms the pattern: within every pool-size decile, Firms B/C/D have rates of $0.0006$–$0.0358$, while Firm A's rate ranges $0.0541$–$0.5958$ across deciles. The firm gap is large within matched pool sizes, not driven by pool composition.
|
||||
|
||||
**Cross-firm hit matrix.** Among Big-4 source signatures whose any-pair rule fires under the inter-CPA candidate-pool counterfactual, the candidate firm of the max-cosine partner is distributed as follows (Script 44):
|
||||
|
||||
| Source firm | Firm A candidate | Firm B | Firm C | Firm D | non-Big-4 | hits |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Firm A | $14{,}447$ | $95$ | $44$ | $19$ | $17$ | $14{,}622$ |
|
||||
| Firm B | $92$ | $371$ | $8$ | $4$ | $9$ | $484$ |
|
||||
| Firm C | $16$ | $7$ | $149$ | $5$ | $1$ | $178$ |
|
||||
| Firm D | $22$ | $2$ | $6$ | $106$ | $1$ | $137$ |
|
||||
|
||||
For the same-pair joint event (a single candidate satisfying both $\text{cos} > 0.95$ and $\text{dHash} \leq 5$), the candidate firm is even more strongly concentrated within the source firm: Firm A source $\to$ Firm A candidate in $11{,}314$ of $11{,}319$ same-pair hits ($99.96\%$); Firm B source $\to$ Firm B candidate in $85$ of $87$ ($97.7\%$); Firm C source $\to$ Firm C candidate in $54$ of $55$ ($98.2\%$); Firm D source $\to$ Firm D candidate in $64$ of $66$ ($97.0\%$).
|
||||
|
||||
**Interpretation.** Under the deployed any-pair rule, the within-firm collision concentration is $98.8\%$ at Firm A and $76.7$–$83.7\%$ at Firms B/C/D — Firm A's pattern is markedly more within-firm-concentrated than the other three firms', though every Big-4 firm still has more than three quarters of its any-pair collisions falling on candidates within the same firm. The stricter same-pair joint event — a single candidate satisfying both cos $> 0.95$ and dHash $\leq 5$ — saturates at $97.0$–$99.96\%$ within-firm across all four firms. This pattern is consistent with — but not by itself diagnostic of — firm-specific template, stamp, or document-production reuse: within-firm scanning workflows, common form templates, and shared report-generation infrastructure could produce visually similar signature crops across different CPAs within the same firm. The byte-level evidence of v3.x §IV-F.1 (Firm A's $145$ pixel-identical signatures across $\sim 50$ distinct certifying partners) provides direct evidence that firm-level template reuse does occur at Firm A; the broader inter-CPA collision pattern in §III-L.4 is consistent with that mechanism extending in milder form to Firms B/C/D. We report this as "inter-CPA collision concentration is within-firm" — a descriptive observation about deployed-rule behaviour — and refrain from inferring that the within-firm hits constitute deliberate or systematic template sharing.
|
||||
|
||||
This connects back to §III-J: the K=3 firm-composition contrast at the accountant level (Firm A dominating C3; Firm C dominating C1) reappears at the deployment level in the cross-firm hit matrix, where the within-firm collision concentration is the dominant pattern at all four Big-4 firms — most strongly at Firm A ($98.8\%$ any-pair, $99.96\%$ same-pair) and at materially lower but still majority levels at Firms B/C/D ($76.7$–$83.7\%$ any-pair; $97.0$–$98.2\%$ same-pair).
|
||||
|
||||
### L.5. Alert-rate sensitivity around inherited thresholds (Script 46)
|
||||
|
||||
To test whether the inherited cosine threshold $0.95$ and dHash threshold $5$ coincide with a low-gradient (plateau-stable) region of the deployed-rule alert-rate surface — which would be weak distributional evidence that the inherited thresholds are stable operating points — we sweep each threshold across a range and report the per-signature alert rate on actual observed Big-4 same-CPA pools (not inter-CPA-replaced pools), comparing the local gradient at the inherited threshold to the median gradient across the sweep (Script 46).
|
||||
|
||||
At the inherited HC operating point cos $> 0.95$ AND dHash $\leq 5$, the local gradient of the per-signature alert rate is substantially larger than the median gradient across the sweep (cosine: ratio $\approx 25\times$ at the $0.95$ point relative to median; dHash: ratio $\approx 3.8\times$ at the $5$ point relative to median; both Script 46). Reading these ratios descriptively, the inherited HC threshold is *locally sensitive* rather than plateau-stable: small threshold perturbations materially change the deployed alert rate (cosine sweep at dHash $\leq 5$ yields rates of $0.5091$ at cos $> 0.945$ vs $0.4789$ at cos $> 0.955$, a $3.0$ pp swing across a $0.01$ cosine perturbation; dHash sweep at cos $> 0.95$ yields rates of $0.4207$ at dHash $\leq 4$ vs $0.5639$ at dHash $\leq 6$, a $14.3$ pp swing across a single integer step). The local-gradient-to-median-gradient ratios are descriptive diagnostics, not formal plateau tests; the primary evidence for "no within-population bimodal antimode at these thresholds" comes from §III-I.4's composition decomposition, not from §III-L.5.
|
||||
|
||||
The MC/HSC boundary at dHash $= 15$, by contrast, *is* in a low-gradient region (ratio $\approx 0.08$ to the median); the plateau-like behaviour around dHash $= 15$ is corroborating evidence that the high-end structural threshold lies in a regime where the rule's alert rate is approximately saturated, consistent with the high-dHash tail behaviour expected once near-identical pairs have been exhausted. The §III-L.5 non-plateau / local-sensitivity finding therefore applies specifically to the HC cutoff (cos $= 0.95$, dHash $= 5$); the MC/HSC sub-band boundary at dHash $= 15$ exhibits the opposite behaviour and is plateau-like.
|
||||
|
||||
We interpret the inherited HC thresholds as **specificity-anchored operating points** chosen for the specificity-vs-alert-yield tradeoff (§III-L.1), *not* as distributional antimodes. Stakeholders requiring different operating points on the tradeoff curve can derive thresholds by inverting the per-comparison or pool-normalised ICCR curves (§III-L.1, §III-L.2) at their preferred specificity target.
|
||||
|
||||
### L.6. Observed deployed alert rate on actual same-CPA pools
|
||||
|
||||
The pool-normalised inter-CPA rates of §III-L.2 and §III-L.3 use the counterfactual of replacing the source signature's same-CPA pool with random non-same-CPA candidates. The **observed deployed alert rate** uses the source's actual same-CPA pool, i.e., the rate at which the deployed rule fires on the real corpus. For Big-4, the inherited HC any-pair rule fires on $49.58\%$ of signatures and $62.28\%$ of documents (Script 46; Script 42 reproduces the per-signature rate at $49.58\%$).
|
||||
|
||||
The per-signature observed-deployed rate is $\sim 4.5\times$ the pool-normalised inter-CPA rate ($0.4958$ vs $0.1102$); the per-document observed-deployed rate is $\sim 3.5\times$ the pool-normalised inter-CPA D1 (HC) rate ($0.6228$ vs $0.1797$). We refer to this multiplicative gap as the **deployed-rate excess over the inter-CPA proxy**:
|
||||
|
||||
- Per-signature: $0.4958 - 0.1102 = 0.3856$ ($38.6$ pp excess)
|
||||
- Per-document HC: $0.6228 - 0.1797 = 0.4431$ ($44.3$ pp excess)
|
||||
|
||||
We *do not* interpret the deployed-rate excess as a presumed true-positive rate; the inferential limits of this interpretation are developed in §III-M. The deployed-rate excess is best read as a *same-CPA repeatability signal* — a quantity that exceeds what random inter-CPA candidate replacement would produce — rather than as an estimate of true replication prevalence.
|
||||
|
||||
### L.7. K=3 not used as classifier
|
||||
|
||||
The K=3 mixture of §III-J is reported in §IV as an accountant-level descriptive summary alongside the per-signature five-way classifier. We do not assign signature-level or document-level labels from the K=3 mixture in any v4.0 result table; the K=3 hard label is used only for the accountant-level firm × cluster cross-tabulation (§III-J; Script 35), and the K=3 *posterior* P(C1) is used (as the continuous Score 1) in the internal-consistency Spearman correlations of §III-K. The operational classifier of §III-L.0 is the inherited v3.x five-way box rule; the calibration evidence in §III-L.1 through §III-L.6 characterises its multi-level coincidence behaviour against the inter-CPA negative anchor.
|
||||
|
||||
## M. Validation Strategy and Limitations under Unsupervised Setting
|
||||
|
||||
The v4.0 corpus lacks signature-level ground-truth replication labels: no signature is annotated as definitively hand-signed or definitively templated. The conservative positive anchor (pixel-identical same-CPA signatures; §III-K.4 and v3.x §IV-F.1) is by construction near $\text{cos} = 1$ and $\text{dHash} = 0$, providing a tautological capture-check rather than a sensitivity estimate for the non-byte-identical replicated class. The corpus therefore does not admit standard supervised classifier validation: we cannot report False Rejection Rate, sensitivity, recall, Equal Error Rate, ROC-AUC, or precision against ground truth.
|
||||
|
||||
In place of supervised validation, v4.0 adopts a **multi-tool collection of partial-evidence diagnostics** (Table XXVII), each with an explicitly disclosed assumption:
|
||||
|
||||
**Table XXVII.** Ten-tool unsupervised-validation collection with disclosed untested assumptions.
|
||||
|
||||
| Tool | What it measures | Untested assumption |
|
||||
|---|---|---|
|
||||
| Composition decomposition (§III-I.4; Scripts 39b–39e) | Whether descriptor multimodality is within-population (mechanism) or between-group (composition + integer artefact); $p_{\text{median}} = 0.35$ under joint firm-mean centring + integer-tie jitter | Integer-tie jitter and firm-mean centring are unbiased over the descriptor support; corroborated by Big-4 per-firm jitter (Script 39d; per-firm dHash rejection disappears under jitter at every Big-4 firm) and Big-4 pooled centred + jittered ($n_{\text{seeds}} = 5$; Script 39e) |
|
||||
| Per-comparison inter-CPA coincidence rate (§III-L.1; Script 40b) | Pair-level specificity proxy under a random-pair negative anchor | Inter-CPA pairs are negative (i.e., not template-related); partially violated by within-firm sharing (§III-L.4) |
|
||||
| Pool-normalised per-signature ICCR (§III-L.2; Script 43) | Deployed-rule specificity proxy at per-signature unit, accounting for pool size | Same as above + that pool replacement preserves the negative-anchor property |
|
||||
| Document-level ICCR (§III-L.3; Script 45) | Operational alarm rate proxy at per-document unit under three alarm definitions | Same as above |
|
||||
| Firm-heterogeneity logistic regression (§III-L.4; Script 44) | Multiplicative effect of firm membership on per-signature rate, controlling for pool size | Per-signature observations are clustered by CPA/firm; naïve standard errors inflated; cluster-robust analysis is a future check |
|
||||
| Cross-firm hit matrix (§III-L.4; Script 44) | Concentration of inter-CPA collisions within source firm | Concentration depends on deployed-rule semantics (the stricter same-pair joint event yields $97.0$–$99.96\%$ within-firm at all four firms versus $76.7$–$98.8\%$ under any-pair; §III-L.4); per-document per-firm assignment uses Script 45's mode-of-firms tie-break (§IV-M.4 footnote) |
|
||||
| Alert-rate sensitivity sweep (§III-L.5; Script 46) | Local sensitivity of deployed rule to threshold perturbation | Gradient comparison is descriptive, not a formal plateau test |
|
||||
| Convergent score Spearman ranking (§III-K.1; Script 38) | Internal-consistency of three feature-derived per-CPA scores | Scores share underlying inputs and are not statistically independent |
|
||||
| Pixel-identical conservative positive capture (§III-K.4; v3.x; Script 40) | Trivial sanity check on the conservative positive anchor | Anchor is tautologically captured by any reasonable threshold |
|
||||
| LOOO firm-level reproducibility (§III-K.3; Scripts 36, 37) | Algorithmic stability of K=2 / K=3 partition across firm folds | Stability is necessary but not sufficient for classification validity |
|
||||
|
||||
No single tool in this collection provides ground-truth validation. Their conjunction constitutes the unsupervised validation ceiling that the v4.0 corpus admits.
|
||||
|
||||
**What v4.0 does not claim.** We do not claim a validated forensic detector or an autonomous classification system. We do not report False Rejection Rate, sensitivity, recall, EER, ROC-AUC, precision, or positive predictive value against ground truth, because no ground truth exists at the signature level. We do not interpret the deployed-rate excess of §III-L.6 as a presumed true-positive rate: that interpretation would require assuming that the within-firm same-CPA pool's collision rate equals the inter-CPA proxy rate in the absence of replication (i.e., that genuine same-CPA hand-signing would produce a collision rate no higher than random inter-CPA pairs). Two factors make the assumption unsafe: (a) a CPA who signs consistently can produce stylistically similar signatures across years that exceed inter-CPA similarity at the cosine axis; (b) within-firm template sharing (§III-L.4 cross-firm hit matrix; v3.x byte-level evidence of Firm A's pixel-identical signatures across partners) places a substantial inter-CPA collision floor that itself reflects template-like reuse rather than independent inter-CPA random matching. We do not infer that the within-firm collision concentration of §III-L.4 constitutes deliberate template sharing; we describe it as "inter-CPA collision concentration is within-firm" and treat the mechanism as an open empirical question.
|
||||
|
||||
**What v4.0 does claim.** The deployed signature-replication screening rule is characterised at three units of analysis (per-comparison, per-signature pool, per-document) against an inter-CPA negative-anchor coincidence-rate calibration. The per-comparison rates ($\leq 0.0006$ at cos $> 0.95$; $\leq 0.0013$ at dHash $\leq 5$; $\leq 0.00014$ jointly) are specificity-proxy-anchored operating points consistent with biometric-verification convention, with the proxy nature recorded in §III-L.0 and §III-M. The per-signature and per-document rates ($0.11$ and $0.34$ respectively under the deployed any-pair HC + MC alarm) are operationally meaningful **alarm-yield** indicators rather than true error rates. Per-firm rates show substantial heterogeneity (Firm A's per-document HC + MC alarm at $0.62$ vs Firm B/C/D at $0.09$–$0.16$), driven by firm-level rather than pool-size effects, and concentrated in within-firm cross-CPA candidate matching. The framework is positioned as a **specificity-proxy-anchored screening tool with human-in-the-loop review**, not as a validated forensic classifier.
|
||||
|
||||
**Specificity-alert-yield tradeoff.** Because sensitivity is unobservable, stakeholders cannot derive an operating point by optimising a ROC criterion. Instead, the specificity-proxy-anchored framework offers a *specificity-alert-yield tradeoff*: tighter operating points (e.g., cos $> 0.98$ AND dHash $\leq 3$) reduce both per-comparison ICCR (to $\approx 5 \times 10^{-5}$; §III-L.1 inversion) and per-signature alert yield (to $\approx 0.05$; §III-L.2), with an unknown effect on actual replication-detection recall. Tighter operating points are not necessarily preferable: any tightening reduces the alert rate but may also miss true replicated signatures whose noise has pushed them outside the tighter envelope. The deployment decision depends on the relative cost of manual review (per alarm) and missed-replication risk (per false negative) — neither directly observable from corpus data.
|
||||
|
||||
## N. Data Source and Firm Anonymization
|
||||
|
||||
**Audit-report corpus.** The 90,282 audit-report PDFs analyzed in this study were obtained from the Market Observation Post System (MOPS) operated by the Taiwan Stock Exchange Corporation.
|
||||
MOPS is the statutory public-disclosure platform for Taiwan-listed companies; every audit report filed on MOPS is already a publicly accessible regulatory document.
|
||||
|
||||
@@ -72,6 +72,9 @@ For observations bounded on $[0,1]$---such as cosine similarity and normalized H
|
||||
Under mild regularity conditions, White's quasi-MLE result [41] supports interpreting maximum-likelihood estimates under a mis-specified parametric family as consistent estimators of the pseudo-true parameter that minimizes the Kullback-Leibler divergence to the data-generating distribution within that family; we use this result to justify the Beta-mixture fit as a principled approximation rather than as a guarantee that the true distribution is Beta.
|
||||
|
||||
The present study combines all three families, using each to produce an independent threshold estimate and treating cross-method convergence---or principled divergence---as evidence of where in the analysis hierarchy the mixture structure is statistically supported.
|
||||
|
||||
*Cross-validation in a small-cluster scope.*
|
||||
Cross-validation methodology in the leave-one-out tradition has been developed extensively in statistics since Stone [42] and Geisser [43], and modern surveys including Vehtari et al. [44] discuss its application to mixture models. In document-forensics calibration the technique has been used selectively, typically with the individual document or signature as the hold-out unit. Our application in §III-K differs in two respects from the standard usage: (i) the hold-out unit is the *firm* (not the individual CPA or signature), so the analysis directly probes cross-firm reproducibility of the fitted mixture rather than within-firm sampling variance; and (ii) the held-out predictions are interpreted as a *composition-sensitivity band* on the candidate mixture boundary, not as a sufficiency claim for the inherited five-way operational classifier (which is calibrated separately; §III-L). We treat LOOO drift as descriptive information about how the mixture characterisation moves when training composition changes, not as a pass/fail test for the operational classifier.
|
||||
<!--
|
||||
REFERENCES for Related Work (see paper_a_references_v3.md for full list):
|
||||
[3] Bromley et al. 1993 — Siamese TDNN (NeurIPS)
|
||||
@@ -101,4 +104,7 @@ REFERENCES for Related Work (see paper_a_references_v3.md for full list):
|
||||
[39] McCrary 2008 — density discontinuity test
|
||||
[40] Dempster, Laird & Rubin 1977 — EM algorithm
|
||||
[41] White 1982 — quasi-MLE consistency
|
||||
[42] Stone 1974 — cross-validatory choice
|
||||
[43] Geisser 1975 — predictive sample reuse
|
||||
[44] Vehtari, Gelman & Gabry 2017 — practical Bayesian LOO/WAIC
|
||||
-->
|
||||
|
||||
+346
-408
@@ -1,5 +1,7 @@
|
||||
# IV. Experiments and Results
|
||||
|
||||
The v4.0 primary analyses (§IV-D through §IV-J) are scoped to the Big-4 sub-corpus (Firms A–D, $n = 437$ CPAs with $n_{\text{sig}} \geq 10$, totalling 150,442 signatures with both descriptors available) per the methodology choice articulated in §III-G. The §IV-K Full-Dataset Robustness section reports the full-dataset (686 CPAs) variant of the K=3 mixture + Paper A box-rule Spearman analysis as a cross-scope robustness check. §IV-A through §IV-C report inherited corpus-wide v3.x material; §IV-L (feature backbone ablation) is also inherited. §IV-M consolidates the v4-new anchor-based ICCR calibration tables.
|
||||
|
||||
## A. Experimental Setup
|
||||
|
||||
Experiments used mixed hardware: YOLOv11n training and inference for signature detection, and ResNet-50 forward inference for feature extraction over all 182,328 detected signatures, were performed on an Nvidia RTX 4090 (CUDA); the downstream statistical analyses (KDE antimode, Hartigan dip test, Beta-mixture EM with logit-Gaussian robustness check, Burgstahler-Dichev/McCrary density-smoothness diagnostic, and pairwise cosine/dHash computations) were performed on an Apple Silicon workstation with Metal Performance Shaders (MPS) acceleration.
|
||||
@@ -25,10 +27,12 @@ The high VLM--YOLO agreement rate (98.8%) further corroborates detection reliabi
|
||||
| Processing rate | 43.1 docs/sec |
|
||||
-->
|
||||
|
||||
The Big-4 subset of the detection output yields 150,442 signatures with both descriptors (cosine and independent dHash) successfully computed; this is the per-signature population used in all §IV v4 primary analyses (§IV-D through §IV-J).
|
||||
|
||||
## C. All-Pairs Intra-vs-Inter Class Distribution Analysis
|
||||
|
||||
Fig. 2 presents the cosine similarity distributions computed over the full set of *pairwise comparisons* under two groupings: intra-class (all signature pairs belonging to the same CPA) and inter-class (signature pairs from different CPAs).
|
||||
This all-pairs analysis is a different unit from the per-signature best-match statistics used in Sections IV-D onward; we report it first because it supplies the reference point for the KDE crossover used in per-document classification (Section III-K).
|
||||
This all-pairs analysis is a different unit from the per-signature best-match statistics used in Sections IV-D onward; we report it first because it supplies the reference point for the KDE crossover used in per-document classification (Section III-L).
|
||||
Table IV summarizes the distributional statistics.
|
||||
|
||||
<!-- TABLE IV: Cosine Similarity Distribution Statistics
|
||||
@@ -54,417 +58,248 @@ We emphasize that pairwise observations are not independent---the same signature
|
||||
We therefore rely primarily on Cohen's $d$ as an effect-size measure that is less sensitive to sample size.
|
||||
A Cohen's $d$ of 0.669 indicates a medium effect size [29], confirming that the distributional difference is practically meaningful, not merely an artifact of the large sample count.
|
||||
|
||||
## D. Signature-Level Distributional Characterisation
|
||||
|
||||
This section applies the threshold-estimator and density-smoothness diagnostic of Section III-I to the per-signature similarity distribution.
|
||||
The joint reading is that per-signature similarity is a continuous quality spectrum rather than a clean two-mechanism mixture, which is why the operational classifier (Section III-K) anchors its cosine cut on the whole-sample Firm A P7.5 percentile rather than on any mixture-fit crossing.
|
||||
|
||||
### 1) Hartigan Dip Test: Unimodality at the Signature Level
|
||||
|
||||
Applying the Hartigan & Hartigan dip test [37] to the per-signature best-match distributions reveals a critical structural finding (Table V).
|
||||
The $N = 168{,}740$ count used in Table V and in the downstream same-CPA per-signature best-match analyses (Tables V and XII, and the Firm-A per-signature rows of Tables XIII and XVIII) is $15$ signatures smaller than the $168{,}755$ CPA-matched count reported in Table III: these $15$ signatures belong to CPAs with exactly one signature in the entire corpus, for whom no same-CPA pairwise best-match statistic can be computed, and are therefore excluded from all same-CPA similarity analyses.
|
||||
|
||||
<!-- TABLE V: Hartigan Dip Test Results
|
||||
| Distribution | N | dip | p-value | Verdict (α=0.05) |
|
||||
|--------------|---|-----|---------|------------------|
|
||||
| Firm A cosine (max-sim) | 60,448 | 0.0019 | 0.169 | Unimodal |
|
||||
| Firm A min dHash (independent) | 60,448 | 0.1051 | <0.001 | Multimodal |
|
||||
| All-CPA cosine (max-sim) | 168,740 | 0.0035 | <0.001 | Multimodal |
|
||||
| All-CPA min dHash (independent) | 168,740 | 0.0468 | <0.001 | Multimodal |
|
||||
-->
|
||||
|
||||
Firm A's per-signature cosine distribution *fails to reject unimodality* ($p = 0.17$), a pattern consistent with a dominant high-similarity regime plus a long left tail attributable to within-firm heterogeneity in signing outputs (Section III-G discusses the scope of partner-level claims).
|
||||
The all-CPA cosine distribution, which mixes many firms with heterogeneous signing practices, is *multimodal* ($p < 0.001$).
|
||||
The Firm A unimodal-long-tail finding is, in conjunction with the byte-identity, partner-ranking, and intra-report evidence reported below, consistent with the replication-dominated framing (Section III-H): a dominant high-similarity regime plus residual within-firm heterogeneity, rather than two cleanly separated mechanisms.
|
||||
|
||||
### 2) Burgstahler-Dichev / McCrary Density-Smoothness Diagnostic
|
||||
|
||||
Applying the BD/McCrary procedure (Section III-I.3) to the per-signature cosine distribution yields a nominally significant $Z^- \rightarrow Z^+$ transition at cosine 0.985 for Firm A and 0.985 for the full sample; the min-dHash distributions exhibit a transition at Hamming distance 2 for both Firm A and the full sample under the bin width ($0.005$ / $1$) used here.
|
||||
Two cautions, however, prevent us from treating these signature-level transitions as thresholds.
|
||||
First, the cosine transition at 0.985 lies *inside* the non-hand-signed mode rather than at the separation between two mechanisms, consistent with the dip-test finding that per-signature cosine is not cleanly bimodal.
|
||||
Second, Appendix A documents that the signature-level transition locations are not bin-width-stable (Firm A cosine drifts across 0.987, 0.985, 0.980, 0.975 as the bin width is widened from 0.003 to 0.015, and full-sample dHash transitions drift across 2, 10, 9 as bin width grows from 1 to 3), which is characteristic of a histogram-resolution artifact rather than of a genuine density discontinuity between two mechanisms.
|
||||
We therefore use BD/McCrary as a density-smoothness diagnostic rather than as an independent threshold estimator.
|
||||
|
||||
### 3) Beta Mixture at Signature Level: A Forced Fit
|
||||
|
||||
Fitting 2- and 3-component Beta mixtures to Firm A's per-signature cosine via EM yields a clear BIC preference for the 3-component fit ($\Delta\text{BIC} = 381$), with a parallel preference under the logit-GMM robustness check.
|
||||
For the full-sample cosine the 3-component fit is likewise strongly preferred ($\Delta\text{BIC} = 10{,}175$).
|
||||
Under the forced 2-component fit the Firm A Beta crossing lies at 0.977 and the logit-GMM crossing at 0.999---values sharply inconsistent with each other, indicating that the 2-component parametric structure is not supported by the data.
|
||||
Under the full-sample 2-component forced fit no Beta crossing is identified; the logit-GMM crossing is at 0.980.
|
||||
|
||||
### 4) Joint Reading of the Three Diagnostics
|
||||
|
||||
The three diagnostics agree that per-signature similarity does not form a clean two-mechanism mixture:
|
||||
(i) the Hartigan dip test fails to reject unimodality for Firm A and rejects it for the heterogeneous-firm pooled sample;
|
||||
(ii) BIC strongly prefers a 3-component over a 2-component Beta fit, so the 2-component crossing is a *forced fit* and the Beta-vs-logit-Gaussian disagreement (0.977 vs 0.999 for Firm A) reflects parametric-form sensitivity rather than a stable two-mechanism boundary;
|
||||
(iii) the BD/McCrary procedure locates its candidate transition *inside* the non-hand-signed mode rather than between modes, and the transition is not bin-width-stable.
|
||||
|
||||
Table VI summarises the signature-level threshold-estimator outputs for cross-method comparison.
|
||||
|
||||
<!-- TABLE VI: Signature-Level Threshold-Estimator Summary
|
||||
| Population | Method | Cosine threshold | dHash threshold | Status |
|
||||
|------------|--------|------------------|-----------------|--------|
|
||||
| **Threshold estimators (signature-level distributional fits)** | | | | |
|
||||
| Firm A signature-level | KDE antimode + Hartigan dip (Section III-I.1) | undefined | — | unimodal at $\alpha=0.05$ ($p=0.169$); antimode not defined for unimodal data |
|
||||
| Firm A signature-level | Beta-2 EM crossing (Section III-I.2) | 0.977 | — | forced fit; BIC strongly prefers $K{=}3$ ($\Delta\text{BIC} = 381$) |
|
||||
| Firm A signature-level | logit-Gaussian-2 crossing (robustness check) | 0.999 | — | forced fit; sharply inconsistent with Beta-2 crossing—reflects parametric-form sensitivity |
|
||||
| Full-sample signature-l. | KDE antimode + Hartigan dip | (multiple modes) | — | multimodal ($p<0.001$); KDE crossover at full-sample is dominated by between-firm heterogeneity |
|
||||
| Full-sample signature-l. | Beta-2 EM crossing | no crossing | — | forced fit; component densities do not cross over $[0,1]$ under recovered parameters |
|
||||
| Full-sample signature-l. | logit-Gaussian-2 crossing | 0.980 | — | forced fit; BIC strongly prefers $K{=}3$ ($\Delta\text{BIC} = 10{,}175$) |
|
||||
| **Density-smoothness diagnostics (not threshold estimators)** | | | | |
|
||||
| Firm A signature-level | BD/McCrary candidate transition (Section III-I.3) | 0.985 (bin 0.005)| 2.0 (bin 1) | bin-unstable across $\{0.003, 0.005, 0.010, 0.015\}$ (Appendix A); transition lies *inside* the non-hand-signed mode |
|
||||
| Full-sample signature-l. | BD/McCrary candidate transition | 0.985 (bin 0.005) | 2.0 (bin 1) | bin-unstable across $\{0.003, 0.005, 0.010, 0.015\}$ (Appendix A) |
|
||||
| **Reference: between-class KDE (different unit of analysis)** | | | | |
|
||||
| All-pairs intra/inter (pair-level; Section IV-C) | KDE crossover | 0.837 | — | reference point for the Uncertain/Likely-hand-signed boundary in the operational classifier |
|
||||
| **Operational classifier anchors and percentile cross-references** | | | | |
|
||||
| Firm A whole-sample | P7.5 (operational anchor; Section III-K) | 0.95 | — | operational cosine cut for the five-way classifier |
|
||||
| Firm A whole-sample | dHash$_\text{indep}$ P75 | — | 4 | informs the $\leq 5$ high-confidence band edge in the classifier |
|
||||
| Firm A whole-sample | dHash$_\text{indep}$ style-consistency ceiling | — | 15 | operational $> 15$ style-consistency boundary |
|
||||
| Firm A calibration fold (70%) | cosine P5 (Section IV-F.2) | 0.9407 | — | calibration-fold cross-reference; held-out fold reports rates at this cut |
|
||||
| Firm A calibration fold (70%) | dHash$_\text{indep}$ P95 | — | 9 | calibration-fold cross-reference (Tables IX and XI report rates at the rounded $\leq 8$ cut for continuity) |
|
||||
|
||||
Read this table by *population × method*: each row reports one method applied to one population.
|
||||
The first three blocks (threshold estimators; density-smoothness diagnostics; between-class KDE) are *characterisation* outputs; the bottom block is the operational anchor set used by the classifier of Section III-K.
|
||||
The disagreement between Firm A Beta-2 (0.977) and Firm A logit-Gaussian-2 (0.999) is the parametric-form sensitivity referenced in the prose of Section IV-D.3; it cannot be resolved from the data because BIC rejects the underlying $K{=}2$ assumption itself.
|
||||
-->
|
||||
|
||||
Non-hand-signed replication quality is therefore best read as a continuous spectrum produced by firm-specific reproduction technologies (administrative stamping in early years, firm-level e-signing later) acting on a common stored exemplar.
|
||||
This finding has a direct methodological pay-off: it is *why* the operational cosine cut is anchored on the whole-sample Firm A P7.5 percentile (Section III-K), and it is *why* the byte-level pixel-identity anchor (Section IV-F.1) is the natural threshold-free positive reference for downstream validation.
|
||||
|
||||
## E. Calibration Validation with Firm A
|
||||
|
||||
Fig. 3 presents the per-signature cosine and dHash distributions of Firm A compared to the overall population.
|
||||
Table IX reports the proportion of Firm A signatures crossing each candidate threshold; these rates play the role of calibration-validation metrics (what fraction of a known replication-dominated population does each threshold capture?).
|
||||
|
||||
<!-- TABLE IX: Firm A Whole-Sample Capture Rates (consistency check, NOT external validation)
|
||||
| Rule | Firm A rate | k / N |
|
||||
|------|-------------|-------|
|
||||
| **Cosine-only marginal rates** | | |
|
||||
| cosine > 0.837 (all-pairs KDE crossover) | 99.93% | 60,408 / 60,448 |
|
||||
| cosine > 0.9407 (calibration-fold P5) | 95.15% | 57,518 / 60,448 |
|
||||
| cosine > 0.945 (calibration-fold P5 rounded) | 94.02% | 56,836 / 60,448 |
|
||||
| cosine > 0.95 (operational; whole-sample Firm A P7.5) | 92.51% | 55,922 / 60,448 |
|
||||
| **dHash-only marginal rates** | | |
|
||||
| dHash_indep ≤ 5 (operational high-confidence cap) | 84.20% | 50,897 / 60,448 |
|
||||
| dHash_indep ≤ 8 (calibration-fold P95 rounded) | 95.17% | 57,527 / 60,448 |
|
||||
| dHash_indep ≤ 15 (operational style-consistency boundary) | 99.83% | 60,348 / 60,448 |
|
||||
| **Operational classifier dual rules (Section III-K)** | | |
|
||||
| cosine > 0.95 AND dHash_indep ≤ 5 (high-confidence non-hand-signed) | 81.70% | 49,389 / 60,448 |
|
||||
| cosine > 0.95 AND 5 < dHash_indep ≤ 15 (moderate-confidence) | 10.76% | 6,503 / 60,448 |
|
||||
| cosine > 0.95 AND dHash_indep ≤ 15 (combined non-hand-signed) | 92.46% | 55,892 / 60,448 |
|
||||
| **Calibration-fold-adjacent cross-reference (not the operational classifier rule)** | | |
|
||||
| cosine > 0.95 AND dHash_indep ≤ 8 | 89.95% | 54,370 / 60,448 |
|
||||
|
||||
All rates computed exactly from the full Firm A sample (N = 60,448 signatures); per-rule counts and codes are available in the supplementary materials.
|
||||
The two operational dHash cuts ($\leq 5$ for the high-confidence cap and $\leq 15$ for the style-consistency boundary) come from the classifier definition in Section III-K and are the rules used by the five-way classifier of Tables XII and XVII; the dHash $\leq 8$ row is *not* an operational classifier rule but a calibration-fold-adjacent reference (Section IV-F.2 calibration-fold dHash P95 = 9; we report the $\leq 8$ rate as the integer-valued threshold immediately below P95, included here so that Firm A capture in the calibration-fold-P95 neighbourhood can be read off the same table).
|
||||
-->
|
||||
|
||||
Table IX is a whole-sample consistency check rather than an external validation: the cosine cut $0.95$ and the operational dHash band edges ($\leq 5$ high-confidence cap and $\leq 15$ style-consistency boundary) are themselves anchored to the whole-sample Firm A distribution described in Section III-K (the 70/30 calibration-fold thresholds of Table XI are separate and slightly different, e.g., calibration-fold cosine P5 = 0.9407 rather than the whole-sample heuristic 0.95).
|
||||
The operational dual rule used by the five-way classifier of Section III-K---cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 15$ (the union of the high-confidence and moderate-confidence non-hand-signed buckets)---captures 92.46% of Firm A; the high-confidence component alone (cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 5$) captures 81.70%.
|
||||
For continuity with prior calibration-fold reporting (Section IV-F.2 reports the calibration-fold rate at the calibration-fold-P95-adjacent cut $\text{dHash}_\text{indep} \leq 8$), Table IX also lists the cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 8$ rate of 89.95%; this is *not* the operational classifier rule but a cross-reference value.
|
||||
Both operational rates are consistent with the dip-test-confirmed unimodal-long-tail shape of Firm A's per-signature cosine distribution (Section IV-D.1) and the 92.5% / 7.5% signature-level split (Section III-H).
|
||||
Section IV-F.2 reports the corresponding rates on the 30% Firm A hold-out fold, which provides the external check these whole-sample rates cannot.
|
||||
|
||||
## F. Pixel-Identity, Inter-CPA, and Held-Out Firm A Validation
|
||||
|
||||
We report three validation analyses corresponding to the anchors of Section III-J.
|
||||
|
||||
### 1) Pixel-Identity Positive Anchor with Inter-CPA Negative Anchor
|
||||
|
||||
Of the 182,328 extracted signatures, 310 have a same-CPA nearest match that is byte-identical after crop and normalization (pixel-identical-to-closest = 1); these form the byte-identity positive anchor---a pair-level proof of image reuse that serves as conservative ground truth for non-hand-signed signatures, subject to the source-template edge case discussed in Section V-G.
|
||||
Within Firm A specifically, 145 of these byte-identical signatures are distributed across 50 distinct partners (of 180 registered Firm A partners), with 35 of the byte-identical pairs spanning different fiscal years; reproduction artifact for this Firm A decomposition is listed in Appendix B.
|
||||
As the gold-negative anchor we sample 50,000 i.i.d. random cross-CPA signature pairs from the full 168,755-signature matched corpus (inter-CPA cosine: mean $= 0.763$, $P_{95} = 0.886$, $P_{99} = 0.915$, max $= 0.992$).
|
||||
Because the positive and negative anchor populations are constructed from different sampling units (byte-identical same-CPA pairs vs random inter-CPA pairs), their relative prevalence in the combined anchor set is arbitrary, and precision / $F_1$ / recall therefore have no meaningful population interpretation.
|
||||
We accordingly report FAR with Wilson 95% confidence intervals against the large inter-CPA negative anchor in Table X.
|
||||
The primary quantity reported by Table X is FAR: the probability that a random pair of signatures from *different* CPAs exceeds the candidate threshold.
|
||||
We do not report an Equal Error Rate: EER is meaningful only when the positive and negative error-rate curves cross in a nontrivial interior region, but byte-identical positives all sit at cosine $\approx 1$ by construction, so FRR against that subset is trivially $0$ at every threshold below $1$. An EER calculation against this anchor would be arithmetic tautology rather than biometric performance, and we therefore omit it.
|
||||
|
||||
<!-- TABLE X: Cosine Threshold Sweep — FAR Against 50,000 Inter-CPA Negative Pairs
|
||||
| Threshold | FAR | FAR 95% Wilson CI |
|
||||
|-----------|-----|-------------------|
|
||||
| 0.837 (all-pairs KDE crossover) | 0.2101 | [0.2066, 0.2137] |
|
||||
| 0.900 | 0.0250 | [0.0237, 0.0264] |
|
||||
| 0.945 (calibration-fold P5 rounded) | 0.0008 | [0.0006, 0.0011] |
|
||||
| 0.950 (whole-sample Firm A P7.5; operational cut) | 0.0005 | [0.0003, 0.0007] |
|
||||
| 0.977 (Firm A Beta-2 forced-fit crossing; Section IV-D) | 0.00014 | [0.00007, 0.00029] |
|
||||
| 0.985 (BD/McCrary candidate transition; Appendix A) | 0.00004 | [0.00001, 0.00015] |
|
||||
|
||||
Table note: We do not include FRR against the byte-identical positive anchor as a column here: the byte-identical subset has cosine $\approx 1$ by construction, so FRR against that subset is trivially $0$ at every threshold below $1$ and carries no biometric information beyond verifying that the threshold does not exceed $1$. The conservative-subset FRR role of the byte-identical anchor is instead discussed qualitatively in Section V-F.
|
||||
-->
|
||||
|
||||
Two caveats apply.
|
||||
First, the byte-identical positive anchor referenced above is a *conservative subset* of the true non-hand-signed population: it captures only those non-hand-signed signatures whose nearest match happens to be byte-identical, not those that are near-identical but not bytewise identical.
|
||||
A would-be FRR computed against this subset is definitionally $0$ at every threshold below $1$ (since byte-identical pairs have cosine $\approx 1$), so such an FRR is a mathematical boundary check rather than an empirical miss-rate estimate; we discuss the generalization limits of this conservative-subset framing in Section V-F.
|
||||
Second, the 0.945 / 0.95 thresholds are derived from the Firm A whole-sample and calibration-fold percentiles rather than from this anchor set, so the FAR values in Table X are post-hoc-fit-free evaluations of thresholds that were not chosen to optimize Table X.
|
||||
The very low FAR at the operational cut is therefore informative about specificity against a realistic inter-CPA negative population.
|
||||
|
||||
### 2) Held-Out Firm A Validation (within-Firm-A sampling variance disclosure)
|
||||
|
||||
We split Firm A CPAs randomly 70 / 30 at the CPA level into a calibration fold (124 CPAs, 45,116 signatures) and a held-out fold (54 CPAs, 15,332 signatures).
|
||||
The total of 178 Firm A CPAs differs from the 180 in the Firm A registry by two registered Firm A partners whose signatures in the corpus are singletons (only one signature each, so the per-signature best-match cosine is undefined and they do not appear in the same-CPA matched-signature table that script `24_validation_recalibration.py` reads); they are therefore not represented in either fold by construction rather than by an explicit exclusion rule.
|
||||
Thresholds are re-derived from calibration-fold percentiles only.
|
||||
Table XI reports both calibration-fold and held-out-fold capture rates with Wilson 95% CIs and a two-proportion $z$-test.
|
||||
|
||||
<!-- TABLE XI: Held-Out vs Calibration Firm A Capture Rates and Generalization Test
|
||||
| Rule | Calibration 70% fold (CI) | Held-out 30% fold (CI) | 2-prop z | p | k/n calib | k/n held |
|
||||
|------|---------------------------|-------------------------|----------|---|-----------|----------|
|
||||
| cosine > 0.837 | 99.94% [99.91%, 99.96%] | 99.93% [99.87%, 99.96%] | +0.31 | 0.756 n.s. | 45,087/45,116 | 15,321/15,332 |
|
||||
| cosine > 0.9407 (calib-fold P5) | 94.99% [94.79%, 95.19%] | 95.63% [95.29%, 95.94%] | -3.19 | 0.001 | 42,856/45,116 | 14,662/15,332 |
|
||||
| cosine > 0.945 (calib-fold P5 rounded) | 93.77% [93.54%, 93.99%] | 94.78% [94.41%, 95.12%] | -4.54 | <0.001 | 42,305/45,116 | 14,531/15,332 |
|
||||
| cosine > 0.950 (whole-sample P7.5; operational cut) | 92.14% [91.89%, 92.38%] | 93.61% [93.21%, 93.98%] | -5.97 | <0.001 | 41,570/45,116 | 14,352/15,332 |
|
||||
| dHash_indep ≤ 5 | 82.96% [82.61%, 83.31%] | 87.84% [87.31%, 88.34%] | -14.29 | <0.001 | 37,430/45,116 | 13,467/15,332 |
|
||||
| dHash_indep ≤ 8 | 94.84% [94.63%, 95.04%] | 96.13% [95.82%, 96.43%] | -6.45 | <0.001 | 42,788/45,116 | 14,739/15,332 |
|
||||
| dHash_indep ≤ 9 (calib-fold P95) | 96.65% [96.48%, 96.81%] | 97.48% [97.22%, 97.71%] | -5.07 | <0.001 | 43,604/45,116 | 14,945/15,332 |
|
||||
| dHash_indep ≤ 15 | 99.83% [99.79%, 99.87%] | 99.84% [99.77%, 99.89%] | -0.31 | 0.754 n.s. | 45,040/45,116 | 15,308/15,332 |
|
||||
| cosine > 0.95 AND dHash_indep ≤ 8 (calibration-fold P95-adjacent reference; P95 = 9) | 89.40% [89.12%, 89.68%] | 91.54% [91.09%, 91.97%] | -7.60 | <0.001 | 40,335/45,116 | 14,035/15,332 |
|
||||
| cosine > 0.95 AND dHash_indep ≤ 15 (operational classifier rule, Section III-K) | 92.09% [91.84%, 92.34%] | 93.56% [93.16%, 93.93%] | -5.93 | <0.001 | 41,548/45,116 | 14,344/15,332 |
|
||||
|
||||
Calibration-fold thresholds: Firm A cosine median = 0.9862, P1 = 0.9067, P5 = 0.9407; dHash_indep median = 2, P95 = 9. Counts and z/p values are reproducible from the supplementary materials (fixed random seed).
|
||||
-->
|
||||
|
||||
We report fold-versus-fold comparisons rather than fold-versus-whole-sample comparisons, because the whole-sample rate is a weighted average of the two folds and therefore cannot, in general, fall inside the Wilson CI of either fold when the folds differ in rate; the correct generalization reference is the calibration fold, which produced the thresholds.
|
||||
|
||||
Under this proper test the two extreme rules agree across folds (cosine $> 0.837$ and $\text{dHash}_\text{indep} \leq 15$; both $p > 0.7$).
|
||||
The operationally relevant rules in the 85–95% capture band differ between folds by 1–5 percentage points ($p < 0.001$ given the $n \approx 45\text{k}/15\text{k}$ fold sizes).
|
||||
Both folds nevertheless sit in the same replication-dominated regime: every calibration-fold rate in the 85–99% range has a held-out counterpart in the 87–99% range, and the calibration-fold-adjacent reference rule cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 8$ (the integer cut immediately below the calibration-fold dHash P95 of 9) captures 89.40% of the calibration fold and 91.54% of the held-out fold; the operational classifier rule cosine $> 0.95$ AND $\text{dHash}_\text{indep} \leq 15$ used by the five-way classifier of Section III-K captures still higher rates in both folds (calibration 92.09%, 41,548 / 45,116; held-out 93.56%, 14,344 / 15,332).
|
||||
The modest fold gap is consistent with within-Firm-A heterogeneity in replication intensity: the random 30% CPA sample evidently contained proportionally more high-replication CPAs.
|
||||
We therefore interpret the held-out fold as confirming the qualitative finding (Firm A is strongly replication-dominated across both folds) while cautioning that exact rates carry fold-level sampling noise that a single 30% split cannot eliminate; the threshold-independent partner-ranking analysis (Section IV-G.2) is the cross-check that is robust to this fold variance.
|
||||
|
||||
### 3) Operational-Threshold Sensitivity: cos $> 0.95$ vs cos $> 0.945$
|
||||
|
||||
The per-signature classifier (Section III-K) uses cos $> 0.95$ as its operational cosine cut, anchored on the whole-sample Firm A P7.5 heuristic (i.e., 7.5% of whole-sample Firm A signatures lie at or below 0.95; see Section III-H).
|
||||
We report a sensitivity check in which this round-number cut is replaced by the slightly stricter calibration-fold P5 rounded value cos $> 0.945$ (calibration-fold P5 = 0.9407, see Table XI).
|
||||
Table XII reports the five-way classifier output under each cut.
|
||||
|
||||
<!-- TABLE XII: Classifier Sensitivity to the Operational Cosine Cut (All-Sample Five-Way Output, N = 168,740 signatures)
|
||||
| Cosine cut | High-confidence | Moderate-confidence | High style consistency | Uncertain | Likely hand-signed |
|
||||
|------------|-----------------|---------------------|------------------------|-----------|--------------------|
|
||||
| cos > 0.940 | 81,069 (48.04%) | 55,308 (32.78%) | 801 (0.47%) | 31,026 (18.39%) | 536 (0.32%) |
|
||||
| cos > 0.945 | 79,278 (46.98%) | 50,001 (29.63%) | 665 (0.39%) | 38,260 (22.67%) | 536 (0.32%) |
|
||||
| cos > 0.950 (operational) | 76,984 (45.62%) | 43,906 (26.02%) | 546 (0.32%) | 46,768 (27.72%) | 536 (0.32%) |
|
||||
| cos > 0.960 | 70,250 (41.63%) | 29,450 (17.45%) | 288 (0.17%) | 68,216 (40.43%) | 536 (0.32%) |
|
||||
| cos > 0.970 | 60,247 (35.70%) | 14,865 ( 8.81%) | 117 (0.07%) | 92,975 (55.10%) | 536 (0.32%) |
|
||||
| cos > 0.985 | 37,368 (22.15%) | 2,231 ( 1.32%) | 10 (0.01%) | 128,595 (76.21%) | 536 (0.32%) |
|
||||
|
||||
The dHash band edges ($\leq 5$ for high-confidence, $5 < \text{dHash}_\text{indep} \leq 15$ for moderate-confidence, $> 15$ for style) are held fixed across the grid; only the cosine cut varies. The Likely-hand-signed count is invariant across the grid because it depends only on the all-pairs KDE crossover cosine $= 0.837$.
|
||||
-->
|
||||
|
||||
At the aggregate firm-level, the calibration-fold-adjacent reference dual rule cos $> 0.95$ AND $\text{dHash}_\text{indep} \leq 8$ captures 89.95% of whole Firm A under the 0.95 cut and 91.14% under the 0.945 cut---a shift of 1.19 percentage points.
|
||||
The operational classifier rule cos $> 0.95$ AND $\text{dHash}_\text{indep} \leq 15$ used by the five-way classifier of Section III-K captures 92.46% under the 0.95 cut and 93.97% under the 0.945 cut---a shift of 1.51 percentage points.
|
||||
|
||||
Reading the wider grid in Table XII: the High-confidence and Moderate-confidence shares shift by less than 5 percentage points across the 0.940-0.950 neighbourhood, while pushing the cosine cut to 0.970 or 0.985 produces qualitatively different classifier behaviour (Moderate-confidence collapses from 26.02% at $0.95$ to 8.81% at $0.97$ and 1.32% at $0.985$, with the displaced mass landing in Uncertain rather than reclassifying out of the corpus).
|
||||
The classifier output is therefore robust to small (~0.005-cosine) perturbations of the operational cut but not to wholesale reanchoring at the threshold-estimator outputs of Section IV-D, which is consistent with our reading that those outputs are not classifier thresholds.
|
||||
At the per-signature categorization level, replacing 0.95 by 0.945 reclassifies 8,508 signatures (5.04% of the corpus) out of the Uncertain band; 6,095 of them migrate to Moderate-confidence non-hand-signed, 2,294 to High-confidence non-hand-signed, and 119 to High style consistency.
|
||||
The Likely-hand-signed category is unaffected because it depends only on the fixed all-pairs KDE crossover cosine $= 0.837$.
|
||||
The High-confidence non-hand-signed share grows from 45.62% to 46.98%.
|
||||
|
||||
We interpret this sensitivity pattern as indicating that the classifier's aggregate and high-confidence output is robust to the choice of operational cut within a 0.005-cosine neighbourhood of the Firm A P7.5 anchor, and that the movement is concentrated at the Uncertain/Moderate-confidence boundary.
|
||||
|
||||
To make the operating-point selection (Section III-K) auditable rather than presented as a single fixed value, Table XII-B reports the capture-vs-FAR tradeoff over the candidate threshold grid spanning the calibration-fold P5 (0.9407), its rounded value (0.945), the operational anchor (0.95), the Firm A Beta-2 forced-fit crossing from Section IV-D.3 (0.977), and the BD/McCrary candidate transition from Section IV-D.2 (0.985).
|
||||
For each grid point we report Firm A capture (under both the cosine-only marginal and the operational dual rule cos $> t$ AND $\text{dHash}_\text{indep} \leq 15$ used by the five-way classifier of Section III-K), non-Firm-A capture (the cosine-only marginal in the 108,292 non-Firm-A matched signatures), and inter-CPA FAR with Wilson 95% CI against the 50,000-pair anchor of Section IV-F.1.
|
||||
|
||||
<!-- TABLE XII-B: Cosine-Threshold Tradeoff: Capture vs Inter-CPA FAR
|
||||
| Cosine cut t | Firm A capture (cos > t) | Firm A capture (cos > t AND dHash_indep ≤ 15) | Non-Firm-A capture (cos > t) | Inter-CPA FAR | Inter-CPA FAR Wilson 95% CI |
|
||||
|--------------|--------------------------|------------------------------------------------|------------------------------|---------------|------------------------------|
|
||||
| 0.9407 (calibration-fold P5) | 95.15% (57,518/60,448) | 95.09% (57,482/60,448) | 72.68% (78,710/108,292) | 0.00126 | [0.00099, 0.00161] |
|
||||
| 0.945 (calibration-fold P5 rounded) | 94.02% (56,836/60,448) | 93.97% (56,804/60,448) | 67.51% (73,108/108,292) | 0.00082 | [0.00061, 0.00111] |
|
||||
| 0.95 (whole-sample Firm A P7.5; **operational cut**) | **92.51%** (55,922/60,448) | **92.46%** (55,892/60,448) | 60.50% (65,514/108,292) | **0.00050** | [0.00034, 0.00074] |
|
||||
| 0.977 (Firm A Beta-2 forced-fit crossing) | 74.53% (45,050/60,448) | 74.51% (45,038/60,448) | 13.14% (14,233/108,292) | 0.00014 | [0.00007, 0.00029] |
|
||||
| 0.985 (BD/McCrary candidate transition) | 55.27% (33,409/60,448) | 55.26% (33,406/60,448) | 5.73% (6,200/108,292) | 0.00004 | [0.00001, 0.00015] |
|
||||
|
||||
Inter-CPA FAR computed against 50,000 i.i.d. inter-CPA pairs (random seed 42, reproducing the anchor of Section IV-F.1 / Table X). Capture and FAR percentages are exact ratios of the displayed integer counts; gap arithmetic in the surrounding prose is computed from those exact counts and rounded to two decimal places. The dual-rule column is the operational classifier rule of Section III-K; for cuts above the dHash-15 saturation point (Firm A dHash$_\text{indep}$ $> 15$ rate is only 0.17%, Table IX), the dual-rule and cosine-only columns coincide to within the dHash$_\text{indep}$ $> 15$ residual.
|
||||
-->
|
||||
|
||||
Reading Table XII-B, three patterns motivate the choice of $0.95$ as the operating point.
|
||||
First, *Firm A capture* on the operational dual rule decays smoothly from 95.09% at $t = 0.9407$ to 55.26% at $t = 0.985$.
|
||||
Relaxing the cut from $0.95$ to $0.945$ buys 1.51 percentage points of additional Firm A capture, and to $0.9407$ buys 2.63 percentage points; tightening from $0.95$ to $0.977$ costs 17.96 percentage points and to $0.985$ costs 37.20 percentage points.
|
||||
The selected cut at $0.95$ is the strictest cut on this grid at which Firm A capture remains above $90\%$ on the operational dual rule.
|
||||
Second, *inter-CPA FAR* is small in absolute terms across the entire candidate grid ($0.00126$ at $0.9407$, falling to $0.00004$ at $0.985$): under any of these operating points the classifier's specificity against random cross-CPA pairs is in the per-mille range or better, so FAR alone does not determine the choice.
|
||||
The marginal FAR cost of relaxing from $0.95$ to $0.945$ is $+0.00032$ ($25 \to 41$ false positives per 50,000 pairs) and to $0.9407$ is $+0.00076$ ($25 \to 63$); the marginal FAR savings from tightening to $0.977$ and $0.985$ are $-0.00036$ and $-0.00046$ respectively.
|
||||
The FAR savings from going stricter are small in absolute terms compared with the corresponding Firm A capture loss, which makes $0.95$ a balanced operating point on this grid rather than a uniquely optimal one.
|
||||
Third, *non-Firm-A capture* (the cosine-only marginal in the 108,292 non-Firm-A signatures) decays from 67.51% at $0.945$ to 60.50% at $0.95$, 13.14% at $0.977$, and 5.73% at $0.985$.
|
||||
The Firm-A-minus-non-Firm-A gap widens with strictness through $0.977$ and then contracts (22.41 percentage points at $0.9407$; 26.46 at $0.945$; 31.97 at $0.95$; 61.36 at $0.977$; 49.54 at $0.985$): on the $0.95 \to 0.977$ segment non-Firm-A capture falls faster than Firm A capture in absolute terms ($-47.35$ vs $-17.96$ percentage points), so the widening is dominated by non-Firm-A removal rather than by an intrinsic property of Firm A; on the $0.977 \to 0.985$ segment Firm A capture falls faster than non-Firm-A's already-low residual, so the gap contracts.
|
||||
We do *not* read the gap pattern as evidence for a particular cut; it is reported here as cross-firm replication heterogeneity rather than as a selection criterion.
|
||||
The operating point at $0.95$ is therefore a defensible---not unique---selection in this neighbourhood, motivated by (i) keeping Firm A capture above $90\%$ on the operational dual rule, (ii) achieving an FAR of $0.0005$ at which marginal further savings from tightening are small relative to the corresponding capture loss, and (iii) preserving the interpretive transparency of the whole-sample Firm A P7.5 reading.
|
||||
It is *not* derived from the threshold-estimator outputs of Section IV-D, which the data do not support as classifier thresholds.
|
||||
|
||||
The paper therefore retains cos $> 0.95$ as the primary operational cut and reports the 0.945 result of Table XII as a sensitivity check rather than as a deployed alternative; downstream document-level rates (Table XVII) and intra-report agreement (Table XVI) are robust to moderate cutoff shifts within the 0.945--0.95 neighbourhood as long as the same cutoff is applied uniformly across firms.
|
||||
|
||||
## G. Additional Firm A Benchmark Validation
|
||||
|
||||
Before presenting the three threshold-robust analyses, Fig. 4 summarises the per-firm yearly per-signature best-match cosine distribution that motivates them.
|
||||
The left panel reports the mean per-signature best-match cosine within each firm bucket and fiscal year (a threshold-free statistic); the right panel reports the share of each firm-bucket-year with per-signature best-match cosine $\geq 0.95$ (the operational cut of Section III-K).
|
||||
Both panels show Firm A above the other Big-4 firms in every year of the 2013-2023 sample, with non-Big-4 firms below all four Big-4 firms throughout, and the cross-firm ordering is stable across the sample period.
|
||||
The mean-cosine separation between Firm A and the other Big-4 firms is on the order of 0.02-0.04 throughout the sample (e.g., 2013: Firm A $0.9733$ vs Firm B $0.9498$, Firm C $0.9464$, Firm D $0.9395$, Non-Big-4 $0.9227$; 2023: $0.9860$ vs $0.9668$, $0.9662$, $0.9525$, $0.9346$); the share-above-0.95 separation is wider (2013: Firm A $87.2\%$ vs $61.8\%$, $56.2\%$, $38.5\%$, $27.5\%$).
|
||||
This visual is the most direct cross-firm evidence in the paper that Firm A's high-similarity behaviour is firm-specific rather than corpus-wide; the three subsections below decompose this gap along three threshold-free or threshold-robust dimensions.
|
||||
|
||||
<!-- FIGURE 4: Per-firm yearly per-signature best-match cosine
|
||||
File: reports/figures/fig_yearly_big4_comparison.png (and .pdf)
|
||||
Generated by: signature_analysis/30_yearly_big4_comparison.py
|
||||
Caption: Per-firm yearly per-signature best-match cosine, 2013-2023.
|
||||
(a) Mean per-signature best-match cosine by firm bucket and fiscal year
|
||||
(threshold-free). (b) Share of per-signature best-match cosine $\geq 0.95$
|
||||
(operational cut of Section III-K). Five lines: Firm A, B, C, D, Non-Big-4.
|
||||
Firm A is above the other Big-4 firms in every year; Non-Big-4 is below all
|
||||
four Big-4 firms in every year. Per-firm signature counts and exact values
|
||||
are in `reports/firm_yearly_comparison/firm_yearly_comparison.{json,md}`.
|
||||
-->
|
||||
|
||||
The capture rates of Section IV-E are an *internal* consistency check: they ask "how much of Firm A does our threshold capture?", but the threshold was itself derived from Firm A's percentiles, so a high capture rate is not surprising.
|
||||
To go beyond this circular check, we report three further analyses, each chosen so that the *informative quantity* does not depend on the threshold's absolute value:
|
||||
|
||||
- **§IV-G.1 (year-by-year stability).** Holds the cosine cutoff fixed at 0.95 and asks whether the share of Firm A below the cutoff is *stable across years*. The information is in the temporal trend, not in the absolute rate; under a noise-only explanation of the left tail, the share should shrink as scan/PDF technology matured.
|
||||
- **§IV-G.2 (partner-level similarity ranking).** Uses *no threshold at all*: every auditor-year is ranked by mean similarity, and we measure Firm A's share of the top decile against its baseline share. The information is in the concentration ratio, which is invariant to the choice of cutoff.
|
||||
- **§IV-G.3 (intra-report agreement).** Applies the calibrated classifier and measures whether the *two co-signing CPAs on the same Firm A report* receive the same classifier label, then compares Firm A's intra-report agreement rate to the other firms'. The information is in the *cross-firm gap*; the absolute agreement rate at any one firm depends on the cutoff, but the gap is robust to moderate cutoff shifts as long as the same cutoff is applied uniformly across firms.
|
||||
|
||||
Together these three analyses provide threshold-free or threshold-robust evidence that complements the within-sample capture rates of Section IV-E.
|
||||
|
||||
### 1) Year-by-Year Stability of the Firm A Left Tail
|
||||
|
||||
Table XIII reports the proportion of Firm A signatures with per-signature best-match cosine below 0.95, disaggregated by fiscal year.
|
||||
Under the replication-dominated interpretation (Section III-H), this signature-level left-tail rate reflects within-firm heterogeneity in signing outputs at Firm A.
|
||||
Consistent with the scope-of-claims framing in Section III-G, we report the rate as a signature-level quantity without disaggregating the underlying mechanism (which may span a minority of hand-signing partners, multi-template replication workflows within the firm, or a combination); partner-level mechanism attribution is not attempted.
|
||||
Under the alternative hypothesis that the left tail is an artifact of scan or compression noise, the share should shrink as scanning and PDF-compression technology improved over 2013-2023.
|
||||
|
||||
<!-- TABLE XIII: Firm A Per-Year Cosine Distribution
|
||||
| Year | N sigs | mean best-match cosine | % below 0.95 |
|
||||
|------|--------|-------------|--------------|
|
||||
| 2013 | 2,167 | 0.9733 | 12.78% |
|
||||
| 2014 | 5,256 | 0.9781 | 8.69% |
|
||||
| 2015 | 5,484 | 0.9793 | 7.46% |
|
||||
| 2016 | 5,739 | 0.9811 | 6.92% |
|
||||
| 2017 | 5,796 | 0.9814 | 6.69% |
|
||||
| 2018 | 5,986 | 0.9808 | 6.58% |
|
||||
| 2019 | 6,122 | 0.9780 | 8.71% |
|
||||
| 2020 | 6,122 | 0.9770 | 9.46% |
|
||||
| 2021 | 5,996 | 0.9792 | 8.37% |
|
||||
| 2022 | 5,918 | 0.9819 | 6.25% |
|
||||
| 2023 | 5,862 | 0.9860 | 3.75% |
|
||||
-->
|
||||
|
||||
The left tail is stable at 6-13% throughout the sample period and shows no pre/post-2020 level shift: the 2013-2019 mean left-tail share is 8.26% and the 2020-2023 mean is 6.96%.
|
||||
The lowest observed share is in 2023 (3.75%), consistent with firm-level electronic signing systems producing more uniform output than earlier manual scanning-and-stamping, not less.
|
||||
This stability supports the replication-dominated framing: a persistent within-firm heterogeneity component is consistent with a Beta left tail that is stable across production technologies, whereas a noise-only explanation would predict a shrinking share as technology improved.
|
||||
|
||||
### 2) Partner-Level Similarity Ranking
|
||||
|
||||
If Firm A applies firm-wide stamping while the other Big-4 firms use stamping only for a subset of partners, Firm A auditor-years should disproportionately occupy the top of the similarity distribution among all auditor-years (across all firms).
|
||||
We test this prediction directly.
|
||||
|
||||
For each auditor-year (CPA $\times$ fiscal year) with at least 5 signatures we compute the mean best-match cosine similarity across the year's signatures, yielding 4,629 auditor-years across 2013-2023.
|
||||
Firm A accounts for 1,287 of these (27.8% baseline share).
|
||||
Table XIV reports per-firm occupancy of the top $K\%$ of the ranked distribution.
|
||||
The per-signature best-match cosine underlying each auditor-year mean is taken over the full same-CPA pool (Section III-G), consistent with the unit-of-analysis framing in Section III-G.
|
||||
|
||||
<!-- TABLE XIV: Top-K Similarity Rank Occupancy by Firm (pooled 2013-2023)
|
||||
| Top-K | k in bucket | Firm A | Firm B | Firm C | Firm D | Non-Big-4 | Firm A share |
|
||||
|-------|-------------|--------|--------|--------|--------|-----------|--------------|
|
||||
| 10% | 462 | 443 | 2 | 3 | 0 | 14 | 95.9% |
|
||||
| 20% | 925 | 877 | 9 | 14 | 2 | 23 | 94.8% |
|
||||
| 25% | 1,157 | 1,043 | 32 | 23 | 9 | 50 | 90.1% |
|
||||
| 30% | 1,388 | 1,129 | 105 | 52 | 25 | 77 | 81.3% |
|
||||
| 50% | 2,314 | 1,220 | 473 | 273 | 102 | 246 | 52.7% |
|
||||
-->
|
||||
|
||||
Firm A occupies 95.9% of the top 10%, 94.8% of the top 20%, 90.1% of the top 25%, and 81.3% of the top 30% of auditor-years by similarity, against its baseline share of 27.8%---a concentration ratio of $3.5\times$ at the top decile, $3.4\times$ at the top quintile, and $2.9\times$ at the top tercile.
|
||||
Firm A's share decays monotonically as the bracket widens (95.9% $\to$ 94.8% $\to$ 90.1% $\to$ 81.3% $\to$ 52.7% across top-10/20/25/30/50%), and only at the top 50% does its share approach its baseline; the over-representation is therefore concentrated in the very top of the distribution rather than spread uniformly through the upper half.
|
||||
Year-by-year (Table XV), the top-10% Firm A share ranges from 88.4% (2020) to 100% (2013, 2014, 2017, 2018, 2019), showing that the concentration is stable across the sample period.
|
||||
|
||||
<!-- TABLE XV: Firm A Share of Top-K Similarity by Year (K = 10%, 20%, 30%)
|
||||
| Year | N auditor-years | Top-10% share | Top-20% share | Top-30% share | Firm A baseline |
|
||||
|------|-----------------|---------------|---------------|---------------|-----------------|
|
||||
| 2013 | 324 | 100.0% (32/32) | 98.4% (63/64) | 89.7% (87/97) | 32.4% |
|
||||
| 2014 | 399 | 100.0% (39/39) | 98.7% (78/79) | 82.4% (98/119) | 27.8% |
|
||||
| 2015 | 394 | 97.4% (38/39) | 96.2% (75/78) | 84.7% (100/118) | 27.7% |
|
||||
| 2016 | 413 | 95.1% (39/41) | 96.3% (79/82) | 81.3% (100/123) | 26.2% |
|
||||
| 2017 | 415 | 100.0% (41/41) | 97.6% (81/83) | 83.9% (104/124) | 27.2% |
|
||||
| 2018 | 434 | 100.0% (43/43) | 97.7% (84/86) | 80.0% (104/130) | 26.5% |
|
||||
| 2019 | 429 | 100.0% (42/42) | 97.6% (83/85) | 78.9% (101/128) | 27.0% |
|
||||
| 2020 | 430 | 88.4% (38/43) | 91.9% (79/86) | 76.0% (98/129) | 27.7% |
|
||||
| 2021 | 450 | 97.8% (44/45) | 96.7% (87/90) | 81.5% (110/135) | 28.7% |
|
||||
| 2022 | 467 | 93.5% (43/46) | 95.7% (89/93) | 84.3% (118/140) | 28.3% |
|
||||
| 2023 | 474 | 97.9% (46/47) | 94.7% (89/94) | 83.8% (119/142) | 27.4% |
|
||||
|
||||
Per-cell entries are "share (k_FirmA / k_total)". Top-25% and top-50% pooled values are reported in Table XIV; per-year top-25/50 columns are omitted from this table to reduce visual width but are reproducible from the supplementary materials.
|
||||
-->
|
||||
|
||||
This over-representation is consistent with firm-wide non-hand-signing practice at Firm A and is not derived from any threshold we subsequently calibrate.
|
||||
It therefore constitutes genuine cross-firm evidence for Firm A's benchmark status.
|
||||
|
||||
### 3) Intra-Report Consistency
|
||||
|
||||
Taiwanese statutory audit reports are co-signed by two engagement partners (a primary and a secondary signer).
|
||||
Under firm-wide stamping practice at a given firm, both signers on the same report should receive the same signature-level classification.
|
||||
Disagreement between the two signers on a report is informative about whether the stamping practice is firm-wide or partner-specific.
|
||||
|
||||
For each report with exactly two signatures and complete per-signature data (84,354 reports total: 83,970 single-firm reports, in which both signers are at the same firm, and 384 mixed-firm reports, in which the two signers are at different firms), we classify each signature using the dual-descriptor rules of Section III-K and record whether the two classifications agree.
|
||||
Table XVI reports per-firm intra-report agreement for the 83,970 single-firm reports only (firm-assignment defined by the common firm identity of both signers); the 384 mixed-firm reports (0.46% of the 2-signature corpus) are excluded from the intra-report analysis because firm-level agreement is not well defined when the two signers are at different firms.
|
||||
|
||||
<!-- TABLE XVI: Intra-Report Classification Agreement by Firm
|
||||
| Firm | Total 2-signer reports | Both non-hand-signed | Both uncertain | Both style | Both hand-signed | Mixed | Agreement rate |
|
||||
|------|-----------------------|----------------------|----------------|------------|------------------|-------|----------------|
|
||||
| Firm A | 30,222 | 26,435 | 734 | 0 | 4 | 3,049 | **89.91%** |
|
||||
| Firm B | 17,121 | 9,260 | 2,159| 5 | 6 | 5,691 | 66.76% |
|
||||
| Firm C | 19,112 | 8,983 | 3,035| 3 | 5 | 7,086 | 62.92% |
|
||||
| Firm D | 8,375 | 3,028 | 2,376| 0 | 3 | 2,968 | 64.56% |
|
||||
| Non-Big-4 | 9,140 | 1,671 | 3,945| 18| 27| 3,479 | 61.94% |
|
||||
|
||||
A report is "in agreement" if both signature labels fall in the same coarse bucket
|
||||
(non-hand-signed = high+moderate; uncertain; style consistency; or likely hand-signed).
|
||||
-->
|
||||
|
||||
Firm A achieves 89.9% intra-report agreement, with 87.5% of Firm A reports having *both* signers classified as non-hand-signed and only 4 reports (0.01%) having both classified as likely hand-signed.
|
||||
The other Big-4 firms (B, C, D) and non-Big-4 firms cluster at 62-67% agreement, a 23-28 percentage-point gap.
|
||||
This 23-28 percentage-point gap in intra-report agreement between Firm A and the other firms is consistent with firm-wide (rather than partner-specific) non-hand-signing practice; we do not claim a sharp discontinuity in the formal sense, since classifier calibration, firm-specific document-production pipelines, and signer-mix differences could each contribute to gap magnitude.
|
||||
|
||||
We note that this test uses the calibrated classifier of Section III-K rather than a threshold-free statistic; the substantive evidence lies in the *cross-firm gap* between Firm A and the other firms rather than in the absolute agreement rate at any single firm, and that gap is robust to moderate shifts in the absolute cutoff so long as the cutoff is applied uniformly across firms.
|
||||
|
||||
## H. Classification Results
|
||||
|
||||
Table XVII presents the final classification results under the dual-descriptor framework with Firm A-calibrated thresholds for 84,386 documents (656 documents excluded from the 85,042-document YOLO-detection cohort because no signature on the document could be matched to a registered CPA; see Table XVII note).
|
||||
We emphasize that the document-level proportions below reflect the *worst-case aggregation rule* of Section III-K: a report carrying one stamped signature and one hand-signed signature is labeled with the most-replication-consistent of the two signature-level verdicts.
|
||||
Document-level rates therefore represent the share of reports in which *at least one* signature is non-hand-signed rather than the share in which *both* are; the intra-report agreement analysis of Section IV-G.3 (Table XVI) reports how frequently the two co-signers share the same signature-level label within each firm, so that readers can judge what fraction of the non-hand-signed document-level share corresponds to fully non-hand-signed reports versus mixed reports.
|
||||
|
||||
<!-- TABLE XVII: Document-Level Classification (Dual-Descriptor: Cosine + dHash)
|
||||
| Verdict | N (PDFs) | % | Firm A | Firm A % |
|
||||
|---------|----------|---|--------|----------|
|
||||
| High-confidence non-hand-signed | 29,529 | 35.0% | 22,970 | 76.0% |
|
||||
| Moderate-confidence non-hand-signed | 36,994 | 43.8% | 6,311 | 20.9% |
|
||||
| High style consistency | 5,133 | 6.1% | 183 | 0.6% |
|
||||
| Uncertain | 12,683 | 15.0% | 758 | 2.5% |
|
||||
| Likely hand-signed | 47 | 0.1% | 4 | 0.0% |
|
||||
|
||||
Per the worst-case aggregation rule of Section III-K, a document with two signatures inherits the most-replication-consistent of the two signature-level labels.
|
||||
The 84,386-document cohort excludes 656 documents (relative to the 85,042 YOLO-detected cohort of Table III) for which no signature could be matched to a registered CPA: the per-document classifier requires at least one CPA-matched signature so that a same-CPA best-match similarity is defined. The exclusion is definitional rather than discretionary; typical causes are auditor's-report-page formats deviating from the standard two-signature layout, or OCR returning a printed CPA name not present in the registry.
|
||||
-->
|
||||
|
||||
Within the 71,656 documents exceeding cosine $0.95$, the dHash dimension stratifies them into three distinct populations:
|
||||
29,529 (41.2%) show converging structural evidence of non-hand-signing (dHash $\leq 5$);
|
||||
36,994 (51.7%) show partial structural similarity (dHash in $[6, 15]$) consistent with replication degraded by scan variations;
|
||||
and 5,133 (7.2%) show no structural corroboration (dHash $> 15$), suggesting high signing consistency rather than image reproduction.
|
||||
A cosine-only classifier would treat all 71,656 identically; the dual-descriptor framework separates them into populations with fundamentally different interpretations.
|
||||
|
||||
### 1) Firm A Capture Profile (Consistency Check)
|
||||
|
||||
96.9% of Firm A's documents fall into the high- or moderate-confidence non-hand-signed categories, 0.6% into high-style-consistency, and 2.5% into uncertain.
|
||||
This pattern is consistent with the replication-dominated framing: the large majority is captured by non-hand-signed rules, while the small residual is consistent with the within-firm heterogeneity implied by the dip-test-confirmed unimodal-long-tail shape of Firm A's per-signature cosine distribution (Section IV-D.1) and the 7.5% signature-level left tail (Section III-H).
|
||||
The near-zero "likely hand-signed" rate (4 of 30,226 Firm A documents, 0.013%; the 30,226 denominator is documents with at least one Firm A signer under the 84,386-document classification cohort, which differs from the 30,222 single-firm two-signer subset of Table XVI by 4 mixed-firm reports excluded from the firm-level intra-report comparison) indicates that the within-firm heterogeneity implied by the 7.5% signature-level left tail (Section IV-D) does not project into the lowest-cosine document-level category under the dual-descriptor rules; it is absorbed instead into the uncertain or high-style-consistency categories at this threshold set.
|
||||
We note that because the non-hand-signed thresholds are themselves calibrated to Firm A's empirical percentiles (Section III-H), these rates are an internal consistency check rather than an external validation; the held-out Firm A validation of Section IV-F.2 is the corresponding external check.
|
||||
|
||||
### 2) Cross-Firm Comparison of Dual-Descriptor Convergence
|
||||
|
||||
Among the 65,514 non-Firm-A signatures with per-signature best-match cosine $> 0.95$, 42.12% have $\text{dHash}_\text{indep} \leq 5$, compared to 88.32% of the 55,922 Firm A signatures meeting the same cosine condition---a $\sim 2.1\times$ difference that the structural-verification layer makes visible.
|
||||
The Firm A denominator (55,922) matches Table IX exactly: both Table IX and the cross-firm decomposition define Firm A membership via the CPA registry (`accountants.firm`), and the cross-firm analysis additionally requires a non-null independent-min dHash record, which all 55,922 Firm A cosine-eligible signatures have in the current database.
|
||||
This cross-firm gap is consistent with firm-wide non-hand-signing practice at Firm A versus partner-specific or per-engagement replication at other firms; it complements the partner-level ranking (Section IV-G.2) and intra-report consistency (Section IV-G.3) findings.
|
||||
Reproduction artifact for these counts is listed in Appendix B.
|
||||
|
||||
## I. Ablation Study: Feature Backbone Comparison
|
||||
## D. Big-4 Accountant-Level Distributional Characterisation
|
||||
|
||||
This section reports the empirical evidence for §III-I's distributional diagnostics at the Big-4 accountant level. All numbers below are direct re-statements from Scripts 32 / 34. The accountant-level dip-test rejection reported in Table V is, per §III-I.4 (Scripts 39b–39e), fully attributable to between-firm location shifts and integer mass-point artefacts rather than to within-population bimodality; the v4-new composition-decomposition diagnostics that establish this finding are tabulated in §IV-M below alongside the anchor-based ICCR calibration.
|
||||
|
||||
**Table V.** Hartigan dip-test results, accountant-level marginals (Big-4 primary; comparison scopes from Script 32).
|
||||
|
||||
| Population | $n$ CPAs | $p_{\text{cos}}$ | $p_{\text{dHash}}$ | Interpretation |
|
||||
|---|---|---|---|---|
|
||||
| **Big-4 pooled (primary)** | 437 | $< 5 \times 10^{-4}$ | $< 5 \times 10^{-4}$ | reject unimodality on both axes |
|
||||
| Firm A pooled alone | 171 | 0.992 | 0.924 | unimodal |
|
||||
| Firms B + C + D pooled | 266 | 0.998 | 0.906 | unimodal |
|
||||
| All non-Firm-A pooled | 515 | 0.998 | 0.907 | unimodal |
|
||||
|
||||
Bootstrap implementation: $n_{\text{boot}} = 2000$; for the Big-4 cells, no bootstrap replicate exceeded the observed dip statistic, so the empirical $p$-value is bounded above by the bootstrap resolution $1 / 2000 = 5 \times 10^{-4}$ (Script 34 reports this as $p = 0.0000$; we report $p < 5 \times 10^{-4}$ to reflect the resolution). Single-firm dip statistics for Firms B, C, and D were not separately computed.
|
||||
|
||||
**Table VI.** Burgstahler-Dichev / McCrary density-smoothness diagnostic on accountant-level marginals (cosine in 0.002 bins; dHash in integer bins; $\alpha = 0.05$, two-sided).
|
||||
|
||||
| Population | Cosine: significant transition? | dHash: significant transition? |
|
||||
|---|---|---|
|
||||
| **Big-4 pooled (primary)** | none ($p > 0.05$) | none ($p > 0.05$) |
|
||||
| Firm A pooled alone | none | none |
|
||||
| Firms B + C + D pooled | none | one transition at $\overline{\text{dHash}} = 10.8$ |
|
||||
| All non-Firm-A pooled | none | one transition at $\overline{\text{dHash}} = 6.6$ |
|
||||
|
||||
The Big-4-scope null on both axes is consistent with the §IV-E mixture evidence: the K=3 components overlap in their tails rather than separating sharply, so a local-discontinuity test does not flag a transition. Outside Big-4, dHash transitions appear in some subsets but no cosine transition is identified in any tested subset (Script 32 sweeps; pre-2018 and post-2020 stratified variants exhibit dHash transitions at varying locations). These off-Big-4 dHash transitions are scope-dependent and are not used as v4.0 operational thresholds; we do not claim a specific structural interpretation for them without an explicit bin-width sensitivity sweep at those scopes.
|
||||
|
||||
## E. Big-4 K=2 / K=3 Mixture Fits
|
||||
|
||||
This section reports the K=2 and K=3 2D Gaussian mixture fits to the Big-4 accountant-level distribution and the bootstrap stability of their marginal crossings.
|
||||
|
||||
**Table VII.** Big-4 K=2 mixture components (descriptive partition; not mechanism clusters per §III-J) and marginal-crossing bootstrap 95% confidence intervals.
|
||||
|
||||
| K=2 component | $\overline{\text{cos}}$ | $\overline{\text{dHash}}$ | weight |
|
||||
|---|---|---|---|
|
||||
| K=2-a (low-cos / high-dHash position) | 0.954 | 7.14 | 0.689 |
|
||||
| K=2-b (high-cos / low-dHash position) | 0.983 | 2.41 | 0.311 |
|
||||
|
||||
Marginal crossings (point + bootstrap 95% CI, $n_{\text{boot}} = 500$):
|
||||
|
||||
| Axis | Point | Bootstrap median | 95% CI | CI half-width |
|
||||
|---|---|---|---|---|
|
||||
| cos | 0.9755 | 0.9754 | $[0.9742, 0.9772]$ | 0.0015 |
|
||||
| dHash | 3.755 | 3.763 | $[3.476, 3.969]$ | 0.246 |
|
||||
|
||||
$\text{BIC}(K{=}2) = -1108.45$ (Script 34).
|
||||
|
||||
**Table VIII.** Big-4 K=3 mixture components (descriptive firm-compositional partition per §III-J; not mechanism clusters).
|
||||
|
||||
| K=3 component | $\overline{\text{cos}}$ | $\overline{\text{dHash}}$ | weight | descriptive position |
|
||||
|---|---|---|---|---|
|
||||
| C1 | 0.9457 | 9.17 | 0.143 | low-cos / high-dHash corner |
|
||||
| C2 | 0.9558 | 6.66 | 0.536 | central region |
|
||||
| C3 | 0.9826 | 2.41 | 0.321 | high-cos / low-dHash corner |
|
||||
|
||||
$\text{BIC}(K{=}3) = -1111.93$, lower than $K{=}2$ by $3.48$ (mild support; not by itself decisive). The full-fit K=3 baseline above is reproduced in Scripts 35, 37, and 38 with identical hyperparameters; Script 37 additionally fits K=3 on each leave-one-firm-out training set (those fold-specific components differ from the full-fit baseline by design and are reported separately in §IV-G Table XIII). Operational use of the K=2 / K=3 fits is governed by §III-J and §III-L; §IV-G reports the LOOO reproducibility evidence that motivates reporting both fits descriptively.
|
||||
|
||||
## F. Convergent Internal-Consistency Checks
|
||||
|
||||
This section reports the empirical evidence for §III-K's three-score internal-consistency analysis. We re-emphasise the §III-K caveat: the three scores are deterministic functions of the same per-CPA descriptor pair $(\overline{\text{cos}}_a, \overline{\text{dHash}}_a)$ and are *not statistically independent measurements*. The pairwise correlations document internal consistency among feature-derived ranks rather than external validation against an independent ground truth.
|
||||
|
||||
**Table IX.** Per-CPA Spearman rank correlations among three feature-derived scores, Big-4, $n = 437$.
|
||||
|
||||
| Score pair | Spearman $\rho$ | $p$-value |
|
||||
|---|---|---|
|
||||
| K=3 P(C1) vs Paper A box-rule less-replication-dominated rate | $+0.9627$ | $< 10^{-248}$ |
|
||||
| Reverse-anchor cosine percentile vs Paper A box-rule less-replication-dominated rate | $+0.8890$ | $< 10^{-149}$ |
|
||||
| K=3 P(C1) vs Reverse-anchor cosine percentile | $+0.8794$ | $< 10^{-142}$ |
|
||||
|
||||
(Source: Script 38.) Reverse-anchor reference: 2D Gaussian fit by MCD (support fraction 0.85) on $n = 249$ non-Big-4 CPAs; reference centre $\overline{\text{cos}} = 0.935$, $\overline{\text{dHash}} = 9.77$.
|
||||
|
||||
**Table X.** Per-firm summary across the three feature-derived scores, Big-4.
|
||||
|
||||
| Firm | $n$ CPAs | mean $P(\text{C1})$ | mean reverse-anchor score | mean Paper A less-replication-dominated rate |
|
||||
|---|---|---|---|---|
|
||||
| Firm A | 171 | 0.0072 | $-0.9726$ | 0.1935 |
|
||||
| Firm B | 112 | 0.1410 | $-0.8201$ | 0.6962 |
|
||||
| Firm C | 102 | 0.3110 | $-0.7672$ | 0.7896 |
|
||||
| Firm D | 52 | 0.2406 | $-0.7125$ | 0.7608 |
|
||||
|
||||
(Source: Script 38 per-firm summary; reverse-anchor score is sign-flipped so that *higher* values indicate deeper into the reference left tail = less replication-dominated relative to the non-Big-4 reference.)
|
||||
|
||||
The three scores agree on placing Firm A as the most replication-dominated and the three non-Firm-A firms as less replication-dominated. The K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Score 1 and Score 3) place Firm C at the least-replication-dominated end of Big-4; the reverse-anchor cosine percentile (Score 2) ranks Firm D fractionally above Firm C. This residual within-Big-4-non-A disagreement is a design feature of the reverse-anchor metric: Score 2 measures only the marginal cosine percentile under the non-Big-4 reference, so a firm with a slightly higher cosine but a markedly different dHash distribution (Firm D vs Firm C) can score higher on Score 2 while scoring lower on Scores 1 and 3, both of which use both descriptors.
|
||||
|
||||
**Table XI.** Per-signature Cohen $\kappa$ (binary collapse, replication-dominated vs less-replication-dominated), $n = 150{,}442$ Big-4 signatures.
|
||||
|
||||
| Pair | Cohen $\kappa$ |
|
||||
|---|---|
|
||||
| Paper A binary high-confidence box rule (cos $> 0.95$ AND dHash $\leq 5$) vs per-CPA K=3 hard label | 0.662 |
|
||||
| Paper A binary high-confidence box rule vs per-signature K=3 hard label | 0.559 |
|
||||
| Per-CPA K=3 hard label vs per-signature K=3 hard label | 0.870 |
|
||||
|
||||
(Source: Script 39; verdict label `SIG_CONVERGENCE_MODERATE`.) Per-signature K=3 components ($n = 150{,}442$) sorted by ascending cosine: $(0.928, 9.75, 0.146)$ / $(0.963, 6.04, 0.582)$ / $(0.989, 1.27, 0.272)$, an absolute cosine drift of $0.018$ in C1 and $0.006$ in C3 relative to the per-CPA fit. These convergence checks cover only the binary high-confidence rule (cos $> 0.95$ AND dHash $\leq 5$); the five-way classifier's moderate-confidence band ($5 < \text{dHash} \leq 15$) inherits its v3.x calibration and capture-rate evaluation (§IV-J).
|
||||
|
||||
## G. Leave-One-Firm-Out Reproducibility
|
||||
|
||||
This section reports the firm-level cross-validation evidence motivating §III-J's "K=3 descriptive, not operational" framing.
|
||||
|
||||
**Table XII.** K=2 leave-one-firm-out across the four Big-4 folds.
|
||||
|
||||
| Held-out firm | $n_{\text{train}}$ | $n_{\text{held}}$ | Fold rule (cos cut, dHash cut) | Held-out classified as templated by fold rule |
|
||||
|---|---|---|---|---|
|
||||
| Firm A | 266 | 171 | cos $> 0.9380$ AND dHash $\leq 8.79$ | $171 / 171 = 100.00\%$ ($95\%$ Wilson $[97.80\%, 100.00\%]$) |
|
||||
| Firm B | 325 | 112 | cos $> 0.9744$ AND dHash $\leq 3.98$ | $0 / 112 = 0\%$ ($95\%$ Wilson $[0\%, 3.32\%]$) |
|
||||
| Firm C | 335 | 102 | cos $> 0.9752$ AND dHash $\leq 3.75$ | $0 / 102 = 0\%$ ($95\%$ Wilson $[0\%, 3.63\%]$) |
|
||||
| Firm D | 385 | 52 | cos $> 0.9756$ AND dHash $\leq 3.74$ | $0 / 52 = 0\%$ ($95\%$ Wilson $[0\%, 6.88\%]$) |
|
||||
|
||||
(Source: Script 36.) Across-fold cosine crossing: pairwise range $[0.9380, 0.9756]$, range = $0.0376$; max absolute deviation from the across-fold mean is $0.028$. This exceeds the report's $0.005$ across-fold stability tolerance by $5.6\times$ and is much larger than the full-Big-4 bootstrap CI half-width of $0.0015$. Together with the all-or-nothing held-out classification pattern (Firm A held out $\Rightarrow$ all held-out CPAs templated; any non-Firm-A firm held out $\Rightarrow$ none templated), this indicates the K=2 boundary is essentially a Firm-A-vs-others separator rather than a within-Big-4 mechanism boundary.
|
||||
|
||||
**Table XIII.** K=3 leave-one-firm-out: C1 component shape and held-out membership.
|
||||
|
||||
| Held-out firm | C1 cos (fit) | C1 dHash (fit) | C1 weight (fit) | Held-out C1 hard-label rate | Full-Big-4 baseline C1% | Absolute difference |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Full-Big-4 baseline | 0.9457 | 9.17 | 0.143 | — | — | — |
|
||||
| Firm A held out | 0.9425 | 10.13 | 0.145 | $4.68\%$ | $0.00\%$ | $4.68$ pp |
|
||||
| Firm B held out | 0.9441 | 9.16 | 0.127 | $7.14\%$ | $8.93\%$ | $1.76$ pp |
|
||||
| Firm C held out | 0.9504 | 8.41 | 0.126 | $36.27\%$ | $23.53\%$ | $12.77$ pp |
|
||||
| Firm D held out | 0.9439 | 9.29 | 0.120 | $17.31\%$ | $11.54\%$ | $5.81$ pp |
|
||||
|
||||
(Source: Script 37; verdict label `P2_PARTIAL`.) Component shape is reproducible across folds: max deviation of C1 cosine = $0.005$, C1 dHash = $0.96$, C1 weight = $0.023$. Hard-posterior membership for the held-out firm varies: max absolute difference from the full-Big-4 baseline is $12.77$ pp at the Firm C held-out fold, exceeding the report's $5$ pp viability bar. We accordingly do not use K=3 hard-posterior membership as an operational classifier label (§III-J, §III-L).
|
||||
|
||||
## H. Pixel-Identity Positive-Anchor Miss Rate
|
||||
|
||||
This section reports the only hard-ground-truth subset analysis available in the corpus: the positive-anchor miss rate against $n = 262$ Big-4 signatures whose nearest same-CPA match is byte-identical after crop and normalisation. Independent hand-signing cannot produce pixel-identical images, so byte-identical signatures are conservative-subset ground truth for the *replicated* class. The analysis is one-sided (positive-anchor only); a paired false-alarm rate against a hand-signed negative anchor is not available because no signature-level hand-signed ground truth exists in the corpus (§III-K item 4).
|
||||
|
||||
**Table XIV.** Positive-anchor miss rate, $n = 262$ Big-4 byte-identical signatures.
|
||||
|
||||
| Classifier | Misclassified as less-replication-dominated | Miss rate | Wilson 95% CI |
|
||||
|---|---|---|---|
|
||||
| Paper A binary high-confidence box rule (cos $> 0.95$ AND dHash $\leq 5$) | $0 / 262$ | $0\%$ | $[0\%, 1.45\%]$ |
|
||||
| K=3 per-CPA hard label (C3 = high-cos / low-dHash; descriptive) | $0 / 262$ | $0\%$ | $[0\%, 1.45\%]$ |
|
||||
| Reverse-anchor (prevalence-calibrated cut) | $0 / 262$ | $0\%$ | $[0\%, 1.45\%]$ |
|
||||
|
||||
(Source: Script 40.) Per-firm breakdown of the byte-identical subset: Firm A 145; Firm B 8; Firm C 107; Firm D 2. All three candidate scores correctly assign every byte-identical signature to the replicated class.
|
||||
|
||||
We caution that for the Paper A box rule this result is close to tautological (byte-identical nearest-neighbour signatures have cosine $\approx 1$ and dHash $\approx 0$, well inside the rule's high-confidence region); v3.20.0 §V-F discusses this conservative-subset caveat at length and we retain that discussion. The reverse-anchor cut is chosen by *prevalence calibration* against the inherited box rule's overall replicated rate of $49.58\%$ across Big-4 signatures; this is a documented v4.0 limitation since no signature-level hand-signed ground truth exists to permit direct ROC optimisation.
|
||||
|
||||
## I. Inter-CPA Pair-Level Coincidence Rate (Big-4 spike + inherited corpus-wide)
|
||||
|
||||
The signature-level inter-CPA pair-level coincidence-rate analysis (reported in v3.20.0 §IV-F.1, Table X as "FAR") is inherited and extended in v4.0. v4.0 retroactively reframes the metric as **inter-CPA pair-level coincidence rate (ICCR)** rather than "False Acceptance Rate" because the corpus does not provide signature-level ground-truth negative labels; the inter-CPA negative-anchor assumption underpinning the metric is itself partially violated by within-firm cross-CPA template-like collision structures (§III-L.4). The v3.20.0 corpus-wide spike on $\sim 50{,}000$ inter-CPA pairs reported a per-comparison rate of $0.0005$ (Wilson 95% CI $[0.0003, 0.0007]$) at the cosine cut $0.95$.
|
||||
|
||||
v4.0 additionally reports the §III-L.1 Big-4-scope spike at higher sample size ($5 \times 10^5$ inter-CPA pairs; Script 40b), which replicates and extends the v3 result and adds the structural dimension (dHash) and joint-rule rates. The §III-L.1 numbers are referenced rather than duplicated here; the consolidated v4-new ICCR calibration appears in §IV-M Tables XXI–XXVI.
|
||||
|
||||
## J. Five-Way Per-Signature + Document-Level Classification Output
|
||||
|
||||
This section reports the §III-L five-way per-signature + document-level worst-case classifier output on the Big-4 sub-corpus. The five-way category definitions are inherited unchanged from v3.20.0 §III-K (now §III-L); see §III-L for the cosine and dHash cuts.
|
||||
|
||||
**Table XV.** Five-way per-signature category counts, Big-4 sub-corpus, $n = 150{,}442$ classified.
|
||||
|
||||
| Category | Long name | $n$ signatures | % of classified |
|
||||
|---|---|---|---|
|
||||
| HC | High-confidence non-hand-signed | 74,593 | 49.58% |
|
||||
| MC | Moderate-confidence non-hand-signed | 39,817 | 26.47% |
|
||||
| HSC | High style consistency | 314 | 0.21% |
|
||||
| UN | Uncertain | 35,480 | 23.58% |
|
||||
| LH | Likely hand-signed | 238 | 0.16% |
|
||||
|
||||
(Source: Script 42; 11 of 150,453 loaded Big-4 signatures lacked one or both descriptors and were excluded. The $150{,}442$ vs $150{,}453$ distinction — descriptor-complete vs vector-complete — recurs across §IV: descriptor-complete analyses (§IV-D through §IV-J, all using accountant-level aggregates or per-signature category counts derived from the same 150,442-signature substrate) use $n = 150{,}442$; vector- or pair-recomputed analyses (§IV-M.2 Table XXI, §IV-M.3 Table XXII, §IV-M.5 Tables XXIV–XXV; Scripts 40b, 43, 44) use $n = 150{,}453$ because their pair- or pool-level computations load all vector-complete signatures including those failing the descriptor-complete filter. See §III-G for the sample-size reconciliation.)
|
||||
|
||||
**Per-firm five-way breakdown (% within firm).**
|
||||
|
||||
| Firm | HC | MC | HSC | UN | LH | total signatures |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Firm A | 81.70% | 10.76% | 0.05% | 7.42% | 0.07% | 60,448 |
|
||||
| Firm B | 34.56% | 35.88% | 0.29% | 29.09% | 0.18% | 34,248 |
|
||||
| Firm C | 23.75% | 41.44% | 0.38% | 34.21% | 0.22% | 38,613 |
|
||||
| Firm D | 24.51% | 29.33% | 0.22% | 45.65% | 0.29% | 17,133 |
|
||||
|
||||
(Source: Script 42 per-firm cross-tab.) The per-firm pattern qualitatively aligns with the K=3 cluster cross-tab of Table XVI: Firm A's signatures concentrate in the HC band (81.70%) while its CPAs concentrate at the accountant level in the K=3 C3 (high-cos / low-dHash) component (82.46%; Table XVI). These two figures address different units (per-signature classification vs per-CPA hard cluster assignment) and are not directly comparable as a like-for-like consistency check; we report the qualitative alignment but do not infer a numerical equivalence. The three non-Firm-A Big-4 firms have markedly lower HC rates than Firm A and substantially higher Uncertain rates, with Firm D having the highest Uncertain rate (45.65%).
|
||||
|
||||
**Document-level worst-case aggregation.** Each audit report typically carries two certifying-CPA signatures. We aggregate signature-level outcomes to document-level labels using the v3.20.0 worst-case rule (HC > MC > HSC > UN > LH; §III-L). v4.0 does not change this aggregation rule; only the population over which it is computed changes (Big-4 subset).
|
||||
|
||||
**Table XIX.** Document-level worst-case category counts, Big-4 sub-corpus, $n = 75{,}233$ unique PDFs.
|
||||
|
||||
| Category | Long name | $n$ documents | % |
|
||||
|---|---|---|---|
|
||||
| HC | High-confidence non-hand-signed | 46,857 | 62.28% |
|
||||
| MC | Moderate-confidence non-hand-signed | 19,667 | 26.14% |
|
||||
| HSC | High style consistency | 167 | 0.22% |
|
||||
| UN | Uncertain | 8,524 | 11.33% |
|
||||
| LH | Likely hand-signed | 18 | 0.02% |
|
||||
|
||||
(Source: Script 42 document-level table; 379 of 75,233 PDFs carried signatures from more than one Big-4 firm and are reported in the single-firm-PDF per-firm breakdown of the script CSV but pooled into the overall counts here.)
|
||||
|
||||
**Per-firm document-level breakdown (single-firm PDFs only).**
|
||||
|
||||
| Firm | HC | MC | HSC | UN | LH | total docs |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Firm A | 27,600 | 1,857 | 7 | 758 | 4 | 30,226 |
|
||||
| Firm B | 8,783 | 6,079 | 57 | 2,202 | 6 | 17,127 |
|
||||
| Firm C | 7,281 | 8,660 | 77 | 3,099 | 5 | 19,122 |
|
||||
| Firm D | 3,100 | 2,838 | 22 | 2,416 | 3 | 8,379 |
|
||||
|
||||
(Source: Script 42; mixed-firm PDFs $n = 379$ excluded from the per-firm rows but included in the overall counts above.)
|
||||
|
||||
The five-way **moderate-confidence non-hand-signed** band (cos $> 0.95$ AND $5 < \text{dHash} \leq 15$) inherits its v3.x calibration; it is **not separately validated by Scripts 38–40**, which evaluated only the binary high-confidence rule (cos $> 0.95$ AND dHash $\leq 5$). v4.0 does not re-derive the moderate-band cuts on the Big-4 subset; we report the Table XV per-firm MC proportions (10.76% / 35.88% / 41.44% / 29.33% across Firms A through D) descriptively. The v3.20.0 capture-rate calibration evidence for the moderate band (v3.20.0 Tables IX, XI, XII, XII-B) is carried into v4.0 by reference and not regenerated on the Big-4 subset. We do not claim that the MC-band per-firm ordering above is a separate validation of the §III-K Spearman convergence, since MC occupancy is not a monotone function of the per-CPA less-replication-dominated ranking (e.g., Firm D's MC fraction is lower than Firm B's while Firm D's reverse-anchor score ranks it as less replication-dominated than Firm B).
|
||||
|
||||
**Table XVI.** Firm × K=3 cluster cross-tabulation, Big-4 sub-corpus.
|
||||
|
||||
| Firm | $n$ | C1 (low-cos / high-dHash) | C2 (central) | C3 (high-cos / low-dHash) | C1 % | C3 % |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Firm A | 171 | 0 | 30 | 141 | $0.00\%$ | $82.46\%$ |
|
||||
| Firm B | 112 | 10 | 102 | 0 | $8.93\%$ | $0.00\%$ |
|
||||
| Firm C | 102 | 24 | 77 | 1 | $23.53\%$ | $0.98\%$ |
|
||||
| Firm D | 52 | 6 | 45 | 1 | $11.54\%$ | $1.92\%$ |
|
||||
|
||||
(Source: Script 35.) The cross-tab is the accountant-level descriptive output of the K=3 mixture (§III-J / §IV-E). It is reported here as a complement to the five-way per-signature classifier (Table XV), not as an operational classifier output. Reading: Firm A's CPAs are concentrated in the C3 (high-cos / low-dHash) component (no Firm A CPAs in C1); Firm C has the highest C1 (low-cos / high-dHash) concentration of the Big-4 (C1 fraction $23.5\%$); Firms B and D sit between A and C on the K=3 hard-label ordering, broadly consistent with the per-firm Spearman ordering of Table X (with the within-Big-4-non-A reverse-anchor disagreement noted there).
|
||||
|
||||
**Document-level worst-case aggregation outputs are reported in Table XIX above.**
|
||||
|
||||
## K. Full-Dataset Robustness (light scope)
|
||||
|
||||
This section reports the v4.0 reproducibility cross-check at the full accountant scope ($n = 686$ CPAs, Big-4 plus mid/small firms). The scope of §IV-K is deliberately narrow: we re-run only the K=3 mixture + Paper A operational-rule per-CPA less-replication-dominated rate analysis, sufficient to demonstrate that the v4.0 K=3 + Paper A convergence reproduces at the wider scope. The §III-L five-way classifier and the §IV-G LOOO analyses are not re-run at the full scope. The five-way moderate-confidence band is documented as inherited from v3.x calibration in §IV-J.
|
||||
|
||||
**Table XVII.** K=3 component comparison, Big-4 sub-corpus vs full dataset.
|
||||
|
||||
| K=3 component | Big-4 (n=437) cos / dHash / weight | Full (n=686) cos / dHash / weight | Drift Big-4 → Full |
|
||||
|---|---|---|---|
|
||||
| C1 (low-cos / high-dHash) | 0.9457 / 9.17 / 0.143 | 0.9278 / 11.17 / 0.284 | $\lvert\Delta\rvert$ cos 0.018, dHash 1.99, wt 0.141 |
|
||||
| C2 (central) | 0.9558 / 6.66 / 0.536 | 0.9535 / 6.99 / 0.512 | $\lvert\Delta\rvert$ cos 0.002, dHash 0.33, wt 0.024 |
|
||||
| C3 (high-cos / low-dHash) | 0.9826 / 2.41 / 0.321 | 0.9826 / 2.40 / 0.205 | $\lvert\Delta\rvert$ cos 0.000, dHash 0.01, wt 0.117 |
|
||||
|
||||
(Source: Script 41; full-dataset $\text{BIC}(K{=}3) = -792.31$ vs Big-4 $\text{BIC}(K{=}3) = -1111.93$; BIC values are not directly comparable across different $n$ and are reported only for completeness.)
|
||||
|
||||
**Table XVIII.** Spearman rank correlation between K=3 P(C1) and Paper A operational less-replication-dominated rate, Big-4 sub-corpus vs full dataset.
|
||||
|
||||
| Scope | $n$ CPAs | Spearman $\rho$ (P(C1) vs Paper A less-replication-dominated rate) | $p$-value |
|
||||
|---|---|---|---|
|
||||
| Big-4 (primary) | 437 | $+0.9627$ | $< 10^{-248}$ |
|
||||
| Full dataset | 686 | $+0.9558$ | $< 10^{-300}$ |
|
||||
| $\lvert\rho_{\text{full}} - \rho_{\text{Big-4}}\rvert$ | — | $0.0069$ | — |
|
||||
|
||||
(Source: Script 41.)
|
||||
|
||||
**Reading.** The K=3 component ordering and the strong Spearman convergence between K=3 P(C1) and the Paper A box-rule less-replication-dominated rate are preserved at the full scope. Component centres shift modestly: C3 (high-cos / low-dHash) is essentially unchanged in centre but loses weight $0.117$ as the full population includes more non-templated CPAs (mid/small firms); C1 (low-cos / high-dHash) gains weight $0.141$ and shifts to lower cosine and higher dHash (centre $(0.928, 11.17)$ vs Big-4 $(0.946, 9.17)$) as the broader population includes mid/small-firm CPAs landing toward the low-cos / high-dHash region that the Big-4-primary scope deliberately excludes. We read this as evidence that the Big-4-primary K=3 + Paper A convergence is not a Big-4-specific artefact; we do **not** read it as an endorsement of using full-dataset K=3 component centres or operational thresholds in place of the Big-4-primary analysis. Mid/small-firm composition shifts the component centres meaningfully and the v4.0 primary methodology is restricted to Big-4 by design (§III-G item 4).
|
||||
|
||||
## L. Ablation Study: Feature Backbone Comparison
|
||||
|
||||
To validate the choice of ResNet-50 as the feature extraction backbone, we conducted an ablation study comparing three pre-trained architectures: ResNet-50 (2048-dim), VGG-16 (4096-dim), and EfficientNet-B0 (1280-dim).
|
||||
All models used ImageNet pre-trained weights without fine-tuning, with identical preprocessing and L2 normalization.
|
||||
Table XVIII presents the comparison.
|
||||
The comparison summary is inherited unchanged from the v3.20.0 backbone-ablation table (v3.20.0 Table XVIII; not the same table as v4 Table XVIII which reports Big-4 vs full-dataset Spearman drift in §IV-K).
|
||||
|
||||
<!-- TABLE XVIII: Backbone Comparison
|
||||
<!-- v3.20.0 TABLE XVIII (inherited unchanged; reproduced here in commented form for v4 splice provenance — actual rendered table is the v3.20.0 backbone-ablation table):
|
||||
| Metric | ResNet-50 | VGG-16 | EfficientNet-B0 |
|
||||
|--------|-----------|--------|-----------------|
|
||||
| Feature dim | 2048 | 4096 | 1280 |
|
||||
@@ -492,3 +327,106 @@ ResNet-50 provides the best overall balance:
|
||||
(2) its tighter distributions yield more reliable individual classifications;
|
||||
(3) the highest Firm A all-pairs 1st percentile (0.543) indicates that known-replication signatures are least likely to produce low-similarity outlier pairs under this backbone; and
|
||||
(4) its 2048-dimensional features offer a practical compromise between discriminative capacity and computational/storage efficiency for processing 182K+ signatures.
|
||||
|
||||
## M. v4-New Anchor-Based ICCR Calibration Results
|
||||
|
||||
This section consolidates the v4-new empirical results that support the §III-L anchor-based threshold calibration framework. Numbers below are direct re-statements from the spike scripts cited per row; the corresponding §III provenance table entries appear in §III's provenance table.
|
||||
|
||||
### M.1 Composition decomposition (Scripts 39b–39e)
|
||||
|
||||
**Table XX.** Within-firm and between-firm decomposition of the Big-4 accountant-level dip-test rejection.
|
||||
|
||||
| Diagnostic | Scope | Statistic | Implication |
|
||||
|---|---|---|---|
|
||||
| Within-firm signature-level cosine dip | Big-4 (4 firms) | $p_{\text{cos}} \in \{0.176, 0.991, 0.551, 0.976\}$ | 0/4 firms reject; cosine within-firm unimodal |
|
||||
| Within-firm signature-level cosine dip | non-Big-4 (10 firms $\geq 500$ sigs) | $p_{\text{cos}} \in [0.59, 0.99]$ | 0/10 firms reject; cosine within-firm unimodal |
|
||||
| Within-firm jittered-dHash dip (5 seeds, median) | Big-4 (4 firms) | $p_{\text{med}} \in \{0.999, 0.996, 0.999, 0.9995\}$ | 0/4 firms reject after integer-jitter; raw rejection was integer-tie artefact |
|
||||
| Big-4 pooled dHash: 2×2 factorial | firm-centred + jittered (5 seeds) | $p_{\text{med}} = 0.35$, 0/5 seeds reject | combined corrections eliminate rejection; multimodality is composition + integer artefact |
|
||||
| Integer-histogram valley near $\text{dHash} \approx 5$ | within each Big-4 firm | none (0/4 firms) | no within-firm dHash antimode at the inherited HC cutoff |
|
||||
|
||||
(Source: Scripts 39b, 39c, 39d, 39e; bootstrap $n_{\text{boot}} = 2000$; jitter $\sim \mathrm{U}[-0.5, +0.5]$.)
|
||||
|
||||
### M.2 Anchor-based inter-CPA pair-level ICCR (Script 40b)
|
||||
|
||||
**Table XXI.** Big-4 inter-CPA per-comparison ICCR sweep, $n = 5 \times 10^5$ pairs (Big-4 scope; v4 new).
|
||||
|
||||
| Threshold | Per-comparison ICCR | 95% Wilson CI |
|
||||
|---|---|---|
|
||||
| cos $> 0.945$ (v3.x published "natural threshold") | $0.00081$ | $[0.00073, 0.00089]$ |
|
||||
| cos $> 0.95$ (inherited operating point) | $0.00060$ | $[0.00053, 0.00067]$ |
|
||||
| cos $> 0.97$ | $0.00024$ | $[0.00020, 0.00029]$ |
|
||||
| cos $> 0.98$ | $0.00009$ | $[0.00007, 0.00012]$ |
|
||||
| dHash $\leq 5$ (inherited operating point) | $0.00129$ | $[0.00120, 0.00140]$ |
|
||||
| dHash $\leq 4$ | $0.00050$ | $[0.00044, 0.00057]$ |
|
||||
| dHash $\leq 3$ | $0.00019$ | $[0.00015, 0.00023]$ |
|
||||
| Joint: cos $> 0.95$ AND dHash $\leq 5$ (any-pair semantics) | $0.00014$ | — |
|
||||
| Joint: cos $> 0.95$ AND dHash $\leq 4$ (any-pair) | $0.00011$ | — |
|
||||
|
||||
Conditional ICCR(dHash $\leq 5$ | cos $> 0.95$) $= 0.234$ (Wilson 95% $[0.190, 0.285]$; $70$ of $299$ pairs).
|
||||
|
||||
The cos $> 0.95$ row replicates v3.20.0 §IV-F.1 Table X (v3 reported $0.0005$ under prior "FAR" terminology). The dHash row and joint row are v4 new.
|
||||
|
||||
### M.3 Pool-normalised per-signature ICCR (Script 43)
|
||||
|
||||
**Table XXII.** Pool-normalised per-signature ICCR under the deployed any-pair HC rule (cos $> 0.95$ AND dHash $\leq 5$); $n_{\text{sig}} = 150{,}453$ (vector-complete Big-4); CPA-block bootstrap $n_{\text{boot}} = 1000$.
|
||||
|
||||
| Scope | Per-signature ICCR | Wilson 95% CI | CPA-bootstrap 95% CI |
|
||||
|---|---|---|---|
|
||||
| Big-4 pooled (any-pair, deployed) | $0.1102$ | $[0.1086, 0.1118]$ | $[0.0908, 0.1330]$ |
|
||||
| Big-4 pooled (same-pair, stricter alternative) | $0.0827$ | $[0.0813, 0.0841]$ | $[0.0668, 0.1021]$ |
|
||||
| Firm A (any-pair) | $0.2594$ | — | — |
|
||||
| Firm B (any-pair) | $0.0147$ | — | — |
|
||||
| Firm C (any-pair) | $0.0053$ | — | — |
|
||||
| Firm D (any-pair) | $0.0110$ | — | — |
|
||||
| Pool-size decile 1 (smallest pools) any-pair | $0.0249$ | — | — |
|
||||
| Pool-size decile 10 (largest pools) any-pair | $0.1905$ | — | — |
|
||||
|
||||
Decile trend is broadly monotone in pool size with two minor reversals (decile 5 and decile 9 dip below their predecessors). Stricter operating point cos $> 0.95$ AND dHash $\leq 3$ (same-pair) gives per-signature ICCR $0.0449$.
|
||||
|
||||
### M.4 Document-level ICCR under three alarm definitions (Script 45)
|
||||
|
||||
**Table XXIII.** Document-level inter-CPA ICCR by alarm definition; $n_{\text{docs}} = 75{,}233$.
|
||||
|
||||
| Alarm definition | Alarm set | Document-level ICCR | Wilson 95% CI |
|
||||
|---|---|---|---|
|
||||
| D1 | HC only | $0.1797$ | $[0.1770, 0.1825]$ |
|
||||
| D2 (operational) | HC + MC | $0.3375$ | $[0.3342, 0.3409]$ |
|
||||
| D3 | HC + MC + HSC | $0.3384$ | $[0.3351, 0.3418]$ |
|
||||
|
||||
Per-firm D2 document-level ICCR: Firm A $0.6201$ ($n = 30{,}226$); Firm B $0.1600$ ($n = 17{,}127$); Firm C $0.1635$ ($n = 19{,}501$); Firm D $0.0863$ ($n = 8{,}379$). The Firm C denominator $n = 19{,}501$ exceeds Table XIX's single-firm Firm C count of $19{,}122$ by exactly the $379$ mixed-firm PDFs: all $379$ are $1{:}1$ Firm C / Firm D mixed-firm documents, and Script 45's mode-of-firms implementation (`np.argmax` over `np.unique`'s alphabetically-sorted firm counts) returns the first-sorted firm on ties, which assigns these tied documents to Firm C rather than to Firm D. The four per-firm denominators here therefore sum to the full $75{,}233$, whereas Table XIX's per-firm rows sum to $74{,}854 = 75{,}233 - 379$.
|
||||
|
||||
### M.5 Firm heterogeneity logistic regression and cross-firm hit matrix (Script 44)
|
||||
|
||||
**Table XXIV.** Logistic regression of per-signature any-pair HC hit indicator on firm dummies and centred log pool size (Firm A reference).
|
||||
|
||||
| Term | Odds ratio (vs Firm A) | Direction |
|
||||
|---|---|---|
|
||||
| Firm B | $0.053$ | $\sim 19\times$ lower odds than Firm A |
|
||||
| Firm C | $0.010$ | $\sim 100\times$ lower odds than Firm A |
|
||||
| Firm D | $0.027$ | $\sim 37\times$ lower odds than Firm A |
|
||||
| log(pool size, centred) | $4.01$ | $\sim 4\times$ higher odds per log unit pool size |
|
||||
|
||||
Per-decile per-firm rates (Table not duplicated here; Script 44 decile table available in the supplementary report): within every pool-size decile, Firms B/C/D show rates of $0.0006$–$0.0358$ while Firm A ranges $0.0541$–$0.5958$. The firm gap survives within matched pool sizes.
|
||||
|
||||
**Table XXV.** Cross-firm hit matrix among Big-4 source signatures with any-pair HC hit; max-cosine partner firm (counts).
|
||||
|
||||
| Source firm | Firm A cand. | Firm B | Firm C | Firm D | non-Big-4 | n hits |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Firm A | $14{,}447$ | $95$ | $44$ | $19$ | $17$ | $14{,}622$ |
|
||||
| Firm B | $92$ | $371$ | $8$ | $4$ | $9$ | $484$ |
|
||||
| Firm C | $16$ | $7$ | $149$ | $5$ | $1$ | $178$ |
|
||||
| Firm D | $22$ | $2$ | $6$ | $106$ | $1$ | $137$ |
|
||||
|
||||
Same-pair joint hits (single candidate satisfying both cos $> 0.95$ AND dHash $\leq 5$) are within-firm at rates $99.96\%$ / $97.7\%$ / $98.2\%$ / $97.0\%$ for Firms A/B/C/D respectively.
|
||||
|
||||
### M.6 Alert-rate sensitivity around inherited HC threshold (Script 46)
|
||||
|
||||
**Table XXVI.** Local-gradient / median-gradient ratio at inherited thresholds (descriptive plateau diagnostic).
|
||||
|
||||
| Threshold | Local / median gradient ratio | Interpretation |
|
||||
|---|---|---|
|
||||
| cos $= 0.95$ (HC) | $\approx 25\times$ | locally sensitive (not plateau-stable) |
|
||||
| dHash $= 5$ (HC) | $\approx 3.8\times$ | locally sensitive (not plateau-stable) |
|
||||
| dHash $= 15$ (MC/HSC boundary) | $\approx 0.08$ | plateau-like (saturating tail) |
|
||||
|
||||
Big-4 observed deployed alert rate on actual same-CPA pools: per-signature HC $= 0.4958$; per-document HC $= 0.6228$. The deployed-rate excess over the inter-CPA proxy is $0.3856$ pp per-signature and $0.4431$ pp per-document; this excess is interpreted as a same-CPA repeatability signal under the §III-M caveats, not as a presumed true-positive rate.
|
||||
|
||||
Reference in New Issue
Block a user