gbanyan/pdf_signature_extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: gbanyan/pdf_signature_extraction:b884d395442939be8b92b0bcbf689fe8e43a2794
Branches Tags
gbanyan/pdf_signature_extraction:master gbanyan/pdf_signature_extraction:paper-a-v4-big4 gbanyan/pdf_signature_extraction:yolo-signature-pipeline gbanyan/pdf_signature_extraction:pp-ocrv5-research
gbanyan/pdf_signature_extraction:v1.0-hybrid-70percent
..
compare: gbanyan/pdf_signature_extraction:4a6f9c5c98e958831b655743d4bef77876cf223a
Branches Tags
gbanyan/pdf_signature_extraction:paper-a-v4-big4 gbanyan/pdf_signature_extraction:yolo-signature-pipeline gbanyan/pdf_signature_extraction:pp-ocrv5-research gbanyan/pdf_signature_extraction:master
gbanyan/pdf_signature_extraction:v1.0-hybrid-70percent
2 Commits
b884d39544 .. 4a6f9c5c98
Author SHA1 Message Date
gbanyanandClaude Opus 4.7 4a6f9c5c98 Apply Phase 5 round-3 splice-blocker fixes from codex round-8 
Closes the three concrete splice blockers codex round-8 surfaced
in the post-round-2 drafts, plus the binary-collapse terminology
residue. No empirical changes.

- Abstract trimmed 261 -> 247 words (3 under IEEE Access <=250
  target). Cut "technically trivial and visually invisible,"
  (S1 motivational redundancy) and the within-firm-rate
  parenthetical "(Firm A 98.8%; Firms B/C/D 76.7-83.7%)" plus
  "between" connector; preserved the corrected 77-99% any-pair
  headline so the M3 substance survives.

- §IV-J Table XV sample-size footnote (line 177) corrected:
  round-2 misclassified §IV-M.5 as descriptor-complete n=150,442;
  Script 44 / Tables XXIV-XXV actually use vector-complete
  n=150,453, same as §IV-M.2 Table XXI (Script 40b) and §IV-M.3
  Table XXII (Script 43). New footnote distinguishes
  descriptor-complete (§IV-D through §IV-J) from
  vector/pair-recomputed (§IV-M.2/M.3/M.5; Scripts 40b/43/44).

- §IV-I (line 161) stale cross-reference: "§IV-M Table XVI" was
  the K=3 firm cross-tab (descriptive), not the v4-new ICCR
  calibration. Replaced with "§IV-M Tables XXI-XXVI" — the full
  ICCR calibration block. Pre-existing error exposed by the
  round-2 cascade.

- §III line 131 + §IV Table XI line 104 binary-collapse label:
  "replicated vs not-replicated" -> "replication-dominated vs
  less-replication-dominated" for consistency with the K=3
  descriptor-position framing. "Replicated class" preserved
  where it refers to byte-identical positive-anchor ground
  truth (§III-K.4, §IV-H lines 143/153/155).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 17:17:30 +08:00
gbanyanandClaude Opus 4.7 4ee2efb5bb Add codex GPT-5.5 Phase 5 round-2 cross-check on post-round-2 drafts 
Verdict: Minor Revision (corroborates Gemini round-1 and Opus round-1).

Round-1 panel finding closure (codex round-8 audit):
- Codex own round-7: 11 Major + 15 Minor → 21 CLOSED, 4 OPEN/PARTIAL
  (mostly splice items); M6 + new-issue-1 (refs [42]-[44]) SUPERSEDED
  (Gemini was right, codex round-7 was wrong about absence)
- Gemini round-1: 5 Major + 3 Minor all CLOSED in main body
- Opus round-1: M1-M4 CLOSED in manuscript body; some minors open

Provenance verification (independent of Opus):
- Within-firm any-pair from Table XXV: 98.8032 / 76.6529 / 83.7079 /
  77.3723% — Opus arithmetic confirmed
- Same-pair joint: 99.9558 / 97.7011 / 98.1818 / 96.9697% — confirms
  the 97.0-99.96% range
- Pooled Big-4 any-pair ICCR 0.1102 verified from Script 43 report
  (16,578 / 150,453); Wilson 95% half-width 0.00158 reconciles
- Per-pair conditional ICCR 0.234 verified from Script 40b (70 / 299)

Round-2-induced / round-2-exposed concrete blockers (fixable):
1. Abstract now 261 words (M3 fix pushed over <=250 IEEE Access target);
   need 11+ word trim
2. §IV line 177 footnote miscategorizes §IV-M.5 as n=150,442 —
   §IV-M.5 / Tables XXIV-XXV actually use 150,453 vector-complete per
   Script 44 report; only §IV-D through §IV-J use 150,442
3. §IV-I line 161 stale cross-reference: "§IV-M Table XVI" should be
   "§IV-M Tables XXI-XXVI" — XVI is the K=3 firm cross-tab,
   pre-existing error exposed by the cascade

Minor copy-edit residue (not blockers): §III line 131 + §IV Table XI
line 104 "replicated vs not-replicated" binary-collapse label;
internal-note staleness at §III lines 438/445, §IV line 3/370.

No empirical reopening: codex confirms Opus M3 does not invalidate
round-7's Major closures of M2 (Big-4 scope) or M11 (cross-scope
reproducibility). Only round-7 minor reopened: m2 abstract margin.

Phase 5 readiness: Partial — empirical core ready, no new statistical
work required; copy-edit / factual-reference splice blockers remain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 17:15:42 +08:00
4 changed files with 143 additions and 5 deletions
Expand all files Collapse all files
paper/codex_review_gpt55_v4_round8.md
+138
View File
@@ -0,0 +1,138 @@
# Paper A Round 28 Review — codex GPT-5.5 v4 round 8
Reviewer: gpt-5.5
Date: 2026-05-14
Target: paper/v4/paper_a_prose_v4_phase4.md + paper/v4/paper_a_methodology_v4_section_iii.md + paper/v4/paper_a_results_v4_section_iv.md (post round-2 fixes, commit b884d39)
Prior reviewer artifacts: paper/codex_review_gpt55_v4_round7.md; paper/gemini_review_v4_round1.md; paper/opus_review_v4_round1.md
Round-2 commit reviewed: b884d39
## Verdict
Minor Revision.
Round-2 closes the empirical substance of Opus M2-M4 and the core of M1/M3: the deployed any-pair vs same-pair within-firm collision semantics are now separated in the body, the §IV K=3 mechanism-label regression is mostly repaired, §V headings now run A-H, and the Table XV-B cascade has been applied in the public §IV table sequence.
However, the round-2 pass introduced or exposed several splice blockers: the abstract is now 261 words against the stated <=250 target; the new §IV-J Table XV sample-size footnote incorrectly says §IV-M.5 uses n=150,442 even though Script 44 / Tables XXIV-XXV use the 150,453 vector-complete substrate; §IV-I still points the ICCR calibration reader to "§IV-M Table XVI" although the relevant tables are now XXI-XXVI; and internal draft notes/checklists still contain stale Table XV-B / >=97% / hand-leaning language. No new statistical work is required, but the manuscript is not ready for splice until those are patched.
## Round-1 panel finding closure table
| Source | Finding | Current status | Evidence / note |
|---|---|---|---|
| codex r7 M1 | Abstract "Three independent feature-derived scores" | CLOSED | Current prose uses "Three feature-derived scores" and the shared-input caveat at paper/v4/paper_a_prose_v4_phase4.md:37 and :97; §III states the scores are not statistically independent at paper/v4/paper_a_methodology_v4_section_iii.md:113. |
| codex r7 M2 | §I overclaimed Big-4 scope | CLOSED | Big-4 primary scope is explicit at §III-G, paper/v4/paper_a_methodology_v4_section_iii.md:19 and Phase 4 paper/v4/paper_a_prose_v4_phase4.md:39; corrected M3 language does not invalidate this closure. |
| codex r7 M3 | Stale §III-D cross-reference | CLOSED | Pipeline / validation references now point to §III-L / §III-M / §III-I.4, e.g. paper/v4/paper_a_prose_v4_phase4.md:29. |
| codex r7 M4 | §I repeated independent-score error | CLOSED | Phase 4 describes internal consistency, not independent validation, at paper/v4/paper_a_prose_v4_phase4.md:37 and :97. |
| codex r7 M5 | §II not submission-ready as standalone | PARTIAL | §II still contains a review-pass note saying only the v4 paragraph is reproduced, paper/v4/paper_a_prose_v4_phase4.md:65. This is a splice-packaging issue. |
| codex r7 M6 | Refs [42]-[44] absent / placeholders | SUPERSEDED | My round-7 claim was wrong against the current reference file: [42]-[44] are present at paper/paper_a_references_v3.md:87-91, and §II cites them at paper/v4/paper_a_prose_v4_phase4.md:67. |
| codex r7 M7 | §V reified CPA mechanism labels | CLOSED | §V now uses descriptor-position language and explicitly rejects latent mechanism classes at paper/v4/paper_a_prose_v4_phase4.md:81 and :93. |
| codex r7 M8 | §V speculative within-CPA unimodality explanation | CLOSED | §V-B restricts the result to composition + integer artefacts, paper/v4/paper_a_prose_v4_phase4.md:81. |
| codex r7 M9 | §V limitations incomplete | CLOSED | §V-H lists nine v4 limitations plus five inherited v3.20.0 limitations at paper/v4/paper_a_prose_v4_phase4.md:111-139. |
| codex r7 M10 | §V full-dataset scope overread | CLOSED | Scope limitation is explicit at paper/v4/paper_a_prose_v4_phase4.md:117 and §IV-K is narrow at paper/v4/paper_a_results_v4_section_iv.md:232-254. |
| codex r7 M11 | §VI overclaimed cross-scope pipeline reproducibility | CLOSED | §VI now limits cross-scope support to K=3/Spearman robustness and leaves full ICCR generalisation to future work, paper/v4/paper_a_prose_v4_phase4.md:147-149. Opus M3 does not reopen this. |
| codex r7 m1 | "candidate classifiers" wording | CLOSED | Current §V-G uses "candidate checks", paper/v4/paper_a_prose_v4_phase4.md:107. |
| codex r7 m2 | Abstract word-count margin | OPEN | Round-2 M3 rewrite pushed the abstract to 261 words by `wc -w`, while the draft target at paper/v4/paper_a_prose_v4_phase4.md:9 is <=250. |
| codex r7 m3 | Abstract omitted operational output | CLOSED | Abstract includes ICCR units and operational HC+MC per-document alarm, paper/v4/paper_a_prose_v4_phase4.md:11. |
| codex r7 m4 | Contribution 4 overclaimed narrower scopes | CLOSED | Contribution 4 now says the threshold path is unsupported by composition decomposition, paper/v4/paper_a_prose_v4_phase4.md:49. |
| codex r7 m5 | Contribution 8 overclaimed full-dataset check | CLOSED | Current contribution 8 is limited to annotation-free positive-anchor and unsupervised validation ceiling, paper/v4/paper_a_prose_v4_phase4.md:57. |
| codex r7 m6 | "External validation" too broad | CLOSED | Current language is specificity-proxy / annotation-free / unsupervised-ceiling, e.g. paper/v4/paper_a_prose_v4_phase4.md:57 and :113. |
| codex r7 m7 | §II LOOO sufficiency wording | CLOSED | §II frames LOOO as composition-sensitivity, not operational-classifier sufficiency, paper/v4/paper_a_prose_v4_phase4.md:67. |
| codex r7 m8 | "Inherits and confirms" too strong | CLOSED | §V-B says v4 strengthens/extends by decomposition but does not overclaim direct validation, paper/v4/paper_a_prose_v4_phase4.md:79-81. |
| codex r7 m9 | Firm A byte-level provenance | CLOSED | Inherited Script 28 provenance is stated at paper/v4/paper_a_prose_v4_phase4.md:85 and §III-H at paper/v4/paper_a_methodology_v4_section_iii.md:37. |
| codex r7 m10 | Firm A alone did not anchor §IV-H | CLOSED | Pixel-identity anchor is Big-4 n=262 with all four firms listed, paper/v4/paper_a_results_v4_section_iv.md:143-153. |
| codex r7 m11 | "Published box rule" not traceable | CLOSED | Current text uses inherited Paper A / v3.x box rule language, e.g. paper/v4/paper_a_results_v4_section_iv.md:215. |
| codex r7 m12 | "Same per-CPA ranking" too strong | CLOSED | Residual Firm D/Firm C disagreement is disclosed at paper/v4/paper_a_prose_v4_phase4.md:97 and §IV-F at paper/v4/paper_a_results_v4_section_iv.md:102. |
| codex r7 m13 | §V "candidate classifiers" residue | CLOSED | Replaced with "candidate checks" at paper/v4/paper_a_prose_v4_phase4.md:107. |
| codex r7 m14 | Future-work audit-quality contrast needed caveat | CLOSED | Future work now keeps the Firm A vs B/C/D contrast descriptive, paper/v4/paper_a_prose_v4_phase4.md:149. |
| codex r7 m15 | Conclusion underplayed operational output | CLOSED | §VI opens with five-way classifier and worst-case aggregation, paper/v4/paper_a_prose_v4_phase4.md:145. |
| codex r7 new issue 1 | §II citation gap | SUPERSEDED | Refs [42]-[44] exist at paper/paper_a_references_v3.md:87-91. |
| codex r7 new issue 2 | Stale close-out metadata | OPEN | Phase 4 close-out still says 243-244 words and "§V-G Limitations", paper/v4/paper_a_prose_v4_phase4.md:157-160; current abstract count is 261 and limitations are §V-H. |
| Gemini M1 | Reject "statistically insignificant" firm heterogeneity framing | CLOSED | Current text says firm effects are large and not pool-size explained at paper/v4/paper_a_methodology_v4_section_iii.md:259-268 and paper/v4/paper_a_prose_v4_phase4.md:35. |
| Gemini M2 | codex refs [42]-[44] error | CLOSED | References are present at paper/paper_a_references_v3.md:87-91. |
| Gemini M3 | ICCR disclaimer adequacy | CLOSED | FAR is explicitly reframed as ICCR / specificity proxy at paper/v4/paper_a_methodology_v4_section_iii.md:185 and paper/v4/paper_a_results_v4_section_iv.md:159. |
| Gemini M4 | K=3 demotion language | CLOSED for main body | §III-J and §V-D are correct at paper/v4/paper_a_methodology_v4_section_iii.md:76-90 and paper/v4/paper_a_prose_v4_phase4.md:89-93. Residual public wording "replicated vs not-replicated" in Table XI is flagged below as minor terminology residue. |
| Gemini M5 | Feature-derived score caveat | CLOSED | Shared-input caveat appears in §III-K, §IV-F, and §V-E: paper/v4/paper_a_methodology_v4_section_iii.md:113; paper/v4/paper_a_results_v4_section_iv.md:79; paper/v4/paper_a_prose_v4_phase4.md:97. |
| Gemini m1 | Internal draft notes/checklists | OPEN | Present in §III, §IV, and Phase 4: paper/v4/paper_a_methodology_v4_section_iii.md:3 and :431-447; paper/v4/paper_a_results_v4_section_iv.md:3 and :365-374; paper/v4/paper_a_prose_v4_phase4.md:3 and :153-162. |
| Gemini m2 | Table XV-B vs XIX numbering | CLOSED in public body | Public document-level table is now Table XIX at paper/v4/paper_a_results_v4_section_iv.md:192 and §IV-M cascades through XX-XXVI at :266, :280, :300, :317, :329, :340, :353. Internal notes still stale. |
| Gemini m3 | Word count note | OPEN | The note remains stale at paper/v4/paper_a_prose_v4_phase4.md:157 and the current abstract is over target. |
| Gemini new issue | Table XV sample-size nuance | PARTIAL | A pointer to §III-G was added at paper/v4/paper_a_results_v4_section_iv.md:177, but the footnote misclassifies §IV-M.5 as n=150,442; Script 44 / Tables XXIV-XXV use the 150,453 vector-complete substrate per §III-G, paper/v4/paper_a_methodology_v4_section_iii.md:31. |
| Opus M1 | §IV K=3 mechanism-label reversion | CLOSED for named tables/prose; MINOR RESIDUE | Tables IX/X/XIV/XVI/XVII now use descriptor-position or less-replication-dominated language at paper/v4/paper_a_results_v4_section_iv.md:81-100, :145-153, :217-226, :234-254. Public residue: Table XI still says "binary collapse, replicated vs not-replicated" at :104; internal §III open question still says hand-leaning at paper/v4/paper_a_methodology_v4_section_iii.md:445. |
| Opus M2 | Table XV-B cascade | CLOSED in public body | Table XIX replaces XV-B at paper/v4/paper_a_results_v4_section_iv.md:192 and §IV-M is XX-XXVI at :266-353. No public Table XV-B reference remains; only internal notes at :3 and :370. |
| Opus M3 | "98-100% within source firm" semantic conflation | CLOSED in body; ABSTRACT SHORT FORM | Body locations now separate deployed any-pair 98.8% / 76.7-83.7% from same-pair 97.0-99.96% at paper/v4/paper_a_prose_v4_phase4.md:35, :53, :87, :115, :147, :149 and §III at paper/v4/paper_a_methodology_v4_section_iii.md:99, :283, :285. The abstract uses a rounded any-pair-only 77-99% headline at paper/v4/paper_a_prose_v4_phase4.md:11, which is not misleading but omits the same-pair subrange. |
| Opus M4 | Duplicate §V-G heading | CLOSED | §V headings now run A-H: paper/v4/paper_a_prose_v4_phase4.md:73, :77, :83, :89, :95, :99, :105, :109. |
| Opus M5 | Stale "seven limitations" close-out note | OPEN internal | Phase 4 checklist still says "seven limitations" and "§V-G Limitations" at paper/v4/paper_a_prose_v4_phase4.md:160; the actual limitations heading is §V-H and has 14 items. |
| Opus M6 | §IV-M composition table partial vs §III factorial | PARTIAL / LOW | §IV-M.1 remains a summary table at paper/v4/paper_a_results_v4_section_iv.md:266-276, while full factorial detail is in §III-I.4 at paper/v4/paper_a_methodology_v4_section_iii.md:61-68. This is acceptable if §IV-D points readers to a summary, but current §IV-D says diagnostics are "tabulated in §IV-M" at paper/v4/paper_a_results_v4_section_iv.md:23. |
| Opus M7 | Mixed Spearman precision | OPEN / COPY-EDIT | §III reports 0.963/0.889/0.879 at paper/v4/paper_a_methodology_v4_section_iii.md:123-127, while §IV uses 0.9627/0.8890/0.8794 at paper/v4/paper_a_results_v4_section_iv.md:81-87. |
| Opus minor 1 | Abstract word-count metadata | OPEN | Current abstract is 261 words; close-out note still says 243-244 at paper/v4/paper_a_prose_v4_phase4.md:157. |
| Opus minor 2 | Internal draft notes | OPEN | Same as Gemini m1. |
| Opus minor 3 | §IV-J Table XV-B pointer | CLOSED in body | Body now says Table XIX at paper/v4/paper_a_results_v4_section_iv.md:228. |
| Opus minor 4 | Mixed decimal / percentage notation | OPEN / COPY-EDIT | Still mixed by design, e.g. 0.34 at paper/v4/paper_a_prose_v4_phase4.md:11 and 33.75% at :33. |
| Opus minor 5 | v3.x §IV-F.1 cross-reference check | OPEN / SPLICE | v3.x §IV-F.1 references remain at paper/v4/paper_a_methodology_v4_section_iii.md:37 and paper/v4/paper_a_prose_v4_phase4.md:85. Verify during master splice. |
| Opus minor 6 | Firm A 50/180 inherited provenance | CLOSED / DISCLOSED | Provenance is disclosed as inherited, not regenerated, at paper/v4/paper_a_methodology_v4_section_iii.md:37. |
| Opus minor 7 | "FAR throughout" historical exception | PARTIAL | v4 framing disclaims FAR, but historical "FAR" appears in paper/v4/paper_a_results_v4_section_iv.md:159 and paper/v4/paper_a_methodology_v4_section_iii.md:185. It is correctly caveated, not an empirical issue. |
| Opus minor 8 | MC band proportions | CLOSED | §IV-J proportions match Table XV rows at paper/v4/paper_a_results_v4_section_iv.md:181-186 and prose at :215. |
| Opus minor 9 | §V-G item count | OPEN internal | Same as Opus M5. |
| Opus minor 10 | LOOO range 1.8-12.8 pp | CLOSED | §IV Table XIII supports 1.76-12.77 pp, paper/v4/paper_a_results_v4_section_iv.md:131-139. |
| Opus minor 11 | Abstract 98-100 public statement | SUPERSEDED | Replaced by rounded any-pair 77-99% at paper/v4/paper_a_prose_v4_phase4.md:11. |
| Opus new issue 1 | Human-in-the-loop not operationalised | OPEN / COPY-EDIT | The positioning remains in Abstract and §III-M, paper/v4/paper_a_prose_v4_phase4.md:11 and paper/v4/paper_a_methodology_v4_section_iii.md:334, but no concrete review workflow is specified. |
| Opus new issue 2 | Feature-derived caveat breaks down in §IV | CLOSED for main §IV | §IV tables and prose were repaired, except the Table XI binary-collapse label noted above. |
| Opus new issue 3 | §III-M nine-tool table unnumbered | OPEN / COPY-EDIT | The validation table at paper/v4/paper_a_methodology_v4_section_iii.md:318-329 remains unnumbered. |
| Opus new issue 4 | §I pipeline-step vs framework-element framing | OPEN / COPY-EDIT | The eight-item enumeration remains at paper/v4/paper_a_prose_v4_phase4.md:29. |
## Round-2 induced issues
1. **Abstract now exceeds the target word count.** `sed -n '11p' paper/v4/paper_a_prose_v4_phase4.md | wc -w` returns 261. The draft note at paper/v4/paper_a_prose_v4_phase4.md:9 sets an IEEE Access <=250 target, and the close-out note at :157 is stale.
2. **The new Table XV sample-size footnote is partly wrong.** The pointer to §III-G is useful, but paper/v4/paper_a_results_v4_section_iv.md:177 says §IV-M.5 uses n=150,442. §III-G says Scripts 40b, 43, and 44 use the 150,453 vector-complete substrate, paper/v4/paper_a_methodology_v4_section_iii.md:31, and Script 44's report states n_big4_sources = 150,453. Correct the footnote to distinguish descriptor-complete sections from vector-complete §IV-M.2 / §IV-M.3 / §IV-M.5.
3. **Public table cross-reference is stale.** §IV-I still says the consolidated v4-new ICCR calibration appears in "§IV-M Table XVI", paper/v4/paper_a_results_v4_section_iv.md:161. Current Table XVI is the K=3 firm cross-tab at :217; the ICCR calibration tables are XXI-XXVI at :280, :300, :317, :329, :340, :353.
4. **Internal notes remain stale.** §IV's draft note still says v3.2 and Table XV-B, paper/v4/paper_a_results_v4_section_iv.md:3; the §IV close-out checklist repeats Table XV-B at :370. §III's internal cross-reference index still says within-firm collision concentration >=97% at paper/v4/paper_a_methodology_v4_section_iii.md:438 and "C1 hand-leaning" at :445. These are internal-only splice items, not empirical blockers, but they must be stripped or updated.
5. **Minor terminology residue remains outside Opus's named M1 sites.** §III and §IV Table XI still call a K=3 / box-rule binary collapse "replicated vs not-replicated", paper/v4/paper_a_methodology_v4_section_iii.md:131 and paper/v4/paper_a_results_v4_section_iv.md:104. Because this is not the byte-identical positive-anchor ground-truth subset, a stricter v4 wording would be "high-cos / low-dHash vs other positions" or "replication-dominated vs less-replication-dominated."
6. **"Less-replication-dominated" is long but not broken.** The phrase is readable in the public replacement sites. The only sentence I would smooth at copy-edit is paper/v4/paper_a_results_v4_section_iv.md:215 ("per-CPA less-replication-dominated ranking"), which could become "per-CPA ranking away from the replication-dominated corner."
## Provenance spot-checks
1. **Within-firm any-pair rates from §IV Table XXV.** From paper/v4/paper_a_results_v4_section_iv.md:340-349:
- Firm A: 14,447 / 14,622 = 98.8032% -> 98.8%.
- Firm B: 371 / 484 = 76.6529% -> 76.7%.
- Firm C: 149 / 178 = 83.7079% -> 83.7%.
- Firm D: 106 / 137 = 77.3723% -> 77.4%.
These match the corrected 76.7-98.8% any-pair range and the B/C/D 76.7-83.7% summary. Script 44's report gives the same matrix at /Volumes/NV2/PDF-Processing/signature-analysis/reports/v4_big4/firm_matched_pool/firm_matched_pool_report.md:36-43.
2. **Same-pair joint range 97.0-99.96%.** §III-L.4 reports 11,314/11,319, 85/87, 54/55, and 64/66 at paper/v4/paper_a_methodology_v4_section_iii.md:281. The arithmetic is 99.9558%, 97.7011%, 98.1818%, and 96.9697%, matching the rounded 99.96% / 97.7% / 98.2% / 97.0%. §IV repeats the rates at paper/v4/paper_a_results_v4_section_iv.md:349.
3. **Pooled Big-4 any-pair per-signature ICCR 0.1102.** Script 43's report gives 16,578 / 150,453 = 0.1102 at /Volumes/NV2/PDF-Processing/signature-analysis/reports/v4_big4/pool_normalized_far/pool_normalized_report.md:42-50. A normal approximation half-width is 1.96 * sqrt(0.1102 * 0.8898 / 150453) = 0.00158, consistent with the reported Wilson [0.1086, 0.1118] in §IV-M.3, paper/v4/paper_a_results_v4_section_iv.md:300-305.
4. **Per-pair conditional ICCR 0.234.** Script 40b's report gives dHash <= 5 conditional on cos > 0.95 as 70 / 299 = 0.23411 at /Volumes/NV2/PDF-Processing/signature-analysis/reports/v4_big4/inter_cpa_far_sweep/far_sweep_report.md:87-99. This matches §III-L.1 at paper/v4/paper_a_methodology_v4_section_iii.md:208 and §IV-M.2 at paper/v4/paper_a_results_v4_section_iv.md:294.
## Updated round-7 closure reassessment
Opus M3 does not invalidate my round-7 closure of M2 (Big-4 scope language) or M11 (cross-scope pipeline reproducibility). Those findings were about overextending Big-4/full-dataset scope and overclaiming cross-scope reproducibility. The corrected current prose keeps the primary scope Big-4, treats full-dataset evidence as a narrow K=3 + Spearman robustness check, and now reports the within-firm collision pattern as any-pair 76.7-98.8% plus same-pair 97.0-99.96%.
If I re-graded round 7 against the corrected current drafts, none of my major closures would move back to PARTIAL on empirical grounds. I would, however, reopen the abstract word-count minor item because the M3 repair pushed the abstract over the <=250-word target.
## Phase 5 readiness
Partial.
The empirical core is ready: no script rerun or statistical redesign is needed. M2-M4 are closed in manuscript body text, and M3 is substantively corrected. Phase 5 is blocked only by splice/copy-edit/factual-reference hygiene:
1. Trim the abstract from 261 to <=250 words.
2. Correct §IV-J line 177's sample-size footnote so §IV-M.2 / M.3 / M.5 are identified as vector-complete / pair-recomputed analyses, not n=150,442 descriptor-complete analyses.
3. Fix §IV-I's stale "§IV-M Table XVI" pointer.
4. Strip or update internal draft notes, checklists, §III cross-reference index, and stale Table XV-B / >=97% / hand-leaning language.
5. Optionally smooth the residual "replicated vs not-replicated" binary-collapse label and the long "less-replication-dominated" phrase in §IV-J.
## Recommended next-step actions
1. **Copy-edit blocker:** trim the abstract by at least 11 words while preserving the corrected any-pair headline. Do not add same-pair detail to the abstract unless other text is removed.
2. **Factual cross-reference blocker:** replace the §IV-J Table XV footnote with a precise version: descriptor-complete analyses use 150,442; vector/pair-recomputed analyses use 150,453, including Scripts 40b, 43, and 44 (§IV-M.2, M.3, M.5).
3. **Cross-reference blocker:** change §IV-I's "§IV-M Table XVI" to "§IV-M.2 Table XXI" or to "§IV-M Tables XXI-XXVI", depending on whether the intended pointer is per-comparison ICCR only or the whole calibration block.
4. **Splice blocker:** remove all internal notes/checklists before manuscript assembly, especially the stale §IV v3.2 / Table XV-B note, §III's >=97% cross-reference-index shorthand, and the Phase 4 "§V-G Limitations / seven limitations" checklist item.
5. **Terminology cleanup:** consider renaming "binary collapse, replicated vs not-replicated" to descriptor-position language in §III-K and §IV-F, while retaining "replicated class" only for the byte-identical positive-anchor ground truth.
paper/v4/paper_a_methodology_v4_section_iii.md
+1 -1
View File
@@ -128,7 +128,7 @@ Pairwise Spearman rank correlations among the three scores, $n = 437$ Big-4 CPAs
We read this as the strongest internal-consistency signal in v4.0: three different summarisations of the same descriptor pair agree on the per-CPA descriptor-position ranking with $\rho > 0.87$. The three scores agree on placing Firm A as the most replication-dominated descriptor position and the three non-Firm-A Big-4 firms further from the templated end, but they do not all rank the non-Firm-A firms identically: the K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Scores 1 and 3) place Firm C at the less-replication-dominated end of Big-4 (mean P(C1) $= 0.311$; mean box-rule less-replication-dominated rate $= 0.790$), while the reverse-anchor cosine percentile (Score 2) places Firm D fractionally higher than Firm C (mean reverse-anchor score $-0.7125$ vs Firm C $-0.7672$, with higher value indicating deeper into the reference left tail). The mean values for Firms B and D sit between Firms A and C on Scores 1 and 3 (Script 38 per-firm summary). We do not claim this constitutes external validation of any operational classifier; the inherited box rule is calibrated separately (§III-L), and the convergence above shows that a mixture-derived score and a reverse-anchor score concur with the box rule's per-CPA-aggregated outputs on the directional ordering, with a modest disagreement at the less-replication-dominated end between the three non-A Big-4 firms. We read this as the strongest internal-consistency signal in v4.0: three different summarisations of the same descriptor pair agree on the per-CPA descriptor-position ranking with $\rho > 0.87$. The three scores agree on placing Firm A as the most replication-dominated descriptor position and the three non-Firm-A Big-4 firms further from the templated end, but they do not all rank the non-Firm-A firms identically: the K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Scores 1 and 3) place Firm C at the less-replication-dominated end of Big-4 (mean P(C1) $= 0.311$; mean box-rule less-replication-dominated rate $= 0.790$), while the reverse-anchor cosine percentile (Score 2) places Firm D fractionally higher than Firm C (mean reverse-anchor score $-0.7125$ vs Firm C $-0.7672$, with higher value indicating deeper into the reference left tail). The mean values for Firms B and D sit between Firms A and C on Scores 1 and 3 (Script 38 per-firm summary). We do not claim this constitutes external validation of any operational classifier; the inherited box rule is calibrated separately (§III-L), and the convergence above shows that a mixture-derived score and a reverse-anchor score concur with the box rule's per-CPA-aggregated outputs on the directional ordering, with a modest disagreement at the less-replication-dominated end between the three non-A Big-4 firms.
**2. Per-signature consistency (Script 39).** Per-CPA aggregation could in principle reflect averaging across within-CPA heterogeneity rather than coherent within-CPA behaviour. We test this by repeating the K=3 fit at the signature level — fitting a fresh K=3 GMM to the 150,442 Big-4 signature-level $(\text{cos}, \text{dHash}_{\text{indep}})$ points (Script 39) — and comparing labels. The per-CPA and per-signature K=3 fits recover a broadly similar three-component ordering; per-CPA C1 is at $\overline{\text{cos}} = 0.946$, $\overline{\text{dHash}} = 9.17$ vs per-signature C1 at $\overline{\text{cos}} = 0.928$, $\overline{\text{dHash}} = 9.75$ (an absolute cosine drift of $0.018$). Cohen $\kappa$ on the binary collapse (replicated vs not-replicated): **2. Per-signature consistency (Script 39).** Per-CPA aggregation could in principle reflect averaging across within-CPA heterogeneity rather than coherent within-CPA behaviour. We test this by repeating the K=3 fit at the signature level — fitting a fresh K=3 GMM to the 150,442 Big-4 signature-level $(\text{cos}, \text{dHash}_{\text{indep}})$ points (Script 39) — and comparing labels. The per-CPA and per-signature K=3 fits recover a broadly similar three-component ordering; per-CPA C1 is at $\overline{\text{cos}} = 0.946$, $\overline{\text{dHash}} = 9.17$ vs per-signature C1 at $\overline{\text{cos}} = 0.928$, $\overline{\text{dHash}} = 9.75$ (an absolute cosine drift of $0.018$). Cohen $\kappa$ on the binary collapse (replication-dominated vs less-replication-dominated):
| Pair | Cohen $\kappa$ | | Pair | Cohen $\kappa$ |
|---|---| |---|---|
paper/v4/paper_a_prose_v4_phase4.md
+1 -1
View File
@@ -8,7 +8,7 @@
> *IEEE Access target: <= 250 words, single paragraph.* > *IEEE Access target: <= 250 words, single paragraph.*
Regulations require Certified Public Accountants (CPAs) to attest each audit report with a signature, but digitization makes reusing a stored signature image across reports — through administrative stamping or firm-level electronic signing — technically trivial and visually invisible, undermining individualized attestation. We build an end-to-end pipeline detecting such *non-hand-signed* signatures at scale: a Vision-Language Model identifies signature pages, YOLOv11 localizes signatures, ResNet-50 supplies deep features, and a dual-descriptor layer combines cosine similarity with an independent-minimum perceptual hash (dHash) to separate *style consistency* from *image reproduction*. Applied to 90,282 Taiwan audit reports (2013–2023), the pipeline yields 182,328 signatures from 758 CPAs; primary analyses are scoped to the Big-4 sub-corpus (437 CPAs; 150,442 signatures). Distributional diagnostics show that the apparent multimodality of the descriptor distribution dissolves under joint firm-mean centring and integer-tie jitter ($p$ rises to $0.35$), so no within-population bimodal antimode anchors the operational thresholds. We instead adopt an anchor-based inter-CPA coincidence-rate (ICCR) calibration at three units: per-comparison ($0.0006$ at cos$>0.95$; $0.0013$ at dHash$\leq 5$; $0.00014$ jointly), pool-normalised per-signature ($0.11$ under the deployed any-pair high-confidence rule), and per-document ($0.34$ for the operational HC+MC alarm). Firm heterogeneity is decisive: Firm A's per-document HC+MC alarm rate is $0.62$ versus $0.09$–$0.16$ at Firms B/C/D after pool-size adjustment, and under the deployed any-pair rule between $77\%$ and $99\%$ of inter-CPA collisions concentrate within the source firm (Firm A $98.8\%$; Firms B/C/D $76.7$–$83.7\%$) — consistent with firm-level template-like reuse. We position the system as a specificity-proxy-anchored screening framework with human-in-the-loop review, not as a validated forensic detector; no calibrated error rates are reportable without signature-level ground truth. Regulations require Certified Public Accountants (CPAs) to attest each audit report with a signature, but digitization makes reusing a stored signature image across reports — through administrative stamping or firm-level electronic signing — undermining individualized attestation. We build an end-to-end pipeline detecting such *non-hand-signed* signatures at scale: a Vision-Language Model identifies signature pages, YOLOv11 localizes signatures, ResNet-50 supplies deep features, and a dual-descriptor layer combines cosine similarity with an independent-minimum perceptual hash (dHash) to separate *style consistency* from *image reproduction*. Applied to 90,282 Taiwan audit reports (2013–2023), the pipeline yields 182,328 signatures from 758 CPAs; primary analyses are scoped to the Big-4 sub-corpus (437 CPAs; 150,442 signatures). Distributional diagnostics show that the apparent multimodality of the descriptor distribution dissolves under joint firm-mean centring and integer-tie jitter ($p$ rises to $0.35$), so no within-population bimodal antimode anchors the operational thresholds. We instead adopt an anchor-based inter-CPA coincidence-rate (ICCR) calibration at three units: per-comparison ($0.0006$ at cos$>0.95$; $0.0013$ at dHash$\leq 5$; $0.00014$ jointly), pool-normalised per-signature ($0.11$ under the deployed any-pair high-confidence rule), and per-document ($0.34$ for the operational HC+MC alarm). Firm heterogeneity is decisive: Firm A's per-document HC+MC alarm rate is $0.62$ versus $0.09$–$0.16$ at Firms B/C/D after pool-size adjustment, and under the deployed any-pair rule $77$–$99\%$ of inter-CPA collisions concentrate within the source firm — consistent with firm-level template-like reuse. We position the system as a specificity-proxy-anchored screening framework with human-in-the-loop review, not as a validated forensic detector; no calibrated error rates are reportable without signature-level ground truth.
--- ---
paper/v4/paper_a_results_v4_section_iv.md
+3 -3
View File
@@ -101,7 +101,7 @@ This section reports the empirical evidence for §III-K's three-score internal-c
The three scores agree on placing Firm A as the most replication-dominated and the three non-Firm-A firms as less replication-dominated. The K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Score 1 and Score 3) place Firm C at the least-replication-dominated end of Big-4; the reverse-anchor cosine percentile (Score 2) ranks Firm D fractionally above Firm C. This residual within-Big-4-non-A disagreement is a design feature of the reverse-anchor metric: Score 2 measures only the marginal cosine percentile under the non-Big-4 reference, so a firm with a slightly higher cosine but a markedly different dHash distribution (Firm D vs Firm C) can score higher on Score 2 while scoring lower on Scores 1 and 3, both of which use both descriptors. The three scores agree on placing Firm A as the most replication-dominated and the three non-Firm-A firms as less replication-dominated. The K=3 posterior P(C1) and the box-rule less-replication-dominated rate (Score 1 and Score 3) place Firm C at the least-replication-dominated end of Big-4; the reverse-anchor cosine percentile (Score 2) ranks Firm D fractionally above Firm C. This residual within-Big-4-non-A disagreement is a design feature of the reverse-anchor metric: Score 2 measures only the marginal cosine percentile under the non-Big-4 reference, so a firm with a slightly higher cosine but a markedly different dHash distribution (Firm D vs Firm C) can score higher on Score 2 while scoring lower on Scores 1 and 3, both of which use both descriptors.
**Table XI.** Per-signature Cohen $\kappa$ (binary collapse, replicated vs not-replicated), $n = 150{,}442$ Big-4 signatures. **Table XI.** Per-signature Cohen $\kappa$ (binary collapse, replication-dominated vs less-replication-dominated), $n = 150{,}442$ Big-4 signatures.
| Pair | Cohen $\kappa$ | | Pair | Cohen $\kappa$ |
|---|---| |---|---|
@@ -158,7 +158,7 @@ We caution that for the Paper A box rule this result is close to tautological (b
The signature-level inter-CPA pair-level coincidence-rate analysis (reported in v3.20.0 §IV-F.1, Table X as "FAR") is inherited and extended in v4.0. v4.0 retroactively reframes the metric as **inter-CPA pair-level coincidence rate (ICCR)** rather than "False Acceptance Rate" because the corpus does not provide signature-level ground-truth negative labels; the inter-CPA negative-anchor assumption underpinning the metric is itself partially violated by within-firm cross-CPA template-like collision structures (§III-L.4). The v3.20.0 corpus-wide spike on $\sim 50{,}000$ inter-CPA pairs reported a per-comparison rate of $0.0005$ (Wilson 95% CI $[0.0003, 0.0007]$) at the cosine cut $0.95$. The signature-level inter-CPA pair-level coincidence-rate analysis (reported in v3.20.0 §IV-F.1, Table X as "FAR") is inherited and extended in v4.0. v4.0 retroactively reframes the metric as **inter-CPA pair-level coincidence rate (ICCR)** rather than "False Acceptance Rate" because the corpus does not provide signature-level ground-truth negative labels; the inter-CPA negative-anchor assumption underpinning the metric is itself partially violated by within-firm cross-CPA template-like collision structures (§III-L.4). The v3.20.0 corpus-wide spike on $\sim 50{,}000$ inter-CPA pairs reported a per-comparison rate of $0.0005$ (Wilson 95% CI $[0.0003, 0.0007]$) at the cosine cut $0.95$.
v4.0 additionally reports the §III-L.1 Big-4-scope spike at higher sample size ($5 \times 10^5$ inter-CPA pairs; Script 40b), which replicates and extends the v3 result and adds the structural dimension (dHash) and joint-rule rates. The §III-L.1 numbers are referenced rather than duplicated here; the consolidated v4-new ICCR calibration appears in §IV-M Table XVI. v4.0 additionally reports the §III-L.1 Big-4-scope spike at higher sample size ($5 \times 10^5$ inter-CPA pairs; Script 40b), which replicates and extends the v3 result and adds the structural dimension (dHash) and joint-rule rates. The §III-L.1 numbers are referenced rather than duplicated here; the consolidated v4-new ICCR calibration appears in §IV-M Tables XXI–XXVI.
## J. Five-Way Per-Signature + Document-Level Classification Output ## J. Five-Way Per-Signature + Document-Level Classification Output
@@ -174,7 +174,7 @@ This section reports the §III-L five-way per-signature + document-level worst-c
| UN | Uncertain | 35,480 | 23.58% | | UN | Uncertain | 35,480 | 23.58% |
| LH | Likely hand-signed | 238 | 0.16% | | LH | Likely hand-signed | 238 | 0.16% |
(Source: Script 42; 11 of 150,453 loaded Big-4 signatures lacked one or both descriptors and were excluded. The $150{,}442$ vs $150{,}453$ distinction — descriptor-complete vs vector-complete — recurs across §IV: §IV-D through §IV-J and §IV-M.5 use $n = 150{,}442$; §IV-M.3 Table XXII uses $n = 150{,}453$ because the pool-normalised ICCR analysis loads all vector-complete signatures including those failing the descriptor-complete filter. See §III-G for the sample-size reconciliation.) (Source: Script 42; 11 of 150,453 loaded Big-4 signatures lacked one or both descriptors and were excluded. The $150{,}442$ vs $150{,}453$ distinction — descriptor-complete vs vector-complete — recurs across §IV: descriptor-complete analyses (§IV-D through §IV-J, all using accountant-level aggregates or per-signature category counts derived from the same 150,442-signature substrate) use $n = 150{,}442$; vector- or pair-recomputed analyses (§IV-M.2 Table XXI, §IV-M.3 Table XXII, §IV-M.5 Tables XXIV–XXV; Scripts 40b, 43, 44) use $n = 150{,}453$ because their pair- or pool-level computations load all vector-complete signatures including those failing the descriptor-complete filter. See §III-G for the sample-size reconciliation.)
**Per-firm five-way breakdown (% within firm).** **Per-firm five-way breakdown (% within firm).**