Paper A v3.20.0: partner Jimmy 2026-04-27 review + DOCX rendering overhaul
Substantive content (addresses partner Jimmy's 2026-04-27 review of v3.19.1): Must-fix items (6/6): - §III-F SSIM/pixel rejection rewritten from first principles (design-level argument from luminance/contrast/structure local-window product, not the prior empirical 0.70 result) - Table VI restructured by population × method; added missing Firm A logit-Gaussian-2 0.999 row; KDE marked undefined (unimodal), BD/McCrary marked bin-unstable (Appendix A) - Tables IX / XI / §IV-F.3 dHash 5/8/15 inconsistency resolved: ≤8 demoted from "operational dual" to "calibration-fold-adjacent reference"; the actual classifier rule cos>0.95 AND dH≤15 = 92.46% added throughout - New Fig. 4 (yearly per-firm best-match cosine, 5 lines, 2013-2023, Firm A on top); script 30_yearly_big4_comparison.py - Tables XIV / XV extended with top-20% (94.8%) and top-30% (81.3%) brackets - §III-K reframed P7.5 from "round-number lower-tail boundary" to operating point; new Table XII-B (cosine-FAR-capture tradeoff at 5 thresholds: 0.9407 / 0.945 / 0.95 / 0.977 / 0.985) Nice-to-have items (3/3): - Table XII expanded to 6-cut classifier sensitivity grid (0.940-0.985) - Defensive parentheticals (84,386 vs 85,042; 30,226 vs 30,222) moved to table notes; cut "invite reviewer skepticism" and "non-load-bearing" Codex 3-pass verification cleanup: - Stale 0.973/0.977/0.979 references unified on canonical 0.977 (Firm A Beta-2 forced-fit crossing from beta_mixture_results.json) - dHash≤8 wording corrected to P95-adjacent (P95 = 9, ≤8 is the integer immediately below) instead of misleading "rounded down" - Table XII-B prose corrected: per-segment qualification of "non-Firm-A capture falls faster" (true on 0.95→0.977 segment but contracts on 0.977→0.985 segment); arithmetic now from exact counts Within-year analyses removed: - Within-year ranking robustness check (Class A) was added in nice-to-have pass but contradicts v3.14 A2-removal stance; removed from §IV-G.2 + the Appendix B provenance row - Within-CPA future-work disclosures (Class B) removed from Discussion limitation #5 and Conclusion future-work paragraph; subsequent limitations renumbered Sixth → Fifth, Seventh → Sixth DOCX rendering pipeline overhaul (paper/export_v3.py): Critical fix - every v3 DOCX since v3.0 was shipping WITHOUT TABLES: strip_comments() was wholesale-deleting HTML comments, but every numerical table is wrapped in <!-- TABLE X: ... -->, so the table body was deleted alongside the wrapper. Now unwraps TABLE comments (emit synthetic __TABLE_CAPTION__: marker + table body) while still stripping non-TABLE editorial comments. Result: 19 tables now render in the DOCX. Other rendering fixes: - LaTeX → Unicode conversion (50+ token replacements: Greek alphabet, ≤≥, ×·≈, →↔⇒, etc.); \frac/\sqrt linearisation; TeX brace tricks ({=}, {,}) - Math-context-scoped sub/superscript via PUA sentinels (/): no more underscore-eating in identifiers like signature_analysis - Display equations rendered via matplotlib mathtext to PNG (3 equations: cosine sim, mixture crossing, BD/McCrary Z statistic), embedded as numbered equation blocks (1), (2), (3); content-addressed cache at paper/equations/ (gitignored, regenerable) - Manual numbered/bulleted list rendering with hanging indent (replaces python-docx style="List Number" which silently drops the number prefix when no numbering definition is bound) - Markdown blockquote (> ...) defensively stripped - Pandoc footnote ([^name]) markers no longer leak (inlined at source) - Heading text cleaned of LaTeX residue + PUA sentinels - File paths in body text (signature_analysis/X.py, reports/Y.json) trimmed to "(reproduction artifact in Appendix B)" pointers New leak linter: paper/lint_paper_v3.py - two-pass markdown source + rendered DOCX leak detector; auto-runs at end of export_v3.py. Script changes: - 21_expanded_validation.py: added 0.9407, 0.977, 0.985 to canonical FAR threshold list so Table XII-B is reproducible from persisted JSON - 30_yearly_big4_comparison.py: NEW; generates Fig. 4 + per-firm yearly data (writes to reports/figures/ and reports/firm_yearly_comparison/) - 31_within_year_ranking_robustness.py: NEW; supports the within-year robustness check (no longer cited in paper but kept as repo-internal due-diligence artifact) Partner handoff DOCX shipped to ~/Downloads/Paper_A_IEEE_Access_Draft_v3.20.0_20260505.docx (536 KB: 19 tables + 4 figures + 3 equation images). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,249 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script 31: Within-Year Same-CPA Ranking Robustness Check
|
||||
==========================================================
|
||||
Recomputes the per-auditor-year mean cosine ranking of Table XIV using
|
||||
within-year same-CPA matching only (instead of cross-year same-CPA pool
|
||||
which Table XIV uses by construction). Reports pooled top-10/20/30%
|
||||
Firm A share under the within-year restriction so the partner-level
|
||||
ranking finding can be checked against the cross-year aggregation
|
||||
choice flagged in Section IV-G.2.
|
||||
|
||||
Definition (within-year statistic):
|
||||
For each signature s, with CPA = c, year = y:
|
||||
cos_within(s) = max cosine(s, s') over s' != s, CPA(s')=c, year(s')=y
|
||||
If a (CPA, year) block has only one signature, cos_within is undefined
|
||||
and that signature is dropped from the auditor-year aggregation
|
||||
(matching the same-CPA pair-existence requirement of Section III-G).
|
||||
|
||||
Outputs:
|
||||
reports/within_year_ranking/within_year_ranking.json
|
||||
reports/within_year_ranking/within_year_ranking.md
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from collections import defaultdict
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
DB = '/Volumes/NV2/PDF-Processing/signature-analysis/signature_analysis.db'
|
||||
OUT = Path('/Volumes/NV2/PDF-Processing/signature-analysis/reports/'
|
||||
'within_year_ranking')
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
FIRM_A = '勤業眾信聯合'
|
||||
MIN_SIGS_PER_AUDITOR_YEAR = 5
|
||||
|
||||
|
||||
def firm_bucket(firm):
|
||||
if firm == '勤業眾信聯合':
|
||||
return 'Firm A'
|
||||
if firm == '安侯建業聯合':
|
||||
return 'Firm B'
|
||||
if firm == '資誠聯合':
|
||||
return 'Firm C'
|
||||
if firm == '安永聯合':
|
||||
return 'Firm D'
|
||||
return 'Non-Big-4'
|
||||
|
||||
|
||||
def load_signatures():
|
||||
conn = sqlite3.connect(DB)
|
||||
cur = conn.cursor()
|
||||
cur.execute("""
|
||||
SELECT s.signature_id, s.assigned_accountant, a.firm,
|
||||
CAST(substr(s.year_month, 1, 4) AS INTEGER) AS year,
|
||||
s.feature_vector
|
||||
FROM signatures s
|
||||
LEFT JOIN accountants a ON s.assigned_accountant = a.name
|
||||
WHERE s.feature_vector IS NOT NULL
|
||||
AND s.assigned_accountant IS NOT NULL
|
||||
AND s.year_month IS NOT NULL
|
||||
""")
|
||||
rows = cur.fetchall()
|
||||
conn.close()
|
||||
return rows
|
||||
|
||||
|
||||
def compute_within_year_max(rows):
|
||||
"""Group by (CPA, year), compute max cosine to other same-block sigs."""
|
||||
blocks = defaultdict(list) # (cpa, year) -> [(sig_id, feat)]
|
||||
for sig_id, cpa, firm, year, blob in rows:
|
||||
if year is None:
|
||||
continue
|
||||
feat = np.frombuffer(blob, dtype=np.float32)
|
||||
blocks[(cpa, int(year))].append((sig_id, feat, firm))
|
||||
|
||||
sig_max_within = {} # sig_id -> max within-year same-CPA cosine
|
||||
sig_meta = {} # sig_id -> (cpa, year, firm)
|
||||
for (cpa, year), entries in blocks.items():
|
||||
if len(entries) < 2:
|
||||
continue # singleton: max-within is undefined
|
||||
feats = np.stack([e[1] for e in entries]) # (n, 2048)
|
||||
sims = feats @ feats.T # (n, n)
|
||||
np.fill_diagonal(sims, -np.inf)
|
||||
maxs = sims.max(axis=1)
|
||||
for i, (sig_id, _, firm) in enumerate(entries):
|
||||
sig_max_within[sig_id] = float(maxs[i])
|
||||
sig_meta[sig_id] = (cpa, year, firm)
|
||||
return sig_max_within, sig_meta
|
||||
|
||||
|
||||
def auditor_year_aggregation(sig_max_within, sig_meta):
|
||||
by_ay = defaultdict(list) # (cpa, year) -> list of cos
|
||||
for sig_id, cos in sig_max_within.items():
|
||||
cpa, year, firm = sig_meta[sig_id]
|
||||
by_ay[(cpa, year)].append(cos)
|
||||
rows = []
|
||||
for (cpa, year), vals in by_ay.items():
|
||||
if len(vals) < MIN_SIGS_PER_AUDITOR_YEAR:
|
||||
continue
|
||||
firm = sig_meta[next(s for s in sig_max_within
|
||||
if sig_meta[s][0] == cpa
|
||||
and sig_meta[s][1] == year)][2]
|
||||
rows.append({
|
||||
'acct': cpa,
|
||||
'year': year,
|
||||
'firm': firm,
|
||||
'cos_mean_within_year': float(np.mean(vals)),
|
||||
'n': len(vals),
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def top_k_breakdown(rows, k_pcts=(10, 20, 25, 30, 50)):
|
||||
sorted_rows = sorted(rows, key=lambda r: -r['cos_mean_within_year'])
|
||||
N = len(sorted_rows)
|
||||
out = {}
|
||||
for k_pct in k_pcts:
|
||||
k = max(1, int(N * k_pct / 100))
|
||||
top = sorted_rows[:k]
|
||||
counts = defaultdict(int)
|
||||
for r in top:
|
||||
counts[firm_bucket(r['firm'])] += 1
|
||||
out[f'top_{k_pct}pct'] = {
|
||||
'k': k,
|
||||
'firm_counts': dict(counts),
|
||||
'firm_a_share': counts['Firm A'] / k,
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def per_year_top_k(rows, k_pcts=(10, 20, 30)):
|
||||
years = sorted(set(r['year'] for r in rows))
|
||||
out = {}
|
||||
for y in years:
|
||||
yr = [r for r in rows if r['year'] == y]
|
||||
if not yr:
|
||||
continue
|
||||
sr = sorted(yr, key=lambda r: -r['cos_mean_within_year'])
|
||||
n_y = len(sr)
|
||||
n_a = sum(1 for r in sr if r['firm'] == FIRM_A)
|
||||
per = {'n_auditor_years': n_y,
|
||||
'firm_a_baseline_share': n_a / n_y,
|
||||
'top_k': {}}
|
||||
for kp in k_pcts:
|
||||
k = max(1, int(n_y * kp / 100))
|
||||
n_a_top = sum(1 for r in sr[:k] if r['firm'] == FIRM_A)
|
||||
per['top_k'][f'top_{kp}pct'] = {
|
||||
'k': k,
|
||||
'firm_a_in_top': n_a_top,
|
||||
'firm_a_share': n_a_top / k,
|
||||
}
|
||||
out[y] = per
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
print('Loading signatures + features...')
|
||||
rows = load_signatures()
|
||||
print(f' loaded {len(rows):,}')
|
||||
|
||||
print('Computing within-year same-CPA max cosine...')
|
||||
sig_max_within, sig_meta = compute_within_year_max(rows)
|
||||
print(f' signatures with within-year pair: {len(sig_max_within):,}')
|
||||
n_dropped = len(rows) - len(sig_max_within)
|
||||
print(f' dropped (singleton within year): {n_dropped:,}')
|
||||
|
||||
ay_rows = auditor_year_aggregation(sig_max_within, sig_meta)
|
||||
print(f' auditor-years (>={MIN_SIGS_PER_AUDITOR_YEAR} sigs '
|
||||
f'with within-year pair): {len(ay_rows):,}')
|
||||
|
||||
pooled = top_k_breakdown(ay_rows)
|
||||
yearly = per_year_top_k(ay_rows)
|
||||
|
||||
payload = {
|
||||
'generated_at': datetime.now().isoformat(timespec='seconds'),
|
||||
'n_signatures_loaded': len(rows),
|
||||
'n_signatures_with_within_year_pair': len(sig_max_within),
|
||||
'n_singleton_dropped': n_dropped,
|
||||
'min_sigs_per_auditor_year': MIN_SIGS_PER_AUDITOR_YEAR,
|
||||
'n_auditor_years': len(ay_rows),
|
||||
'n_firm_a_auditor_years': sum(1 for r in ay_rows
|
||||
if r['firm'] == FIRM_A),
|
||||
'pooled_top_k': pooled,
|
||||
'yearly_top_k': yearly,
|
||||
}
|
||||
json_path = OUT / 'within_year_ranking.json'
|
||||
json_path.write_text(json.dumps(payload, indent=2, ensure_ascii=False),
|
||||
encoding='utf-8')
|
||||
print(f'\nWrote {json_path}')
|
||||
|
||||
# Markdown
|
||||
md = ['# Within-Year Same-CPA Ranking Robustness',
|
||||
'',
|
||||
f"Generated: {payload['generated_at']}",
|
||||
'',
|
||||
('Per-signature best-match cosine recomputed using within-year '
|
||||
'same-CPA matching only. See Script 31 docstring for the '
|
||||
'precise definition.'),
|
||||
'',
|
||||
f"- Signatures loaded: {len(rows):,}",
|
||||
f"- Signatures with at least one within-year same-CPA pair: "
|
||||
f"{len(sig_max_within):,}",
|
||||
f"- Singletons dropped (no within-year pair): {n_dropped:,}",
|
||||
f"- Auditor-years with >= {MIN_SIGS_PER_AUDITOR_YEAR} sigs: "
|
||||
f"{len(ay_rows):,}",
|
||||
f"- Firm A auditor-years: {payload['n_firm_a_auditor_years']:,} "
|
||||
f"({100*payload['n_firm_a_auditor_years']/len(ay_rows):.1f}% baseline)",
|
||||
'',
|
||||
'## Pooled (2013-2023) top-K Firm A share',
|
||||
'',
|
||||
'| Top-K | k | Firm A share | A | B | C | D | NB4 |',
|
||||
'|-------|---|--------------|---|---|---|---|-----|']
|
||||
for kp in [10, 20, 25, 30, 50]:
|
||||
d = pooled[f'top_{kp}pct']
|
||||
c = d['firm_counts']
|
||||
md.append(f"| {kp}% | {d['k']:,} | "
|
||||
f"{100*d['firm_a_share']:.1f}% | "
|
||||
f"{c.get('Firm A', 0)} | {c.get('Firm B', 0)} | "
|
||||
f"{c.get('Firm C', 0)} | {c.get('Firm D', 0)} | "
|
||||
f"{c.get('Non-Big-4', 0)} |")
|
||||
|
||||
md.extend(['',
|
||||
'## Year-by-year top-K Firm A share',
|
||||
'',
|
||||
'| Year | n AY | Top-10% share | Top-20% share | '
|
||||
'Top-30% share | A baseline |',
|
||||
'|------|------|---------------|---------------|'
|
||||
'---------------|------------|'])
|
||||
for y in sorted(yearly):
|
||||
per = yearly[y]
|
||||
line = (f"| {y} | {per['n_auditor_years']:,} ")
|
||||
for kp in [10, 20, 30]:
|
||||
d = per['top_k'][f'top_{kp}pct']
|
||||
line += (f"| {100*d['firm_a_share']:.1f}% "
|
||||
f"({d['firm_a_in_top']}/{d['k']}) ")
|
||||
line += f"| {100*per['firm_a_baseline_share']:.1f}% |"
|
||||
md.append(line)
|
||||
|
||||
md_path = OUT / 'within_year_ranking.md'
|
||||
md_path.write_text('\n'.join(md) + '\n', encoding='utf-8')
|
||||
print(f'Wrote {md_path}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user