Paper A v3.20.0: partner Jimmy 2026-04-27 review + DOCX rendering overhaul
Substantive content (addresses partner Jimmy's 2026-04-27 review of v3.19.1): Must-fix items (6/6): - §III-F SSIM/pixel rejection rewritten from first principles (design-level argument from luminance/contrast/structure local-window product, not the prior empirical 0.70 result) - Table VI restructured by population × method; added missing Firm A logit-Gaussian-2 0.999 row; KDE marked undefined (unimodal), BD/McCrary marked bin-unstable (Appendix A) - Tables IX / XI / §IV-F.3 dHash 5/8/15 inconsistency resolved: ≤8 demoted from "operational dual" to "calibration-fold-adjacent reference"; the actual classifier rule cos>0.95 AND dH≤15 = 92.46% added throughout - New Fig. 4 (yearly per-firm best-match cosine, 5 lines, 2013-2023, Firm A on top); script 30_yearly_big4_comparison.py - Tables XIV / XV extended with top-20% (94.8%) and top-30% (81.3%) brackets - §III-K reframed P7.5 from "round-number lower-tail boundary" to operating point; new Table XII-B (cosine-FAR-capture tradeoff at 5 thresholds: 0.9407 / 0.945 / 0.95 / 0.977 / 0.985) Nice-to-have items (3/3): - Table XII expanded to 6-cut classifier sensitivity grid (0.940-0.985) - Defensive parentheticals (84,386 vs 85,042; 30,226 vs 30,222) moved to table notes; cut "invite reviewer skepticism" and "non-load-bearing" Codex 3-pass verification cleanup: - Stale 0.973/0.977/0.979 references unified on canonical 0.977 (Firm A Beta-2 forced-fit crossing from beta_mixture_results.json) - dHash≤8 wording corrected to P95-adjacent (P95 = 9, ≤8 is the integer immediately below) instead of misleading "rounded down" - Table XII-B prose corrected: per-segment qualification of "non-Firm-A capture falls faster" (true on 0.95→0.977 segment but contracts on 0.977→0.985 segment); arithmetic now from exact counts Within-year analyses removed: - Within-year ranking robustness check (Class A) was added in nice-to-have pass but contradicts v3.14 A2-removal stance; removed from §IV-G.2 + the Appendix B provenance row - Within-CPA future-work disclosures (Class B) removed from Discussion limitation #5 and Conclusion future-work paragraph; subsequent limitations renumbered Sixth → Fifth, Seventh → Sixth DOCX rendering pipeline overhaul (paper/export_v3.py): Critical fix - every v3 DOCX since v3.0 was shipping WITHOUT TABLES: strip_comments() was wholesale-deleting HTML comments, but every numerical table is wrapped in <!-- TABLE X: ... -->, so the table body was deleted alongside the wrapper. Now unwraps TABLE comments (emit synthetic __TABLE_CAPTION__: marker + table body) while still stripping non-TABLE editorial comments. Result: 19 tables now render in the DOCX. Other rendering fixes: - LaTeX → Unicode conversion (50+ token replacements: Greek alphabet, ≤≥, ×·≈, →↔⇒, etc.); \frac/\sqrt linearisation; TeX brace tricks ({=}, {,}) - Math-context-scoped sub/superscript via PUA sentinels (/): no more underscore-eating in identifiers like signature_analysis - Display equations rendered via matplotlib mathtext to PNG (3 equations: cosine sim, mixture crossing, BD/McCrary Z statistic), embedded as numbered equation blocks (1), (2), (3); content-addressed cache at paper/equations/ (gitignored, regenerable) - Manual numbered/bulleted list rendering with hanging indent (replaces python-docx style="List Number" which silently drops the number prefix when no numbering definition is bound) - Markdown blockquote (> ...) defensively stripped - Pandoc footnote ([^name]) markers no longer leak (inlined at source) - Heading text cleaned of LaTeX residue + PUA sentinels - File paths in body text (signature_analysis/X.py, reports/Y.json) trimmed to "(reproduction artifact in Appendix B)" pointers New leak linter: paper/lint_paper_v3.py - two-pass markdown source + rendered DOCX leak detector; auto-runs at end of export_v3.py. Script changes: - 21_expanded_validation.py: added 0.9407, 0.977, 0.985 to canonical FAR threshold list so Table XII-B is reproducible from persisted JSON - 30_yearly_big4_comparison.py: NEW; generates Fig. 4 + per-firm yearly data (writes to reports/figures/ and reports/firm_yearly_comparison/) - 31_within_year_ranking_robustness.py: NEW; supports the within-year robustness check (no longer cited in paper but kept as repo-internal due-diligence artifact) Partner handoff DOCX shipped to ~/Downloads/Paper_A_IEEE_Access_Draft_v3.20.0_20260505.docx (536 KB: 19 tables + 4 figures + 3 equation images). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script 30: Yearly Per-Firm Cosine Similarity Comparison
|
||||
========================================================
|
||||
Generates the per-firm year-by-year per-signature best-match cosine
|
||||
distribution: Firm A (Deloitte), Firm B (KPMG), Firm C (PwC),
|
||||
Firm D (EY), Non-Big-4. The two-panel figure (mean cosine; share above
|
||||
0.95) is the headline cross-firm visual requested in partner review of
|
||||
v3.19.1 (2026-04-27): five lines, X-axis 2013-2023, Firm A at the top.
|
||||
|
||||
Outputs:
|
||||
reports/figures/fig_yearly_big4_comparison.png
|
||||
reports/figures/fig_yearly_big4_comparison.pdf
|
||||
reports/firm_yearly_comparison/firm_yearly_comparison.json
|
||||
reports/firm_yearly_comparison/firm_yearly_comparison.md
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
DB = '/Volumes/NV2/PDF-Processing/signature-analysis/signature_analysis.db'
|
||||
FIG_OUT = Path('/Volumes/NV2/PDF-Processing/signature-analysis/reports/'
|
||||
'figures')
|
||||
DATA_OUT = Path('/Volumes/NV2/PDF-Processing/signature-analysis/reports/'
|
||||
'firm_yearly_comparison')
|
||||
FIG_OUT.mkdir(parents=True, exist_ok=True)
|
||||
DATA_OUT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
FIRM_BUCKETS = [
|
||||
('Firm A', '勤業眾信聯合'),
|
||||
('Firm B', '安侯建業聯合'),
|
||||
('Firm C', '資誠聯合'),
|
||||
('Firm D', '安永聯合'),
|
||||
]
|
||||
|
||||
FIRM_COLORS = {
|
||||
'Firm A': '#d62728',
|
||||
'Firm B': '#1f77b4',
|
||||
'Firm C': '#2ca02c',
|
||||
'Firm D': '#9467bd',
|
||||
'Non-Big-4': '#7f7f7f',
|
||||
}
|
||||
FIRM_MARKERS = {
|
||||
'Firm A': 'o',
|
||||
'Firm B': 's',
|
||||
'Firm C': '^',
|
||||
'Firm D': 'D',
|
||||
'Non-Big-4': 'v',
|
||||
}
|
||||
COSINE_CUT = 0.95
|
||||
|
||||
|
||||
def firm_bucket(firm):
|
||||
for label, name in FIRM_BUCKETS:
|
||||
if firm == name:
|
||||
return label
|
||||
return 'Non-Big-4'
|
||||
|
||||
|
||||
def load_rows(conn):
|
||||
cur = conn.cursor()
|
||||
cur.execute("""
|
||||
SELECT a.firm,
|
||||
CAST(substr(s.year_month, 1, 4) AS INTEGER) AS year,
|
||||
s.max_similarity_to_same_accountant
|
||||
FROM signatures s
|
||||
LEFT JOIN accountants a ON s.assigned_accountant = a.name
|
||||
WHERE s.max_similarity_to_same_accountant IS NOT NULL
|
||||
AND s.year_month IS NOT NULL
|
||||
AND s.assigned_accountant IS NOT NULL
|
||||
""")
|
||||
return cur.fetchall()
|
||||
|
||||
|
||||
def aggregate(rows):
|
||||
"""Returns dict keyed by (firm_label, year) -> {n, mean_cos, share_ge_cut}."""
|
||||
by_firm_year = {}
|
||||
for firm, year, cos in rows:
|
||||
if year is None or year < 2013 or year > 2023:
|
||||
continue
|
||||
label = firm_bucket(firm)
|
||||
key = (label, int(year))
|
||||
by_firm_year.setdefault(key, []).append(float(cos))
|
||||
|
||||
summary = {}
|
||||
for (label, year), vals in by_firm_year.items():
|
||||
arr = np.array(vals, dtype=float)
|
||||
summary[(label, year)] = {
|
||||
'n': int(arr.size),
|
||||
'mean_cos': float(arr.mean()),
|
||||
'share_ge_cut': float(np.mean(arr >= COSINE_CUT)),
|
||||
}
|
||||
return summary
|
||||
|
||||
|
||||
def plot_figure(summary, years, firm_labels, fig_path_png, fig_path_pdf):
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
|
||||
ax = axes[0]
|
||||
for label in firm_labels:
|
||||
ys = [summary[(label, y)]['mean_cos']
|
||||
if (label, y) in summary else np.nan
|
||||
for y in years]
|
||||
ax.plot(years, ys,
|
||||
marker=FIRM_MARKERS[label], color=FIRM_COLORS[label],
|
||||
lw=2.0, ms=6, label=label,
|
||||
zorder=3 if label == 'Firm A' else 2)
|
||||
ax.set_xlabel('Fiscal year')
|
||||
ax.set_ylabel('Mean per-signature best-match cosine')
|
||||
ax.set_title('(a) Mean per-signature best-match cosine, by firm and year')
|
||||
ax.set_xticks(years)
|
||||
ax.tick_params(axis='x', rotation=0)
|
||||
ax.grid(True, ls=':', alpha=0.4)
|
||||
ax.legend(loc='lower right', framealpha=0.95)
|
||||
|
||||
ax = axes[1]
|
||||
for label in firm_labels:
|
||||
ys = [100.0 * summary[(label, y)]['share_ge_cut']
|
||||
if (label, y) in summary else np.nan
|
||||
for y in years]
|
||||
ax.plot(years, ys,
|
||||
marker=FIRM_MARKERS[label], color=FIRM_COLORS[label],
|
||||
lw=2.0, ms=6, label=label,
|
||||
zorder=3 if label == 'Firm A' else 2)
|
||||
ax.set_xlabel('Fiscal year')
|
||||
ax.set_ylabel(f'% signatures with best-match cosine $\\geq$ {COSINE_CUT}')
|
||||
ax.set_title(f'(b) Share with cosine $\\geq$ {COSINE_CUT}, '
|
||||
'by firm and year')
|
||||
ax.set_xticks(years)
|
||||
ax.tick_params(axis='x', rotation=0)
|
||||
ax.grid(True, ls=':', alpha=0.4)
|
||||
ax.legend(loc='lower right', framealpha=0.95)
|
||||
ax.set_ylim(0, 100)
|
||||
|
||||
fig.suptitle('Per-firm yearly per-signature best-match cosine '
|
||||
'(operational cut shown as 0.95)',
|
||||
fontsize=12, y=1.02)
|
||||
fig.tight_layout()
|
||||
fig.savefig(fig_path_png, dpi=200, bbox_inches='tight')
|
||||
fig.savefig(fig_path_pdf, bbox_inches='tight')
|
||||
plt.close(fig)
|
||||
|
||||
|
||||
def write_markdown(summary, years, firm_labels, md_path):
|
||||
lines = ['# Per-Firm Yearly Cosine Comparison',
|
||||
'',
|
||||
f"Generated: {datetime.now().isoformat(timespec='seconds')}",
|
||||
'',
|
||||
('Per-signature best-match cosine '
|
||||
'(`max_similarity_to_same_accountant`), aggregated by firm '
|
||||
'bucket and fiscal year. Firm bucket via CPA registry '
|
||||
'(`accountants.firm`).'),
|
||||
'']
|
||||
|
||||
lines.append('## Mean per-signature best-match cosine')
|
||||
lines.append('')
|
||||
header = '| Year | ' + ' | '.join(firm_labels) + ' |'
|
||||
sep = '|------|' + '|'.join(['------'] * len(firm_labels)) + '|'
|
||||
lines.append(header)
|
||||
lines.append(sep)
|
||||
for y in years:
|
||||
row = f'| {y} | '
|
||||
cells = []
|
||||
for lab in firm_labels:
|
||||
if (lab, y) in summary:
|
||||
cells.append(f"{summary[(lab, y)]['mean_cos']:.4f}")
|
||||
else:
|
||||
cells.append('---')
|
||||
row += ' | '.join(cells) + ' |'
|
||||
lines.append(row)
|
||||
|
||||
lines.append('')
|
||||
lines.append(f'## Share with cosine $\\geq$ {COSINE_CUT}')
|
||||
lines.append('')
|
||||
lines.append(header)
|
||||
lines.append(sep)
|
||||
for y in years:
|
||||
row = f'| {y} | '
|
||||
cells = []
|
||||
for lab in firm_labels:
|
||||
if (lab, y) in summary:
|
||||
cells.append(f"{100*summary[(lab, y)]['share_ge_cut']:.1f}%")
|
||||
else:
|
||||
cells.append('---')
|
||||
row += ' | '.join(cells) + ' |'
|
||||
lines.append(row)
|
||||
|
||||
lines.append('')
|
||||
lines.append('## Per-firm signature counts')
|
||||
lines.append('')
|
||||
lines.append(header)
|
||||
lines.append(sep)
|
||||
for y in years:
|
||||
row = f'| {y} | '
|
||||
cells = []
|
||||
for lab in firm_labels:
|
||||
if (lab, y) in summary:
|
||||
cells.append(f"{summary[(lab, y)]['n']:,}")
|
||||
else:
|
||||
cells.append('---')
|
||||
row += ' | '.join(cells) + ' |'
|
||||
lines.append(row)
|
||||
|
||||
md_path.write_text('\n'.join(lines) + '\n', encoding='utf-8')
|
||||
|
||||
|
||||
def main():
|
||||
conn = sqlite3.connect(DB)
|
||||
try:
|
||||
rows = load_rows(conn)
|
||||
finally:
|
||||
conn.close()
|
||||
print(f'Loaded {len(rows):,} signatures with cosine + year + firm.')
|
||||
|
||||
summary = aggregate(rows)
|
||||
years = sorted({y for (_, y) in summary})
|
||||
firm_labels = ['Firm A', 'Firm B', 'Firm C', 'Firm D', 'Non-Big-4']
|
||||
|
||||
fig_png = FIG_OUT / 'fig_yearly_big4_comparison.png'
|
||||
fig_pdf = FIG_OUT / 'fig_yearly_big4_comparison.pdf'
|
||||
plot_figure(summary, years, firm_labels, fig_png, fig_pdf)
|
||||
print(f'Wrote {fig_png}')
|
||||
print(f'Wrote {fig_pdf}')
|
||||
|
||||
payload = {
|
||||
'generated_at': datetime.now().isoformat(timespec='seconds'),
|
||||
'database_path': DB,
|
||||
'cosine_cut': COSINE_CUT,
|
||||
'firm_buckets': dict(FIRM_BUCKETS) | {'Non-Big-4': 'all other'},
|
||||
'years': years,
|
||||
'rows': [
|
||||
{'firm': lab, 'year': y, **summary[(lab, y)]}
|
||||
for lab in firm_labels for y in years
|
||||
if (lab, y) in summary
|
||||
],
|
||||
}
|
||||
json_path = DATA_OUT / 'firm_yearly_comparison.json'
|
||||
json_path.write_text(json.dumps(payload, indent=2, ensure_ascii=False),
|
||||
encoding='utf-8')
|
||||
print(f'Wrote {json_path}')
|
||||
|
||||
md_path = DATA_OUT / 'firm_yearly_comparison.md'
|
||||
write_markdown(summary, years, firm_labels, md_path)
|
||||
print(f'Wrote {md_path}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user