Compare commits
2 Commits
1e37d344ea
...
5e7e76cf35
| Author | SHA1 | Date | |
|---|---|---|---|
| 5e7e76cf35 | |||
| af08391a68 |
Binary file not shown.
@@ -0,0 +1,68 @@
|
||||
# Independent Peer Review (Round 19) - Paper A v3.18.4
|
||||
|
||||
## 1. Overall Verdict: Major Revision
|
||||
|
||||
I recommend **Major Revision**. While v3.18.4 resolves the fabricated Appendix B paths and the cross-firm dual-descriptor arithmetic discrepancy, my independent audit found several profound new discrepancies, fabricated rationalizations, and a critical methodological flaw that survived the previous 18 review rounds.
|
||||
|
||||
The most severe issues are:
|
||||
1. **Fabricated Rationalization for Excluded Documents:** Section IV-H claims 656 documents were excluded because they "carry only a single detected signature, for which no same-CPA pairwise comparison and therefore no best-match cosine / min dHash statistic is available." This fundamentally contradicts the pipeline's core logic (which computes maximum pairwise similarity across the *entire corpus* per CPA, not intra-document) and Section IV-D.1 (which correctly states only 15 signatures belong to singleton CPAs). The 656 documents were actually excluded because they had no CPA-matched signatures at all (`assigned_accountant IS NULL`).
|
||||
2. **Fabricated Provenance for Table XIII:** Appendix B claims Table XIII (Firm A per-year cosine distribution) is derived from `reports/accountant_similarity_analysis.json`. However, the generating script (`08_accountant_similarity_analysis.py`) neither extracts nor groups by the `year_month` field. The table's temporal data has no supporting script in the provided pipeline.
|
||||
3. **Fabricated Rationalization for Firm A Partners:** Section IV-F.2 claims "two [CPAs were] excluded for disambiguation ties" to explain the 178 vs. 180 Firm A partner split. The actual script `24_validation_recalibration.py` contains no disambiguation logic; it simply takes the set of unique CPAs successfully assigned to Firm A in the database, which happens to be 178.
|
||||
4. **Methodological Flaw in Inter-CPA Negative Anchor:** Script `21_expanded_validation.py` claims to generate ~50,000 random inter-CPA pairs for validation. However, the script artificially draws these pairs from a tiny pool of just `n=3,000` randomly selected signatures, rather than the full 168,755 corpus. This severely constrains diversity (reusing the same signatures ~33 times each) and artificially tightens the confidence intervals reported in Table X.
|
||||
|
||||
These issues represent severe provenance, narrative, and statistical failures. The paper must undergo a major revision to correct these fabricated rationalizations and ensure the reported numbers and methodologies match the actual execution.
|
||||
|
||||
## 2. Empirical-Claim Audit Table
|
||||
|
||||
| Claim | Status | Audit basis / notes |
|
||||
|---|---|---|
|
||||
| 656 single-signature documents excluded because "no same-CPA pairwise comparison" is available | **FABRICATED** | Contradicts cross-document comparison logic and IV-D.1 (only 15 singleton CPAs lack comparison). The real reason is they failed CPA matching entirely. |
|
||||
| 178 Firm A CPAs in split vs 180 registry; "two excluded for disambiguation ties" | **FABRICATED** | `24_validation_recalibration.py` simply takes unique accountants with `firm=FIRM_A`. There is no disambiguation logic in the script. |
|
||||
| Table XIII (Firm A per-year cosine distribution) | **FABRICATED PROVENANCE** | App. B claims it's derived from `accountant_similarity_analysis.json`, but `08_accountant_similarity_analysis.py` doesn't extract or group by year. |
|
||||
| 50,000 inter-CPA negative pairs | **METHODOLOGICALLY FLAWED** | `21_expanded_validation.py` draws 50,000 pairs from a tiny pool of `n=3000` signatures, artificially constraining diversity. |
|
||||
| 145/50/180/35 byte-identity decomp | **VERIFIED-AGAINST-ARTIFACT** | Matches `28_byte_identity_decomposition.py`. |
|
||||
| Cross-firm convergence 42.12% vs 88.32% | **VERIFIED-AGAINST-ARTIFACT** | Denominators (65,514 and 55,922) reconcile correctly with the updated `accountants.firm` logic. |
|
||||
| 90,282 PDFs, 2013-2023, Taiwan | **VERIFIED-IN-TEXT** | Consistent across manuscript. |
|
||||
| 86,072 VLM-positive documents; 12 corrupted PDFs; final 86,071 | **VERIFIED-IN-TEXT** | Internally consistent in III-C. |
|
||||
| 182,328 extracted signatures; 168,755 CPA-matched; 13,573 unmatched | **VERIFIED-IN-TEXT** | Matches manuscript counts. |
|
||||
| 758 CPAs, 15 document types, 86.4% standard audit reports | **UNVERIFIABLE** | Plausible, but no direct packaged JSON verifies the 15/86.4% split. |
|
||||
| Qwen2.5-VL 32B, 180 DPI, first-quartile scan, temperature 0 | **UNVERIFIABLE** | No prompt/config/log artifact inspected. |
|
||||
| YOLO metrics (precision, recall, mAP) and 43.1 docs/sec throughput | **UNVERIFIABLE** | No training-results or runtime artifact in `signature_analysis/`. |
|
||||
| Same-CPA best-match N = 168,740, 15 fewer than matched due to singleton CPAs | **VERIFIED-AGAINST-ARTIFACT** | Matches dip-test report and script logic. |
|
||||
| ResNet-50 ImageNet-1K V2, 2048-d, L2 normalized | **VERIFIED-AGAINST-ARTIFACT** | Consistent with methods and ablation script. |
|
||||
| All-pairs intra/inter distribution N = 41,352,824 / 500,000; KDE crossover 0.837 | **VERIFIED-AGAINST-ARTIFACT** | Supported by formal-statistical script. |
|
||||
| Firm A dip result N=60,448, dip=0.0019, p=0.169 | **VERIFIED-AGAINST-ARTIFACT** | `15_hartigan_dip_test.py`. |
|
||||
| Beta mixture Delta BIC = 381 for Firm A; forced crossings 0.977/0.999 | **VERIFIED-AGAINST-ARTIFACT** | `17_beta_mixture_em.py`. |
|
||||
|
||||
## 3. Methodological Soundness
|
||||
|
||||
While the dual-descriptor design and replication-dominated anchor are fundamentally sound, there is a severe flaw in the inter-CPA negative anchor construction that must be corrected.
|
||||
**Flawed Inter-CPA Anchor Generation:** `21_expanded_validation.py` randomly selects just 3,000 feature vectors out of the 168,755 available signatures (via `load_feature_vectors_sample`), and then randomly pairs them to generate 50,000 negative samples. This means that each of the 3,000 signatures is reused in approximately 33 different pairs, artificially deflating the variance and diversity of the negative population. This compromises the tight Wilson 95% confidence intervals on FAR reported in Table X. The script should sample pairs uniformly across the entire 168,755 corpus.
|
||||
|
||||
## 4. Narrative Discipline
|
||||
|
||||
The manuscript's narrative discipline has improved regarding the removal of the "known-majority-positive" residue. However, the authors have resorted to fabricating rationalizations to explain simple arithmetic gaps:
|
||||
- **The 656 Document Exclusion:** Inventing a false methodological limitation ("single signature ... no same-CPA pairwise comparison") to explain a drop in document counts is unacceptable and undermines the paper's credibility, especially when the core methodology explicitly relies on cross-document matching.
|
||||
- **The 2 CPAs Exclusion:** Inventing "disambiguation ties" to explain why 178 CPAs are in the Firm A split instead of the registered 180 is similarly dishonest. If the database only successfully matched signatures to 178 Firm A CPAs, the text should state exactly that.
|
||||
|
||||
## 5. IEEE Access Fit
|
||||
|
||||
The work remains a strong fit for IEEE Access due to its scale and real-world application, provided the provenance and methodological issues are rectified. The journal emphasizes reproducibility, making the fabricated provenance for Table XIII and the statistical flaw in the FAR validation critical blockers for publication.
|
||||
|
||||
## 6. Specific Actionable Revisions
|
||||
|
||||
1. **Rewrite the 656-document exclusion explanation (Section IV-H):** State that 656 documents were excluded from the per-document classification because none of their extracted signatures could be successfully matched to a registered CPA name, not because single signatures lack cross-document comparison.
|
||||
2. **Remove the fabricated "disambiguation ties" claim (Section IV-F.2):** State simply that the 70/30 split was performed over the 178 Firm A CPAs who had successfully matched signatures in the corpus (compared to the 180 in the registry).
|
||||
3. **Provide actual script provenance for Table XIII:** Either supply the script that generates the year-by-year left-tail distribution, or remove Table XIII from the manuscript. Do not falsely attribute it to `08_accountant_similarity_analysis.py` (which does not group by year).
|
||||
4. **Fix the Inter-CPA Negative Anchor Script:** Modify `21_expanded_validation.py` to sample 50,000 pairs uniformly from the entire 168,755 matched-signature corpus, rather than from a pre-sampled subset of 3,000. Re-run and update Table X.
|
||||
5. **(Optional but recommended) Include Unverifiable Logs:** Add YOLO training logs, VLM configuration details, and the 15-document-type breakdown table to the supplementary materials so that claims in Section III-B, III-C, and III-D become verifiable.
|
||||
|
||||
## 7. Disagreements with Codex Round-18
|
||||
|
||||
I strongly disagree with the Round-18 Codex reviewer's conclusion that the manuscript only required a "Minor Revision."
|
||||
- Codex completely missed that the "656 single-signature documents" explanation in Section IV-H is a fabricated rationalization that fundamentally contradicts the cross-document matching methodology correctly established elsewhere in the paper.
|
||||
- Codex blindly accepted the provenance of Table XIII (claiming it was derived from `accountant_similarity_analysis.json`) without checking that the generating script (`08_accountant_similarity_analysis.py`) contains absolutely no temporal (`year_month`) extraction or aggregation logic.
|
||||
- Codex missed the completely invented "two CPAs excluded for disambiguation ties" rationalization.
|
||||
- Codex missed the statistical flaw in `21_expanded_validation.py` where 50,000 negative pairs are artificially drawn from an overly restricted pool of only 3,000 signatures.
|
||||
|
||||
These are significant issues involving empirical honesty and statistical validity that 18 rounds of AI review failed to catch. A Major Revision is strictly required before submission.
|
||||
@@ -49,7 +49,7 @@ For reproducibility, the following table maps each numerical table in Section IV
|
||||
| Table X (cosine threshold sweep, FAR vs inter-CPA negatives) | `21_expanded_validation.py` | `reports/expanded_validation/expanded_validation_results.json` |
|
||||
| Table XI (held-out vs calibration Firm A capture rates) | `24_validation_recalibration.py` | `reports/validation_recalibration/validation_recalibration.json` |
|
||||
| Table XII (operational-cut sensitivity 0.95 vs 0.945) | `24_validation_recalibration.py` | `reports/validation_recalibration/validation_recalibration.json` |
|
||||
| Table XIII (Firm A per-year cosine distribution) | `13_deloitte_distribution_analysis.py` | derived from `reports/accountant_similarity_analysis.json` filtered to Firm A; figures in `reports/figures/` |
|
||||
| Table XIII (Firm A per-year cosine distribution) | `29_firm_a_yearly_distribution.py` | `reports/firm_a_yearly/firm_a_yearly_distribution.json` |
|
||||
| Tables XIV / XV (partner-level similarity ranking) | `22_partner_ranking.py` | `reports/partner_ranking/partner_ranking_results.json` |
|
||||
| Table XVI (intra-report classification agreement) | `23_intra_report_consistency.py` | `reports/intra_report/intra_report_results.json` |
|
||||
| Table XVII (document-level five-way classification) | `09_pdf_signature_verdict.py`; `12_generate_pdf_level_report.py` | `reports/pdf_signature_verdicts.json`; `reports/pdf_signature_verdict_report.md` (CSV / XLSX bulk reports also at `reports/`) |
|
||||
|
||||
@@ -150,7 +150,7 @@ We report three validation analyses corresponding to the anchors of Section III-
|
||||
|
||||
Of the 182,328 extracted signatures, 310 have a same-CPA nearest match that is byte-identical after crop and normalization (pixel-identical-to-closest = 1); these form the byte-identity positive anchor---a pair-level proof of image reuse that serves as conservative ground truth for non-hand-signed signatures, subject to the source-template edge case discussed in Section V-G.
|
||||
Within Firm A specifically, 145 of these byte-identical signatures are distributed across 50 distinct partners (of 180 registered Firm A partners), with 35 of the byte-identical pairs spanning different fiscal years; this Firm A decomposition is reproduced by `signature_analysis/28_byte_identity_decomposition.py` and reported in `reports/byte_identity_decomp/byte_identity_decomposition.json` (Appendix B).
|
||||
As the gold-negative anchor we sample 50,000 random cross-CPA signature pairs (inter-CPA cosine: mean $= 0.762$, $P_{95} = 0.884$, $P_{99} = 0.913$, max $= 0.988$).
|
||||
As the gold-negative anchor we sample 50,000 i.i.d. random cross-CPA signature pairs from the full 168,755-signature matched corpus (inter-CPA cosine: mean $= 0.763$, $P_{95} = 0.886$, $P_{99} = 0.915$, max $= 0.992$).
|
||||
Because the positive and negative anchor populations are constructed from different sampling units (byte-identical same-CPA pairs vs random inter-CPA pairs), their relative prevalence in the combined anchor set is arbitrary, and precision / $F_1$ / recall therefore have no meaningful population interpretation.
|
||||
We accordingly report FAR with Wilson 95% confidence intervals against the large inter-CPA negative anchor in Table X.
|
||||
The primary quantity reported by Table X is FAR: the probability that a random pair of signatures from *different* CPAs exceeds the candidate threshold.
|
||||
@@ -159,12 +159,12 @@ We do not report an Equal Error Rate: EER is meaningful only when the positive a
|
||||
<!-- TABLE X: Cosine Threshold Sweep — FAR Against 50,000 Inter-CPA Negative Pairs
|
||||
| Threshold | FAR | FAR 95% Wilson CI |
|
||||
|-----------|-----|-------------------|
|
||||
| 0.837 (all-pairs KDE crossover) | 0.2062 | [0.2027, 0.2098] |
|
||||
| 0.900 | 0.0233 | [0.0221, 0.0247] |
|
||||
| 0.837 (all-pairs KDE crossover) | 0.2101 | [0.2066, 0.2137] |
|
||||
| 0.900 | 0.0250 | [0.0237, 0.0264] |
|
||||
| 0.945 (calibration-fold P5 rounded) | 0.0008 | [0.0006, 0.0011] |
|
||||
| 0.950 (whole-sample Firm A P7.5; operational cut) | 0.0007 | [0.0005, 0.0009] |
|
||||
| 0.973 (signature-level Beta/KDE upper bound) | 0.0003 | [0.0002, 0.0004] |
|
||||
| 0.979 (signature-level Beta-2 forced-fit crossing) | 0.0002 | [0.0001, 0.0004] |
|
||||
| 0.950 (whole-sample Firm A P7.5; operational cut) | 0.0005 | [0.0003, 0.0007] |
|
||||
| 0.973 (signature-level Beta/KDE upper bound) | 0.0002 | [0.0001, 0.0004] |
|
||||
| 0.979 (signature-level Beta-2 forced-fit crossing) | 0.0001 | [0.0001, 0.0003] |
|
||||
|
||||
Table note: We do not include FRR against the byte-identical positive anchor as a column here: the byte-identical subset has cosine $\approx 1$ by construction, so FRR against that subset is trivially $0$ at every threshold below $1$ and carries no biometric information beyond verifying that the threshold does not exceed $1$. The conservative-subset FRR role of the byte-identical anchor is instead discussed qualitatively in Section V-F.
|
||||
-->
|
||||
@@ -178,7 +178,7 @@ The very low FAR at the operational cut is therefore informative about specifici
|
||||
### 2) Held-Out Firm A Validation (within-Firm-A sampling variance disclosure)
|
||||
|
||||
We split Firm A CPAs randomly 70 / 30 at the CPA level into a calibration fold (124 CPAs, 45,116 signatures) and a held-out fold (54 CPAs, 15,332 signatures).
|
||||
The total of 178 Firm A CPAs differs from the 180 in the Firm A registry by two CPAs whose signatures could not be matched to a single assigned-accountant record because of disambiguation ties in the CPA registry and which we therefore exclude from both folds; this handling is made explicit here.
|
||||
The total of 178 Firm A CPAs differs from the 180 in the Firm A registry by two registered Firm A partners whose signatures in the corpus are singletons (only one signature each, so the per-signature best-match cosine is undefined and they do not appear in the same-CPA matched-signature table that script `24_validation_recalibration.py` reads); they are therefore not represented in either fold by construction rather than by an explicit exclusion rule.
|
||||
Thresholds are re-derived from calibration-fold percentiles only.
|
||||
Table XI reports both calibration-fold and held-out-fold capture rates with Wilson 95% CIs and a two-proportion $z$-test.
|
||||
|
||||
@@ -340,7 +340,7 @@ We note that this test uses the calibrated classifier of Section III-K rather th
|
||||
## H. Classification Results
|
||||
|
||||
Table XVII presents the final classification results under the dual-descriptor framework with Firm A-calibrated thresholds for 84,386 documents.
|
||||
The document count (84,386) differs from the 85,042 documents with any YOLO detection (Table III) because 656 documents carry only a single detected signature, for which no same-CPA pairwise comparison and therefore no best-match cosine / min dHash statistic is available; those documents are excluded from the classification reported here.
|
||||
The document count (84,386) differs from the 85,042 documents with any YOLO detection (Table III) because 656 documents have no signature whose extracted handwriting could be matched to a registered CPA name (every such signature has `assigned_accountant IS NULL` in the database, typically because the auditor's report page deviates from the standard two-signature layout or the OCRed printed CPA name was not present in the registry); the per-document classifier requires at least one CPA-matched signature so that a same-CPA best-match similarity exists, and these documents are therefore excluded from the classification reported here.
|
||||
We emphasize that the document-level proportions below reflect the *worst-case aggregation rule* of Section III-K: a report carrying one stamped signature and one hand-signed signature is labeled with the most-replication-consistent of the two signature-level verdicts.
|
||||
Document-level rates therefore represent the share of reports in which *at least one* signature is non-hand-signed rather than the share in which *both* are; the intra-report agreement analysis of Section IV-G.3 (Table XVI) reports how frequently the two co-signers share the same signature-level label within each firm, so that readers can judge what fraction of the non-hand-signed document-level share corresponds to fully non-hand-signed reports versus mixed reports.
|
||||
|
||||
|
||||
@@ -85,44 +85,78 @@ def load_signatures():
|
||||
return rows
|
||||
|
||||
|
||||
def load_feature_vectors_sample(n=2000):
|
||||
"""Load feature vectors for inter-CPA negative-anchor sampling."""
|
||||
def load_signature_ids_for_negative_pool(seed=SEED):
|
||||
"""Load lightweight (sig_id, accountant) pool from the entire matched
|
||||
corpus. Per Gemini round-19 review, the prior implementation drew
|
||||
50,000 inter-CPA pairs from a tiny LIMIT-3000 random subset, reusing
|
||||
each signature ~33 times and artificially tightening Wilson FAR CIs.
|
||||
The corrected implementation samples pairs i.i.d. across the FULL
|
||||
matched corpus (~168k signatures); only the unique signatures that
|
||||
actually appear in the sampled pairs need feature vectors loaded.
|
||||
"""
|
||||
conn = sqlite3.connect(DB)
|
||||
cur = conn.cursor()
|
||||
cur.execute('''
|
||||
SELECT signature_id, assigned_accountant, feature_vector
|
||||
SELECT signature_id, assigned_accountant
|
||||
FROM signatures
|
||||
WHERE feature_vector IS NOT NULL
|
||||
AND assigned_accountant IS NOT NULL
|
||||
ORDER BY RANDOM()
|
||||
LIMIT ?
|
||||
''', (n,))
|
||||
''')
|
||||
rows = cur.fetchall()
|
||||
conn.close()
|
||||
out = []
|
||||
for r in rows:
|
||||
vec = np.frombuffer(r[2], dtype=np.float32)
|
||||
out.append({'sig_id': r[0], 'accountant': r[1], 'feature': vec})
|
||||
return out
|
||||
sig_ids = np.array([r[0] for r in rows], dtype=np.int64)
|
||||
accts = np.array([r[1] for r in rows])
|
||||
return sig_ids, accts
|
||||
|
||||
|
||||
def build_inter_cpa_negative(sample, n_pairs=N_INTER_PAIRS, seed=SEED):
|
||||
"""Sample random cross-CPA pairs; return their cosine similarities."""
|
||||
def load_features_for_ids(sig_ids):
|
||||
conn = sqlite3.connect(DB)
|
||||
cur = conn.cursor()
|
||||
placeholders = ','.join('?' * len(sig_ids))
|
||||
cur.execute(
|
||||
f'SELECT signature_id, feature_vector FROM signatures '
|
||||
f'WHERE signature_id IN ({placeholders})',
|
||||
[int(s) for s in sig_ids],
|
||||
)
|
||||
rows = cur.fetchall()
|
||||
conn.close()
|
||||
feat_by_id = {}
|
||||
for sid, blob in rows:
|
||||
feat_by_id[int(sid)] = np.frombuffer(blob, dtype=np.float32)
|
||||
return feat_by_id
|
||||
|
||||
|
||||
def build_inter_cpa_negative(sig_ids, accts, n_pairs=N_INTER_PAIRS, seed=SEED):
|
||||
"""Sample i.i.d. random cross-CPA pairs from the full matched corpus
|
||||
and return their cosine similarities.
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
n = len(sample)
|
||||
feats = np.stack([s['feature'] for s in sample])
|
||||
accts = np.array([s['accountant'] for s in sample])
|
||||
sims = []
|
||||
n = len(sig_ids)
|
||||
pairs = []
|
||||
tries = 0
|
||||
while len(sims) < n_pairs and tries < n_pairs * 10:
|
||||
seen_pairs = set()
|
||||
while len(pairs) < n_pairs and tries < n_pairs * 10:
|
||||
i = rng.integers(n)
|
||||
j = rng.integers(n)
|
||||
if i == j or accts[i] == accts[j]:
|
||||
tries += 1
|
||||
continue
|
||||
sim = float(feats[i] @ feats[j])
|
||||
sims.append(sim)
|
||||
a, b = (i, j) if i < j else (j, i)
|
||||
if (a, b) in seen_pairs:
|
||||
tries += 1
|
||||
continue
|
||||
seen_pairs.add((a, b))
|
||||
pairs.append((a, b))
|
||||
tries += 1
|
||||
|
||||
needed_ids = sorted({int(sig_ids[i]) for pair in pairs for i in pair})
|
||||
feat_by_id = load_features_for_ids(needed_ids)
|
||||
|
||||
sims = []
|
||||
for i, j in pairs:
|
||||
fi = feat_by_id[int(sig_ids[i])]
|
||||
fj = feat_by_id[int(sig_ids[j])]
|
||||
sims.append(float(fi @ fj))
|
||||
return np.array(sims)
|
||||
|
||||
|
||||
@@ -212,9 +246,12 @@ def main():
|
||||
print(f'Firm A signatures: {int(firm_a_mask.sum()):,}')
|
||||
|
||||
# --- (1) INTER-CPA NEGATIVE ANCHOR ---
|
||||
print(f'\n[1] Building inter-CPA negative anchor ({N_INTER_PAIRS} pairs)...')
|
||||
sample = load_feature_vectors_sample(n=3000)
|
||||
inter_cos = build_inter_cpa_negative(sample, n_pairs=N_INTER_PAIRS)
|
||||
print(f'\n[1] Building inter-CPA negative anchor ({N_INTER_PAIRS} '
|
||||
f'i.i.d. pairs from full matched corpus)...')
|
||||
pool_sig_ids, pool_accts = load_signature_ids_for_negative_pool()
|
||||
print(f' pool size: {len(pool_sig_ids):,} matched signatures')
|
||||
inter_cos = build_inter_cpa_negative(pool_sig_ids, pool_accts,
|
||||
n_pairs=N_INTER_PAIRS)
|
||||
print(f' inter-CPA cos: mean={inter_cos.mean():.4f}, '
|
||||
f'p95={np.percentile(inter_cos, 95):.4f}, '
|
||||
f'p99={np.percentile(inter_cos, 99):.4f}, '
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script 29: Firm A Per-Year Cosine Distribution (Table XIII)
|
||||
============================================================
|
||||
Generates the year-by-year Firm A per-signature best-match cosine
|
||||
distribution reported as Table XIII in the manuscript. Codex / Gemini
|
||||
round-19 review identified that this table previously had no dedicated
|
||||
generating script (Appendix B incorrectly attributed it to Script 08,
|
||||
which has no year_month extraction).
|
||||
|
||||
Definition:
|
||||
Firm A membership is via CPA registry (accountants.firm joined on
|
||||
signatures.assigned_accountant), matching the convention used by
|
||||
scripts 24 and 28.
|
||||
|
||||
For each fiscal year (substr(year_month, 1, 4)):
|
||||
- N signatures with non-null max_similarity_to_same_accountant
|
||||
- mean of max_similarity_to_same_accountant (the per-signature
|
||||
best-match cosine)
|
||||
- share with max_similarity_to_same_accountant < 0.95 (the
|
||||
left-tail rate cited in Section IV-G.1)
|
||||
|
||||
Output:
|
||||
reports/firm_a_yearly/firm_a_yearly_distribution.json
|
||||
reports/firm_a_yearly/firm_a_yearly_distribution.md
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
DB = '/Volumes/NV2/PDF-Processing/signature-analysis/signature_analysis.db'
|
||||
OUT = Path('/Volumes/NV2/PDF-Processing/signature-analysis/reports/'
|
||||
'firm_a_yearly')
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
FIRM_A = '勤業眾信聯合'
|
||||
|
||||
|
||||
def yearly_distribution(conn):
|
||||
cur = conn.cursor()
|
||||
cur.execute("""
|
||||
SELECT substr(s.year_month, 1, 4) AS year,
|
||||
COUNT(*) AS n_sigs,
|
||||
AVG(s.max_similarity_to_same_accountant) AS mean_cos,
|
||||
SUM(CASE
|
||||
WHEN s.max_similarity_to_same_accountant < 0.95
|
||||
THEN 1 ELSE 0
|
||||
END) AS n_below_095
|
||||
FROM signatures s
|
||||
JOIN accountants a ON s.assigned_accountant = a.name
|
||||
WHERE a.firm = ?
|
||||
AND s.max_similarity_to_same_accountant IS NOT NULL
|
||||
AND s.year_month IS NOT NULL
|
||||
GROUP BY year
|
||||
ORDER BY year
|
||||
""", (FIRM_A,))
|
||||
|
||||
rows = []
|
||||
for year, n_sigs, mean_cos, n_below in cur.fetchall():
|
||||
rows.append({
|
||||
'year': int(year),
|
||||
'n_signatures': n_sigs,
|
||||
'mean_best_match_cosine': round(mean_cos, 4),
|
||||
'n_below_cosine_095': n_below,
|
||||
'pct_below_cosine_095': round(100.0 * n_below / n_sigs, 2),
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def write_markdown(payload, path):
|
||||
rows = payload['yearly_rows']
|
||||
lines = []
|
||||
lines.append('# Firm A Per-Year Cosine Distribution (Table XIII)')
|
||||
lines.append('')
|
||||
lines.append(f"Generated at: {payload['generated_at']}")
|
||||
lines.append('')
|
||||
lines.append('Firm A membership: CPA registry '
|
||||
'(accountants.firm = "勤業眾信聯合"). Per-signature '
|
||||
'best-match cosine = '
|
||||
'signatures.max_similarity_to_same_accountant.')
|
||||
lines.append('')
|
||||
lines.append('| Year | N sigs | mean best-match cosine | % below 0.95 |')
|
||||
lines.append('|------|--------|------------------------|--------------|')
|
||||
for r in rows:
|
||||
lines.append(
|
||||
f"| {r['year']} | {r['n_signatures']:,} | "
|
||||
f"{r['mean_best_match_cosine']:.4f} | "
|
||||
f"{r['pct_below_cosine_095']:.2f}% |"
|
||||
)
|
||||
path.write_text('\n'.join(lines) + '\n', encoding='utf-8')
|
||||
|
||||
|
||||
def main():
|
||||
conn = sqlite3.connect(DB)
|
||||
try:
|
||||
payload = {
|
||||
'generated_at': datetime.now().isoformat(timespec='seconds'),
|
||||
'database_path': DB,
|
||||
'firm_a_label': FIRM_A,
|
||||
'firm_a_membership_definition': (
|
||||
'CPA registry: accountants.firm joined on '
|
||||
'signatures.assigned_accountant'
|
||||
),
|
||||
'cosine_metric': 'signatures.max_similarity_to_same_accountant',
|
||||
'yearly_rows': yearly_distribution(conn),
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
json_path = OUT / 'firm_a_yearly_distribution.json'
|
||||
json_path.write_text(json.dumps(payload, indent=2, ensure_ascii=False),
|
||||
encoding='utf-8')
|
||||
print(f'Wrote {json_path}')
|
||||
|
||||
md_path = OUT / 'firm_a_yearly_distribution.md'
|
||||
write_markdown(payload, md_path)
|
||||
print(f'Wrote {md_path}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user