gbanyan
6b64eabbfb
Codex round-18 (paper/codex_review_gpt55_v3_18_3.md) caught a falsified provenance claim I introduced in v3.18.3 plus four cleaner narrative items that survived the prior 17 rounds. Verdict was Minor Revision; this commit closes all 5 actionable items. - Harmonize signature_analysis/28_byte_identity_decomposition.py to use accountants.firm (joined on signatures.assigned_accountant) for Firm A membership, matching the convention in 24_validation_recalibration.py. Regenerated reports/byte_identity_decomp/byte_identity_decomposition.json. Cross-firm convergence now reports Firm A 49,389 / 55,922 = 88.32% and Non-Firm-A 27,595 / 65,514 = 42.12% (percentages unchanged at two decimal places; counts now match Table IX exactly). - Replace the Section IV-H.2 reconciliation note. The previous note speculated that the one-record discrepancy was a snapshot/floating-point artifact, which codex round-18 falsified by direct DB queries: the real cause was that script 28 used signatures.excel_firm while Table IX uses accountants.firm. With script 28 now harmonized, Table IX and the cross-firm artifact agree exactly at 55,922; the new note documents the Firm A grouping convention plus the dHash-non-null filter. - Replace residual "known-majority-positive" wording with "replication-dominated" in Introduction (contributions 4 and 6) and Methodology III-I (anchor-rationale paragraph). - Correct Methodology III-G's auditor-year description: the per-signature best-match cosine that feeds each auditor-year mean is computed against the full same-CPA cross-year pool, not within-year only. The aggregation unit is within-year, but the underlying similarity statistic is not. - Add the 145 / 50 / 180 / 35 Firm A byte-decomposition sentence to Results IV-F.1 with explicit pointer to script 28 and the JSON artifact; this resolves the round-18 finding that several manuscript locations cited IV-F.1 for a decomposition that was not actually reported there. - Rebuild Paper_A_IEEE_Access_Draft_v3.docx. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PDF Signature Extraction System
Automated extraction of handwritten Chinese signatures from PDF documents using hybrid VLM + Computer Vision approach.
Quick Start
Step 1: Extract Pages from CSV
cd /Volumes/NV2/pdf_recognize
source venv/bin/activate
python extract_pages_from_csv.py
Step 2: Extract Signatures
python extract_signatures_hybrid.py
Documentation
- PROJECT_DOCUMENTATION.md - Complete project history, all approaches tested, detailed results
- README_page_extraction.md - Page extraction documentation
- README_hybrid_extraction.md - Hybrid signature extraction documentation
Current Performance
Test Dataset: 5 PDF pages
- Signatures expected: 10
- Signatures found: 7
- Precision: 100% (no false positives)
- Recall: 70%
Key Features
✅ Hybrid Approach: VLM name extraction + CV detection + VLM verification
✅ Name-Based: Signatures saved as signature_周寶蓮.png
✅ No False Positives: Name-specific verification filters out dates, text, stamps
✅ Duplicate Prevention: Only one signature per person
✅ Handles Both: PDFs with/without text layer
File Structure
extract_pages_from_csv.py # Step 1: Extract pages
extract_signatures_hybrid.py # Step 2: Extract signatures (CURRENT)
README.md # This file
PROJECT_DOCUMENTATION.md # Complete documentation
README_page_extraction.md # Page extraction guide
README_hybrid_extraction.md # Signature extraction guide
Requirements
- Python 3.9+
- PyMuPDF, OpenCV, NumPy, Requests
- Ollama with qwen2.5vl:32b model
- Ollama instance: http://192.168.30.36:11434
Data
- Input:
/Volumes/NV2/PDF-Processing/master_signatures.csv(86,073 rows) - PDFs:
/Volumes/NV2/PDF-Processing/total-pdf/batch_*/ - Output:
/Volumes/NV2/PDF-Processing/signature-image-output/
Status
✅ Page extraction: Tested with 100 files, working ✅ Signature extraction: Tested with 5 files, 70% recall, 100% precision ⏳ Large-scale testing: Pending ⏳ Full dataset (86K files): Pending
See PROJECT_DOCUMENTATION.md for complete details.