gbanyan
128a91433f
Closes the two factual / provenance issues codex round-9 caught in the round-4 fixes. Text-only patches; no script reruns. Patch A — N1 wording corrected: §IV-M.4 line 325 had said the 379 mixed-firm PDFs "resolve to Firm C as the majority firm" (propagated from Opus round-2's incorrect inference from reading the Script 45 source). Codex DB-verified all 379 are actually 1:1 Firm C / Firm D ties, assigned to Firm C only because `np.argmax` over `np.unique`'s alphabetically-sorted firm counts returns the first-sorted firm on ties. Corrected to the actual tie-break explanation. Patch B — N2 Table XXVII row 1 narrowed: composition-decomposition row's untested-assumption cell previously claimed "within-firm dip tests on every firm with n >= 500 (Script 39c) corroborate absence of within-population bimodality." Codex verified Script 39c on raw dHash actually REJECTS unimodality in all 10 firms (integer ties); only Big-4 per-firm jittered (Script 39d) and Big-4 pooled centred+jittered (Script 39e) are emitted. Narrowed to those two diagnostics — no overreach to non-Big-4 jittered evidence. Patch C — §III line 59 + provenance table line 382: replaced the unreproducible $[0.71, 1.00]$ non-Big-4 jittered-dHash range with codex's read-only verified range $[0.38, 1.00]$, attributed as a "codex-verified read-only spike on Script 39c substrate." The qualitative claim (0/10 non-Big-4 firms reject after jitter) is preserved and confirmed by codex's independent rerun; only the specific manuscript range was unverifiable from the committed script reports. Verification: - `rg -n "majority firm |nine-tool|9 tools"` in paper/v4/ returns 0 matches in published prose; only 2 matches in internal strip-at-splice text (Phase 4 draft note + §III internal checklist). - All Script 39c citations now technically accurate (cosine for per-firm; codex-verified for jittered-dHash spike). - Abstract still 247 words. Phase 5 convergence: 3/3 reviewers in Accept/Minor band remains intact. With these factual corrections applied, the manuscript text is now consistent with the committed script outputs. Remaining work: splice-time strip of internal notes / checklists, then proceed to Phase 6 partner Jimmy review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PDF Signature Extraction System
Automated extraction of handwritten Chinese signatures from PDF documents using hybrid VLM + Computer Vision approach.
Quick Start
Step 1: Extract Pages from CSV
cd /Volumes/NV2/pdf_recognize
source venv/bin/activate
python extract_pages_from_csv.py
Step 2: Extract Signatures
python extract_signatures_hybrid.py
Documentation
- PROJECT_DOCUMENTATION.md - Complete project history, all approaches tested, detailed results
- README_page_extraction.md - Page extraction documentation
- README_hybrid_extraction.md - Hybrid signature extraction documentation
Current Performance
Test Dataset: 5 PDF pages
- Signatures expected: 10
- Signatures found: 7
- Precision: 100% (no false positives)
- Recall: 70%
Key Features
✅ Hybrid Approach: VLM name extraction + CV detection + VLM verification
✅ Name-Based: Signatures saved as signature_周寶蓮.png
✅ No False Positives: Name-specific verification filters out dates, text, stamps
✅ Duplicate Prevention: Only one signature per person
✅ Handles Both: PDFs with/without text layer
File Structure
extract_pages_from_csv.py # Step 1: Extract pages
extract_signatures_hybrid.py # Step 2: Extract signatures (CURRENT)
README.md # This file
PROJECT_DOCUMENTATION.md # Complete documentation
README_page_extraction.md # Page extraction guide
README_hybrid_extraction.md # Signature extraction guide
Requirements
- Python 3.9+
- PyMuPDF, OpenCV, NumPy, Requests
- Ollama with qwen2.5vl:32b model
- Ollama instance: http://192.168.30.36:11434
Data
- Input:
/Volumes/NV2/PDF-Processing/master_signatures.csv(86,073 rows) - PDFs:
/Volumes/NV2/PDF-Processing/total-pdf/batch_*/ - Output:
/Volumes/NV2/PDF-Processing/signature-image-output/
Status
✅ Page extraction: Tested with 100 files, working ✅ Signature extraction: Tested with 5 files, 70% recall, 100% precision ⏳ Large-scale testing: Pending ⏳ Full dataset (86K files): Pending
See PROJECT_DOCUMENTATION.md for complete details.