Complete OpenCV Method 3 implementation with 86.5% handwriting retention
- Implemented comprehensive feature analysis based on size, stroke length, and regularity - Size-based scoring: height >50px indicates handwriting - Stroke length ratio: >0.4 indicates handwriting - Irregularity metrics: low compactness/solidity indicates handwriting - Successfully tested on sample PDF with 2 signatures (楊智惠, 張志銘) - Created detailed documentation: CURRENT_STATUS.md and NEW_SESSION_HANDOFF.md - Stable PaddleOCR 2.7.3 configuration documented (numpy 1.26.4, opencv 4.6.0.66) - Prepared research plan for PP-OCRv5 upgrade investigation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
252
CURRENT_STATUS.md
Normal file
252
CURRENT_STATUS.md
Normal file
@@ -0,0 +1,252 @@
|
|||||||
|
# 项目当前状态
|
||||||
|
|
||||||
|
**更新时间**: 2025-10-29
|
||||||
|
**分支**: `paddleocr-improvements`
|
||||||
|
**PaddleOCR版本**: 2.7.3 (稳定版本)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 当前进度总结
|
||||||
|
|
||||||
|
### ✅ 已完成
|
||||||
|
|
||||||
|
1. **PaddleOCR服务器部署** (192.168.30.36:5555)
|
||||||
|
- 版本: PaddleOCR 2.7.3
|
||||||
|
- GPU: 启用
|
||||||
|
- 语言: 中文
|
||||||
|
- 状态: 稳定运行
|
||||||
|
|
||||||
|
2. **基础Pipeline实现**
|
||||||
|
- ✅ PDF → 图像渲染 (DPI=300)
|
||||||
|
- ✅ PaddleOCR文字检测 (26个区域/页)
|
||||||
|
- ✅ 文本区域遮罩 (padding=25px)
|
||||||
|
- ✅ 候选区域检测
|
||||||
|
- ✅ 区域合并算法 (12→4 regions)
|
||||||
|
|
||||||
|
3. **OpenCV分离方法测试**
|
||||||
|
- Method 1: 笔画宽度分析 - ❌ 效果差
|
||||||
|
- Method 2: 连通组件基础分析 - ⚠️ 中等效果
|
||||||
|
- Method 3: 综合特征分析 - ✅ **最佳方案** (86.5%手写保留率)
|
||||||
|
|
||||||
|
4. **测试结果**
|
||||||
|
- 测试文件: `201301_1324_AI1_page3.pdf`
|
||||||
|
- 预期签名: 2个 (楊智惠, 張志銘)
|
||||||
|
- 检测结果: 2个签名区域成功合并
|
||||||
|
- 保留率: 86.5% 手写内容
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 技术架构
|
||||||
|
|
||||||
|
```
|
||||||
|
PDF文档
|
||||||
|
↓
|
||||||
|
1. 渲染 (PyMuPDF, 300 DPI)
|
||||||
|
↓
|
||||||
|
2. PaddleOCR检测 (识别印刷文字)
|
||||||
|
↓
|
||||||
|
3. 遮罩印刷文字 (黑色填充, padding=25px)
|
||||||
|
↓
|
||||||
|
4. 区域检测 (OpenCV形态学)
|
||||||
|
↓
|
||||||
|
5. 区域合并 (距离阈值: H≤100px, V≤50px)
|
||||||
|
↓
|
||||||
|
6. 特征分析 (大小+笔画长度+规律性)
|
||||||
|
↓
|
||||||
|
7. [TODO] VLM验证
|
||||||
|
↓
|
||||||
|
签名提取结果
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 核心文件
|
||||||
|
|
||||||
|
| 文件 | 说明 | 状态 |
|
||||||
|
|------|------|------|
|
||||||
|
| `paddleocr_client.py` | PaddleOCR REST客户端 | ✅ 稳定 |
|
||||||
|
| `test_mask_and_detect.py` | 基础遮罩+检测测试 | ✅ 完成 |
|
||||||
|
| `test_opencv_separation.py` | OpenCV方法1+2测试 | ✅ 完成 |
|
||||||
|
| `test_opencv_advanced.py` | OpenCV方法3(最佳) | ✅ 完成 |
|
||||||
|
| `extract_signatures_paddleocr_improved.py` | 完整Pipeline (Method B+E) | ⚠️ Method E有问题 |
|
||||||
|
| `PADDLEOCR_STATUS.md` | 详细技术文档 | ✅ 完成 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Method 3: 综合特征分析 (当前最佳方案)
|
||||||
|
|
||||||
|
### 判断依据
|
||||||
|
|
||||||
|
**您的观察** (非常准确):
|
||||||
|
1. ✅ **手写字比印刷字大** - height > 50px
|
||||||
|
2. ✅ **手写笔画长度更长** - stroke_ratio > 0.4
|
||||||
|
3. ✅ **印刷体规律，手写潦草** - compactness, solidity
|
||||||
|
|
||||||
|
### 评分系统
|
||||||
|
|
||||||
|
```python
|
||||||
|
handwriting_score = 0
|
||||||
|
|
||||||
|
# 大小评分
|
||||||
|
if height > 50: score += 3
|
||||||
|
elif height > 35: score += 2
|
||||||
|
|
||||||
|
# 笔画长度评分
|
||||||
|
if stroke_ratio > 0.5: score += 2
|
||||||
|
elif stroke_ratio > 0.35: score += 1
|
||||||
|
|
||||||
|
# 规律性评分
|
||||||
|
if is_irregular: score += 1 # 不规律 = 手写
|
||||||
|
else: score -= 1 # 规律 = 印刷
|
||||||
|
|
||||||
|
# 面积评分
|
||||||
|
if area > 2000: score += 2
|
||||||
|
elif area < 500: score -= 1
|
||||||
|
|
||||||
|
# 分类: score > 0 → 手写
|
||||||
|
```
|
||||||
|
|
||||||
|
### 效果
|
||||||
|
|
||||||
|
- 手写像素保留: **86.5%** ✅
|
||||||
|
- 印刷像素过滤: 13.5%
|
||||||
|
- Top 10组件全部正确分类
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 已识别问题
|
||||||
|
|
||||||
|
### 1. Method E (两阶段OCR) 失效 ❌
|
||||||
|
|
||||||
|
**原因**: PaddleOCR无法区分"印刷"和"手写"，第二次OCR会把手写也识别并删除
|
||||||
|
|
||||||
|
**解决方案**:
|
||||||
|
- ❌ 不使用Method E
|
||||||
|
- ✅ 使用Method B (区域合并) + OpenCV Method 3
|
||||||
|
|
||||||
|
### 2. 印刷名字与手写签名重叠
|
||||||
|
|
||||||
|
**现象**: 区域包含"楊 智 惠"(印刷) + 手写签名
|
||||||
|
**策略**: 接受少量印刷残留，优先保证手写完整性
|
||||||
|
**后续**: 用VLM最终验证
|
||||||
|
|
||||||
|
### 3. Masking padding 矛盾
|
||||||
|
|
||||||
|
**小padding (5-10px)**: 印刷残留多，但不伤手写
|
||||||
|
**大padding (25px)**: 印刷删除干净，但可能遮住手写边缘
|
||||||
|
**当前**: 使用 25px，依赖OpenCV Method 3过滤残留
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 下一步计划
|
||||||
|
|
||||||
|
### 短期 (继续当前方案)
|
||||||
|
|
||||||
|
- [ ] 整合 Method B + OpenCV Method 3 为完整Pipeline
|
||||||
|
- [ ] 添加VLM验证步骤
|
||||||
|
- [ ] 在10个样本上测试
|
||||||
|
- [ ] 调优参数 (height阈值, merge距离等)
|
||||||
|
|
||||||
|
### 中期 (PP-OCRv5研究)
|
||||||
|
|
||||||
|
**新branch**: `pp-ocrv5-research`
|
||||||
|
|
||||||
|
- [ ] 研究PaddleOCR 3.3.0新API
|
||||||
|
- [ ] 测试PP-OCRv5手写检测能力
|
||||||
|
- [ ] 对比性能: v4 vs v5
|
||||||
|
- [ ] 评估是否升级
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 服务器配置
|
||||||
|
|
||||||
|
### PaddleOCR服务器 (Linux)
|
||||||
|
|
||||||
|
```
|
||||||
|
Host: 192.168.30.36:5555
|
||||||
|
SSH: ssh gblinux
|
||||||
|
路径: ~/Project/paddleocr-server/
|
||||||
|
版本: PaddleOCR 2.7.3, numpy 1.26.4, opencv-contrib 4.6.0.66
|
||||||
|
启动: cd ~/Project/paddleocr-server && source venv/bin/activate && python paddleocr_server.py
|
||||||
|
日志: ~/Project/paddleocr-server/server_stable.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### VLM服务器 (Ollama)
|
||||||
|
|
||||||
|
```
|
||||||
|
Host: 192.168.30.36:11434
|
||||||
|
模型: qwen2.5vl:32b
|
||||||
|
状态: 未在当前Pipeline中使用
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 测试数据
|
||||||
|
|
||||||
|
### 样本文件
|
||||||
|
|
||||||
|
```
|
||||||
|
/Volumes/NV2/PDF-Processing/signature-image-output/201301_1324_AI1_page3.pdf
|
||||||
|
- 页面: 第3页
|
||||||
|
- 预期签名: 2个 (楊智惠, 張志銘)
|
||||||
|
- 尺寸: 2481x3510 pixels
|
||||||
|
```
|
||||||
|
|
||||||
|
### 输出目录
|
||||||
|
|
||||||
|
```
|
||||||
|
/Volumes/NV2/PDF-Processing/signature-image-output/
|
||||||
|
├── mask_test/ # 基础遮罩测试结果
|
||||||
|
├── paddleocr_improved/ # Method B+E测试 (E失败)
|
||||||
|
├── opencv_separation_test/ # Method 1+2测试
|
||||||
|
└── opencv_advanced_test/ # Method 3测试 (最佳)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 性能对比
|
||||||
|
|
||||||
|
| 方法 | 手写保留 | 印刷去除 | 总评 |
|
||||||
|
|------|---------|---------|------|
|
||||||
|
| 基础遮罩 | 100% | 低 | ⚠️ 太多印刷残留 |
|
||||||
|
| Method 1 (笔画宽度) | 0% | - | ❌ 完全失败 |
|
||||||
|
| Method 2 (连通组件) | 1% | 中 | ❌ 丢失太多手写 |
|
||||||
|
| Method 3 (综合特征) | **86.5%** | 高 | ✅ **最佳** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Git状态
|
||||||
|
|
||||||
|
```
|
||||||
|
当前分支: paddleocr-improvements
|
||||||
|
基于: PaddleOCR-Cover
|
||||||
|
标签: paddleocr-v1-basic (基础遮罩版本)
|
||||||
|
|
||||||
|
待提交:
|
||||||
|
- OpenCV高级分离方法 (Method 3)
|
||||||
|
- 完整测试脚本和结果
|
||||||
|
- 文档更新
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 已知限制
|
||||||
|
|
||||||
|
1. **参数需调优**: height阈值、merge距离等在不同文档可能需要调整
|
||||||
|
2. **依赖文档质量**: 模糊、倾斜的文档可能效果变差
|
||||||
|
3. **计算性能**: OpenCV处理较快，但完整Pipeline需要优化
|
||||||
|
4. **泛化能力**: 仅在1个样本测试，需要更多样本验证
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 联系与协作
|
||||||
|
|
||||||
|
**主要开发者**: Claude Code
|
||||||
|
**协作方式**: 会话式开发
|
||||||
|
**代码仓库**: 本地Git仓库
|
||||||
|
**测试环境**: macOS (本地) + Linux (服务器)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**状态**: ✅ 当前方案稳定，可继续开发
|
||||||
|
**建议**: 先在更多样本测试Method 3，再考虑PP-OCRv5升级
|
||||||
432
NEW_SESSION_HANDOFF.md
Normal file
432
NEW_SESSION_HANDOFF.md
Normal file
@@ -0,0 +1,432 @@
|
|||||||
|
# 新对话交接文档 - PP-OCRv5研究
|
||||||
|
|
||||||
|
**日期**: 2025-10-29
|
||||||
|
**前序对话**: PaddleOCR-Cover分支开发
|
||||||
|
**当前分支**: `paddleocr-improvements` (稳定)
|
||||||
|
**新分支**: `pp-ocrv5-research` (待创建)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 任务目标
|
||||||
|
|
||||||
|
研究和实现 **PP-OCRv5** 的手写签名检测功能
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 背景信息
|
||||||
|
|
||||||
|
### 当前状况
|
||||||
|
|
||||||
|
✅ **已有稳定方案** (`paddleocr-improvements` 分支):
|
||||||
|
- PaddleOCR 2.7.3 + OpenCV Method 3
|
||||||
|
- 86.5%手写保留率
|
||||||
|
- 区域合并算法工作良好
|
||||||
|
- 测试: 1个PDF成功检测2个签名
|
||||||
|
|
||||||
|
⚠️ **PP-OCRv5升级遇到问题**:
|
||||||
|
- PaddleOCR 3.3.0 API完全改变
|
||||||
|
- 旧服务器代码不兼容
|
||||||
|
- 需要深入研究新API
|
||||||
|
|
||||||
|
### 为什么要研究PP-OCRv5？
|
||||||
|
|
||||||
|
**文档显示**: https://www.paddleocr.ai/main/en/version3.x/algorithm/PP-OCRv5/PP-OCRv5.html
|
||||||
|
|
||||||
|
PP-OCRv5性能提升:
|
||||||
|
- 手写中文检测: **0.706 → 0.803** (+13.7%)
|
||||||
|
- 手写英文检测: **0.249 → 0.841** (+237%)
|
||||||
|
- 可能支持直接输出手写区域坐标
|
||||||
|
|
||||||
|
**潜在优势**:
|
||||||
|
1. 更好的手写识别能力
|
||||||
|
2. 可能内置手写/印刷分类
|
||||||
|
3. 更准确的坐标输出
|
||||||
|
4. 减少复杂的后处理
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 技术栈
|
||||||
|
|
||||||
|
### 服务器环境
|
||||||
|
|
||||||
|
```
|
||||||
|
Host: 192.168.30.36 (Linux GPU服务器)
|
||||||
|
SSH: ssh gblinux
|
||||||
|
目录: ~/Project/paddleocr-server/
|
||||||
|
```
|
||||||
|
|
||||||
|
**当前稳定版本**:
|
||||||
|
- PaddleOCR: 2.7.3
|
||||||
|
- numpy: 1.26.4
|
||||||
|
- opencv-contrib-python: 4.6.0.66
|
||||||
|
- 服务器文件: `paddleocr_server.py`
|
||||||
|
|
||||||
|
**已安装但未使用**:
|
||||||
|
- PaddleOCR 3.3.0 (PP-OCRv5)
|
||||||
|
- 临时服务器: `paddleocr_server_v5.py` (未完成)
|
||||||
|
|
||||||
|
### 本地环境
|
||||||
|
|
||||||
|
```
|
||||||
|
macOS
|
||||||
|
Python: 3.14
|
||||||
|
虚拟环境: venv/
|
||||||
|
客户端: paddleocr_client.py
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 核心问题
|
||||||
|
|
||||||
|
### 1. API变更
|
||||||
|
|
||||||
|
**旧API (2.7.3)**:
|
||||||
|
```python
|
||||||
|
from paddleocr import PaddleOCR
|
||||||
|
ocr = PaddleOCR(lang='ch')
|
||||||
|
result = ocr.ocr(image_np, cls=False)
|
||||||
|
|
||||||
|
# 返回格式:
|
||||||
|
# [[[box], (text, confidence)], ...]
|
||||||
|
```
|
||||||
|
|
||||||
|
**新API (3.3.0)** - ⚠️ 未完全理解:
|
||||||
|
```python
|
||||||
|
# 方式1: 传统方式 (Deprecated)
|
||||||
|
result = ocr.ocr(image_np) # 警告: Please use predict instead
|
||||||
|
|
||||||
|
# 方式2: 新方式
|
||||||
|
from paddlex import create_model
|
||||||
|
model = create_model("???") # 模型名未知
|
||||||
|
result = model.predict(image_np)
|
||||||
|
|
||||||
|
# 返回格式: ???
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. 遇到的错误
|
||||||
|
|
||||||
|
**错误1**: `cls` 参数不再支持
|
||||||
|
```python
|
||||||
|
# 错误: PaddleOCR.predict() got an unexpected keyword argument 'cls'
|
||||||
|
result = ocr.ocr(image_np, cls=False) # ❌
|
||||||
|
```
|
||||||
|
|
||||||
|
**错误2**: 返回格式改变
|
||||||
|
```python
|
||||||
|
# 旧代码解析失败:
|
||||||
|
text = item[1][0] # ❌ IndexError
|
||||||
|
confidence = item[1][1] # ❌ IndexError
|
||||||
|
```
|
||||||
|
|
||||||
|
**错误3**: 模型名称错误
|
||||||
|
```python
|
||||||
|
model = create_model("PP-OCRv5_server") # ❌ Model not supported
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 研究任务清单
|
||||||
|
|
||||||
|
### Phase 1: API研究 (优先级高)
|
||||||
|
|
||||||
|
- [ ] **阅读官方文档**
|
||||||
|
- PP-OCRv5完整文档
|
||||||
|
- PaddleX API文档
|
||||||
|
- 迁移指南 (如果有)
|
||||||
|
|
||||||
|
- [ ] **理解新API**
|
||||||
|
```python
|
||||||
|
# 需要搞清楚:
|
||||||
|
1. 正确的导入方式
|
||||||
|
2. 模型初始化方法
|
||||||
|
3. predict()参数和返回格式
|
||||||
|
4. 如何区分手写/印刷
|
||||||
|
5. 是否有手写检测专用功能
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **编写测试脚本**
|
||||||
|
- `test_pp_ocrv5_api.py` - 测试基础API调用
|
||||||
|
- 打印完整的result数据结构
|
||||||
|
- 对比v4和v5的返回差异
|
||||||
|
|
||||||
|
### Phase 2: 服务器适配
|
||||||
|
|
||||||
|
- [ ] **重写服务器代码**
|
||||||
|
- 适配新API
|
||||||
|
- 正确解析返回数据
|
||||||
|
- 保持REST接口兼容
|
||||||
|
|
||||||
|
- [ ] **测试稳定性**
|
||||||
|
- 测试10个PDF样本
|
||||||
|
- 检查GPU利用率
|
||||||
|
- 对比v4性能
|
||||||
|
|
||||||
|
### Phase 3: 手写检测功能
|
||||||
|
|
||||||
|
- [ ] **查找手写检测能力**
|
||||||
|
```python
|
||||||
|
# 可能的方式:
|
||||||
|
1. result中是否有 text_type 字段?
|
||||||
|
2. 是否有专门的 handwriting_detection 模型?
|
||||||
|
3. 是否有置信度差异可以利用?
|
||||||
|
4. PP-Structure 的 layout 分析?
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **对比测试**
|
||||||
|
- v4 (当前方案) vs v5
|
||||||
|
- 准确率、召回率、速度
|
||||||
|
- 手写检测能力
|
||||||
|
|
||||||
|
### Phase 4: 集成决策
|
||||||
|
|
||||||
|
- [ ] **性能评估**
|
||||||
|
- 如果v5更好 → 升级
|
||||||
|
- 如果改进不明显 → 保持v4
|
||||||
|
|
||||||
|
- [ ] **文档更新**
|
||||||
|
- 记录v5使用方法
|
||||||
|
- 更新PADDLEOCR_STATUS.md
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 调试技巧
|
||||||
|
|
||||||
|
### 1. 查看完整返回数据
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pprint
|
||||||
|
result = model.predict(image)
|
||||||
|
pprint.pprint(result) # 完整输出所有字段
|
||||||
|
|
||||||
|
# 或者
|
||||||
|
import json
|
||||||
|
print(json.dumps(result, indent=2, ensure_ascii=False))
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. 查找官方示例
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 在服务器上查找PaddleOCR安装示例
|
||||||
|
find ~/Project/paddleocr-server/venv/lib/python3.12/site-packages/paddleocr -name "*.py" | grep example
|
||||||
|
|
||||||
|
# 查看源码
|
||||||
|
less ~/Project/paddleocr-server/venv/lib/python3.12/site-packages/paddleocr/paddleocr.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. 查看可用模型
|
||||||
|
|
||||||
|
```python
|
||||||
|
from paddlex.inference.models import OFFICIAL_MODELS
|
||||||
|
print(OFFICIAL_MODELS) # 列出所有支持的模型名
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Web文档搜索
|
||||||
|
|
||||||
|
重点查看:
|
||||||
|
- https://github.com/PaddlePaddle/PaddleOCR
|
||||||
|
- https://www.paddleocr.ai
|
||||||
|
- https://github.com/PaddlePaddle/PaddleX
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📂 文件结构
|
||||||
|
|
||||||
|
```
|
||||||
|
/Volumes/NV2/pdf_recognize/
|
||||||
|
├── CURRENT_STATUS.md # 当前状态文档 ✅
|
||||||
|
├── NEW_SESSION_HANDOFF.md # 本文件 ✅
|
||||||
|
├── PADDLEOCR_STATUS.md # 详细技术文档 ✅
|
||||||
|
├── SESSION_INIT.md # 初始会话信息
|
||||||
|
│
|
||||||
|
├── paddleocr_client.py # 稳定客户端 (v2.7.3) ✅
|
||||||
|
├── paddleocr_server_v5.py # v5服务器 (未完成) ⚠️
|
||||||
|
│
|
||||||
|
├── test_paddleocr_client.py # 基础测试
|
||||||
|
├── test_mask_and_detect.py # 遮罩+检测
|
||||||
|
├── test_opencv_separation.py # Method 1+2
|
||||||
|
├── test_opencv_advanced.py # Method 3 (最佳) ✅
|
||||||
|
├── extract_signatures_paddleocr_improved.py # 完整Pipeline
|
||||||
|
│
|
||||||
|
└── check_rejected_for_missing.py # 诊断脚本
|
||||||
|
```
|
||||||
|
|
||||||
|
**服务器端** (`ssh gblinux`):
|
||||||
|
```
|
||||||
|
~/Project/paddleocr-server/
|
||||||
|
├── paddleocr_server.py # v2.7.3稳定版 ✅
|
||||||
|
├── paddleocr_server_v5.py # v5版本 (待完成) ⚠️
|
||||||
|
├── paddleocr_server_backup.py # 备份
|
||||||
|
├── server_stable.log # 当前运行日志
|
||||||
|
└── venv/ # 虚拟环境
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚡ 快速启动
|
||||||
|
|
||||||
|
### 启动稳定服务器 (v2.7.3)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh gblinux
|
||||||
|
cd ~/Project/paddleocr-server
|
||||||
|
source venv/bin/activate
|
||||||
|
python paddleocr_server.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 测试连接
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 本地Mac
|
||||||
|
cd /Volumes/NV2/pdf_recognize
|
||||||
|
source venv/bin/activate
|
||||||
|
python test_paddleocr_client.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 创建新研究分支
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Volumes/NV2/pdf_recognize
|
||||||
|
git checkout -b pp-ocrv5-research
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 注意事项
|
||||||
|
|
||||||
|
### 1. 不要破坏稳定版本
|
||||||
|
|
||||||
|
- `paddleocr-improvements` 分支保持稳定
|
||||||
|
- 所有v5实验在新分支 `pp-ocrv5-research`
|
||||||
|
- 服务器保留 `paddleocr_server.py` (v2.7.3)
|
||||||
|
- 新代码命名: `paddleocr_server_v5.py`
|
||||||
|
|
||||||
|
### 2. 环境隔离
|
||||||
|
|
||||||
|
- 服务器虚拟环境可能需要重建
|
||||||
|
- 或者用Docker隔离v4和v5
|
||||||
|
- 避免版本冲突
|
||||||
|
|
||||||
|
### 3. 性能测试
|
||||||
|
|
||||||
|
- 记录v4和v5的具体指标
|
||||||
|
- 至少测试10个样本
|
||||||
|
- 包括速度、准确率、召回率
|
||||||
|
|
||||||
|
### 4. 文档驱动
|
||||||
|
|
||||||
|
- 每个发现记录到文档
|
||||||
|
- API用法写清楚
|
||||||
|
- 便于未来维护
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 成功标准
|
||||||
|
|
||||||
|
### 最低目标
|
||||||
|
|
||||||
|
- [ ] 成功运行PP-OCRv5基础OCR
|
||||||
|
- [ ] 理解新API调用方式
|
||||||
|
- [ ] 服务器稳定运行
|
||||||
|
- [ ] 记录完整文档
|
||||||
|
|
||||||
|
### 理想目标
|
||||||
|
|
||||||
|
- [ ] 发现手写检测功能
|
||||||
|
- [ ] 性能超过v4方案
|
||||||
|
- [ ] 简化Pipeline复杂度
|
||||||
|
- [ ] 提升准确率 > 90%
|
||||||
|
|
||||||
|
### 决策点
|
||||||
|
|
||||||
|
**如果v5明显更好** → 升级到v5，废弃v4
|
||||||
|
**如果v5改进不明显** → 保持v4，v5仅作研究记录
|
||||||
|
**如果v5有bug** → 等待官方修复，暂用v4
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 问题排查
|
||||||
|
|
||||||
|
### 遇到问题时
|
||||||
|
|
||||||
|
1. **先查日志**: `tail -f ~/Project/paddleocr-server/server_stable.log`
|
||||||
|
2. **查看源码**: 在venv里找PaddleOCR代码
|
||||||
|
3. **搜索Issues**: https://github.com/PaddlePaddle/PaddleOCR/issues
|
||||||
|
4. **降级测试**: 确认v2.7.3是否还能用
|
||||||
|
|
||||||
|
### 常见问题
|
||||||
|
|
||||||
|
**Q: 服务器启动失败?**
|
||||||
|
A: 检查numpy版本 (需要 < 2.0)
|
||||||
|
|
||||||
|
**Q: 找不到模型?**
|
||||||
|
A: 模型名可能变化，查看OFFICIAL_MODELS
|
||||||
|
|
||||||
|
**Q: API调用失败?**
|
||||||
|
A: 对比官方文档，可能参数格式变化
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎓 学习资源
|
||||||
|
|
||||||
|
### 官方文档
|
||||||
|
|
||||||
|
1. **PP-OCRv5**: https://www.paddleocr.ai/main/en/version3.x/algorithm/PP-OCRv5/PP-OCRv5.html
|
||||||
|
2. **PaddleOCR GitHub**: https://github.com/PaddlePaddle/PaddleOCR
|
||||||
|
3. **PaddleX**: https://github.com/PaddlePaddle/PaddleX
|
||||||
|
|
||||||
|
### 相关技术
|
||||||
|
|
||||||
|
- PaddlePaddle深度学习框架
|
||||||
|
- PP-Structure文档结构分析
|
||||||
|
- 手写识别 (Handwriting Recognition)
|
||||||
|
- 版面分析 (Layout Analysis)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 💡 提示
|
||||||
|
|
||||||
|
### 如果发现内置手写检测
|
||||||
|
|
||||||
|
可能的用法:
|
||||||
|
```python
|
||||||
|
# 猜测1: 返回结果包含类型
|
||||||
|
for item in result:
|
||||||
|
text_type = item.get('type') # 'printed' or 'handwritten'?
|
||||||
|
|
||||||
|
# 猜测2: 专门的layout模型
|
||||||
|
from paddlex import create_model
|
||||||
|
layout_model = create_model("PP-Structure")
|
||||||
|
layout_result = layout_model.predict(image)
|
||||||
|
# 可能返回: text, handwriting, figure, table...
|
||||||
|
|
||||||
|
# 猜测3: 置信度差异
|
||||||
|
# 手写文字置信度可能更低
|
||||||
|
```
|
||||||
|
|
||||||
|
### 如果没有内置手写检测
|
||||||
|
|
||||||
|
那么当前OpenCV Method 3仍然是最佳方案，v5仅提供更好的OCR准确度。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ 完成检查清单
|
||||||
|
|
||||||
|
研究完成后，确保:
|
||||||
|
|
||||||
|
- [ ] 新API用法完全理解并文档化
|
||||||
|
- [ ] 服务器代码重写并测试通过
|
||||||
|
- [ ] 性能对比数据记录
|
||||||
|
- [ ] 决策文档 (升级 vs 保持v4)
|
||||||
|
- [ ] 代码提交到 `pp-ocrv5-research` 分支
|
||||||
|
- [ ] 更新 `CURRENT_STATUS.md`
|
||||||
|
- [ ] 如果升级: 合并到main分支
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**祝研究顺利！** 🚀
|
||||||
|
|
||||||
|
有问题随时查阅:
|
||||||
|
- `CURRENT_STATUS.md` - 当前方案详情
|
||||||
|
- `PADDLEOCR_STATUS.md` - 技术细节和问题分析
|
||||||
|
|
||||||
|
**最重要**: 记录所有发现，无论成功或失败，都是宝贵经验！
|
||||||
415
extract_signatures_paddleocr_improved.py
Normal file
415
extract_signatures_paddleocr_improved.py
Normal file
@@ -0,0 +1,415 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
PaddleOCR Signature Extraction - Improved Pipeline
|
||||||
|
|
||||||
|
Implements:
|
||||||
|
- Method B: Region Merging (merge nearby regions to avoid splits)
|
||||||
|
- Method E: Two-Stage Approach (second OCR pass on regions)
|
||||||
|
|
||||||
|
Pipeline:
|
||||||
|
1. PaddleOCR detects printed text on full page
|
||||||
|
2. Mask printed text with padding
|
||||||
|
3. Detect candidate regions
|
||||||
|
4. Merge nearby regions (METHOD B)
|
||||||
|
5. For each region: Run OCR again to remove remaining printed text (METHOD E)
|
||||||
|
6. VLM verification (optional)
|
||||||
|
7. Save cleaned handwriting regions
|
||||||
|
"""
|
||||||
|
|
||||||
|
import fitz # PyMuPDF
|
||||||
|
import numpy as np
|
||||||
|
import cv2
|
||||||
|
from pathlib import Path
|
||||||
|
from paddleocr_client import create_ocr_client
|
||||||
|
from typing import List, Dict, Tuple
|
||||||
|
import base64
|
||||||
|
import requests
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
TEST_PDF = "/Volumes/NV2/PDF-Processing/signature-image-output/201301_1324_AI1_page3.pdf"
|
||||||
|
OUTPUT_DIR = "/Volumes/NV2/PDF-Processing/signature-image-output/paddleocr_improved"
|
||||||
|
DPI = 300
|
||||||
|
|
||||||
|
# PaddleOCR Settings
|
||||||
|
MASKING_PADDING = 25 # Pixels to expand text boxes when masking
|
||||||
|
|
||||||
|
# Region Detection Parameters
|
||||||
|
MIN_REGION_AREA = 3000
|
||||||
|
MAX_REGION_AREA = 300000
|
||||||
|
MIN_ASPECT_RATIO = 0.3
|
||||||
|
MAX_ASPECT_RATIO = 15.0
|
||||||
|
|
||||||
|
# Region Merging Parameters (METHOD B)
|
||||||
|
MERGE_DISTANCE_HORIZONTAL = 100 # pixels
|
||||||
|
MERGE_DISTANCE_VERTICAL = 50 # pixels
|
||||||
|
|
||||||
|
# VLM Settings (optional)
|
||||||
|
USE_VLM_VERIFICATION = False # Set to True to enable VLM filtering
|
||||||
|
OLLAMA_URL = "http://192.168.30.36:11434"
|
||||||
|
OLLAMA_MODEL = "qwen2.5vl:32b"
|
||||||
|
|
||||||
|
|
||||||
|
def merge_nearby_regions(regions: List[Dict],
|
||||||
|
h_distance: int = 100,
|
||||||
|
v_distance: int = 50) -> List[Dict]:
|
||||||
|
"""
|
||||||
|
Merge regions that are close to each other (METHOD B).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
regions: List of region dicts with 'box': (x, y, w, h)
|
||||||
|
h_distance: Maximum horizontal distance between regions to merge
|
||||||
|
v_distance: Maximum vertical distance between regions to merge
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of merged regions
|
||||||
|
"""
|
||||||
|
if not regions:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Sort regions by y-coordinate (top to bottom)
|
||||||
|
regions = sorted(regions, key=lambda r: r['box'][1])
|
||||||
|
|
||||||
|
merged = []
|
||||||
|
skip_indices = set()
|
||||||
|
|
||||||
|
for i, region1 in enumerate(regions):
|
||||||
|
if i in skip_indices:
|
||||||
|
continue
|
||||||
|
|
||||||
|
x1, y1, w1, h1 = region1['box']
|
||||||
|
|
||||||
|
# Find all regions that should merge with this one
|
||||||
|
merge_group = [region1]
|
||||||
|
|
||||||
|
for j, region2 in enumerate(regions[i+1:], start=i+1):
|
||||||
|
if j in skip_indices:
|
||||||
|
continue
|
||||||
|
|
||||||
|
x2, y2, w2, h2 = region2['box']
|
||||||
|
|
||||||
|
# Calculate distances
|
||||||
|
# Horizontal distance: gap between boxes horizontally
|
||||||
|
h_dist = max(0, max(x1, x2) - min(x1 + w1, x2 + w2))
|
||||||
|
|
||||||
|
# Vertical distance: gap between boxes vertically
|
||||||
|
v_dist = max(0, max(y1, y2) - min(y1 + h1, y2 + h2))
|
||||||
|
|
||||||
|
# Check if regions are close enough to merge
|
||||||
|
if h_dist <= h_distance and v_dist <= v_distance:
|
||||||
|
merge_group.append(region2)
|
||||||
|
skip_indices.add(j)
|
||||||
|
# Update bounding box to include new region
|
||||||
|
x1 = min(x1, x2)
|
||||||
|
y1 = min(y1, y2)
|
||||||
|
w1 = max(x1 + w1, x2 + w2) - x1
|
||||||
|
h1 = max(y1 + h1, y2 + h2) - y1
|
||||||
|
|
||||||
|
# Create merged region
|
||||||
|
merged_box = (x1, y1, w1, h1)
|
||||||
|
merged_area = w1 * h1
|
||||||
|
merged_aspect = w1 / h1 if h1 > 0 else 0
|
||||||
|
|
||||||
|
merged.append({
|
||||||
|
'box': merged_box,
|
||||||
|
'area': merged_area,
|
||||||
|
'aspect_ratio': merged_aspect,
|
||||||
|
'merged_count': len(merge_group)
|
||||||
|
})
|
||||||
|
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
def clean_region_with_ocr(region_image: np.ndarray,
|
||||||
|
ocr_client,
|
||||||
|
padding: int = 10) -> np.ndarray:
|
||||||
|
"""
|
||||||
|
Remove printed text from a region using second OCR pass (METHOD E).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
region_image: The region image to clean
|
||||||
|
ocr_client: PaddleOCR client
|
||||||
|
padding: Padding around detected text boxes
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Cleaned region with printed text masked
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Run OCR on this specific region
|
||||||
|
text_boxes = ocr_client.get_text_boxes(region_image)
|
||||||
|
|
||||||
|
if not text_boxes:
|
||||||
|
return region_image # No text found, return as-is
|
||||||
|
|
||||||
|
# Mask detected printed text
|
||||||
|
cleaned = region_image.copy()
|
||||||
|
for (x, y, w, h) in text_boxes:
|
||||||
|
# Add padding
|
||||||
|
x_pad = max(0, x - padding)
|
||||||
|
y_pad = max(0, y - padding)
|
||||||
|
w_pad = min(cleaned.shape[1] - x_pad, w + 2*padding)
|
||||||
|
h_pad = min(cleaned.shape[0] - y_pad, h + 2*padding)
|
||||||
|
|
||||||
|
cv2.rectangle(cleaned, (x_pad, y_pad),
|
||||||
|
(x_pad + w_pad, y_pad + h_pad),
|
||||||
|
(255, 255, 255), -1) # Fill with white
|
||||||
|
|
||||||
|
return cleaned
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f" Warning: OCR cleaning failed: {e}")
|
||||||
|
return region_image
|
||||||
|
|
||||||
|
|
||||||
|
def verify_handwriting_with_vlm(image: np.ndarray) -> Tuple[bool, float]:
|
||||||
|
"""
|
||||||
|
Use VLM to verify if image contains handwriting.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
image: Region image (RGB numpy array)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(is_handwriting: bool, confidence: float)
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Convert image to base64
|
||||||
|
from PIL import Image
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
pil_image = Image.fromarray(image.astype(np.uint8))
|
||||||
|
buffered = BytesIO()
|
||||||
|
pil_image.save(buffered, format="PNG")
|
||||||
|
image_base64 = base64.b64encode(buffered.getvalue()).decode('utf-8')
|
||||||
|
|
||||||
|
# Ask VLM
|
||||||
|
prompt = """Does this image contain handwritten text or a handwritten signature?
|
||||||
|
|
||||||
|
Answer only 'yes' or 'no', followed by a confidence score 0-100.
|
||||||
|
Format: yes 95 OR no 80"""
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": OLLAMA_MODEL,
|
||||||
|
"prompt": prompt,
|
||||||
|
"images": [image_base64],
|
||||||
|
"stream": False
|
||||||
|
}
|
||||||
|
|
||||||
|
response = requests.post(f"{OLLAMA_URL}/api/generate",
|
||||||
|
json=payload, timeout=30)
|
||||||
|
response.raise_for_status()
|
||||||
|
answer = response.json()['response'].strip().lower()
|
||||||
|
|
||||||
|
# Parse answer
|
||||||
|
is_handwriting = 'yes' in answer
|
||||||
|
|
||||||
|
# Try to extract confidence
|
||||||
|
confidence = 0.5
|
||||||
|
parts = answer.split()
|
||||||
|
for part in parts:
|
||||||
|
try:
|
||||||
|
conf = float(part)
|
||||||
|
if 0 <= conf <= 100:
|
||||||
|
confidence = conf / 100
|
||||||
|
break
|
||||||
|
except:
|
||||||
|
continue
|
||||||
|
|
||||||
|
return is_handwriting, confidence
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f" Warning: VLM verification failed: {e}")
|
||||||
|
return True, 0.5 # Default to accepting the region
|
||||||
|
|
||||||
|
|
||||||
|
print("="*80)
|
||||||
|
print("PaddleOCR Improved Pipeline - Region Merging + Two-Stage Cleaning")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
# Create output directory
|
||||||
|
Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Step 1: Connect to PaddleOCR
|
||||||
|
print("\n1. Connecting to PaddleOCR server...")
|
||||||
|
try:
|
||||||
|
ocr_client = create_ocr_client()
|
||||||
|
print(f" ✅ Connected: {ocr_client.server_url}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 2: Render PDF
|
||||||
|
print("\n2. Rendering PDF...")
|
||||||
|
try:
|
||||||
|
doc = fitz.open(TEST_PDF)
|
||||||
|
page = doc[0]
|
||||||
|
mat = fitz.Matrix(DPI/72, DPI/72)
|
||||||
|
pix = page.get_pixmap(matrix=mat)
|
||||||
|
original_image = np.frombuffer(pix.samples, dtype=np.uint8).reshape(
|
||||||
|
pix.height, pix.width, pix.n)
|
||||||
|
|
||||||
|
if pix.n == 4:
|
||||||
|
original_image = cv2.cvtColor(original_image, cv2.COLOR_RGBA2RGB)
|
||||||
|
|
||||||
|
print(f" ✅ Rendered: {original_image.shape[1]}x{original_image.shape[0]}")
|
||||||
|
doc.close()
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 3: Detect printed text (Stage 1)
|
||||||
|
print("\n3. Detecting printed text (Stage 1 OCR)...")
|
||||||
|
try:
|
||||||
|
text_boxes = ocr_client.get_text_boxes(original_image)
|
||||||
|
print(f" ✅ Detected {len(text_boxes)} text regions")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 4: Mask printed text with padding
|
||||||
|
print(f"\n4. Masking printed text (padding={MASKING_PADDING}px)...")
|
||||||
|
try:
|
||||||
|
masked_image = original_image.copy()
|
||||||
|
|
||||||
|
for (x, y, w, h) in text_boxes:
|
||||||
|
# Add padding
|
||||||
|
x_pad = max(0, x - MASKING_PADDING)
|
||||||
|
y_pad = max(0, y - MASKING_PADDING)
|
||||||
|
w_pad = min(masked_image.shape[1] - x_pad, w + 2*MASKING_PADDING)
|
||||||
|
h_pad = min(masked_image.shape[0] - y_pad, h + 2*MASKING_PADDING)
|
||||||
|
|
||||||
|
cv2.rectangle(masked_image, (x_pad, y_pad),
|
||||||
|
(x_pad + w_pad, y_pad + h_pad), (0, 0, 0), -1)
|
||||||
|
|
||||||
|
print(f" ✅ Masked {len(text_boxes)} regions")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 5: Detect candidate regions
|
||||||
|
print("\n5. Detecting candidate regions...")
|
||||||
|
try:
|
||||||
|
gray = cv2.cvtColor(masked_image, cv2.COLOR_RGB2GRAY)
|
||||||
|
_, binary = cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY_INV)
|
||||||
|
|
||||||
|
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
|
||||||
|
morphed = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel, iterations=2)
|
||||||
|
|
||||||
|
contours, _ = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||||
|
|
||||||
|
candidate_regions = []
|
||||||
|
for contour in contours:
|
||||||
|
x, y, w, h = cv2.boundingRect(contour)
|
||||||
|
area = w * h
|
||||||
|
aspect_ratio = w / h if h > 0 else 0
|
||||||
|
|
||||||
|
if (MIN_REGION_AREA <= area <= MAX_REGION_AREA and
|
||||||
|
MIN_ASPECT_RATIO <= aspect_ratio <= MAX_ASPECT_RATIO):
|
||||||
|
candidate_regions.append({
|
||||||
|
'box': (x, y, w, h),
|
||||||
|
'area': area,
|
||||||
|
'aspect_ratio': aspect_ratio
|
||||||
|
})
|
||||||
|
|
||||||
|
print(f" ✅ Found {len(candidate_regions)} candidate regions")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 6: Merge nearby regions (METHOD B)
|
||||||
|
print(f"\n6. Merging nearby regions (h_dist<={MERGE_DISTANCE_HORIZONTAL}, v_dist<={MERGE_DISTANCE_VERTICAL})...")
|
||||||
|
try:
|
||||||
|
merged_regions = merge_nearby_regions(
|
||||||
|
candidate_regions,
|
||||||
|
h_distance=MERGE_DISTANCE_HORIZONTAL,
|
||||||
|
v_distance=MERGE_DISTANCE_VERTICAL
|
||||||
|
)
|
||||||
|
print(f" ✅ Merged {len(candidate_regions)} → {len(merged_regions)} regions")
|
||||||
|
|
||||||
|
for i, region in enumerate(merged_regions):
|
||||||
|
if region['merged_count'] > 1:
|
||||||
|
print(f" Region {i+1}: Merged {region['merged_count']} sub-regions")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ❌ Error: {e}")
|
||||||
|
import traceback
|
||||||
|
traceback.print_exc()
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
# Step 7: Extract and clean each region (METHOD E)
|
||||||
|
print("\n7. Extracting and cleaning regions (Stage 2 OCR)...")
|
||||||
|
final_signatures = []
|
||||||
|
|
||||||
|
for i, region in enumerate(merged_regions):
|
||||||
|
x, y, w, h = region['box']
|
||||||
|
print(f"\n Region {i+1}/{len(merged_regions)}: ({x}, {y}, {w}, {h})")
|
||||||
|
|
||||||
|
# Extract region from ORIGINAL image (not masked)
|
||||||
|
padding = 10
|
||||||
|
x_pad = max(0, x - padding)
|
||||||
|
y_pad = max(0, y - padding)
|
||||||
|
w_pad = min(original_image.shape[1] - x_pad, w + 2*padding)
|
||||||
|
h_pad = min(original_image.shape[0] - y_pad, h + 2*padding)
|
||||||
|
|
||||||
|
region_img = original_image[y_pad:y_pad+h_pad, x_pad:x_pad+w_pad].copy()
|
||||||
|
|
||||||
|
print(f" - Extracted: {region_img.shape[1]}x{region_img.shape[0]}px")
|
||||||
|
|
||||||
|
# Clean with second OCR pass
|
||||||
|
print(f" - Running Stage 2 OCR to remove printed text...")
|
||||||
|
cleaned_region = clean_region_with_ocr(region_img, ocr_client, padding=5)
|
||||||
|
|
||||||
|
# VLM verification (optional)
|
||||||
|
if USE_VLM_VERIFICATION:
|
||||||
|
print(f" - VLM verification...")
|
||||||
|
is_handwriting, confidence = verify_handwriting_with_vlm(cleaned_region)
|
||||||
|
print(f" - VLM says: {'✅ Handwriting' if is_handwriting else '❌ Not handwriting'} (confidence: {confidence:.2f})")
|
||||||
|
|
||||||
|
if not is_handwriting:
|
||||||
|
print(f" - Skipping (not handwriting)")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Save
|
||||||
|
final_signatures.append({
|
||||||
|
'image': cleaned_region,
|
||||||
|
'box': region['box'],
|
||||||
|
'original_image': region_img
|
||||||
|
})
|
||||||
|
|
||||||
|
print(f" ✅ Kept as signature candidate")
|
||||||
|
|
||||||
|
print(f"\n ✅ Final signatures: {len(final_signatures)}")
|
||||||
|
|
||||||
|
# Step 8: Save results
|
||||||
|
print("\n8. Saving results...")
|
||||||
|
|
||||||
|
for i, sig in enumerate(final_signatures):
|
||||||
|
# Save cleaned signature
|
||||||
|
sig_path = Path(OUTPUT_DIR) / f"signature_{i+1:02d}_cleaned.png"
|
||||||
|
cv2.imwrite(str(sig_path), cv2.cvtColor(sig['image'], cv2.COLOR_RGB2BGR))
|
||||||
|
|
||||||
|
# Save original region for comparison
|
||||||
|
orig_path = Path(OUTPUT_DIR) / f"signature_{i+1:02d}_original.png"
|
||||||
|
cv2.imwrite(str(orig_path), cv2.cvtColor(sig['original_image'], cv2.COLOR_RGB2BGR))
|
||||||
|
|
||||||
|
print(f" 📁 Signature {i+1}: {sig_path.name}")
|
||||||
|
|
||||||
|
# Save visualizations
|
||||||
|
vis_merged = original_image.copy()
|
||||||
|
for region in merged_regions:
|
||||||
|
x, y, w, h = region['box']
|
||||||
|
color = (255, 0, 0) if region in [{'box': s['box']} for s in final_signatures] else (128, 128, 128)
|
||||||
|
cv2.rectangle(vis_merged, (x, y), (x + w, y + h), color, 3)
|
||||||
|
|
||||||
|
vis_path = Path(OUTPUT_DIR) / "visualization_merged_regions.png"
|
||||||
|
cv2.imwrite(str(vis_path), cv2.cvtColor(vis_merged, cv2.COLOR_RGB2BGR))
|
||||||
|
print(f" 📁 Visualization: {vis_path.name}")
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("Pipeline completed!")
|
||||||
|
print(f"Results: {OUTPUT_DIR}")
|
||||||
|
print("="*80)
|
||||||
|
print(f"\nSummary:")
|
||||||
|
print(f" - Stage 1 OCR: {len(text_boxes)} text regions masked")
|
||||||
|
print(f" - Initial candidates: {len(candidate_regions)}")
|
||||||
|
print(f" - After merging: {len(merged_regions)}")
|
||||||
|
print(f" - Final signatures: {len(final_signatures)}")
|
||||||
|
print(f" - Expected signatures: 2 (楊智惠, 張志銘)")
|
||||||
|
print("="*80)
|
||||||
91
paddleocr_server_v5.py
Normal file
91
paddleocr_server_v5.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
PaddleOCR Server v5 (PP-OCRv5)
|
||||||
|
Flask HTTP server exposing PaddleOCR v3.3.0 functionality
|
||||||
|
"""
|
||||||
|
|
||||||
|
from paddlex import create_model
|
||||||
|
import base64
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
from io import BytesIO
|
||||||
|
from flask import Flask, request, jsonify
|
||||||
|
import traceback
|
||||||
|
|
||||||
|
app = Flask(__name__)
|
||||||
|
|
||||||
|
# Initialize PP-OCRv5 model
|
||||||
|
print("Initializing PP-OCRv5 model...")
|
||||||
|
model = create_model("PP-OCRv5_server")
|
||||||
|
print("PP-OCRv5 model loaded successfully!")
|
||||||
|
|
||||||
|
@app.route('/health', methods=['GET'])
|
||||||
|
def health():
|
||||||
|
"""Health check endpoint."""
|
||||||
|
return jsonify({
|
||||||
|
'status': 'ok',
|
||||||
|
'service': 'paddleocr-server-v5',
|
||||||
|
'version': '3.3.0',
|
||||||
|
'model': 'PP-OCRv5_server',
|
||||||
|
'gpu_enabled': True
|
||||||
|
})
|
||||||
|
|
||||||
|
@app.route('/ocr', methods=['POST'])
|
||||||
|
def ocr_endpoint():
|
||||||
|
"""
|
||||||
|
OCR endpoint using PP-OCRv5.
|
||||||
|
|
||||||
|
Accepts: {"image": "base64_encoded_image"}
|
||||||
|
Returns: {"success": true, "count": N, "results": [...]}
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Parse request
|
||||||
|
data = request.get_json()
|
||||||
|
image_base64 = data['image']
|
||||||
|
|
||||||
|
# Decode image
|
||||||
|
image_bytes = base64.b64decode(image_base64)
|
||||||
|
image = Image.open(BytesIO(image_bytes))
|
||||||
|
image_np = np.array(image)
|
||||||
|
|
||||||
|
# Run OCR with PP-OCRv5
|
||||||
|
result = model.predict(image_np)
|
||||||
|
|
||||||
|
# Format results
|
||||||
|
formatted_results = []
|
||||||
|
|
||||||
|
if result and 'dt_polys' in result[0] and 'rec_text' in result[0]:
|
||||||
|
dt_polys = result[0]['dt_polys']
|
||||||
|
rec_texts = result[0]['rec_text']
|
||||||
|
rec_scores = result[0]['rec_score']
|
||||||
|
|
||||||
|
for i in range(len(dt_polys)):
|
||||||
|
box = dt_polys[i].tolist() # Convert to list
|
||||||
|
text = rec_texts[i]
|
||||||
|
confidence = float(rec_scores[i])
|
||||||
|
|
||||||
|
formatted_results.append({
|
||||||
|
'box': box,
|
||||||
|
'text': text,
|
||||||
|
'confidence': confidence
|
||||||
|
})
|
||||||
|
|
||||||
|
return jsonify({
|
||||||
|
'success': True,
|
||||||
|
'count': len(formatted_results),
|
||||||
|
'results': formatted_results
|
||||||
|
})
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Error during OCR: {str(e)}")
|
||||||
|
traceback.print_exc()
|
||||||
|
return jsonify({
|
||||||
|
'success': False,
|
||||||
|
'error': str(e)
|
||||||
|
}), 500
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
print("Starting PP-OCRv5 server on port 5555...")
|
||||||
|
print("Model: PP-OCRv5_server")
|
||||||
|
print("Version: 3.3.0")
|
||||||
|
app.run(host='0.0.0.0', port=5555, debug=False)
|
||||||
256
test_opencv_advanced.py
Normal file
256
test_opencv_advanced.py
Normal file
@@ -0,0 +1,256 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Advanced OpenCV separation based on key observations:
|
||||||
|
1. 手写字比印刷字大 (Handwriting is LARGER)
|
||||||
|
2. 手写笔画长度更长 (Handwriting strokes are LONGER)
|
||||||
|
3. 印刷标楷体规律，手写潦草 (Printed is regular, handwriting is messy)
|
||||||
|
"""
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
from pathlib import Path
|
||||||
|
from scipy import ndimage
|
||||||
|
|
||||||
|
# Test image
|
||||||
|
TEST_IMAGE = "/Volumes/NV2/PDF-Processing/signature-image-output/paddleocr_improved/signature_02_original.png"
|
||||||
|
OUTPUT_DIR = "/Volumes/NV2/PDF-Processing/signature-image-output/opencv_advanced_test"
|
||||||
|
|
||||||
|
print("="*80)
|
||||||
|
print("Advanced OpenCV Separation - Size + Stroke Length + Regularity")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Load and preprocess
|
||||||
|
image = cv2.imread(TEST_IMAGE)
|
||||||
|
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
|
||||||
|
_, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
|
||||||
|
|
||||||
|
print(f"\nImage: {image.shape[1]}x{image.shape[0]}")
|
||||||
|
|
||||||
|
# Save binary
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "00_binary.png"), binary)
|
||||||
|
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("METHOD 3: Comprehensive Feature Analysis")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
# Find connected components
|
||||||
|
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(binary, connectivity=8)
|
||||||
|
|
||||||
|
print(f"\nFound {num_labels - 1} connected components")
|
||||||
|
print("\nAnalyzing each component...")
|
||||||
|
|
||||||
|
# Store analysis for each component
|
||||||
|
components_analysis = []
|
||||||
|
|
||||||
|
for i in range(1, num_labels):
|
||||||
|
x, y, w, h, area = stats[i]
|
||||||
|
|
||||||
|
# Extract component mask
|
||||||
|
component_mask = (labels == i).astype(np.uint8) * 255
|
||||||
|
|
||||||
|
# ============================================
|
||||||
|
# FEATURE 1: Size (手写字比印刷字大)
|
||||||
|
# ============================================
|
||||||
|
bbox_area = w * h
|
||||||
|
font_height = h # Character height is a good indicator
|
||||||
|
|
||||||
|
# ============================================
|
||||||
|
# FEATURE 2: Stroke Length (笔画长度)
|
||||||
|
# ============================================
|
||||||
|
# Skeletonize to get the actual stroke centerline
|
||||||
|
from skimage.morphology import skeletonize
|
||||||
|
skeleton = skeletonize(component_mask // 255)
|
||||||
|
stroke_length = np.sum(skeleton) # Total length of strokes
|
||||||
|
|
||||||
|
# Stroke length ratio (length relative to area)
|
||||||
|
stroke_length_ratio = stroke_length / area if area > 0 else 0
|
||||||
|
|
||||||
|
# ============================================
|
||||||
|
# FEATURE 3: Regularity vs Messiness
|
||||||
|
# ============================================
|
||||||
|
# 3a. Compactness (regular shapes are more compact)
|
||||||
|
contours, _ = cv2.findContours(component_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
||||||
|
if contours:
|
||||||
|
perimeter = cv2.arcLength(contours[0], True)
|
||||||
|
compactness = (4 * np.pi * area) / (perimeter * perimeter) if perimeter > 0 else 0
|
||||||
|
else:
|
||||||
|
compactness = 0
|
||||||
|
|
||||||
|
# 3b. Solidity (ratio of area to convex hull area)
|
||||||
|
if contours:
|
||||||
|
hull = cv2.convexHull(contours[0])
|
||||||
|
hull_area = cv2.contourArea(hull)
|
||||||
|
solidity = area / hull_area if hull_area > 0 else 0
|
||||||
|
else:
|
||||||
|
solidity = 0
|
||||||
|
|
||||||
|
# 3c. Extent (ratio of area to bounding box area)
|
||||||
|
extent = area / bbox_area if bbox_area > 0 else 0
|
||||||
|
|
||||||
|
# 3d. Edge roughness (measure irregularity)
|
||||||
|
# More irregular edges = more "messy" = likely handwriting
|
||||||
|
edges = cv2.Canny(component_mask, 50, 150)
|
||||||
|
edge_pixels = np.sum(edges > 0)
|
||||||
|
edge_roughness = edge_pixels / perimeter if perimeter > 0 else 0
|
||||||
|
|
||||||
|
# ============================================
|
||||||
|
# CLASSIFICATION LOGIC
|
||||||
|
# ============================================
|
||||||
|
|
||||||
|
# Large characters are likely handwriting
|
||||||
|
is_large = font_height > 40 # Threshold for "large" characters
|
||||||
|
|
||||||
|
# Long strokes relative to area indicate handwriting
|
||||||
|
is_long_stroke = stroke_length_ratio > 0.4 # Handwriting has higher ratio
|
||||||
|
|
||||||
|
# Regular shapes (high compactness, high solidity) = printed
|
||||||
|
# Irregular shapes (low compactness, low solidity) = handwriting
|
||||||
|
is_irregular = compactness < 0.3 or solidity < 0.7 or extent < 0.5
|
||||||
|
|
||||||
|
# DECISION RULES
|
||||||
|
handwriting_score = 0
|
||||||
|
|
||||||
|
# Size-based scoring (重要!)
|
||||||
|
if font_height > 50:
|
||||||
|
handwriting_score += 3 # Very large = likely handwriting
|
||||||
|
elif font_height > 35:
|
||||||
|
handwriting_score += 2 # Medium-large = possibly handwriting
|
||||||
|
elif font_height < 25:
|
||||||
|
handwriting_score -= 2 # Small = likely printed
|
||||||
|
|
||||||
|
# Stroke length scoring
|
||||||
|
if stroke_length_ratio > 0.5:
|
||||||
|
handwriting_score += 2 # Long strokes
|
||||||
|
elif stroke_length_ratio > 0.35:
|
||||||
|
handwriting_score += 1
|
||||||
|
|
||||||
|
# Regularity scoring (标楷体 is regular, 手写 is messy)
|
||||||
|
if is_irregular:
|
||||||
|
handwriting_score += 1 # Irregular = handwriting
|
||||||
|
else:
|
||||||
|
handwriting_score -= 1 # Regular = printed
|
||||||
|
|
||||||
|
# Area scoring
|
||||||
|
if area > 2000:
|
||||||
|
handwriting_score += 2 # Large area = handwriting
|
||||||
|
elif area < 500:
|
||||||
|
handwriting_score -= 1 # Small area = printed
|
||||||
|
|
||||||
|
# Final classification
|
||||||
|
is_handwriting = handwriting_score > 0
|
||||||
|
|
||||||
|
components_analysis.append({
|
||||||
|
'id': i,
|
||||||
|
'box': (x, y, w, h),
|
||||||
|
'area': area,
|
||||||
|
'height': font_height,
|
||||||
|
'stroke_length': stroke_length,
|
||||||
|
'stroke_ratio': stroke_length_ratio,
|
||||||
|
'compactness': compactness,
|
||||||
|
'solidity': solidity,
|
||||||
|
'extent': extent,
|
||||||
|
'edge_roughness': edge_roughness,
|
||||||
|
'handwriting_score': handwriting_score,
|
||||||
|
'is_handwriting': is_handwriting,
|
||||||
|
'mask': component_mask
|
||||||
|
})
|
||||||
|
|
||||||
|
# Sort by area (largest first)
|
||||||
|
components_analysis.sort(key=lambda c: c['area'], reverse=True)
|
||||||
|
|
||||||
|
# Print analysis
|
||||||
|
print("\n" + "-"*80)
|
||||||
|
print("Top 10 Components Analysis:")
|
||||||
|
print("-"*80)
|
||||||
|
print(f"{'ID':<4} {'Area':<6} {'H':<4} {'StrokeLen':<9} {'StrokeR':<7} {'Compact':<7} "
|
||||||
|
f"{'Solid':<6} {'Score':<5} {'Type':<12}")
|
||||||
|
print("-"*80)
|
||||||
|
|
||||||
|
for i, comp in enumerate(components_analysis[:10]):
|
||||||
|
comp_type = "✅ Handwriting" if comp['is_handwriting'] else "❌ Printed"
|
||||||
|
print(f"{comp['id']:<4} {comp['area']:<6} {comp['height']:<4} "
|
||||||
|
f"{comp['stroke_length']:<9.0f} {comp['stroke_ratio']:<7.3f} "
|
||||||
|
f"{comp['compactness']:<7.3f} {comp['solidity']:<6.3f} "
|
||||||
|
f"{comp['handwriting_score']:>+5} {comp_type:<12}")
|
||||||
|
|
||||||
|
# Create masks
|
||||||
|
handwriting_mask = np.zeros_like(binary)
|
||||||
|
printed_mask = np.zeros_like(binary)
|
||||||
|
|
||||||
|
for comp in components_analysis:
|
||||||
|
if comp['is_handwriting']:
|
||||||
|
handwriting_mask = cv2.bitwise_or(handwriting_mask, comp['mask'])
|
||||||
|
else:
|
||||||
|
printed_mask = cv2.bitwise_or(printed_mask, comp['mask'])
|
||||||
|
|
||||||
|
# Statistics
|
||||||
|
hw_count = sum(1 for c in components_analysis if c['is_handwriting'])
|
||||||
|
pr_count = sum(1 for c in components_analysis if not c['is_handwriting'])
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("Classification Results:")
|
||||||
|
print("="*80)
|
||||||
|
print(f" Handwriting components: {hw_count}")
|
||||||
|
print(f" Printed components: {pr_count}")
|
||||||
|
print(f" Total: {len(components_analysis)}")
|
||||||
|
|
||||||
|
# Apply to original image
|
||||||
|
result_handwriting = cv2.bitwise_and(image, image, mask=handwriting_mask)
|
||||||
|
result_printed = cv2.bitwise_and(image, image, mask=printed_mask)
|
||||||
|
|
||||||
|
# Save results
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method3_handwriting_mask.png"), handwriting_mask)
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method3_printed_mask.png"), printed_mask)
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method3_handwriting_result.png"), result_handwriting)
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method3_printed_result.png"), result_printed)
|
||||||
|
|
||||||
|
# Create visualization
|
||||||
|
vis_overlay = image.copy()
|
||||||
|
vis_overlay[handwriting_mask > 0] = [0, 255, 0] # Green for handwriting
|
||||||
|
vis_overlay[printed_mask > 0] = [0, 0, 255] # Red for printed
|
||||||
|
vis_final = cv2.addWeighted(image, 0.6, vis_overlay, 0.4, 0)
|
||||||
|
|
||||||
|
# Add labels to visualization
|
||||||
|
for comp in components_analysis[:15]: # Label top 15
|
||||||
|
x, y, w, h = comp['box']
|
||||||
|
cx, cy = x + w//2, y + h//2
|
||||||
|
|
||||||
|
color = (0, 255, 0) if comp['is_handwriting'] else (0, 0, 255)
|
||||||
|
label = f"H{comp['handwriting_score']:+d}" if comp['is_handwriting'] else f"P{comp['handwriting_score']:+d}"
|
||||||
|
|
||||||
|
cv2.putText(vis_final, label, (cx-15, cy), cv2.FONT_HERSHEY_SIMPLEX, 0.4, color, 1)
|
||||||
|
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method3_visualization.png"), vis_final)
|
||||||
|
|
||||||
|
print("\n📁 Saved results:")
|
||||||
|
print(" - method3_handwriting_mask.png")
|
||||||
|
print(" - method3_printed_mask.png")
|
||||||
|
print(" - method3_handwriting_result.png")
|
||||||
|
print(" - method3_printed_result.png")
|
||||||
|
print(" - method3_visualization.png")
|
||||||
|
|
||||||
|
# Calculate content pixels
|
||||||
|
hw_pixels = np.count_nonzero(handwriting_mask)
|
||||||
|
pr_pixels = np.count_nonzero(printed_mask)
|
||||||
|
total_pixels = np.count_nonzero(binary)
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("Pixel Distribution:")
|
||||||
|
print("="*80)
|
||||||
|
print(f" Total foreground: {total_pixels:6d} pixels (100.0%)")
|
||||||
|
print(f" Handwriting: {hw_pixels:6d} pixels ({hw_pixels/total_pixels*100:5.1f}%)")
|
||||||
|
print(f" Printed: {pr_pixels:6d} pixels ({pr_pixels/total_pixels*100:5.1f}%)")
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("Test completed!")
|
||||||
|
print(f"Results: {OUTPUT_DIR}")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
print("\n📊 Feature Analysis Summary:")
|
||||||
|
print(" ✅ Size-based classification: Large characters → Handwriting")
|
||||||
|
print(" ✅ Stroke length analysis: Long stroke ratio → Handwriting")
|
||||||
|
print(" ✅ Regularity analysis: Irregular shapes → Handwriting")
|
||||||
|
print("\nNext: Review visualization to tune thresholds if needed")
|
||||||
272
test_opencv_separation.py
Normal file
272
test_opencv_separation.py
Normal file
@@ -0,0 +1,272 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test OpenCV methods to separate handwriting from printed text
|
||||||
|
|
||||||
|
Tests two methods:
|
||||||
|
1. Stroke Width Analysis (笔画宽度分析)
|
||||||
|
2. Connected Components + Shape Features (连通组件+形状特征)
|
||||||
|
"""
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Test image - contains both printed and handwritten
|
||||||
|
TEST_IMAGE = "/Volumes/NV2/PDF-Processing/signature-image-output/paddleocr_improved/signature_02_original.png"
|
||||||
|
OUTPUT_DIR = "/Volumes/NV2/PDF-Processing/signature-image-output/opencv_separation_test"
|
||||||
|
|
||||||
|
print("="*80)
|
||||||
|
print("OpenCV Handwriting Separation Test")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
# Create output directory
|
||||||
|
Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Load image
|
||||||
|
print(f"\nLoading test image: {Path(TEST_IMAGE).name}")
|
||||||
|
image = cv2.imread(TEST_IMAGE)
|
||||||
|
if image is None:
|
||||||
|
print(f"Error: Cannot load image from {TEST_IMAGE}")
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
|
||||||
|
print(f"Image size: {image.shape[1]}x{image.shape[0]}")
|
||||||
|
|
||||||
|
# Convert to grayscale
|
||||||
|
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
|
||||||
|
|
||||||
|
# Binarize
|
||||||
|
_, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
|
||||||
|
|
||||||
|
# Save binary for reference
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "00_binary.png"), binary)
|
||||||
|
print("\n📁 Saved: 00_binary.png")
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("METHOD 1: Stroke Width Analysis (笔画宽度分析)")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
def method1_stroke_width(binary_img, threshold_values=[2.0, 3.0, 4.0, 5.0]):
|
||||||
|
"""
|
||||||
|
Method 1: Separate by stroke width using distance transform
|
||||||
|
|
||||||
|
Args:
|
||||||
|
binary_img: Binary image (foreground = 255, background = 0)
|
||||||
|
threshold_values: List of distance thresholds to test
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of (threshold, result_image) tuples
|
||||||
|
"""
|
||||||
|
results = []
|
||||||
|
|
||||||
|
# Calculate distance transform
|
||||||
|
dist_transform = cv2.distanceTransform(binary_img, cv2.DIST_L2, 5)
|
||||||
|
|
||||||
|
# Normalize for visualization
|
||||||
|
dist_normalized = cv2.normalize(dist_transform, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
|
||||||
|
results.append(('distance_transform', dist_normalized))
|
||||||
|
|
||||||
|
print("\n Distance transform statistics:")
|
||||||
|
print(f" Min: {dist_transform.min():.2f}")
|
||||||
|
print(f" Max: {dist_transform.max():.2f}")
|
||||||
|
print(f" Mean: {dist_transform.mean():.2f}")
|
||||||
|
print(f" Median: {np.median(dist_transform):.2f}")
|
||||||
|
|
||||||
|
# Test different thresholds
|
||||||
|
print("\n Testing different stroke width thresholds:")
|
||||||
|
|
||||||
|
for threshold in threshold_values:
|
||||||
|
# Pixels with distance > threshold are considered "thick strokes" (handwriting)
|
||||||
|
handwriting_mask = (dist_transform > threshold).astype(np.uint8) * 255
|
||||||
|
|
||||||
|
# Count pixels
|
||||||
|
total_foreground = np.count_nonzero(binary_img)
|
||||||
|
handwriting_pixels = np.count_nonzero(handwriting_mask)
|
||||||
|
percentage = (handwriting_pixels / total_foreground * 100) if total_foreground > 0 else 0
|
||||||
|
|
||||||
|
print(f" Threshold {threshold:.1f}: {handwriting_pixels} pixels ({percentage:.1f}% of foreground)")
|
||||||
|
|
||||||
|
results.append((f'threshold_{threshold:.1f}', handwriting_mask))
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
# Run Method 1
|
||||||
|
method1_results = method1_stroke_width(binary, threshold_values=[2.0, 2.5, 3.0, 3.5, 4.0, 5.0])
|
||||||
|
|
||||||
|
# Save Method 1 results
|
||||||
|
print("\n Saving results...")
|
||||||
|
for name, result_img in method1_results:
|
||||||
|
output_path = Path(OUTPUT_DIR) / f"method1_{name}.png"
|
||||||
|
cv2.imwrite(str(output_path), result_img)
|
||||||
|
print(f" 📁 {output_path.name}")
|
||||||
|
|
||||||
|
# Apply best threshold result to original image
|
||||||
|
best_threshold = 3.0 # Will adjust based on visual inspection
|
||||||
|
_, best_mask = [(n, r) for n, r in method1_results if f'threshold_{best_threshold}' in n][0]
|
||||||
|
|
||||||
|
# Dilate mask slightly to connect nearby strokes
|
||||||
|
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
|
||||||
|
best_mask_dilated = cv2.dilate(best_mask, kernel, iterations=1)
|
||||||
|
|
||||||
|
# Apply to color image
|
||||||
|
result_method1 = cv2.bitwise_and(image, image, mask=best_mask_dilated)
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method1_final_result.png"), result_method1)
|
||||||
|
print(f"\n 📁 Final result: method1_final_result.png (threshold={best_threshold})")
|
||||||
|
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("METHOD 2: Connected Components + Shape Features (连通组件分析)")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
def method2_component_analysis(binary_img, original_img):
|
||||||
|
"""
|
||||||
|
Method 2: Analyze each connected component's shape features
|
||||||
|
|
||||||
|
Printed text characteristics:
|
||||||
|
- Regular bounding box (aspect ratio ~1:1)
|
||||||
|
- Medium size (200-2000 pixels)
|
||||||
|
- High circularity/compactness
|
||||||
|
|
||||||
|
Handwriting characteristics:
|
||||||
|
- Irregular shapes
|
||||||
|
- May be large (connected strokes)
|
||||||
|
- Variable aspect ratios
|
||||||
|
"""
|
||||||
|
# Find connected components
|
||||||
|
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(binary_img, connectivity=8)
|
||||||
|
|
||||||
|
print(f"\n Found {num_labels - 1} connected components")
|
||||||
|
|
||||||
|
# Create masks for different categories
|
||||||
|
handwriting_mask = np.zeros_like(binary_img)
|
||||||
|
printed_mask = np.zeros_like(binary_img)
|
||||||
|
|
||||||
|
# Analyze each component
|
||||||
|
component_info = []
|
||||||
|
|
||||||
|
for i in range(1, num_labels): # Skip background (0)
|
||||||
|
x, y, w, h, area = stats[i]
|
||||||
|
|
||||||
|
# Calculate features
|
||||||
|
aspect_ratio = w / h if h > 0 else 0
|
||||||
|
perimeter = cv2.arcLength(cv2.findContours((labels == i).astype(np.uint8),
|
||||||
|
cv2.RETR_EXTERNAL,
|
||||||
|
cv2.CHAIN_APPROX_SIMPLE)[0][0], True)
|
||||||
|
compactness = (4 * np.pi * area) / (perimeter * perimeter) if perimeter > 0 else 0
|
||||||
|
|
||||||
|
# Classification logic
|
||||||
|
# Printed text: medium size, regular aspect ratio, compact
|
||||||
|
is_printed = (
|
||||||
|
(200 < area < 3000) and # Medium size
|
||||||
|
(0.3 < aspect_ratio < 3.0) and # Not too elongated
|
||||||
|
(area < 1000) # Small to medium
|
||||||
|
)
|
||||||
|
|
||||||
|
# Handwriting: larger, or irregular, or very wide/tall
|
||||||
|
is_handwriting = (
|
||||||
|
(area >= 3000) or # Large components (likely handwriting)
|
||||||
|
(aspect_ratio > 3.0) or # Very elongated (连笔)
|
||||||
|
(aspect_ratio < 0.3) or # Very tall
|
||||||
|
not is_printed # Default to handwriting if not clearly printed
|
||||||
|
)
|
||||||
|
|
||||||
|
component_info.append({
|
||||||
|
'id': i,
|
||||||
|
'area': area,
|
||||||
|
'aspect_ratio': aspect_ratio,
|
||||||
|
'compactness': compactness,
|
||||||
|
'is_printed': is_printed,
|
||||||
|
'is_handwriting': is_handwriting
|
||||||
|
})
|
||||||
|
|
||||||
|
# Assign to mask
|
||||||
|
if is_handwriting:
|
||||||
|
handwriting_mask[labels == i] = 255
|
||||||
|
if is_printed:
|
||||||
|
printed_mask[labels == i] = 255
|
||||||
|
|
||||||
|
# Print statistics
|
||||||
|
print("\n Component statistics:")
|
||||||
|
handwriting_components = [c for c in component_info if c['is_handwriting']]
|
||||||
|
printed_components = [c for c in component_info if c['is_printed']]
|
||||||
|
|
||||||
|
print(f" Handwriting components: {len(handwriting_components)}")
|
||||||
|
print(f" Printed components: {len(printed_components)}")
|
||||||
|
|
||||||
|
# Show top 5 largest components
|
||||||
|
print("\n Top 5 largest components:")
|
||||||
|
sorted_components = sorted(component_info, key=lambda c: c['area'], reverse=True)
|
||||||
|
for i, comp in enumerate(sorted_components[:5], 1):
|
||||||
|
comp_type = "Handwriting" if comp['is_handwriting'] else "Printed"
|
||||||
|
print(f" {i}. Area: {comp['area']:5d}, Aspect: {comp['aspect_ratio']:.2f}, "
|
||||||
|
f"Type: {comp_type}")
|
||||||
|
|
||||||
|
return handwriting_mask, printed_mask, component_info
|
||||||
|
|
||||||
|
# Run Method 2
|
||||||
|
handwriting_mask_m2, printed_mask_m2, components = method2_component_analysis(binary, image)
|
||||||
|
|
||||||
|
# Save Method 2 results
|
||||||
|
print("\n Saving results...")
|
||||||
|
|
||||||
|
# Handwriting mask
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method2_handwriting_mask.png"), handwriting_mask_m2)
|
||||||
|
print(f" 📁 method2_handwriting_mask.png")
|
||||||
|
|
||||||
|
# Printed mask
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method2_printed_mask.png"), printed_mask_m2)
|
||||||
|
print(f" 📁 method2_printed_mask.png")
|
||||||
|
|
||||||
|
# Apply to original image
|
||||||
|
result_handwriting = cv2.bitwise_and(image, image, mask=handwriting_mask_m2)
|
||||||
|
result_printed = cv2.bitwise_and(image, image, mask=printed_mask_m2)
|
||||||
|
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method2_handwriting_result.png"), result_handwriting)
|
||||||
|
print(f" 📁 method2_handwriting_result.png")
|
||||||
|
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method2_printed_result.png"), result_printed)
|
||||||
|
print(f" 📁 method2_printed_result.png")
|
||||||
|
|
||||||
|
# Create visualization with component labels
|
||||||
|
vis_components = cv2.cvtColor(binary, cv2.COLOR_GRAY2BGR)
|
||||||
|
vis_components = cv2.cvtColor(vis_components, cv2.COLOR_BGR2RGB)
|
||||||
|
|
||||||
|
# Color code: green = handwriting, red = printed
|
||||||
|
vis_overlay = image.copy()
|
||||||
|
vis_overlay[handwriting_mask_m2 > 0] = [0, 255, 0] # Green for handwriting
|
||||||
|
vis_overlay[printed_mask_m2 > 0] = [0, 0, 255] # Red for printed
|
||||||
|
|
||||||
|
# Blend with original
|
||||||
|
vis_final = cv2.addWeighted(image, 0.6, vis_overlay, 0.4, 0)
|
||||||
|
cv2.imwrite(str(Path(OUTPUT_DIR) / "method2_visualization.png"), vis_final)
|
||||||
|
print(f" 📁 method2_visualization.png (green=handwriting, red=printed)")
|
||||||
|
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("COMPARISON")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
# Count non-white pixels in each result
|
||||||
|
def count_content_pixels(img):
|
||||||
|
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) if len(img.shape) == 3 else img
|
||||||
|
return np.count_nonzero(gray > 10)
|
||||||
|
|
||||||
|
original_pixels = count_content_pixels(image)
|
||||||
|
method1_pixels = count_content_pixels(result_method1)
|
||||||
|
method2_pixels = count_content_pixels(result_handwriting)
|
||||||
|
|
||||||
|
print(f"\nContent pixels retained:")
|
||||||
|
print(f" Original image: {original_pixels:6d} pixels")
|
||||||
|
print(f" Method 1 (stroke): {method1_pixels:6d} pixels ({method1_pixels/original_pixels*100:.1f}%)")
|
||||||
|
print(f" Method 2 (component): {method2_pixels:6d} pixels ({method2_pixels/original_pixels*100:.1f}%)")
|
||||||
|
|
||||||
|
print("\n" + "="*80)
|
||||||
|
print("Test completed!")
|
||||||
|
print(f"Results saved to: {OUTPUT_DIR}")
|
||||||
|
print("="*80)
|
||||||
|
|
||||||
|
print("\nNext steps:")
|
||||||
|
print(" 1. Review the output images")
|
||||||
|
print(" 2. Check which method better preserves handwriting")
|
||||||
|
print(" 3. Adjust thresholds if needed")
|
||||||
|
print(" 4. Choose the best method for production pipeline")
|
||||||
Reference in New Issue
Block a user