Compare commits
4 Commits
9079f7a8a9
...
5571076406
| Author | SHA1 | Date | |
|---|---|---|---|
| 5571076406 | |||
| 8777e27cbb | |||
| 43785db595 | |||
| baea210109 |
277
ARCHITECTURE_ANALYSIS.md
Normal file
277
ARCHITECTURE_ANALYSIS.md
Normal file
@@ -0,0 +1,277 @@
|
|||||||
|
# novelty-seeking 系統流程與耦合度分析
|
||||||
|
|
||||||
|
> 生成日期: 2025-12-04
|
||||||
|
|
||||||
|
## 一、系統整體架構概覽
|
||||||
|
|
||||||
|
novelty-seeking 是一個創新思維引導系統，由三個核心 Agent 組成：
|
||||||
|
- **Attribute Agent**：從查詢到屬性節點的映射
|
||||||
|
- **Transformation Agent**：屬性到新關鍵字的轉換
|
||||||
|
- **Expert Transformation Agent**：多視角專家角度的屬性轉換
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、完整資料流程
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ Attribute Agent │
|
||||||
|
├─────────────────────────────────────────────────────────────────────┤
|
||||||
|
│ 用戶輸入 Query (如「腳踏車」) │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 0: 類別分析 (category_mode 決定) │
|
||||||
|
│ → 產出: CategoryDefinition[] (如 材料/功能/用途/使用族群) │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 1: 屬性列表生成 │
|
||||||
|
│ → 產出: {材料: [鋼,木,碳纖維], 功能: [搬運,儲存], ...} │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 2: 關係生成 (DAG 邊) │
|
||||||
|
│ → 產出: AttributeDAG (nodes + edges) │
|
||||||
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
|
↓ (高耦合)
|
||||||
|
┌─────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ Expert Transformation Agent │
|
||||||
|
├─────────────────────────────────────────────────────────────────────┤
|
||||||
|
│ 輸入: Query + Category + Attributes (來自 Attribute Agent) │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 0: 專家團隊生成 │
|
||||||
|
│ → expert_source 決定: llm / curated / dbpedia / wikidata │
|
||||||
|
│ → 產出: ExpertProfile[] (如 會計師/心理師/生態學家) │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 1: 專家視角關鍵字生成 (對每個 attribute) │
|
||||||
|
│ → 產出: ExpertKeyword[] (關鍵字 + 來源專家 + 來源屬性) │
|
||||||
|
│ ↓ │
|
||||||
|
│ Step 2: 描述生成 (對每個 keyword) │
|
||||||
|
│ → 產出: ExpertTransformationDescription[] │
|
||||||
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、Attribute Agent 詳細流程
|
||||||
|
|
||||||
|
### 3.1 流程架構
|
||||||
|
|
||||||
|
```
|
||||||
|
用戶查詢 (Query)
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 0: 類別分析 (Category Mode 決定) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 輸入: query, suggested_category_count │
|
||||||
|
│ 處理: │
|
||||||
|
│ - FIXED_ONLY: 使用 4 個固定類別 │
|
||||||
|
│ - FIXED_PLUS_CUSTOM: 固定 + 用戶自訂 │
|
||||||
|
│ - FIXED_PLUS_DYNAMIC: 固定 + LLM 推薦 │
|
||||||
|
│ - CUSTOM_ONLY: 僅 LLM 推薦 │
|
||||||
|
│ - DYNAMIC_AUTO: 純 LLM 推薦 (預設) │
|
||||||
|
│ 輸出: Step0Result (recommended categories) │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 1: 屬性列表生成 (Attributes) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 輸入: query, final_categories │
|
||||||
|
│ LLM 處理: │
|
||||||
|
│ - 分析 query 在各類別下的屬性 │
|
||||||
|
│ - 每個類別生成 3-5 個屬性 │
|
||||||
|
│ 輸出: DynamicStep1Result │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 2: 關係映射 (Relationships → DAG) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 輸入: query, categories, attributes │
|
||||||
|
│ LLM 處理: │
|
||||||
|
│ - 分析相鄰類別之間的因果關係 │
|
||||||
|
│ - 生成 (source, target) 關係對 │
|
||||||
|
│ 輸出: AttributeDAG │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 關鍵輸入變數
|
||||||
|
|
||||||
|
| 變數 | 來源 | 影響範圍 | 作用 |
|
||||||
|
|------|------|--------|------|
|
||||||
|
| `query` | 用戶輸入 | Step 0-2 全部 | 決定分析的物件 |
|
||||||
|
| `category_mode` | 用戶選擇 | Step 0 | 決定使用哪些類別 |
|
||||||
|
| `suggested_category_count` | 用戶設定 | Step 0 | LLM 推薦類別的數量 |
|
||||||
|
| `temperature` | 用戶設定 | Step 0-2 | 控制 LLM 輸出的多樣性 |
|
||||||
|
| `model` | 用戶選擇 | Step 0-2 | 選擇不同的 LLM 模型 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、Expert Transformation Agent 詳細流程
|
||||||
|
|
||||||
|
### 4.1 流程架構
|
||||||
|
|
||||||
|
```
|
||||||
|
屬性列表 (attributes from Attribute Agent)
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 0: 專家團隊生成 (Expert Generation) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 決定因素: │
|
||||||
|
│ - expert_source = 'llm' → LLM 生成 │
|
||||||
|
│ - expert_source ∈ ['curated', 'dbpedia', │
|
||||||
|
│ 'wikidata'] → 本地檔案隨機選取 │
|
||||||
|
│ - 有 custom_experts → 結合 LLM │
|
||||||
|
│ │
|
||||||
|
│ 輸出: ExpertProfile[] │
|
||||||
|
│ [{id, name, domain, perspective}] │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 1: 專家視角關鍵字生成 (Keywords) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 迴圈: for each attribute in attributes: │
|
||||||
|
│ LLM 為每個專家生成 keywords_per_expert │
|
||||||
|
│ 個關鍵字 │
|
||||||
|
│ │
|
||||||
|
│ 輸出: ExpertKeyword[] │
|
||||||
|
│ [{keyword, expert_id, expert_name, │
|
||||||
|
│ source_attribute}] │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Step 2: 描述生成 (Descriptions) │
|
||||||
|
├─────────────────────────────────────────────┤
|
||||||
|
│ 迴圈: for each expert_keyword: │
|
||||||
|
│ LLM 生成 15-30 字的創新應用描述 │
|
||||||
|
│ │
|
||||||
|
│ 輸出: ExpertTransformationDescription[] │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 關鍵輸入變數
|
||||||
|
|
||||||
|
| 變數 | 來源 | 影響範圍 | 作用 |
|
||||||
|
|------|------|--------|------|
|
||||||
|
| `expert_source` | 用戶選擇 | Step 0 | 決定專家來源 (llm/curated/dbpedia/wikidata) |
|
||||||
|
| `expert_count` | 用戶設定 | Step 0 | 專家數量 (2-8) |
|
||||||
|
| `keywords_per_expert` | 用戶設定 | Step 1 | 每專家每屬性關鍵字數 (1-3) |
|
||||||
|
| `custom_experts` | 用戶輸入 | Step 0 | 用戶指定的專家名稱 |
|
||||||
|
| `temperature` | 用戶設定 | Step 0-2 | 控制多樣性 |
|
||||||
|
|
||||||
|
### 4.3 關鍵字生成公式
|
||||||
|
|
||||||
|
```
|
||||||
|
總關鍵字數 = len(attributes) × expert_count × keywords_per_expert
|
||||||
|
|
||||||
|
範例計算:
|
||||||
|
├─ 3 個屬性 (搬運, 儲存, 展示)
|
||||||
|
├─ 3 位專家 (會計師, 心理師, 生態學家)
|
||||||
|
├─ 1 個關鍵字/專家
|
||||||
|
└─ = 3 × 3 × 1 = 9 個關鍵字
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 五、關鍵字生成影響因素
|
||||||
|
|
||||||
|
| 階段 | 影響變數 | 影響程度 | 說明 |
|
||||||
|
|------|---------|---------|------|
|
||||||
|
| **屬性生成** | `query` | 極高 | 決定 LLM 分析的語義基礎 |
|
||||||
|
| | `category_mode` | 高 | 決定類別維度 |
|
||||||
|
| | `temperature` | 中 | 越高越多樣 |
|
||||||
|
| | `model` | 中 | 不同模型知識基礎不同 |
|
||||||
|
| **專家生成** | `expert_source` | 高 | 決定專家來源與品質 |
|
||||||
|
| | `expert_count` | 中 | 2-8 位專家 |
|
||||||
|
| | `custom_experts` | 中 | 與 LLM 結合 |
|
||||||
|
| **關鍵字生成** | `experts[].domain` | 極高 | 直接決定關鍵字視角 |
|
||||||
|
| | `keywords_per_expert` | 低 | 控制數量 |
|
||||||
|
| | `source_attribute` | 高 | 決定思考起點 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 六、耦合度分析
|
||||||
|
|
||||||
|
### 6.1 高耦合連接 ⚠️
|
||||||
|
|
||||||
|
| 連接 | 耦合度 | 原因 | 風險 |
|
||||||
|
|------|--------|------|------|
|
||||||
|
| Attribute → Expert Transform | 高 | Expert 依賴 Attribute 輸出 | 結構變更需同步修改 |
|
||||||
|
| Expert 生成 → Keyword 生成 | 高 | domain 直接影響關鍵字 | domain 品質差→關鍵字無關 |
|
||||||
|
| Prompt → LLM 輸出結構 | 高 | prompt 定義 JSON 格式 | 改 prompt 需改 schema |
|
||||||
|
|
||||||
|
### 6.2 低耦合連接 ✓
|
||||||
|
|
||||||
|
| 連接 | 耦合度 | 原因 | 優點 |
|
||||||
|
|------|--------|------|------|
|
||||||
|
| curated/dbpedia/wikidata | 低 | 獨立本地檔案 | 可單獨更新 |
|
||||||
|
| SSE 通信格式 | 低 | 標準化解耦 | 向後相容 |
|
||||||
|
| useAttribute/useExpertTransformation | 低 | 獨立 hook | 可單獨複用 |
|
||||||
|
|
||||||
|
### 6.3 耦合度矩陣
|
||||||
|
|
||||||
|
| | Attribute | Transformation | Expert Transform |
|
||||||
|
|----|-----------|---------------|----|
|
||||||
|
| **Attribute** | - | 低 | 高 |
|
||||||
|
| **Transformation** | 低 | - | 低 |
|
||||||
|
| **Expert Transform** | 高 | 低 | - |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 七、專家來源比較
|
||||||
|
|
||||||
|
| 來源 | 檔案 | 筆數 | Domain 品質 | 特點 |
|
||||||
|
|------|------|------|------------|------|
|
||||||
|
| `llm` | - | 動態 | 高 | LLM 根據 query 生成相關專家 |
|
||||||
|
| `curated` | curated_occupations_zh/en.json | 210 | 高 | 精選職業，含具體領域 |
|
||||||
|
| `dbpedia` | dbpedia_occupations_en.json | 2164 | 低 | 全是 "Professional Field" |
|
||||||
|
| `wikidata` | - | - | - | 未實作本地化 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 八、決策變化追蹤範例
|
||||||
|
|
||||||
|
```
|
||||||
|
Query: "腳踏車"
|
||||||
|
↓
|
||||||
|
Category Mode: DYNAMIC_AUTO
|
||||||
|
→ LLM 建議 [材料, 功能, 用途, 使用族群]
|
||||||
|
↓
|
||||||
|
Expert Source: "curated"
|
||||||
|
→ 隨機選取 [外科醫師(醫療與健康), 軟體工程師(資訊科技), 主廚(餐飲與服務)]
|
||||||
|
↓
|
||||||
|
Attribute "搬運" + Expert "外科醫師"
|
||||||
|
→ LLM 思考: 醫療視角看搬運
|
||||||
|
→ Keyword: "器官運輸", "急救物流"
|
||||||
|
↓
|
||||||
|
Description 生成:
|
||||||
|
→ "從急救醫療角度，腳踏車可改良為緊急醫療運輸工具..."
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 九、改進建議
|
||||||
|
|
||||||
|
| 問題 | 現狀 | 建議改進 |
|
||||||
|
|------|------|---------|
|
||||||
|
| domain 品質 | DBpedia 全是通用值 | ✅ 已建立精選職業 |
|
||||||
|
| 重複計算 Expert | 每類別重新生成 | 考慮 Expert 全局化 |
|
||||||
|
| Temperature 統一 | 整流程同一值 | 可按 Step 分開設定 |
|
||||||
|
| 缺乏快取 | 每次重新分析 | 加入 Attribute 快取層 |
|
||||||
|
| 語言支援 | 主要中文 | ✅ 已建立英文版 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 十、關鍵檔案清單
|
||||||
|
|
||||||
|
### Backend
|
||||||
|
- `app/routers/analyze.py` - Attribute Agent 路由
|
||||||
|
- `app/routers/expert_transformation.py` - Expert Transformation 路由
|
||||||
|
- `app/prompts/step_prompts.py` - Attribute Agent 提示詞
|
||||||
|
- `app/prompts/expert_transformation_prompt.py` - Expert Transformation 提示詞
|
||||||
|
- `app/services/expert_source_service.py` - 專家來源服務
|
||||||
|
- `app/services/llm_service.py` - LLM 調用服務
|
||||||
|
- `app/data/curated_occupations_zh.json` - 精選職業（中文）
|
||||||
|
- `app/data/curated_occupations_en.json` - 精選職業（英文）
|
||||||
|
- `app/data/dbpedia_occupations_en.json` - DBpedia 職業
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
- `src/App.tsx` - 主狀態管理
|
||||||
|
- `src/hooks/useAttribute.ts` - Attribute Agent Hook
|
||||||
|
- `src/hooks/useExpertTransformation.ts` - Expert Transformation Hook
|
||||||
|
- `src/components/TransformationInputPanel.tsx` - 轉換控制面板
|
||||||
|
- `src/types/index.ts` - 類型定義
|
||||||
9
backend/app/data/conceptnet_occupations_en.json
Normal file
9
backend/app/data/conceptnet_occupations_en.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"source": "conceptnet",
|
||||||
|
"language": "en",
|
||||||
|
"fetched_at": "2025-12-04T07:26:30.695936+00:00",
|
||||||
|
"total_count": 0
|
||||||
|
},
|
||||||
|
"occupations": []
|
||||||
|
}
|
||||||
9
backend/app/data/conceptnet_occupations_zh.json
Normal file
9
backend/app/data/conceptnet_occupations_zh.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"source": "conceptnet",
|
||||||
|
"language": "zh",
|
||||||
|
"fetched_at": "2025-12-04T07:26:26.994914+00:00",
|
||||||
|
"total_count": 0
|
||||||
|
},
|
||||||
|
"occupations": []
|
||||||
|
}
|
||||||
216
backend/app/data/curated_occupations_en.json
Normal file
216
backend/app/data/curated_occupations_en.json
Normal file
@@ -0,0 +1,216 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"source": "curated",
|
||||||
|
"language": "en",
|
||||||
|
"created_at": "2025-12-04",
|
||||||
|
"total_count": 210,
|
||||||
|
"description": "Curated common professional occupations with specific domains"
|
||||||
|
},
|
||||||
|
"occupations": [
|
||||||
|
{"name": "Surgeon", "domain": "Healthcare"},
|
||||||
|
{"name": "Internist", "domain": "Healthcare"},
|
||||||
|
{"name": "Dentist", "domain": "Healthcare"},
|
||||||
|
{"name": "Ophthalmologist", "domain": "Healthcare"},
|
||||||
|
{"name": "Psychiatrist", "domain": "Healthcare"},
|
||||||
|
{"name": "Pediatrician", "domain": "Healthcare"},
|
||||||
|
{"name": "Nurse", "domain": "Healthcare"},
|
||||||
|
{"name": "Pharmacist", "domain": "Healthcare"},
|
||||||
|
{"name": "Clinical Psychologist", "domain": "Healthcare"},
|
||||||
|
{"name": "Physical Therapist", "domain": "Healthcare"},
|
||||||
|
{"name": "Occupational Therapist", "domain": "Healthcare"},
|
||||||
|
{"name": "Nutritionist", "domain": "Healthcare"},
|
||||||
|
{"name": "Traditional Chinese Medicine Doctor", "domain": "Healthcare"},
|
||||||
|
{"name": "Veterinarian", "domain": "Healthcare"},
|
||||||
|
|
||||||
|
{"name": "Software Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "Frontend Developer", "domain": "Information Technology"},
|
||||||
|
{"name": "Backend Developer", "domain": "Information Technology"},
|
||||||
|
{"name": "Data Scientist", "domain": "Information Technology"},
|
||||||
|
{"name": "Data Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "Machine Learning Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "Cybersecurity Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "DevOps Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "UI Designer", "domain": "Information Technology"},
|
||||||
|
{"name": "UX Designer", "domain": "Information Technology"},
|
||||||
|
{"name": "Product Manager", "domain": "Information Technology"},
|
||||||
|
{"name": "Systems Analyst", "domain": "Information Technology"},
|
||||||
|
{"name": "Network Engineer", "domain": "Information Technology"},
|
||||||
|
{"name": "Cloud Architect", "domain": "Information Technology"},
|
||||||
|
|
||||||
|
{"name": "Accountant", "domain": "Finance & Business"},
|
||||||
|
{"name": "Financial Analyst", "domain": "Finance & Business"},
|
||||||
|
{"name": "Investment Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Risk Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Actuary", "domain": "Finance & Business"},
|
||||||
|
{"name": "Bank Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Securities Analyst", "domain": "Finance & Business"},
|
||||||
|
{"name": "Tax Consultant", "domain": "Finance & Business"},
|
||||||
|
{"name": "Business Consultant", "domain": "Finance & Business"},
|
||||||
|
{"name": "HR Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Marketing Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Sales Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Procurement Manager", "domain": "Finance & Business"},
|
||||||
|
{"name": "Entrepreneur", "domain": "Finance & Business"},
|
||||||
|
|
||||||
|
{"name": "Lawyer", "domain": "Law & Policy"},
|
||||||
|
{"name": "Judge", "domain": "Law & Policy"},
|
||||||
|
{"name": "Prosecutor", "domain": "Law & Policy"},
|
||||||
|
{"name": "Notary", "domain": "Law & Policy"},
|
||||||
|
{"name": "Legal Counsel", "domain": "Law & Policy"},
|
||||||
|
{"name": "IP Attorney", "domain": "Law & Policy"},
|
||||||
|
{"name": "Policy Analyst", "domain": "Law & Policy"},
|
||||||
|
{"name": "Diplomat", "domain": "Law & Policy"},
|
||||||
|
{"name": "Civil Servant", "domain": "Law & Policy"},
|
||||||
|
{"name": "Legislator", "domain": "Law & Policy"},
|
||||||
|
{"name": "Mediator", "domain": "Law & Policy"},
|
||||||
|
{"name": "Legal Scholar", "domain": "Law & Policy"},
|
||||||
|
|
||||||
|
{"name": "University Professor", "domain": "Education & Academia"},
|
||||||
|
{"name": "High School Teacher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Middle School Teacher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Elementary School Teacher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Preschool Teacher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Special Education Teacher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Tutor", "domain": "Education & Academia"},
|
||||||
|
{"name": "Researcher", "domain": "Education & Academia"},
|
||||||
|
{"name": "Librarian", "domain": "Education & Academia"},
|
||||||
|
{"name": "Education Administrator", "domain": "Education & Academia"},
|
||||||
|
{"name": "Academic Editor", "domain": "Education & Academia"},
|
||||||
|
{"name": "Education Consultant", "domain": "Education & Academia"},
|
||||||
|
{"name": "Speech Therapist", "domain": "Education & Academia"},
|
||||||
|
|
||||||
|
{"name": "Painter", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Sculptor", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Musician", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Composer", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Conductor", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Dancer", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Actor", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Film Director", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Screenwriter", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Photographer", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Illustrator", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Animator", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Graphic Designer", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Fashion Designer", "domain": "Arts & Creativity"},
|
||||||
|
{"name": "Jewelry Designer", "domain": "Arts & Creativity"},
|
||||||
|
|
||||||
|
{"name": "Mechanical Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Electrical Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Electronics Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Chemical Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Materials Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Industrial Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Automation Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Quality Control Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Process Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "R&D Engineer", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Production Manager", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Factory Manager", "domain": "Engineering & Manufacturing"},
|
||||||
|
{"name": "Technician", "domain": "Engineering & Manufacturing"},
|
||||||
|
|
||||||
|
{"name": "Architect", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Interior Designer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Landscape Designer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Urban Planner", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Structural Engineer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Civil Engineer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Construction Engineer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Site Supervisor", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Surveyor", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Architectural Drafter", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Exhibition Designer", "domain": "Architecture & Space"},
|
||||||
|
{"name": "Lighting Designer", "domain": "Architecture & Space"},
|
||||||
|
|
||||||
|
{"name": "Journalist", "domain": "Media & Communications"},
|
||||||
|
{"name": "News Anchor", "domain": "Media & Communications"},
|
||||||
|
{"name": "Editor", "domain": "Media & Communications"},
|
||||||
|
{"name": "Copy Editor", "domain": "Media & Communications"},
|
||||||
|
{"name": "Video Editor", "domain": "Media & Communications"},
|
||||||
|
{"name": "PR Specialist", "domain": "Media & Communications"},
|
||||||
|
{"name": "Advertising Planner", "domain": "Media & Communications"},
|
||||||
|
{"name": "Social Media Manager", "domain": "Media & Communications"},
|
||||||
|
{"name": "Content Creator", "domain": "Media & Communications"},
|
||||||
|
{"name": "Podcast Host", "domain": "Media & Communications"},
|
||||||
|
{"name": "Publisher", "domain": "Media & Communications"},
|
||||||
|
{"name": "Translator", "domain": "Media & Communications"},
|
||||||
|
{"name": "Interpreter", "domain": "Media & Communications"},
|
||||||
|
|
||||||
|
{"name": "Agronomist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Horticulturist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Livestock Specialist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Aquaculture Specialist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Environmental Engineer", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Ecologist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Forest Ranger", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Meteorologist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Geologist", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Environmental Inspector", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Sustainability Consultant", "domain": "Agriculture & Environment"},
|
||||||
|
{"name": "Organic Farmer", "domain": "Agriculture & Environment"},
|
||||||
|
|
||||||
|
{"name": "Executive Chef", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Pastry Chef", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Bartender", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Sommelier", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Restaurant Manager", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Hotel Manager", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Travel Planner", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Tour Guide", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Barista", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Food Critic", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Wedding Planner", "domain": "Hospitality & Service"},
|
||||||
|
{"name": "Event Planner", "domain": "Hospitality & Service"},
|
||||||
|
|
||||||
|
{"name": "Sports Coach", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Personal Trainer", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Yoga Instructor", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Athletic Trainer", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Physical Education Teacher", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Sports Psychologist", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Sports Nutritionist", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Professional Athlete", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Referee", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Strength Coach", "domain": "Sports & Fitness"},
|
||||||
|
{"name": "Sports Agent", "domain": "Sports & Fitness"},
|
||||||
|
|
||||||
|
{"name": "Social Worker", "domain": "Social Services"},
|
||||||
|
{"name": "Counselor", "domain": "Social Services"},
|
||||||
|
{"name": "Guidance Counselor", "domain": "Social Services"},
|
||||||
|
{"name": "Volunteer Coordinator", "domain": "Social Services"},
|
||||||
|
{"name": "Nonprofit Manager", "domain": "Social Services"},
|
||||||
|
{"name": "Community Organizer", "domain": "Social Services"},
|
||||||
|
{"name": "Elderly Care Worker", "domain": "Social Services"},
|
||||||
|
{"name": "Youth Counselor", "domain": "Social Services"},
|
||||||
|
{"name": "Family Therapist", "domain": "Social Services"},
|
||||||
|
{"name": "Career Counselor", "domain": "Social Services"},
|
||||||
|
{"name": "Addiction Counselor", "domain": "Social Services"},
|
||||||
|
|
||||||
|
{"name": "Pilot", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Ship Captain", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Train Operator", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Air Traffic Controller", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Logistics Manager", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Supply Chain Manager", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Warehouse Manager", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Customs Broker", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Traffic Engineer", "domain": "Transportation & Logistics"},
|
||||||
|
{"name": "Port Authority Officer", "domain": "Transportation & Logistics"},
|
||||||
|
|
||||||
|
{"name": "Physicist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Chemist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Biologist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Astronomer", "domain": "Scientific Research"},
|
||||||
|
{"name": "Mathematician", "domain": "Scientific Research"},
|
||||||
|
{"name": "Statistician", "domain": "Scientific Research"},
|
||||||
|
{"name": "Geneticist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Neuroscientist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Oceanographer", "domain": "Scientific Research"},
|
||||||
|
{"name": "Archaeologist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Anthropologist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Sociologist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Economist", "domain": "Scientific Research"},
|
||||||
|
{"name": "Historian", "domain": "Scientific Research"},
|
||||||
|
{"name": "Philosopher", "domain": "Scientific Research"}
|
||||||
|
]
|
||||||
|
}
|
||||||
216
backend/app/data/curated_occupations_zh.json
Normal file
216
backend/app/data/curated_occupations_zh.json
Normal file
@@ -0,0 +1,216 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"source": "curated",
|
||||||
|
"language": "zh",
|
||||||
|
"created_at": "2025-12-04",
|
||||||
|
"total_count": 210,
|
||||||
|
"description": "精選常見專家職業，含具體專業領域"
|
||||||
|
},
|
||||||
|
"occupations": [
|
||||||
|
{"name": "外科醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "內科醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "牙醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "眼科醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "精神科醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "小兒科醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "護理師", "domain": "醫療與健康"},
|
||||||
|
{"name": "藥師", "domain": "醫療與健康"},
|
||||||
|
{"name": "臨床心理師", "domain": "醫療與健康"},
|
||||||
|
{"name": "物理治療師", "domain": "醫療與健康"},
|
||||||
|
{"name": "職能治療師", "domain": "醫療與健康"},
|
||||||
|
{"name": "營養師", "domain": "醫療與健康"},
|
||||||
|
{"name": "中醫師", "domain": "醫療與健康"},
|
||||||
|
{"name": "獸醫師", "domain": "醫療與健康"},
|
||||||
|
|
||||||
|
{"name": "軟體工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "前端工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "後端工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "資料科學家", "domain": "資訊科技"},
|
||||||
|
{"name": "資料工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "機器學習工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "資安工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "DevOps工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "UI設計師", "domain": "資訊科技"},
|
||||||
|
{"name": "UX設計師", "domain": "資訊科技"},
|
||||||
|
{"name": "產品經理", "domain": "資訊科技"},
|
||||||
|
{"name": "系統分析師", "domain": "資訊科技"},
|
||||||
|
{"name": "網路工程師", "domain": "資訊科技"},
|
||||||
|
{"name": "雲端架構師", "domain": "資訊科技"},
|
||||||
|
|
||||||
|
{"name": "會計師", "domain": "金融與商業"},
|
||||||
|
{"name": "財務分析師", "domain": "金融與商業"},
|
||||||
|
{"name": "投資經理", "domain": "金融與商業"},
|
||||||
|
{"name": "風險管理師", "domain": "金融與商業"},
|
||||||
|
{"name": "精算師", "domain": "金融與商業"},
|
||||||
|
{"name": "銀行經理", "domain": "金融與商業"},
|
||||||
|
{"name": "證券分析師", "domain": "金融與商業"},
|
||||||
|
{"name": "稅務顧問", "domain": "金融與商業"},
|
||||||
|
{"name": "企業顧問", "domain": "金融與商業"},
|
||||||
|
{"name": "人資經理", "domain": "金融與商業"},
|
||||||
|
{"name": "行銷經理", "domain": "金融與商業"},
|
||||||
|
{"name": "業務經理", "domain": "金融與商業"},
|
||||||
|
{"name": "採購經理", "domain": "金融與商業"},
|
||||||
|
{"name": "創業家", "domain": "金融與商業"},
|
||||||
|
|
||||||
|
{"name": "律師", "domain": "法律與政策"},
|
||||||
|
{"name": "法官", "domain": "法律與政策"},
|
||||||
|
{"name": "檢察官", "domain": "法律與政策"},
|
||||||
|
{"name": "公證人", "domain": "法律與政策"},
|
||||||
|
{"name": "法務專員", "domain": "法律與政策"},
|
||||||
|
{"name": "智財律師", "domain": "法律與政策"},
|
||||||
|
{"name": "政策分析師", "domain": "法律與政策"},
|
||||||
|
{"name": "外交官", "domain": "法律與政策"},
|
||||||
|
{"name": "公務員", "domain": "法律與政策"},
|
||||||
|
{"name": "立法委員", "domain": "法律與政策"},
|
||||||
|
{"name": "調解員", "domain": "法律與政策"},
|
||||||
|
{"name": "法律學者", "domain": "法律與政策"},
|
||||||
|
|
||||||
|
{"name": "大學教授", "domain": "教育與學術"},
|
||||||
|
{"name": "高中教師", "domain": "教育與學術"},
|
||||||
|
{"name": "國中教師", "domain": "教育與學術"},
|
||||||
|
{"name": "小學教師", "domain": "教育與學術"},
|
||||||
|
{"name": "幼教老師", "domain": "教育與學術"},
|
||||||
|
{"name": "特教老師", "domain": "教育與學術"},
|
||||||
|
{"name": "補習班老師", "domain": "教育與學術"},
|
||||||
|
{"name": "研究員", "domain": "教育與學術"},
|
||||||
|
{"name": "圖書館員", "domain": "教育與學術"},
|
||||||
|
{"name": "教育行政人員", "domain": "教育與學術"},
|
||||||
|
{"name": "學術編輯", "domain": "教育與學術"},
|
||||||
|
{"name": "教育顧問", "domain": "教育與學術"},
|
||||||
|
{"name": "語言治療師", "domain": "教育與學術"},
|
||||||
|
|
||||||
|
{"name": "畫家", "domain": "藝術與創意"},
|
||||||
|
{"name": "雕塑家", "domain": "藝術與創意"},
|
||||||
|
{"name": "音樂家", "domain": "藝術與創意"},
|
||||||
|
{"name": "作曲家", "domain": "藝術與創意"},
|
||||||
|
{"name": "指揮家", "domain": "藝術與創意"},
|
||||||
|
{"name": "舞蹈家", "domain": "藝術與創意"},
|
||||||
|
{"name": "演員", "domain": "藝術與創意"},
|
||||||
|
{"name": "導演", "domain": "藝術與創意"},
|
||||||
|
{"name": "編劇", "domain": "藝術與創意"},
|
||||||
|
{"name": "攝影師", "domain": "藝術與創意"},
|
||||||
|
{"name": "插畫家", "domain": "藝術與創意"},
|
||||||
|
{"name": "動畫師", "domain": "藝術與創意"},
|
||||||
|
{"name": "平面設計師", "domain": "藝術與創意"},
|
||||||
|
{"name": "時尚設計師", "domain": "藝術與創意"},
|
||||||
|
{"name": "珠寶設計師", "domain": "藝術與創意"},
|
||||||
|
|
||||||
|
{"name": "機械工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "電機工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "電子工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "化學工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "材料工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "工業工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "自動化工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "品管工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "製程工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "研發工程師", "domain": "工程與製造"},
|
||||||
|
{"name": "生產經理", "domain": "工程與製造"},
|
||||||
|
{"name": "工廠廠長", "domain": "工程與製造"},
|
||||||
|
{"name": "技師", "domain": "工程與製造"},
|
||||||
|
|
||||||
|
{"name": "建築師", "domain": "建築與空間"},
|
||||||
|
{"name": "室內設計師", "domain": "建築與空間"},
|
||||||
|
{"name": "景觀設計師", "domain": "建築與空間"},
|
||||||
|
{"name": "都市規劃師", "domain": "建築與空間"},
|
||||||
|
{"name": "結構工程師", "domain": "建築與空間"},
|
||||||
|
{"name": "土木工程師", "domain": "建築與空間"},
|
||||||
|
{"name": "營造工程師", "domain": "建築與空間"},
|
||||||
|
{"name": "工地主任", "domain": "建築與空間"},
|
||||||
|
{"name": "測量師", "domain": "建築與空間"},
|
||||||
|
{"name": "建築繪圖員", "domain": "建築與空間"},
|
||||||
|
{"name": "展場設計師", "domain": "建築與空間"},
|
||||||
|
{"name": "燈光設計師", "domain": "建築與空間"},
|
||||||
|
|
||||||
|
{"name": "記者", "domain": "媒體與傳播"},
|
||||||
|
{"name": "主播", "domain": "媒體與傳播"},
|
||||||
|
{"name": "編輯", "domain": "媒體與傳播"},
|
||||||
|
{"name": "文字編輯", "domain": "媒體與傳播"},
|
||||||
|
{"name": "影片剪輯師", "domain": "媒體與傳播"},
|
||||||
|
{"name": "公關專員", "domain": "媒體與傳播"},
|
||||||
|
{"name": "廣告企劃", "domain": "媒體與傳播"},
|
||||||
|
{"name": "社群經理", "domain": "媒體與傳播"},
|
||||||
|
{"name": "內容創作者", "domain": "媒體與傳播"},
|
||||||
|
{"name": "播客主持人", "domain": "媒體與傳播"},
|
||||||
|
{"name": "出版人", "domain": "媒體與傳播"},
|
||||||
|
{"name": "翻譯師", "domain": "媒體與傳播"},
|
||||||
|
{"name": "口譯員", "domain": "媒體與傳播"},
|
||||||
|
|
||||||
|
{"name": "農藝師", "domain": "農業與環境"},
|
||||||
|
{"name": "園藝師", "domain": "農業與環境"},
|
||||||
|
{"name": "畜牧專家", "domain": "農業與環境"},
|
||||||
|
{"name": "水產養殖師", "domain": "農業與環境"},
|
||||||
|
{"name": "環境工程師", "domain": "農業與環境"},
|
||||||
|
{"name": "生態學家", "domain": "農業與環境"},
|
||||||
|
{"name": "森林保育員", "domain": "農業與環境"},
|
||||||
|
{"name": "氣象學家", "domain": "農業與環境"},
|
||||||
|
{"name": "地質學家", "domain": "農業與環境"},
|
||||||
|
{"name": "環保稽查員", "domain": "農業與環境"},
|
||||||
|
{"name": "永續發展顧問", "domain": "農業與環境"},
|
||||||
|
{"name": "有機農場主", "domain": "農業與環境"},
|
||||||
|
|
||||||
|
{"name": "主廚", "domain": "餐飲與服務"},
|
||||||
|
{"name": "西點師傅", "domain": "餐飲與服務"},
|
||||||
|
{"name": "調酒師", "domain": "餐飲與服務"},
|
||||||
|
{"name": "侍酒師", "domain": "餐飲與服務"},
|
||||||
|
{"name": "餐廳經理", "domain": "餐飲與服務"},
|
||||||
|
{"name": "飯店經理", "domain": "餐飲與服務"},
|
||||||
|
{"name": "旅遊規劃師", "domain": "餐飲與服務"},
|
||||||
|
{"name": "導遊", "domain": "餐飲與服務"},
|
||||||
|
{"name": "咖啡師", "domain": "餐飲與服務"},
|
||||||
|
{"name": "美食評論家", "domain": "餐飲與服務"},
|
||||||
|
{"name": "婚禮策劃師", "domain": "餐飲與服務"},
|
||||||
|
{"name": "活動企劃", "domain": "餐飲與服務"},
|
||||||
|
|
||||||
|
{"name": "運動教練", "domain": "運動與健身"},
|
||||||
|
{"name": "健身教練", "domain": "運動與健身"},
|
||||||
|
{"name": "瑜珈老師", "domain": "運動與健身"},
|
||||||
|
{"name": "運動防護員", "domain": "運動與健身"},
|
||||||
|
{"name": "體育老師", "domain": "運動與健身"},
|
||||||
|
{"name": "運動心理師", "domain": "運動與健身"},
|
||||||
|
{"name": "運動營養師", "domain": "運動與健身"},
|
||||||
|
{"name": "職業運動員", "domain": "運動與健身"},
|
||||||
|
{"name": "裁判", "domain": "運動與健身"},
|
||||||
|
{"name": "體能訓練師", "domain": "運動與健身"},
|
||||||
|
{"name": "運動經紀人", "domain": "運動與健身"},
|
||||||
|
|
||||||
|
{"name": "社工師", "domain": "社會服務"},
|
||||||
|
{"name": "心理諮商師", "domain": "社會服務"},
|
||||||
|
{"name": "輔導員", "domain": "社會服務"},
|
||||||
|
{"name": "志工協調員", "domain": "社會服務"},
|
||||||
|
{"name": "非營利組織經理", "domain": "社會服務"},
|
||||||
|
{"name": "社區營造員", "domain": "社會服務"},
|
||||||
|
{"name": "長照服務員", "domain": "社會服務"},
|
||||||
|
{"name": "青少年輔導員", "domain": "社會服務"},
|
||||||
|
{"name": "家庭治療師", "domain": "社會服務"},
|
||||||
|
{"name": "職涯諮詢師", "domain": "社會服務"},
|
||||||
|
{"name": "戒癮輔導員", "domain": "社會服務"},
|
||||||
|
|
||||||
|
{"name": "飛行員", "domain": "交通與物流"},
|
||||||
|
{"name": "船長", "domain": "交通與物流"},
|
||||||
|
{"name": "火車駕駛", "domain": "交通與物流"},
|
||||||
|
{"name": "航空管制員", "domain": "交通與物流"},
|
||||||
|
{"name": "物流經理", "domain": "交通與物流"},
|
||||||
|
{"name": "供應鏈經理", "domain": "交通與物流"},
|
||||||
|
{"name": "倉儲經理", "domain": "交通與物流"},
|
||||||
|
{"name": "報關員", "domain": "交通與物流"},
|
||||||
|
{"name": "交通工程師", "domain": "交通與物流"},
|
||||||
|
{"name": "港務人員", "domain": "交通與物流"},
|
||||||
|
|
||||||
|
{"name": "物理學家", "domain": "科學研究"},
|
||||||
|
{"name": "化學家", "domain": "科學研究"},
|
||||||
|
{"name": "生物學家", "domain": "科學研究"},
|
||||||
|
{"name": "天文學家", "domain": "科學研究"},
|
||||||
|
{"name": "數學家", "domain": "科學研究"},
|
||||||
|
{"name": "統計學家", "domain": "科學研究"},
|
||||||
|
{"name": "基因學家", "domain": "科學研究"},
|
||||||
|
{"name": "神經科學家", "domain": "科學研究"},
|
||||||
|
{"name": "海洋學家", "domain": "科學研究"},
|
||||||
|
{"name": "考古學家", "domain": "科學研究"},
|
||||||
|
{"name": "人類學家", "domain": "科學研究"},
|
||||||
|
{"name": "社會學家", "domain": "科學研究"},
|
||||||
|
{"name": "經濟學家", "domain": "科學研究"},
|
||||||
|
{"name": "歷史學家", "domain": "科學研究"},
|
||||||
|
{"name": "哲學家", "domain": "科學研究"}
|
||||||
|
]
|
||||||
|
}
|
||||||
8666
backend/app/data/dbpedia_occupations_en.json
Normal file
8666
backend/app/data/dbpedia_occupations_en.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -206,6 +206,14 @@ class ExpertTransformationDAGResult(BaseModel):
|
|||||||
results: List[ExpertTransformationCategoryResult]
|
results: List[ExpertTransformationCategoryResult]
|
||||||
|
|
||||||
|
|
||||||
|
class ExpertSource(str, Enum):
|
||||||
|
"""專家來源類型"""
|
||||||
|
LLM = "llm"
|
||||||
|
CURATED = "curated" # 精選職業（210筆，含具體領域）
|
||||||
|
DBPEDIA = "dbpedia"
|
||||||
|
WIKIDATA = "wikidata"
|
||||||
|
|
||||||
|
|
||||||
class ExpertTransformationRequest(BaseModel):
|
class ExpertTransformationRequest(BaseModel):
|
||||||
"""Expert Transformation Agent 請求"""
|
"""Expert Transformation Agent 請求"""
|
||||||
query: str
|
query: str
|
||||||
@@ -217,6 +225,10 @@ class ExpertTransformationRequest(BaseModel):
|
|||||||
keywords_per_expert: int = 1 # 每個專家為每個屬性生成幾個關鍵字 (1-3)
|
keywords_per_expert: int = 1 # 每個專家為每個屬性生成幾個關鍵字 (1-3)
|
||||||
custom_experts: Optional[List[str]] = None # 用戶指定專家 ["藥師", "工程師"]
|
custom_experts: Optional[List[str]] = None # 用戶指定專家 ["藥師", "工程師"]
|
||||||
|
|
||||||
|
# Expert source parameters
|
||||||
|
expert_source: ExpertSource = ExpertSource.LLM # 專家來源
|
||||||
|
expert_language: str = "en" # 外部來源的語言 (目前只有英文資料)
|
||||||
|
|
||||||
# LLM parameters
|
# LLM parameters
|
||||||
model: Optional[str] = None
|
model: Optional[str] = None
|
||||||
temperature: Optional[float] = 0.7
|
temperature: Optional[float] = 0.7
|
||||||
|
|||||||
@@ -10,20 +10,40 @@ def get_expert_generation_prompt(
|
|||||||
custom_experts: Optional[List[str]] = None
|
custom_experts: Optional[List[str]] = None
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Step 0: 生成專家團隊（不依賴主題，純隨機多元）"""
|
"""Step 0: 生成專家團隊（不依賴主題，純隨機多元）"""
|
||||||
|
import time
|
||||||
|
import random
|
||||||
|
|
||||||
custom_text = ""
|
custom_text = ""
|
||||||
if custom_experts and len(custom_experts) > 0:
|
if custom_experts and len(custom_experts) > 0:
|
||||||
custom_text = f"（已指定：{', '.join(custom_experts[:expert_count])}）"
|
custom_text = f"（已指定：{', '.join(custom_experts[:expert_count])}）"
|
||||||
|
|
||||||
|
# 加入時間戳和隨機數來增加多樣性
|
||||||
|
seed = int(time.time() * 1000) % 10000
|
||||||
|
diversity_hints = [
|
||||||
|
"冷門、非主流、跨領域",
|
||||||
|
"罕見職業、新興領域、邊緣學科",
|
||||||
|
"非傳統、創新、小眾專業",
|
||||||
|
"未來趨向、實驗性、非常規",
|
||||||
|
"跨文化、混合領域、獨特視角"
|
||||||
|
]
|
||||||
|
hint = random.choice(diversity_hints)
|
||||||
|
|
||||||
return f"""/no_think
|
return f"""/no_think
|
||||||
隨機組建 {expert_count} 個來自完全不同領域的專家團隊{custom_text}。
|
隨機組建 {expert_count} 個來自完全不同領域的專家團隊{custom_text}。
|
||||||
|
|
||||||
|
【創新要求】（隨機種子:{seed}）
|
||||||
|
- 優先選擇{hint}的專家
|
||||||
|
- 避免常見職業（如醫生、工程師、教師、律師等）
|
||||||
|
- 每個專家必須來自完全不相關的領域
|
||||||
|
- 越罕見、越創新越好
|
||||||
|
|
||||||
回傳 JSON：
|
回傳 JSON：
|
||||||
{{"experts": [{{"id": "expert-0", "name": "職業", "domain": "領域", "perspective": "角度"}}, ...]}}
|
{{"experts": [{{"id": "expert-0", "name": "職業", "domain": "領域", "perspective": "角度"}}, ...]}}
|
||||||
|
|
||||||
規則：
|
規則：
|
||||||
- id 為 expert-0 到 expert-{expert_count - 1}
|
- id 為 expert-0 到 expert-{expert_count - 1}
|
||||||
- name 填寫職業名稱（非人名），2-5字
|
- name 填寫職業名稱（非人名），2-5字
|
||||||
- 各專家的 domain 必須來自截然不同的領域，越多元越好"""
|
- domain 要具體且獨特，不可重複類型"""
|
||||||
|
|
||||||
|
|
||||||
def get_expert_keyword_generation_prompt(
|
def get_expert_keyword_generation_prompt(
|
||||||
@@ -33,46 +53,53 @@ def get_expert_keyword_generation_prompt(
|
|||||||
keywords_per_expert: int = 1
|
keywords_per_expert: int = 1
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Step 1: 專家視角關鍵字生成"""
|
"""Step 1: 專家視角關鍵字生成"""
|
||||||
experts_info = ", ".join([f"{exp['id']}:{exp['name']}({exp['domain']})" for exp in experts])
|
# 建立專家列表，格式更清晰
|
||||||
|
experts_list = "\n".join([f"- {exp['id']}: {exp['name']}" for exp in experts])
|
||||||
|
|
||||||
return f"""/no_think
|
return f"""/no_think
|
||||||
專家團隊：{experts_info}
|
你需要扮演以下專家，為屬性生成創新關鍵字：
|
||||||
屬性：「{attribute}」（{category}）
|
|
||||||
|
|
||||||
每位專家從自己的專業視角為此屬性生成 {keywords_per_expert} 個創新關鍵字（2-6字）。
|
【專家名單】
|
||||||
關鍵字要反映該專家領域的獨特思考方式。
|
{experts_list}
|
||||||
|
|
||||||
|
【任務】
|
||||||
|
屬性：「{attribute}」（類別：{category}）
|
||||||
|
|
||||||
|
請為每位專家：
|
||||||
|
1. 先理解該職業的專業背景、知識領域、工作內容
|
||||||
|
2. 從該職業的獨特視角思考「{attribute}」
|
||||||
|
3. 生成 {keywords_per_expert} 個與該專業相關的創新關鍵字（2-6字）
|
||||||
|
|
||||||
|
關鍵字必須反映該專家的專業思維方式，例如：
|
||||||
|
- 會計師 看「移動」→「資金流動」「成本效益」
|
||||||
|
- 建築師 看「移動」→「動線設計」「空間流動」
|
||||||
|
- 心理師 看「移動」→「行為動機」「情緒轉變」
|
||||||
|
|
||||||
回傳 JSON：
|
回傳 JSON：
|
||||||
{{"keywords": [{{"keyword": "詞彙", "expert_id": "expert-X", "expert_name": "名稱"}}, ...]}}
|
{{"keywords": [{{"keyword": "詞彙", "expert_id": "expert-X", "expert_name": "名稱"}}, ...]}}
|
||||||
|
|
||||||
共需 {len(experts) * keywords_per_expert} 個關鍵字。"""
|
共需 {len(experts) * keywords_per_expert} 個關鍵字，每個關鍵字必須明顯與對應專家的專業領域相關。"""
|
||||||
|
|
||||||
|
|
||||||
def get_expert_batch_description_prompt(
|
def get_single_description_prompt(
|
||||||
query: str,
|
query: str,
|
||||||
category: str,
|
keyword: str,
|
||||||
expert_keywords: List[dict] # List[ExpertKeyword]
|
expert_id: str,
|
||||||
|
expert_name: str,
|
||||||
|
expert_domain: str
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Step 2: 批次生成專家關鍵字的描述"""
|
"""Step 2: 為單一關鍵字生成描述"""
|
||||||
keywords_info = ", ".join([
|
# 如果 domain 是通用的，就只用職業名稱
|
||||||
f"{kw['expert_name']}:{kw['keyword']}"
|
domain_text = f"（{expert_domain}）" if expert_domain and expert_domain != "Professional Field" else ""
|
||||||
for kw in expert_keywords
|
|
||||||
])
|
|
||||||
|
|
||||||
# 建立 keyword -> (expert_id, expert_name) 的對照
|
|
||||||
keyword_expert_map = ", ".join([
|
|
||||||
f"{kw['keyword']}→{kw['expert_id']}/{kw['expert_name']}"
|
|
||||||
for kw in expert_keywords
|
|
||||||
])
|
|
||||||
|
|
||||||
return f"""/no_think
|
return f"""/no_think
|
||||||
物件：「{query}」
|
物件：「{query}」
|
||||||
關鍵字（專家:詞彙）：{keywords_info}
|
專家：{expert_name}{domain_text}
|
||||||
對照：{keyword_expert_map}
|
關鍵字：{keyword}
|
||||||
|
|
||||||
為每個關鍵字生成創新描述（15-30字），說明如何將該概念應用到「{query}」上。
|
你是一位{expert_name}。從你的專業視角，生成一段創新應用描述（15-30字），說明如何將「{keyword}」的概念應用到「{query}」上。
|
||||||
|
|
||||||
|
描述要體現{expert_name}的專業思維和獨特觀點。
|
||||||
|
|
||||||
回傳 JSON：
|
回傳 JSON：
|
||||||
{{"descriptions": [{{"keyword": "詞彙", "expert_id": "expert-X", "expert_name": "名稱", "description": "應用描述"}}, ...]}}
|
{{"description": "應用描述"}}"""
|
||||||
|
|
||||||
共需 {len(expert_keywords)} 個描述。"""
|
|
||||||
|
|||||||
@@ -13,13 +13,15 @@ from ..models.schemas import (
|
|||||||
ExpertKeyword,
|
ExpertKeyword,
|
||||||
ExpertTransformationCategoryResult,
|
ExpertTransformationCategoryResult,
|
||||||
ExpertTransformationDescription,
|
ExpertTransformationDescription,
|
||||||
|
ExpertSource,
|
||||||
)
|
)
|
||||||
from ..prompts.expert_transformation_prompt import (
|
from ..prompts.expert_transformation_prompt import (
|
||||||
get_expert_generation_prompt,
|
get_expert_generation_prompt,
|
||||||
get_expert_keyword_generation_prompt,
|
get_expert_keyword_generation_prompt,
|
||||||
get_expert_batch_description_prompt,
|
get_single_description_prompt,
|
||||||
)
|
)
|
||||||
from ..services.llm_service import ollama_provider, extract_json_from_response
|
from ..services.llm_service import ollama_provider, extract_json_from_response
|
||||||
|
from ..services.expert_source_service import expert_source_service
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
router = APIRouter(prefix="/api/expert-transformation", tags=["expert-transformation"])
|
router = APIRouter(prefix="/api/expert-transformation", tags=["expert-transformation"])
|
||||||
@@ -35,16 +37,38 @@ async def generate_expert_transformation_events(
|
|||||||
model = request.model
|
model = request.model
|
||||||
|
|
||||||
# ========== Step 0: Generate expert team ==========
|
# ========== Step 0: Generate expert team ==========
|
||||||
yield f"event: expert_start\ndata: {json.dumps({'message': '正在組建專家團隊...'}, ensure_ascii=False)}\n\n"
|
logger.info(f"[DEBUG] expert_source from request: {request.expert_source}")
|
||||||
|
logger.info(f"[DEBUG] expert_source value: {request.expert_source.value}")
|
||||||
|
logger.info(f"[DEBUG] custom_experts: {request.custom_experts}")
|
||||||
|
|
||||||
|
yield f"event: expert_start\ndata: {json.dumps({'message': '正在組建專家團隊...', 'source': request.expert_source.value}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
experts: List[ExpertProfile] = []
|
experts: List[ExpertProfile] = []
|
||||||
|
actual_source = request.expert_source.value
|
||||||
|
|
||||||
|
# 過濾出實際有內容的自訂專家（排除空字串）
|
||||||
|
actual_custom_experts = [
|
||||||
|
e.strip() for e in (request.custom_experts or [])
|
||||||
|
if e and e.strip()
|
||||||
|
]
|
||||||
|
logger.info(f"[DEBUG] actual_custom_experts (filtered): {actual_custom_experts}")
|
||||||
|
|
||||||
|
# 決定使用哪種來源生成專家
|
||||||
|
# 只有在明確選擇 LLM 或有實際自訂專家時才使用 LLM
|
||||||
|
use_llm = (
|
||||||
|
request.expert_source == ExpertSource.LLM or
|
||||||
|
len(actual_custom_experts) > 0 # 有實際自訂專家時，使用 LLM 補充
|
||||||
|
)
|
||||||
|
logger.info(f"[DEBUG] use_llm decision: {use_llm}")
|
||||||
|
|
||||||
|
if use_llm:
|
||||||
|
# LLM 生成專家
|
||||||
try:
|
try:
|
||||||
expert_prompt = get_expert_generation_prompt(
|
expert_prompt = get_expert_generation_prompt(
|
||||||
query=request.query,
|
query=request.query,
|
||||||
categories=all_categories,
|
categories=all_categories,
|
||||||
expert_count=request.expert_count,
|
expert_count=request.expert_count,
|
||||||
custom_experts=request.custom_experts
|
custom_experts=actual_custom_experts if actual_custom_experts else None
|
||||||
)
|
)
|
||||||
logger.info(f"Expert prompt: {expert_prompt[:200]}")
|
logger.info(f"Expert prompt: {expert_prompt[:200]}")
|
||||||
|
|
||||||
@@ -60,11 +84,64 @@ async def generate_expert_transformation_events(
|
|||||||
if isinstance(exp, dict) and all(k in exp for k in ["id", "name", "domain"]):
|
if isinstance(exp, dict) and all(k in exp for k in ["id", "name", "domain"]):
|
||||||
experts.append(ExpertProfile(**exp))
|
experts.append(ExpertProfile(**exp))
|
||||||
|
|
||||||
|
actual_source = "llm"
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Failed to generate experts: {e}")
|
logger.error(f"Failed to generate experts via LLM: {e}")
|
||||||
yield f"event: error\ndata: {json.dumps({'error': f'專家團隊生成失敗: {str(e)}'}, ensure_ascii=False)}\n\n"
|
yield f"event: error\ndata: {json.dumps({'error': f'專家團隊生成失敗: {str(e)}'}, ensure_ascii=False)}\n\n"
|
||||||
return
|
return
|
||||||
|
else:
|
||||||
|
# 外部來源生成專家 (本地檔案，同步)
|
||||||
|
try:
|
||||||
|
experts_data, actual_source = expert_source_service.get_experts(
|
||||||
|
source=request.expert_source.value,
|
||||||
|
count=request.expert_count,
|
||||||
|
language=request.expert_language
|
||||||
|
)
|
||||||
|
|
||||||
|
for i, exp_data in enumerate(experts_data):
|
||||||
|
experts.append(ExpertProfile(
|
||||||
|
id=f"expert-{i}",
|
||||||
|
name=exp_data["name"],
|
||||||
|
domain=exp_data["domain"],
|
||||||
|
perspective=f"從{exp_data['domain']}角度思考"
|
||||||
|
))
|
||||||
|
|
||||||
|
logger.info(f"Generated {len(experts)} experts from {actual_source}")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
# 外部來源失敗，fallback 到 LLM
|
||||||
|
logger.warning(f"External source failed: {e}, falling back to LLM")
|
||||||
|
yield f"event: expert_fallback\ndata: {json.dumps({'original': request.expert_source.value, 'fallback': 'llm', 'reason': str(e)}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
|
try:
|
||||||
|
expert_prompt = get_expert_generation_prompt(
|
||||||
|
query=request.query,
|
||||||
|
categories=all_categories,
|
||||||
|
expert_count=request.expert_count,
|
||||||
|
custom_experts=actual_custom_experts if actual_custom_experts else None
|
||||||
|
)
|
||||||
|
|
||||||
|
expert_response = await ollama_provider.generate(
|
||||||
|
expert_prompt, model=model, temperature=temperature
|
||||||
|
)
|
||||||
|
|
||||||
|
expert_data = extract_json_from_response(expert_response)
|
||||||
|
experts_raw = expert_data.get("experts", [])
|
||||||
|
|
||||||
|
for exp in experts_raw:
|
||||||
|
if isinstance(exp, dict) and all(k in exp for k in ["id", "name", "domain"]):
|
||||||
|
experts.append(ExpertProfile(**exp))
|
||||||
|
|
||||||
|
actual_source = "llm"
|
||||||
|
|
||||||
|
except Exception as llm_error:
|
||||||
|
logger.error(f"LLM fallback also failed: {llm_error}")
|
||||||
|
yield f"event: error\ndata: {json.dumps({'error': f'專家團隊生成失敗: {str(llm_error)}'}, ensure_ascii=False)}\n\n"
|
||||||
|
return
|
||||||
|
|
||||||
|
# 回報來源資訊
|
||||||
|
yield f"event: expert_source\ndata: {json.dumps({'source': actual_source}, ensure_ascii=False)}\n\n"
|
||||||
yield f"event: expert_complete\ndata: {json.dumps({'experts': [e.model_dump() for e in experts]}, ensure_ascii=False)}\n\n"
|
yield f"event: expert_complete\ndata: {json.dumps({'experts': [e.model_dump() for e in experts]}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
if not experts:
|
if not experts:
|
||||||
@@ -119,34 +196,48 @@ async def generate_expert_transformation_events(
|
|||||||
yield f"event: error\ndata: {json.dumps({'error': '無法生成關鍵字'}, ensure_ascii=False)}\n\n"
|
yield f"event: error\ndata: {json.dumps({'error': '無法生成關鍵字'}, ensure_ascii=False)}\n\n"
|
||||||
return
|
return
|
||||||
|
|
||||||
# ========== Step 2: Generate descriptions for each expert keyword ==========
|
# ========== Step 2: Generate descriptions one by one ==========
|
||||||
yield f"event: description_start\ndata: {json.dumps({'message': '為專家關鍵字生成創新應用描述...'}, ensure_ascii=False)}\n\n"
|
yield f"event: description_start\ndata: {json.dumps({'message': '為專家關鍵字生成創新應用描述...', 'total': len(all_expert_keywords)}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
descriptions: List[ExpertTransformationDescription] = []
|
descriptions: List[ExpertTransformationDescription] = []
|
||||||
|
|
||||||
|
# Build expert lookup for domain info
|
||||||
|
expert_lookup = {exp.id: exp for exp in experts}
|
||||||
|
|
||||||
|
for idx, kw in enumerate(all_expert_keywords):
|
||||||
try:
|
try:
|
||||||
desc_prompt = get_expert_batch_description_prompt(
|
expert = expert_lookup.get(kw.expert_id)
|
||||||
|
expert_domain = expert.domain if expert else ""
|
||||||
|
|
||||||
|
desc_prompt = get_single_description_prompt(
|
||||||
query=request.query,
|
query=request.query,
|
||||||
category=request.category,
|
keyword=kw.keyword,
|
||||||
expert_keywords=[kw.model_dump() for kw in all_expert_keywords]
|
expert_id=kw.expert_id,
|
||||||
|
expert_name=kw.expert_name,
|
||||||
|
expert_domain=expert_domain
|
||||||
)
|
)
|
||||||
logger.info(f"Description prompt: {desc_prompt[:300]}")
|
|
||||||
|
|
||||||
desc_response = await ollama_provider.generate(
|
desc_response = await ollama_provider.generate(
|
||||||
desc_prompt, model=model, temperature=temperature
|
desc_prompt, model=model, temperature=temperature
|
||||||
)
|
)
|
||||||
logger.info(f"Description response: {desc_response[:500]}")
|
|
||||||
|
|
||||||
desc_data = extract_json_from_response(desc_response)
|
desc_data = extract_json_from_response(desc_response)
|
||||||
descriptions_raw = desc_data.get("descriptions", [])
|
desc_text = desc_data.get("description", "")
|
||||||
|
|
||||||
for desc in descriptions_raw:
|
if desc_text:
|
||||||
if isinstance(desc, dict) and all(k in desc for k in ["keyword", "expert_id", "expert_name", "description"]):
|
descriptions.append(ExpertTransformationDescription(
|
||||||
descriptions.append(ExpertTransformationDescription(**desc))
|
keyword=kw.keyword,
|
||||||
|
expert_id=kw.expert_id,
|
||||||
|
expert_name=kw.expert_name,
|
||||||
|
description=desc_text
|
||||||
|
))
|
||||||
|
|
||||||
|
# Send progress update
|
||||||
|
yield f"event: description_progress\ndata: {json.dumps({'current': idx + 1, 'total': len(all_expert_keywords), 'keyword': kw.keyword}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"Failed to generate descriptions: {e}")
|
logger.warning(f"Failed to generate description for '{kw.keyword}': {e}")
|
||||||
# Continue without descriptions - at least we have keywords
|
# Continue with next keyword
|
||||||
|
|
||||||
yield f"event: description_complete\ndata: {json.dumps({'count': len(descriptions)}, ensure_ascii=False)}\n\n"
|
yield f"event: description_complete\ndata: {json.dumps({'count': len(descriptions)}, ensure_ascii=False)}\n\n"
|
||||||
|
|
||||||
|
|||||||
201
backend/app/services/expert_source_service.py
Normal file
201
backend/app/services/expert_source_service.py
Normal file
@@ -0,0 +1,201 @@
|
|||||||
|
"""Expert 本地資料來源服務
|
||||||
|
|
||||||
|
從本地 JSON 檔案讀取職業資料，提供隨機選取功能。
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import random
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import List, Tuple
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# 資料目錄
|
||||||
|
DATA_DIR = Path(__file__).parent.parent / "data"
|
||||||
|
|
||||||
|
|
||||||
|
class LocalDataProvider:
|
||||||
|
"""從本地 JSON 檔案讀取職業資料"""
|
||||||
|
|
||||||
|
def __init__(self, source: str):
|
||||||
|
"""
|
||||||
|
Args:
|
||||||
|
source: 資料來源名稱 (dbpedia/wikidata)
|
||||||
|
"""
|
||||||
|
self.source = source
|
||||||
|
self._cache: dict = {} # 記憶體快取
|
||||||
|
|
||||||
|
def load_occupations(self, language: str = "en") -> List[dict]:
|
||||||
|
"""
|
||||||
|
載入職業資料
|
||||||
|
|
||||||
|
Args:
|
||||||
|
language: 語言代碼 (en/zh)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
職業列表 [{"name": "...", "domain": "..."}, ...]
|
||||||
|
"""
|
||||||
|
cache_key = f"{self.source}:{language}"
|
||||||
|
|
||||||
|
# 檢查記憶體快取
|
||||||
|
if cache_key in self._cache:
|
||||||
|
return self._cache[cache_key]
|
||||||
|
|
||||||
|
# 讀取檔案
|
||||||
|
file_path = DATA_DIR / f"{self.source}_occupations_{language}.json"
|
||||||
|
|
||||||
|
if not file_path.exists():
|
||||||
|
logger.warning(f"資料檔案不存在: {file_path}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(file_path, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
|
||||||
|
occupations = data.get("occupations", [])
|
||||||
|
logger.info(f"載入 {len(occupations)} 筆 {self.source} {language} 職業")
|
||||||
|
|
||||||
|
# 存入快取
|
||||||
|
self._cache[cache_key] = occupations
|
||||||
|
return occupations
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"讀取職業資料失敗: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
def random_select(self, count: int, language: str = "en") -> List[dict]:
|
||||||
|
"""
|
||||||
|
隨機選取指定數量的職業
|
||||||
|
|
||||||
|
Args:
|
||||||
|
count: 需要的數量
|
||||||
|
language: 語言代碼
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
隨機選取的職業列表
|
||||||
|
"""
|
||||||
|
all_occupations = self.load_occupations(language)
|
||||||
|
|
||||||
|
if not all_occupations:
|
||||||
|
return []
|
||||||
|
|
||||||
|
if len(all_occupations) <= count:
|
||||||
|
return all_occupations
|
||||||
|
|
||||||
|
return random.sample(all_occupations, count)
|
||||||
|
|
||||||
|
|
||||||
|
class ExpertSourceService:
|
||||||
|
"""統一的專家來源服務"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.curated = LocalDataProvider("curated") # 精選職業
|
||||||
|
self.dbpedia = LocalDataProvider("dbpedia")
|
||||||
|
self.wikidata = LocalDataProvider("wikidata")
|
||||||
|
|
||||||
|
def get_experts(
|
||||||
|
self,
|
||||||
|
source: str,
|
||||||
|
count: int,
|
||||||
|
language: str = "en",
|
||||||
|
fallback_to_llm: bool = True
|
||||||
|
) -> Tuple[List[dict], str]:
|
||||||
|
"""
|
||||||
|
從指定來源獲取專家資料
|
||||||
|
|
||||||
|
Args:
|
||||||
|
source: 來源類型 ("dbpedia" | "wikidata")
|
||||||
|
count: 需要的專家數量
|
||||||
|
language: 語言代碼
|
||||||
|
fallback_to_llm: 失敗時是否允許 fallback（由呼叫者處理）
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(專家資料列表, 實際使用的來源)
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: 當獲取失敗且資料為空時
|
||||||
|
"""
|
||||||
|
# 選擇 provider
|
||||||
|
if source == "curated":
|
||||||
|
provider = self.curated
|
||||||
|
# 精選職業支援 zh 和 en，預設使用 zh
|
||||||
|
if language not in ["zh", "en"]:
|
||||||
|
language = "zh"
|
||||||
|
elif source == "wikidata":
|
||||||
|
provider = self.wikidata
|
||||||
|
else:
|
||||||
|
# 預設使用 dbpedia
|
||||||
|
provider = self.dbpedia
|
||||||
|
source = "dbpedia"
|
||||||
|
|
||||||
|
experts = provider.random_select(count, language)
|
||||||
|
|
||||||
|
if not experts:
|
||||||
|
raise ValueError(f"No occupations found from {source} ({language})")
|
||||||
|
|
||||||
|
logger.info(f"從 {source} 取得 {len(experts)} 位專家")
|
||||||
|
return experts, source
|
||||||
|
|
||||||
|
def get_available_sources(self) -> List[dict]:
|
||||||
|
"""
|
||||||
|
取得可用的資料來源資訊
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
來源資訊列表
|
||||||
|
"""
|
||||||
|
sources = []
|
||||||
|
|
||||||
|
# 檢查精選職業（中文）
|
||||||
|
curated_zh = DATA_DIR / "curated_occupations_zh.json"
|
||||||
|
if curated_zh.exists():
|
||||||
|
with open(curated_zh, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
sources.append({
|
||||||
|
"source": "curated",
|
||||||
|
"language": "zh",
|
||||||
|
"count": data["metadata"]["total_count"],
|
||||||
|
"created_at": data["metadata"]["created_at"]
|
||||||
|
})
|
||||||
|
|
||||||
|
# 檢查精選職業（英文）
|
||||||
|
curated_en = DATA_DIR / "curated_occupations_en.json"
|
||||||
|
if curated_en.exists():
|
||||||
|
with open(curated_en, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
sources.append({
|
||||||
|
"source": "curated",
|
||||||
|
"language": "en",
|
||||||
|
"count": data["metadata"]["total_count"],
|
||||||
|
"created_at": data["metadata"]["created_at"]
|
||||||
|
})
|
||||||
|
|
||||||
|
# 檢查 DBpedia
|
||||||
|
dbpedia_en = DATA_DIR / "dbpedia_occupations_en.json"
|
||||||
|
if dbpedia_en.exists():
|
||||||
|
with open(dbpedia_en, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
sources.append({
|
||||||
|
"source": "dbpedia",
|
||||||
|
"language": "en",
|
||||||
|
"count": data["metadata"]["total_count"],
|
||||||
|
"fetched_at": data["metadata"]["fetched_at"]
|
||||||
|
})
|
||||||
|
|
||||||
|
# 檢查 Wikidata
|
||||||
|
wikidata_zh = DATA_DIR / "wikidata_occupations_zh.json"
|
||||||
|
if wikidata_zh.exists():
|
||||||
|
with open(wikidata_zh, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
sources.append({
|
||||||
|
"source": "wikidata",
|
||||||
|
"language": "zh",
|
||||||
|
"count": data["metadata"]["total_count"],
|
||||||
|
"fetched_at": data["metadata"]["fetched_at"]
|
||||||
|
})
|
||||||
|
|
||||||
|
return sources
|
||||||
|
|
||||||
|
|
||||||
|
# 全域服務實例
|
||||||
|
expert_source_service = ExpertSourceService()
|
||||||
386
backend/scripts/fetch_occupations.py
Normal file
386
backend/scripts/fetch_occupations.py
Normal file
@@ -0,0 +1,386 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
職業資料抓取腳本
|
||||||
|
|
||||||
|
從 Wikidata SPARQL 和 ConceptNet API 抓取職業資料，
|
||||||
|
儲存為本地 JSON 檔案供應用程式使用。
|
||||||
|
|
||||||
|
使用方式:
|
||||||
|
cd backend
|
||||||
|
python scripts/fetch_occupations.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import List
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
# 輸出目錄
|
||||||
|
DATA_DIR = Path(__file__).parent.parent / "app" / "data"
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_wikidata_occupations(language: str) -> List[dict]:
|
||||||
|
"""
|
||||||
|
從 Wikidata SPARQL 端點抓取所有職業（使用分頁）
|
||||||
|
|
||||||
|
Args:
|
||||||
|
language: 語言代碼 (zh/en)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
職業列表 [{"name": "...", "domain": "..."}, ...]
|
||||||
|
"""
|
||||||
|
print(f"[Wikidata] 正在抓取 {language} 職業資料（分頁模式）...")
|
||||||
|
|
||||||
|
endpoint = "https://query.wikidata.org/sparql"
|
||||||
|
page_size = 500 # 每頁筆數
|
||||||
|
all_bindings = []
|
||||||
|
offset = 0
|
||||||
|
|
||||||
|
try:
|
||||||
|
with httpx.Client(timeout=120.0) as client:
|
||||||
|
while True:
|
||||||
|
# SPARQL 查詢 - 使用 SERVICE wikibase:label (更高效)
|
||||||
|
query = f"""
|
||||||
|
SELECT DISTINCT ?occupation ?occupationLabel ?fieldLabel WHERE {{
|
||||||
|
?occupation wdt:P31 wd:Q28640.
|
||||||
|
OPTIONAL {{ ?occupation wdt:P425 ?field. }}
|
||||||
|
SERVICE wikibase:label {{ bd:serviceParam wikibase:language "{language},en". }}
|
||||||
|
}}
|
||||||
|
LIMIT {page_size}
|
||||||
|
OFFSET {offset}
|
||||||
|
"""
|
||||||
|
|
||||||
|
print(f"[Wikidata] 抓取第 {offset // page_size + 1} 頁 (offset={offset})...")
|
||||||
|
|
||||||
|
response = client.get(
|
||||||
|
endpoint,
|
||||||
|
params={"query": query, "format": "json"},
|
||||||
|
headers={
|
||||||
|
"Accept": "application/sparql-results+json",
|
||||||
|
"User-Agent": "NoveltySeeking/1.0",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
|
||||||
|
bindings = data.get("results", {}).get("bindings", [])
|
||||||
|
print(f"[Wikidata] 取得 {len(bindings)} 筆")
|
||||||
|
|
||||||
|
if not bindings:
|
||||||
|
# 沒有更多資料了
|
||||||
|
break
|
||||||
|
|
||||||
|
all_bindings.extend(bindings)
|
||||||
|
offset += page_size
|
||||||
|
|
||||||
|
# 如果取得的筆數少於 page_size，表示已經是最後一頁
|
||||||
|
if len(bindings) < page_size:
|
||||||
|
break
|
||||||
|
|
||||||
|
print(f"[Wikidata] 總共取得 {len(all_bindings)} 筆原始資料")
|
||||||
|
|
||||||
|
# 解析回應
|
||||||
|
occupations = []
|
||||||
|
for item in all_bindings:
|
||||||
|
name = item.get("occupationLabel", {}).get("value", "")
|
||||||
|
field = item.get("fieldLabel", {}).get("value", "")
|
||||||
|
|
||||||
|
if name and len(name) >= 2:
|
||||||
|
occupations.append({
|
||||||
|
"name": name,
|
||||||
|
"domain": field if field else infer_domain(name, language),
|
||||||
|
})
|
||||||
|
|
||||||
|
# 去重
|
||||||
|
seen = set()
|
||||||
|
unique = []
|
||||||
|
for occ in occupations:
|
||||||
|
if occ["name"] not in seen:
|
||||||
|
seen.add(occ["name"])
|
||||||
|
unique.append(occ)
|
||||||
|
|
||||||
|
print(f"[Wikidata] 去重後: {len(unique)} 筆職業")
|
||||||
|
return unique
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[Wikidata] 錯誤: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_conceptnet_occupations(language: str) -> List[dict]:
|
||||||
|
"""
|
||||||
|
從 ConceptNet API 抓取職業相關概念（使用分頁）
|
||||||
|
|
||||||
|
Args:
|
||||||
|
language: 語言代碼 (zh/en)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
職業列表 [{"name": "...", "domain": "..."}, ...]
|
||||||
|
"""
|
||||||
|
print(f"[ConceptNet] 正在抓取 {language} 職業資料（分頁模式）...")
|
||||||
|
|
||||||
|
endpoint = "https://api.conceptnet.io"
|
||||||
|
lang_code = language
|
||||||
|
page_size = 100 # ConceptNet 建議的 limit
|
||||||
|
|
||||||
|
# 起始概念
|
||||||
|
start_concepts = {
|
||||||
|
"zh": ["/c/zh/職業", "/c/zh/專業", "/c/zh/工作", "/c/zh/職務"],
|
||||||
|
"en": ["/c/en/occupation", "/c/en/profession", "/c/en/job", "/c/en/career"],
|
||||||
|
}
|
||||||
|
|
||||||
|
# 要查詢的關係類型
|
||||||
|
relations = ["/r/IsA", "/r/RelatedTo", "/r/HasA", "/r/AtLocation"]
|
||||||
|
|
||||||
|
all_occupations = []
|
||||||
|
|
||||||
|
try:
|
||||||
|
with httpx.Client(timeout=60.0) as client:
|
||||||
|
for concept in start_concepts.get(lang_code, start_concepts["zh"]):
|
||||||
|
for rel in relations:
|
||||||
|
offset = 0
|
||||||
|
max_pages = 5 # 每個組合最多抓 5 頁
|
||||||
|
|
||||||
|
for page in range(max_pages):
|
||||||
|
try:
|
||||||
|
print(f"[ConceptNet] 查詢 {concept} {rel} (offset={offset})...")
|
||||||
|
|
||||||
|
# 查詢 start 參數
|
||||||
|
response = client.get(
|
||||||
|
f"{endpoint}/query",
|
||||||
|
params={
|
||||||
|
"start": concept,
|
||||||
|
"rel": rel,
|
||||||
|
"limit": page_size,
|
||||||
|
"offset": offset,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
if response.status_code != 200:
|
||||||
|
print(f"[ConceptNet] HTTP {response.status_code}, 跳過")
|
||||||
|
break
|
||||||
|
|
||||||
|
data = response.json()
|
||||||
|
edges = data.get("edges", [])
|
||||||
|
|
||||||
|
if not edges:
|
||||||
|
break
|
||||||
|
|
||||||
|
parsed = parse_conceptnet_response(data, lang_code)
|
||||||
|
all_occupations.extend(parsed)
|
||||||
|
print(f"[ConceptNet] 取得 {len(parsed)} 筆")
|
||||||
|
|
||||||
|
if len(edges) < page_size:
|
||||||
|
break
|
||||||
|
|
||||||
|
offset += page_size
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[ConceptNet] 錯誤: {e}")
|
||||||
|
break
|
||||||
|
|
||||||
|
# 去重
|
||||||
|
seen = set()
|
||||||
|
unique = []
|
||||||
|
for occ in all_occupations:
|
||||||
|
if occ["name"] not in seen:
|
||||||
|
seen.add(occ["name"])
|
||||||
|
unique.append(occ)
|
||||||
|
|
||||||
|
print(f"[ConceptNet] 去重後: {len(unique)} 筆概念")
|
||||||
|
return unique
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"[ConceptNet] 錯誤: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def parse_conceptnet_response(data: dict, lang_code: str) -> List[dict]:
|
||||||
|
"""解析 ConceptNet API 回應"""
|
||||||
|
results = []
|
||||||
|
edges = data.get("edges", [])
|
||||||
|
|
||||||
|
for edge in edges:
|
||||||
|
start = edge.get("start", {})
|
||||||
|
end = edge.get("end", {})
|
||||||
|
|
||||||
|
# 嘗試從兩端取得有意義的概念
|
||||||
|
for node in [start, end]:
|
||||||
|
node_id = node.get("@id", "")
|
||||||
|
label = node.get("label", "")
|
||||||
|
|
||||||
|
# 過濾：確保是目標語言且有意義
|
||||||
|
if f"/c/{lang_code}/" in node_id and label and len(label) >= 2:
|
||||||
|
# 排除過於泛用的詞
|
||||||
|
if label not in ["職業", "工作", "專業", "occupation", "job", "profession"]:
|
||||||
|
results.append({
|
||||||
|
"name": label,
|
||||||
|
"domain": infer_domain(label, lang_code),
|
||||||
|
})
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def infer_domain(occupation_name: str, language: str) -> str:
|
||||||
|
"""根據職業名稱推斷領域"""
|
||||||
|
if language == "zh":
|
||||||
|
domain_keywords = {
|
||||||
|
"醫": "醫療健康",
|
||||||
|
"護": "醫療健康",
|
||||||
|
"藥": "醫療健康",
|
||||||
|
"師": "專業服務",
|
||||||
|
"工程": "工程技術",
|
||||||
|
"技術": "工程技術",
|
||||||
|
"設計": "設計創意",
|
||||||
|
"藝術": "藝術文化",
|
||||||
|
"音樂": "藝術文化",
|
||||||
|
"運動": "體育運動",
|
||||||
|
"農": "農業",
|
||||||
|
"漁": "漁業",
|
||||||
|
"商": "商業貿易",
|
||||||
|
"銷": "商業貿易",
|
||||||
|
"法": "法律",
|
||||||
|
"律": "法律",
|
||||||
|
"教": "教育",
|
||||||
|
"研究": "學術研究",
|
||||||
|
"科學": "學術研究",
|
||||||
|
"廚": "餐飲服務",
|
||||||
|
"烹": "餐飲服務",
|
||||||
|
"建築": "建築營造",
|
||||||
|
"軍": "軍事國防",
|
||||||
|
"警": "公共安全",
|
||||||
|
"消防": "公共安全",
|
||||||
|
"記者": "媒體傳播",
|
||||||
|
"編輯": "媒體傳播",
|
||||||
|
"作家": "文學創作",
|
||||||
|
"程式": "資訊科技",
|
||||||
|
"軟體": "資訊科技",
|
||||||
|
"電腦": "資訊科技",
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
domain_keywords = {
|
||||||
|
"doctor": "Healthcare",
|
||||||
|
"nurse": "Healthcare",
|
||||||
|
"medical": "Healthcare",
|
||||||
|
"engineer": "Engineering",
|
||||||
|
"technical": "Engineering",
|
||||||
|
"design": "Design & Creative",
|
||||||
|
"artist": "Arts & Culture",
|
||||||
|
"music": "Arts & Culture",
|
||||||
|
"sport": "Sports",
|
||||||
|
"athletic": "Sports",
|
||||||
|
"farm": "Agriculture",
|
||||||
|
"fish": "Fishery",
|
||||||
|
"business": "Business",
|
||||||
|
"sales": "Business",
|
||||||
|
"law": "Legal",
|
||||||
|
"attorney": "Legal",
|
||||||
|
"teach": "Education",
|
||||||
|
"professor": "Education",
|
||||||
|
"research": "Academic Research",
|
||||||
|
"scien": "Academic Research",
|
||||||
|
"chef": "Culinary",
|
||||||
|
"cook": "Culinary",
|
||||||
|
"architect": "Architecture",
|
||||||
|
"military": "Military",
|
||||||
|
"police": "Public Safety",
|
||||||
|
"fire": "Public Safety",
|
||||||
|
"journal": "Media",
|
||||||
|
"editor": "Media",
|
||||||
|
"writer": "Literature",
|
||||||
|
"author": "Literature",
|
||||||
|
"program": "Information Technology",
|
||||||
|
"software": "Information Technology",
|
||||||
|
"computer": "Information Technology",
|
||||||
|
"develop": "Information Technology",
|
||||||
|
}
|
||||||
|
|
||||||
|
name_lower = occupation_name.lower()
|
||||||
|
for keyword, domain in domain_keywords.items():
|
||||||
|
if keyword in name_lower:
|
||||||
|
return domain
|
||||||
|
|
||||||
|
return "專業領域" if language == "zh" else "Professional Field"
|
||||||
|
|
||||||
|
|
||||||
|
def save_json(data: List[dict], source: str, language: str) -> None:
|
||||||
|
"""儲存資料到 JSON 檔案"""
|
||||||
|
filename = f"{source}_occupations_{language}.json"
|
||||||
|
filepath = DATA_DIR / filename
|
||||||
|
|
||||||
|
output = {
|
||||||
|
"metadata": {
|
||||||
|
"source": source,
|
||||||
|
"language": language,
|
||||||
|
"fetched_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"total_count": len(data),
|
||||||
|
},
|
||||||
|
"occupations": data,
|
||||||
|
}
|
||||||
|
|
||||||
|
with open(filepath, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(output, f, ensure_ascii=False, indent=2)
|
||||||
|
|
||||||
|
print(f"[儲存] {filepath} ({len(data)} 筆)")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""主程式"""
|
||||||
|
print("=" * 60)
|
||||||
|
print("職業資料抓取腳本")
|
||||||
|
print(f"輸出目錄: {DATA_DIR}")
|
||||||
|
print("=" * 60)
|
||||||
|
print()
|
||||||
|
|
||||||
|
# 確保輸出目錄存在
|
||||||
|
DATA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# 抓取 Wikidata
|
||||||
|
print("--- Wikidata ---")
|
||||||
|
try:
|
||||||
|
wikidata_zh = fetch_wikidata_occupations("zh")
|
||||||
|
save_json(wikidata_zh, "wikidata", "zh")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Wikidata 中文抓取失敗: {e}")
|
||||||
|
wikidata_zh = []
|
||||||
|
|
||||||
|
try:
|
||||||
|
wikidata_en = fetch_wikidata_occupations("en")
|
||||||
|
save_json(wikidata_en, "wikidata", "en")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Wikidata 英文抓取失敗: {e}")
|
||||||
|
wikidata_en = []
|
||||||
|
|
||||||
|
print()
|
||||||
|
|
||||||
|
# 抓取 ConceptNet
|
||||||
|
print("--- ConceptNet ---")
|
||||||
|
try:
|
||||||
|
conceptnet_zh = fetch_conceptnet_occupations("zh")
|
||||||
|
save_json(conceptnet_zh, "conceptnet", "zh")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"ConceptNet 中文抓取失敗: {e}")
|
||||||
|
conceptnet_zh = []
|
||||||
|
|
||||||
|
try:
|
||||||
|
conceptnet_en = fetch_conceptnet_occupations("en")
|
||||||
|
save_json(conceptnet_en, "conceptnet", "en")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"ConceptNet 英文抓取失敗: {e}")
|
||||||
|
conceptnet_en = []
|
||||||
|
|
||||||
|
print()
|
||||||
|
print("=" * 60)
|
||||||
|
print("抓取完成！")
|
||||||
|
print(f" Wikidata 中文: {len(wikidata_zh)} 筆")
|
||||||
|
print(f" Wikidata 英文: {len(wikidata_en)} 筆")
|
||||||
|
print(f" ConceptNet 中文: {len(conceptnet_zh)} 筆")
|
||||||
|
print(f" ConceptNet 英文: {len(conceptnet_en)} 筆")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -10,7 +10,7 @@ import { useAttribute } from './hooks/useAttribute';
|
|||||||
import { getModels } from './services/api';
|
import { getModels } from './services/api';
|
||||||
import type { MindmapDAGRef } from './components/MindmapDAG';
|
import type { MindmapDAGRef } from './components/MindmapDAG';
|
||||||
import type { TransformationDAGRef } from './components/TransformationDAG';
|
import type { TransformationDAGRef } from './components/TransformationDAG';
|
||||||
import type { CategoryMode } from './types';
|
import type { CategoryMode, ExpertSource } from './types';
|
||||||
|
|
||||||
const { Header, Sider, Content } = Layout;
|
const { Header, Sider, Content } = Layout;
|
||||||
const { Title } = Typography;
|
const { Title } = Typography;
|
||||||
@@ -33,7 +33,7 @@ function App() {
|
|||||||
|
|
||||||
// Transformation Agent settings
|
// Transformation Agent settings
|
||||||
const [transformModel, setTransformModel] = useState<string>('');
|
const [transformModel, setTransformModel] = useState<string>('');
|
||||||
const [transformTemperature, setTransformTemperature] = useState<number>(0.7);
|
const [transformTemperature, setTransformTemperature] = useState<number>(0.95);
|
||||||
const [expertConfig, setExpertConfig] = useState<{
|
const [expertConfig, setExpertConfig] = useState<{
|
||||||
expert_count: number;
|
expert_count: number;
|
||||||
keywords_per_expert: number;
|
keywords_per_expert: number;
|
||||||
@@ -44,6 +44,7 @@ function App() {
|
|||||||
custom_experts: undefined,
|
custom_experts: undefined,
|
||||||
});
|
});
|
||||||
const [customExpertsInput, setCustomExpertsInput] = useState('');
|
const [customExpertsInput, setCustomExpertsInput] = useState('');
|
||||||
|
const [expertSource, setExpertSource] = useState<ExpertSource>('llm');
|
||||||
const [shouldStartTransform, setShouldStartTransform] = useState(false);
|
const [shouldStartTransform, setShouldStartTransform] = useState(false);
|
||||||
const [transformLoading, setTransformLoading] = useState(false);
|
const [transformLoading, setTransformLoading] = useState(false);
|
||||||
|
|
||||||
@@ -186,6 +187,7 @@ function App() {
|
|||||||
model={transformModel}
|
model={transformModel}
|
||||||
temperature={transformTemperature}
|
temperature={transformTemperature}
|
||||||
expertConfig={expertConfig}
|
expertConfig={expertConfig}
|
||||||
|
expertSource={expertSource}
|
||||||
shouldStartTransform={shouldStartTransform}
|
shouldStartTransform={shouldStartTransform}
|
||||||
onTransformComplete={() => setShouldStartTransform(false)}
|
onTransformComplete={() => setShouldStartTransform(false)}
|
||||||
onLoadingChange={setTransformLoading}
|
onLoadingChange={setTransformLoading}
|
||||||
@@ -226,10 +228,12 @@ function App() {
|
|||||||
temperature={transformTemperature}
|
temperature={transformTemperature}
|
||||||
expertConfig={expertConfig}
|
expertConfig={expertConfig}
|
||||||
customExpertsInput={customExpertsInput}
|
customExpertsInput={customExpertsInput}
|
||||||
|
expertSource={expertSource}
|
||||||
onModelChange={setTransformModel}
|
onModelChange={setTransformModel}
|
||||||
onTemperatureChange={setTransformTemperature}
|
onTemperatureChange={setTransformTemperature}
|
||||||
onExpertConfigChange={setExpertConfig}
|
onExpertConfigChange={setExpertConfig}
|
||||||
onCustomExpertsInputChange={setCustomExpertsInput}
|
onCustomExpertsInputChange={setCustomExpertsInput}
|
||||||
|
onExpertSourceChange={setExpertSource}
|
||||||
availableModels={availableModels}
|
availableModels={availableModels}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|||||||
@@ -1,9 +1,17 @@
|
|||||||
import { Card, Select, Slider, Typography, Space, Button, Divider } from 'antd';
|
import { Card, Select, Slider, Typography, Space, Button, Divider } from 'antd';
|
||||||
import { ThunderboltOutlined } from '@ant-design/icons';
|
import { ThunderboltOutlined } from '@ant-design/icons';
|
||||||
import { ExpertConfigPanel } from './transformation';
|
import { ExpertConfigPanel } from './transformation';
|
||||||
|
import type { ExpertSource } from '../types';
|
||||||
|
|
||||||
const { Title, Text } = Typography;
|
const { Title, Text } = Typography;
|
||||||
|
|
||||||
|
const EXPERT_SOURCE_OPTIONS = [
|
||||||
|
{ label: 'LLM 生成', value: 'llm' as ExpertSource, description: '使用 AI 模型生成專家' },
|
||||||
|
{ label: '精選職業', value: 'curated' as ExpertSource, description: '從 210 個常見職業隨機選取（含具體領域）' },
|
||||||
|
{ label: 'DBpedia', value: 'dbpedia' as ExpertSource, description: '從 DBpedia 隨機選取職業 (2164 筆)' },
|
||||||
|
{ label: 'Wikidata', value: 'wikidata' as ExpertSource, description: '從 Wikidata 查詢職業 (需等待 API)' },
|
||||||
|
];
|
||||||
|
|
||||||
interface TransformationInputPanelProps {
|
interface TransformationInputPanelProps {
|
||||||
onTransform: () => void;
|
onTransform: () => void;
|
||||||
loading: boolean;
|
loading: boolean;
|
||||||
@@ -17,6 +25,7 @@ interface TransformationInputPanelProps {
|
|||||||
custom_experts?: string[];
|
custom_experts?: string[];
|
||||||
};
|
};
|
||||||
customExpertsInput: string;
|
customExpertsInput: string;
|
||||||
|
expertSource: ExpertSource;
|
||||||
onModelChange: (model: string) => void;
|
onModelChange: (model: string) => void;
|
||||||
onTemperatureChange: (temperature: number) => void;
|
onTemperatureChange: (temperature: number) => void;
|
||||||
onExpertConfigChange: (config: {
|
onExpertConfigChange: (config: {
|
||||||
@@ -25,6 +34,7 @@ interface TransformationInputPanelProps {
|
|||||||
custom_experts?: string[];
|
custom_experts?: string[];
|
||||||
}) => void;
|
}) => void;
|
||||||
onCustomExpertsInputChange: (value: string) => void;
|
onCustomExpertsInputChange: (value: string) => void;
|
||||||
|
onExpertSourceChange: (source: ExpertSource) => void;
|
||||||
availableModels: string[];
|
availableModels: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -37,10 +47,12 @@ export const TransformationInputPanel: React.FC<TransformationInputPanelProps> =
|
|||||||
temperature,
|
temperature,
|
||||||
expertConfig,
|
expertConfig,
|
||||||
customExpertsInput,
|
customExpertsInput,
|
||||||
|
expertSource,
|
||||||
onModelChange,
|
onModelChange,
|
||||||
onTemperatureChange,
|
onTemperatureChange,
|
||||||
onExpertConfigChange,
|
onExpertConfigChange,
|
||||||
onCustomExpertsInputChange,
|
onCustomExpertsInputChange,
|
||||||
|
onExpertSourceChange,
|
||||||
availableModels,
|
availableModels,
|
||||||
}) => {
|
}) => {
|
||||||
return (
|
return (
|
||||||
@@ -108,6 +120,31 @@ export const TransformationInputPanel: React.FC<TransformationInputPanelProps> =
|
|||||||
</Space>
|
</Space>
|
||||||
</Card>
|
</Card>
|
||||||
|
|
||||||
|
{/* Expert Source Selection */}
|
||||||
|
<Card
|
||||||
|
size="small"
|
||||||
|
title="專家來源"
|
||||||
|
style={{
|
||||||
|
background: isDark ? '#1f1f1f' : '#fafafa',
|
||||||
|
border: `1px solid ${isDark ? '#434343' : '#d9d9d9'}`,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Space direction="vertical" size="small" style={{ width: '100%' }}>
|
||||||
|
<Select
|
||||||
|
value={expertSource}
|
||||||
|
onChange={onExpertSourceChange}
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
options={EXPERT_SOURCE_OPTIONS.map((opt) => ({
|
||||||
|
label: opt.label,
|
||||||
|
value: opt.value,
|
||||||
|
}))}
|
||||||
|
/>
|
||||||
|
<Text type="secondary" style={{ fontSize: 11 }}>
|
||||||
|
{EXPERT_SOURCE_OPTIONS.find((opt) => opt.value === expertSource)?.description}
|
||||||
|
</Text>
|
||||||
|
</Space>
|
||||||
|
</Card>
|
||||||
|
|
||||||
{/* Expert Configuration */}
|
{/* Expert Configuration */}
|
||||||
<ExpertConfigPanel
|
<ExpertConfigPanel
|
||||||
expertCount={expertConfig.expert_count}
|
expertCount={expertConfig.expert_count}
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
import { forwardRef, useMemo, useCallback, useEffect } from 'react';
|
import { forwardRef, useMemo, useCallback, useEffect } from 'react';
|
||||||
import { Empty, Spin, Button, Progress, Card, Space, Typography, Tag } from 'antd';
|
import { Empty, Spin, Button, Progress, Card, Space, Typography, Tag } from 'antd';
|
||||||
import { ReloadOutlined } from '@ant-design/icons';
|
import { ReloadOutlined } from '@ant-design/icons';
|
||||||
import type { AttributeDAG, ExpertTransformationInput } from '../types';
|
import type { AttributeDAG, ExpertTransformationInput, ExpertSource } from '../types';
|
||||||
import { TransformationDAG } from './TransformationDAG';
|
import { TransformationDAG } from './TransformationDAG';
|
||||||
import type { TransformationDAGRef } from './TransformationDAG';
|
import type { TransformationDAGRef } from './TransformationDAG';
|
||||||
import { useExpertTransformation } from '../hooks/useExpertTransformation';
|
import { useExpertTransformation } from '../hooks/useExpertTransformation';
|
||||||
@@ -18,20 +18,21 @@ interface TransformationPanelProps {
|
|||||||
keywords_per_expert: number;
|
keywords_per_expert: number;
|
||||||
custom_experts?: string[];
|
custom_experts?: string[];
|
||||||
};
|
};
|
||||||
|
expertSource: ExpertSource;
|
||||||
shouldStartTransform: boolean;
|
shouldStartTransform: boolean;
|
||||||
onTransformComplete: () => void;
|
onTransformComplete: () => void;
|
||||||
onLoadingChange: (loading: boolean) => void;
|
onLoadingChange: (loading: boolean) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
export const TransformationPanel = forwardRef<TransformationDAGRef, TransformationPanelProps>(
|
export const TransformationPanel = forwardRef<TransformationDAGRef, TransformationPanelProps>(
|
||||||
({ attributeData, isDark, model, temperature, expertConfig, shouldStartTransform, onTransformComplete, onLoadingChange }, ref) => {
|
({ attributeData, isDark, model, temperature, expertConfig, expertSource, shouldStartTransform, onTransformComplete, onLoadingChange }, ref) => {
|
||||||
const {
|
const {
|
||||||
loading,
|
loading,
|
||||||
progress,
|
progress,
|
||||||
results,
|
results,
|
||||||
transformAll,
|
transformAll,
|
||||||
clearResults,
|
clearResults,
|
||||||
} = useExpertTransformation({ model, temperature });
|
} = useExpertTransformation({ model, temperature, expertSource });
|
||||||
|
|
||||||
// Notify parent of loading state changes
|
// Notify parent of loading state changes
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
|
|||||||
@@ -7,11 +7,13 @@ import type {
|
|||||||
ExpertTransformationDAGResult,
|
ExpertTransformationDAGResult,
|
||||||
ExpertProfile,
|
ExpertProfile,
|
||||||
CategoryDefinition,
|
CategoryDefinition,
|
||||||
|
ExpertSource,
|
||||||
} from '../types';
|
} from '../types';
|
||||||
|
|
||||||
interface UseExpertTransformationOptions {
|
interface UseExpertTransformationOptions {
|
||||||
model?: string;
|
model?: string;
|
||||||
temperature?: number;
|
temperature?: number;
|
||||||
|
expertSource?: ExpertSource;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function useExpertTransformation(options: UseExpertTransformationOptions = {}) {
|
export function useExpertTransformation(options: UseExpertTransformationOptions = {}) {
|
||||||
@@ -60,6 +62,7 @@ export function useExpertTransformation(options: UseExpertTransformationOptions
|
|||||||
expert_count: expertConfig.expert_count,
|
expert_count: expertConfig.expert_count,
|
||||||
keywords_per_expert: expertConfig.keywords_per_expert,
|
keywords_per_expert: expertConfig.keywords_per_expert,
|
||||||
custom_experts: expertConfig.custom_experts,
|
custom_experts: expertConfig.custom_experts,
|
||||||
|
expert_source: options.expertSource,
|
||||||
model: options.model,
|
model: options.model,
|
||||||
temperature: options.temperature,
|
temperature: options.temperature,
|
||||||
},
|
},
|
||||||
@@ -152,7 +155,7 @@ export function useExpertTransformation(options: UseExpertTransformationOptions
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
},
|
},
|
||||||
[options.model, options.temperature]
|
[options.model, options.temperature, options.expertSource]
|
||||||
);
|
);
|
||||||
|
|
||||||
const transformAll = useCallback(
|
const transformAll = useCallback(
|
||||||
|
|||||||
@@ -230,6 +230,8 @@ export interface ExpertTransformationDAGResult {
|
|||||||
results: ExpertTransformationCategoryResult[];
|
results: ExpertTransformationCategoryResult[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type ExpertSource = 'llm' | 'curated' | 'dbpedia' | 'wikidata';
|
||||||
|
|
||||||
export interface ExpertTransformationRequest {
|
export interface ExpertTransformationRequest {
|
||||||
query: string;
|
query: string;
|
||||||
category: string;
|
category: string;
|
||||||
@@ -237,6 +239,8 @@ export interface ExpertTransformationRequest {
|
|||||||
expert_count: number; // 2-8
|
expert_count: number; // 2-8
|
||||||
keywords_per_expert: number; // 1-3
|
keywords_per_expert: number; // 1-3
|
||||||
custom_experts?: string[]; // ["藥師", "工程師"]
|
custom_experts?: string[]; // ["藥師", "工程師"]
|
||||||
|
expert_source?: ExpertSource; // 專家來源 (default: 'llm')
|
||||||
|
expert_language?: string; // 外部來源語言 (default: 'en')
|
||||||
model?: string;
|
model?: string;
|
||||||
temperature?: number;
|
temperature?: number;
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user