Implement a new Deduplication Agent that identifies and groups similar
transformation descriptions. Supports two deduplication methods:
- Embedding: Fast vector similarity comparison using cosine similarity
- LLM: Accurate pairwise semantic comparison (slower but more precise)
Backend changes:
- Add deduplication router with /deduplicate endpoint
- Add embedding_service for vector-based similarity
- Add llm_deduplication_service for LLM-based comparison
- Improve expert_transformation error handling and progress reporting
Frontend changes:
- Add DeduplicationPanel with interactive group visualization
- Add useDeduplication hook for state management
- Integrate deduplication tab in main App
- Add threshold slider and method selector in sidebar
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add expert_cache.py: TTL-based in-memory cache (1 hour default)
- Add expert_source_service.py: WikidataProvider and ConceptNetProvider
- Wikidata SPARQL queries for occupations with Chinese labels
- ConceptNet API queries for occupation-related concepts
- Random selection from cached pool
- Update schemas.py: Add ExpertSource enum (llm/wikidata/conceptnet)
- Update ExpertTransformationRequest with expert_source and expert_language
- Update router: Conditionally use external sources with LLM fallback
- New SSE events: expert_source, expert_fallback
- Update frontend types with ExpertSource
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add random seed and diversity hints to expert generation prompt
- Explicitly avoid common professions (醫生、工程師、教師、律師等)
- Change description generation from batch to one-by-one for reliability
- Increase default temperature from 0.7 to 0.95 for more creative output
- Add description_progress SSE event for real-time feedback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Backend:
- Add CategoryMode enum with 4 modes (fixed_only, fixed_plus_custom, custom_only, dynamic_auto)
- Add Step 0 for LLM category analysis before attribute generation
- Implement dynamic prompts for Step 1/2 that work with N categories
- Add execute_step0(), resolve_final_categories(), assemble_dynamic_attribute_tree()
- Update SSE events to include step0_start, step0_complete, categories_resolved
Frontend:
- Add CategorySelector component with mode selection, custom category input, and category count slider
- Update types with CategoryDefinition, Step0Result, DynamicStep1Result, DynamicCausalChain
- Update api.ts with new SSE event handlers
- Update useAttribute hook with category parameters
- Integrate CategorySelector into InputPanel
- Fix mindmap to dynamically extract and display N categories (was hardcoded to 4)
- Add CSS styles for depth 5-8 to support more category levels
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
