210 lines
9.6 KiB
Markdown
210 lines
9.6 KiB
Markdown
# Literature Review: Expert-Augmented LLM Ideation
|
|
|
|
## 1. Core Directly-Related Work
|
|
|
|
### 1.1 Wisdom of Crowds via Role Assumption
|
|
**Bringing the Wisdom of the Crowd to an Individual by Having the Individual Assume Different Roles** (ACM C&C 2017)
|
|
|
|
Groups of people tend to generate more diverse ideas than individuals because each group member brings a different perspective. This study showed it's possible to help individuals think more like a group by asking them to approach a problem from different perspectives. In an experiment with 54 crowd workers, participants who assumed different expert roles came up with more creative ideas than those who did not.
|
|
|
|
**Gap for our work**: This was human-based role-playing. Our system automates this with LLM-powered expert perspectives.
|
|
|
|
### 1.2 PersonaFlow: LLM-Simulated Expert Perspectives
|
|
**PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation** (2024)
|
|
|
|
PersonaFlow provides multiple perspectives by using LLMs to simulate domain-specific experts. User studies showed it increased the perceived relevance and creativity of ideated research directions and promoted users' critical thinking activities without increasing perceived cognitive load.
|
|
|
|
**Gap for our work**: PersonaFlow focuses on research ideation. Our system applies to product/innovation ideation with structured attribute decomposition.
|
|
|
|
### 1.3 PopBlends: Conceptual Blending with LLMs
|
|
**PopBlends: Strategies for Conceptual Blending with Large Language Models** (CHI 2023)
|
|
|
|
PopBlends automatically suggests conceptual blends using both traditional knowledge extraction and LLMs. Studies showed people found twice as many blend suggestions with the system, with half the mental demand.
|
|
|
|
**Gap for our work**: We structure blending through expert domain knowledge rather than direct concept pairing.
|
|
|
|
### 1.4 BILLY: Persona Vector Merging
|
|
**BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation** (2025)
|
|
|
|
Proposes fusing persona vectors in activation space to steer LLM output towards multiple perspectives simultaneously, requiring only a single additive operation during inference.
|
|
|
|
**Gap for our work**: We use sequential multi-expert generation rather than vector fusion, allowing more explicit control and interpretability.
|
|
|
|
---
|
|
|
|
## 2. Theoretical Foundations
|
|
|
|
### 2.1 Semantic Distance Theory
|
|
|
|
**Core Insight** (Mednick, 1962): Creative thinking involves connecting weakly related, remote concepts in semantic memory. The farther one "moves away" from a conventional idea, the more creative the new idea will likely be.
|
|
|
|
**Key Research**:
|
|
- Semantic distance plays an important role in the creative process
|
|
- A more "flexible" semantic memory structure (higher connectivity, shorter distances) facilitates creative idea generation
|
|
- Quantitative measures using LSA and semantic networks can objectively examine creative output
|
|
- Divergent Semantic Integration (DSI) correlates strongly with human creativity ratings (72% variance explained)
|
|
|
|
**Application to Our Work**: Expert perspectives force semantic "jumps" to distant domains that LLMs wouldn't naturally traverse.
|
|
|
|
```
|
|
Without Expert: "Chair" → furniture, sitting, comfort (short semantic distance)
|
|
With Expert: "Chair" + Marine Biologist → pressure, buoyancy, coral (long semantic distance)
|
|
```
|
|
|
|
### 2.2 Conceptual Blending Theory
|
|
|
|
**Core Insight** (Fauconnier & Turner, 2002): Creative products emerge from blending elements of two input spaces into a novel integrated space.
|
|
|
|
**Key Research**:
|
|
- Blending process: (1) find connecting concept between inputs, (2) map elements that can be blended
|
|
- Generative AI demonstrates ability to blend and integrate concepts (bisociation)
|
|
- Trisociation (three-concept blending) is being used for AI-augmented idea generation
|
|
- Conceptual blending provides terminology for describing creative products
|
|
|
|
**Limitation**: Blending theory doesn't explain where inputs originate - the "inspiration problem."
|
|
|
|
**Application to Our Work**: Each expert provides a distinct "input space" enabling systematic multi-space blending. Our attribute decomposition provides structured inputs for blending.
|
|
|
|
### 2.3 Design Fixation
|
|
|
|
**Core Insight** (Jansson & Smith, 1991): Design fixation is "blind adherence to a set of ideas or concepts limiting the output of conceptual design."
|
|
|
|
**Key Research**:
|
|
- Fixation results from categorical knowledge organization around prototypes
|
|
- Accessing prototypes requires less cognitive effort than processing exemplars
|
|
- Diverse teams, model-making, and facilitation help prevent fixation
|
|
- Reflecting on prior fixation episodes is most effective prevention
|
|
|
|
**Neural Evidence**: fMRI studies show distinct patterns during fixated vs. creative ideation.
|
|
|
|
**Application to Our Work**: LLMs exhibit "semantic fixation" on high-probability outputs. Expert perspectives break this by forcing activation of non-prototype knowledge.
|
|
|
|
### 2.4 Constraint-Based Creativity
|
|
|
|
**Core Insight**: Paradoxically, constraints can enhance creativity by pushing beyond the path of least resistance.
|
|
|
|
**Key Research**:
|
|
- Constraints push people to search for more distant ideas in semantic memory
|
|
- Extreme constraints may require different types of creative problem-solving
|
|
- Not all constraints promote creativity for all individuals/tasks
|
|
- A "constraint-leveraging mindset" can be developed through experience
|
|
|
|
**Application to Our Work**: Expert role = productive constraint that expands rather than limits creative space. The expert perspective forces exploration of non-obvious solution spaces.
|
|
|
|
---
|
|
|
|
## 3. LLM Limitations in Creative Generation
|
|
|
|
### 3.1 Design Fixation from AI
|
|
**The Effects of Generative AI on Design Fixation and Divergent Thinking** (CHI 2024)
|
|
|
|
Key finding: AI exposure during ideation leads to HIGHER fixation. Participants who used AI produced:
|
|
- Fewer ideas
|
|
- Less variety
|
|
- Lower originality
|
|
|
|
compared to baseline (no AI assistance).
|
|
|
|
### 3.2 Dual Mechanisms: Inspiration vs. Fixation
|
|
**Inspiration Booster or Creative Fixation?** (Nature Humanities & Social Sciences, 2025)
|
|
|
|
- LLMs help in **simple** creative tasks (inspiration stimulation)
|
|
- LLMs **hurt** in **complex** creative tasks (creative fixation)
|
|
|
|
**Application to Our Work**: Our structured decomposition manages complexity, while multi-expert approach maintains inspiration benefits.
|
|
|
|
### 3.3 Statistical Pattern Perpetuation
|
|
**Bias and Fairness in Large Language Models: A Survey** (MIT Press, 2024)
|
|
|
|
LLMs learn, perpetuate, and amplify patterns from training data. This applies to creative outputs - LLMs generate what is statistically common/expected.
|
|
|
|
### 3.4 Generalization Bias
|
|
**Generalization Bias in LLM Summarization** (Royal Society, 2025)
|
|
|
|
LLMs' overgeneralization tendency produces outputs that lack sufficient empirical support. This suggests a bias toward "safe" middle-ground outputs rather than novel extremes.
|
|
|
|
---
|
|
|
|
## 4. Role-Playing and Perspective-Taking
|
|
|
|
### 4.1 Creativity Enhancement
|
|
Research on tabletop role-playing games (TTRPGs) demonstrates:
|
|
- Significant positive impact on creativity potential through divergent thinking
|
|
- TTRPG players exhibit significantly higher creativity than non-players
|
|
- Perspective-taking is closely linked to empathy and cognitive flexibility
|
|
|
|
### 4.2 Therapeutic and Educational Applications
|
|
- Role-playing develops perspective-taking, storytelling, creativity, and self-expression
|
|
- Physiological, emotional, and mental well-being from play enables creative ideation
|
|
- Play signals psychological safety, which is essential for creativity
|
|
|
|
### 4.3 Design Research Applications
|
|
- Role-playing stimulates creativity by exploring alternative solutions
|
|
- Offers safe environment to explore failure modes and challenge assumptions
|
|
- Well-suited for early-stage ideation and empathy-critical moments
|
|
|
|
---
|
|
|
|
## 5. Creativity Support Tools (CSTs)
|
|
|
|
### 5.1 Current State
|
|
- CSTs primarily support **divergent** thinking
|
|
- **Convergent** thinking often neglected
|
|
- Ideal CST should offer tailored support for both
|
|
|
|
### 5.2 AI as Creative Partner
|
|
- Collaborative ideation systems expose users to different ideas
|
|
- Competing theories on when/whether such exposure helps
|
|
- Tool-mediated expert activity view: computers as "mediating artifacts people act through"
|
|
|
|
### 5.3 Evaluation Methods
|
|
**Consensual Assessment Technique (CAT)**:
|
|
- Pool of experts independently evaluate artifacts
|
|
- Creative if high evaluations + high interrater reliability (Cronbach's alpha > 0.7)
|
|
|
|
**Semantic Distance Measures**:
|
|
- SemDis platform for automated creativity assessment
|
|
- Overcomes labor cost and subjectivity of human rating
|
|
- Uses NLP to quantify semantic relatedness
|
|
|
|
---
|
|
|
|
## 6. Our Theoretical Contribution
|
|
|
|
### The "Semantic Gravity" Problem
|
|
|
|
```
|
|
Direct LLM Generation:
|
|
P(idea | query)
|
|
→ Samples from high-probability region
|
|
→ Ideas cluster around training distribution modes
|
|
→ "Semantic gravity" pulls toward conventional associations
|
|
```
|
|
|
|
### Expert Transformation Solution
|
|
|
|
```
|
|
Conditioned Generation:
|
|
P(idea | query, expert)
|
|
→ Expert perspective activates distant semantic regions
|
|
→ Forces conceptual blending across domains
|
|
→ Breaks design fixation through productive constraints
|
|
```
|
|
|
|
### Multi-Expert Aggregation
|
|
|
|
```
|
|
Diverse Experts → Semantic Coverage
|
|
→ "Inner crowd" wisdom without actual crowd
|
|
→ Systematic exploration of idea space
|
|
→ Deduplication ensures non-redundant novelty
|
|
```
|
|
|
|
### Theoretical Model
|
|
|
|
1. **Attribute Decomposition**: Structures the problem space (categories, attributes)
|
|
2. **Expert Perspectives**: Forces semantic jumps to distant domains
|
|
3. **Multi-Expert Aggregation**: Achieves crowd-like diversity individually
|
|
4. **Deduplication**: Ensures generated ideas are truly distinct
|
|
5. **Patent Validation**: Grounds novelty in real-world uniqueness
|