201 lines
11 KiB
Markdown
201 lines
11 KiB
Markdown
---
|
|
phase: 05-output-cli
|
|
plan: 03
|
|
type: execute
|
|
wave: 2
|
|
depends_on: ["05-01", "05-02"]
|
|
files_modified:
|
|
- src/usher_pipeline/cli/report_cmd.py
|
|
- src/usher_pipeline/cli/main.py
|
|
- tests/test_report_cmd.py
|
|
autonomous: true
|
|
|
|
must_haves:
|
|
truths:
|
|
- "usher-pipeline report command generates tiered candidate list, visualizations, and reproducibility report in one invocation"
|
|
- "Report command reads scored_genes from DuckDB, applies tiering, adds evidence summary, writes dual-format output, generates plots, and creates reproducibility report"
|
|
- "Report command supports --output-dir, --force, --skip-viz, and --skip-report flags"
|
|
- "Unified CLI provides subcommands for setup, evidence, score, and report with consistent --config and --verbose flags"
|
|
artifacts:
|
|
- path: "src/usher_pipeline/cli/report_cmd.py"
|
|
provides: "CLI report command wiring tiering + viz + reproducibility"
|
|
exports: ["report"]
|
|
- path: "src/usher_pipeline/cli/main.py"
|
|
provides: "Updated CLI entry point with report command registered"
|
|
contains: "report"
|
|
- path: "tests/test_report_cmd.py"
|
|
provides: "CliRunner integration tests for report command"
|
|
key_links:
|
|
- from: "src/usher_pipeline/cli/report_cmd.py"
|
|
to: "src/usher_pipeline/output/tiers.py"
|
|
via: "import assign_tiers"
|
|
pattern: "from usher_pipeline\\.output.*import.*assign_tiers"
|
|
- from: "src/usher_pipeline/cli/report_cmd.py"
|
|
to: "src/usher_pipeline/output/evidence_summary.py"
|
|
via: "import add_evidence_summary"
|
|
pattern: "from usher_pipeline\\.output.*import.*add_evidence_summary"
|
|
- from: "src/usher_pipeline/cli/report_cmd.py"
|
|
to: "src/usher_pipeline/output/writers.py"
|
|
via: "import write_candidate_output"
|
|
pattern: "from usher_pipeline\\.output.*import.*write_candidate_output"
|
|
- from: "src/usher_pipeline/cli/report_cmd.py"
|
|
to: "src/usher_pipeline/output/visualizations.py"
|
|
via: "import generate_all_plots"
|
|
pattern: "from usher_pipeline\\.output.*import.*generate_all_plots"
|
|
- from: "src/usher_pipeline/cli/report_cmd.py"
|
|
to: "src/usher_pipeline/output/reproducibility.py"
|
|
via: "import generate_reproducibility_report"
|
|
pattern: "from usher_pipeline\\.output.*import.*generate_reproducibility_report"
|
|
- from: "src/usher_pipeline/cli/main.py"
|
|
to: "src/usher_pipeline/cli/report_cmd.py"
|
|
via: "cli.add_command(report)"
|
|
pattern: "add_command.*report"
|
|
---
|
|
|
|
<objective>
|
|
Create the CLI `report` command that orchestrates the full output pipeline: reads scored_genes from DuckDB, applies tiering and evidence summary, writes TSV+Parquet output, generates visualizations, and creates the reproducibility report.
|
|
|
|
Purpose: This is the user-facing entry point that ties together all output modules into a single invocation. After running `usher-pipeline score`, the user runs `usher-pipeline report` to get all deliverables.
|
|
Output: `src/usher_pipeline/cli/report_cmd.py` registered in main.py, with CliRunner integration tests.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/Users/gbanyan/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/Users/gbanyan/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/05-output-cli/05-01-SUMMARY.md
|
|
@.planning/phases/05-output-cli/05-02-SUMMARY.md
|
|
@src/usher_pipeline/cli/main.py
|
|
@src/usher_pipeline/cli/score_cmd.py
|
|
@src/usher_pipeline/cli/evidence_cmd.py
|
|
@src/usher_pipeline/persistence/duckdb_store.py
|
|
@src/usher_pipeline/config/schema.py
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto">
|
|
<name>Task 1: Report CLI command</name>
|
|
<files>
|
|
src/usher_pipeline/cli/report_cmd.py
|
|
src/usher_pipeline/cli/main.py
|
|
</files>
|
|
<action>
|
|
**report_cmd.py**: Create CLI report command following the established pattern from score_cmd.py and evidence_cmd.py.
|
|
|
|
```
|
|
@click.command('report')
|
|
@click.option('--output-dir', type=click.Path(path_type=Path), default=None, help='Output directory (default: {data_dir}/report)')
|
|
@click.option('--force', is_flag=True, help='Overwrite existing report files')
|
|
@click.option('--skip-viz', is_flag=True, help='Skip visualization generation')
|
|
@click.option('--skip-report', is_flag=True, help='Skip reproducibility report generation')
|
|
@click.option('--high-threshold', type=float, default=0.7, help='Minimum score for HIGH tier (default: 0.7)')
|
|
@click.option('--medium-threshold', type=float, default=0.4, help='Minimum score for MEDIUM tier (default: 0.4)')
|
|
@click.option('--low-threshold', type=float, default=0.2, help='Minimum score for LOW tier (default: 0.2)')
|
|
@click.option('--min-evidence-high', type=int, default=3, help='Minimum evidence layers for HIGH tier (default: 3)')
|
|
@click.option('--min-evidence-medium', type=int, default=2, help='Minimum evidence layers for MEDIUM tier (default: 2)')
|
|
@click.pass_context
|
|
def report(ctx, output_dir, force, skip_viz, skip_report, high_threshold, medium_threshold, low_threshold, min_evidence_high, min_evidence_medium):
|
|
```
|
|
|
|
Follow the established CLI command pattern: load config -> init store/provenance -> check prerequisites -> execute steps -> display summary -> cleanup.
|
|
|
|
Pipeline steps (echoed with click.style like score_cmd.py):
|
|
1. Load configuration and initialize storage (same pattern as score_cmd.py)
|
|
2. Check scored_genes table exists (error if not: "Run 'usher-pipeline score' first")
|
|
3. Load scored_genes DataFrame from DuckDB via store.load_dataframe('scored_genes')
|
|
4. Build tier thresholds from CLI options into dict: {"high": {"score": high_threshold, "evidence_count": min_evidence_high}, "medium": {"score": medium_threshold, "evidence_count": min_evidence_medium}, "low": {"score": low_threshold}}
|
|
5. Apply tiering: `tiered_df = assign_tiers(scored_df, thresholds=thresholds)`
|
|
6. Add evidence summary: `tiered_df = add_evidence_summary(tiered_df)`
|
|
7. Write dual-format output: `paths = write_candidate_output(tiered_df, output_dir, "candidates")`
|
|
8. Echo tier counts: "HIGH: N, MEDIUM: N, LOW: N (total: N candidates from M scored genes)"
|
|
9. If not --skip-viz: `plot_paths = generate_all_plots(tiered_df, output_dir / "plots")` -- echo each plot file created
|
|
10. If not --skip-report: Load validation result if available (try store.load_dataframe for validation metadata or call validate_known_gene_ranking if scored_genes has known gene data). Call `report = generate_reproducibility_report(config, tiered_df, provenance, validation_result)`. Write report.to_json() and report.to_markdown() to output_dir.
|
|
11. Save provenance sidecar for the report command itself
|
|
12. Display final summary: output directory, file list, tier counts
|
|
|
|
Default output_dir: `Path(config.data_dir) / "report"` if not specified via --output-dir.
|
|
|
|
If output files already exist and --force not set, echo warning and skip (checkpoint pattern).
|
|
|
|
Ensure store.close() in finally block.
|
|
|
|
**main.py**: Add report command registration.
|
|
|
|
Add import: `from usher_pipeline.cli.report_cmd import report`
|
|
Add registration: `cli.add_command(report)`
|
|
|
|
The CLI now has 4 top-level commands: setup, evidence, score, report (plus the existing info command).
|
|
</action>
|
|
<verify>
|
|
Run: `cd /Users/gbanyan/Project/usher-exploring && usher-pipeline report --help` -- should show all options including --output-dir, --force, --skip-viz, --skip-report, tier thresholds
|
|
Run: `usher-pipeline --help` -- should list report in available commands
|
|
</verify>
|
|
<done>
|
|
report command is registered and shows all expected options in --help output. CLI entry point lists setup, evidence, score, report, and info commands.
|
|
</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: CliRunner integration tests for report command</name>
|
|
<files>
|
|
tests/test_report_cmd.py
|
|
</files>
|
|
<action>
|
|
**tests/test_report_cmd.py**: Create CliRunner integration tests.
|
|
|
|
Follow the established test pattern from test_scoring_integration.py: create synthetic data in a tmp_path DuckDB, invoke CLI commands via CliRunner.
|
|
|
|
Create test fixtures:
|
|
- `test_config` fixture: Write minimal config YAML to tmp_path, pointing duckdb_path and data_dir to tmp_path
|
|
- `populated_db` fixture: Create DuckDB at tmp_path, populate with:
|
|
- gene_universe table (20 synthetic genes with gene_id and gene_symbol)
|
|
- scored_genes table with all required columns (gene_id, gene_symbol, composite_score, evidence_count, quality_flag, all 6 layer score columns + 6 contribution columns, available_weight, weighted_sum)
|
|
- Design data so: 3 genes HIGH tier (score 0.7-0.95, evidence_count 3-5), 5 MEDIUM, 5 LOW, 4 EXCLUDED (score < 0.2), 3 NULL composite_score
|
|
- Register in _checkpoints table so has_checkpoint('scored_genes') returns True
|
|
|
|
Tests:
|
|
1. test_report_help: Invoke `report --help`, assert exit_code 0, assert "--output-dir" in output
|
|
2. test_report_generates_files: Invoke report with populated_db and test_config, assert exit_code 0, verify candidates.tsv exists, candidates.parquet exists, candidates.provenance.yaml exists
|
|
3. test_report_tier_counts_in_output: Invoke report, assert "HIGH: 3" (or similar) appears in CLI output
|
|
4. test_report_with_viz: Invoke report (no --skip-viz), verify plots/ directory contains score_distribution.png, layer_contributions.png, tier_breakdown.png
|
|
5. test_report_skip_viz: Invoke report with --skip-viz, verify no plots/ directory created
|
|
6. test_report_skip_report: Invoke report with --skip-report, verify no reproducibility .json/.md files
|
|
7. test_report_custom_thresholds: Invoke with --high-threshold 0.8 --medium-threshold 0.5, verify different tier counts
|
|
8. test_report_no_scored_genes_error: Invoke report with empty DuckDB (no scored_genes table), assert exit_code != 0, assert "Run 'usher-pipeline score' first" in output
|
|
9. test_report_output_dir_option: Invoke with --output-dir custom_path, verify files created in custom_path
|
|
</action>
|
|
<verify>
|
|
Run: `cd /Users/gbanyan/Project/usher-exploring && python -m pytest tests/test_report_cmd.py -v`
|
|
</verify>
|
|
<done>
|
|
All 9 CliRunner integration tests pass. Report command correctly generates tiered candidates in TSV+Parquet, visualizations (unless --skip-viz), and reproducibility report (unless --skip-report). Custom tier thresholds work. Missing scored_genes table produces clear error message. All file paths are verified.
|
|
</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
- `usher-pipeline --help` lists setup, evidence, score, report, info commands
|
|
- `usher-pipeline report --help` shows all options
|
|
- `python -m pytest tests/test_report_cmd.py -v` -- all 9 tests pass
|
|
- End-to-end: scored_genes data -> tiered candidates.tsv + candidates.parquet + provenance.yaml + plots/ + reproducibility report
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- CLI `report` command orchestrates full output pipeline in one invocation
|
|
- Supports --output-dir, --force, --skip-viz, --skip-report, and configurable tier thresholds
|
|
- Follows established CLI patterns (config loading, store init, checkpoint, provenance, summary, cleanup)
|
|
- All CliRunner integration tests pass
|
|
- Unified CLI has all subcommands: setup, evidence, score, report, info
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/05-output-cli/05-03-SUMMARY.md`
|
|
</output>
|