Ai product canvas

--- name: ai-product-canvas description: "Structure AI and ML product decisions with the same rigor as any other product decision. Use it when building AI-powered features, evaluating LLM integrations, designing AI products, or assessing AI readiness. It produces a comprehensive AI product canvas covering problem definition, model approach, data requirements, evaluation framework, UX design, a responsible AI checklist, and a launch monitoring plan." --- # AI Product Canvas Skill Define AI products with the same rigor as any product decision:but with additional layers for data, model, evaluation, and responsible AI. This canvas prevents the most common AI product failure: building a technically impressive feature that doesn’t solve a real problem. ## AI Product Anti-Patterns to Check First Before building, identify if any of these apply: - ❌ “We should add AI to [existing feature]”:without defining a user problem - ❌ Accuracy target undefined before development begins - ❌ No plan for what happens when the model is wrong - ❌ User-facing AI output with no human review or fallback - ❌ Training data not audited for bias or quality - ❌ No evaluation metric : “we’ll know it when we see it” --- ## AI Product Canvas Output Format ### AI Product Canvas : [Feature Name] : [Date] PM Owner: [Name] ML/AI Lead: [Name] Status: Discovery / Design / Build / Evaluation / Live --- #### 1. Problem Definition User problem being solved: > [What specific situation is the user in? What task are they trying to complete?] Why AI? > [What makes this problem require AI rather than a deterministic solution? If the answer is “because we can,” stop here.] Success for the user looks like: > [What outcome does the user experience when the AI feature is working well?] --- #### 2. AI Approach Task type: - [ ] Classification - [ ] Generation (text, image, code) - [ ] Summarization / extraction - [ ] Recommendation - [ ] Search / retrieval - [ ] Prediction/forecasting - [ ] Conversation/agent Model approach: - [ ] LLMAPIs (GPT-4, Claude, Gemini, etc.) : specify: [Model name + version] - [ ] Fine-tuned model on own data - [ ] Custom model trained from scratch - [ ] RAG (retrieval-augmented generation) - [ ] Embedding + vector search Rationale for chosen approach: [Why this, not alternatives] --- #### 3. Data Requirements | Data Type | Source | Volume | Quality Status | Bias Risk | |---|---|---|---|---| | [Training data] | [Source] | [Volume] | [Audit status] | H/M/L | | [Evaluation data] | [Source] | [Volume] | [Audit status] | H/M/L | Data gaps: [What’s missing and plan to obtain it] Privacy considerations: [Any PII in training or inference data] Data ownership: [Do we own this data? Can we use it for training?] --- #### 4. Evaluation Framework Primary metric: [The number that defines success : accuracy, F1, BLEU, user rating, task completion rate] Minimum acceptable threshold: [Below X, the feature does not ship] Human evaluation plan: [How will humans review model outputs? Sampling rate? Review panel?] | Evaluation Type | Method | Frequency | Owner | |---|---|---|---| | Offline (pre-launch) | [Test set, benchmark] | Pre-launch | ML Lead | | Online (post-launch) | [A/B test, user feedback] | Weekly | PM + ML | | Adversarial | [Red-team, edge cases] | Pre-launch | Safety reviewer | --- #### 5. User Experience Design How is AI output presented? - [ ] Direct output shown to user (high trust required) - [ ] AI-assisted with user confirmation - [ ] Suggestion that user can accept or reject - [ ] Background action with audit log Confidence and uncertainty handling: - What happens when confidence is low? [Show alternative, ask for clarification, fallback to manual] - How is uncertainty communicated to the user? [UI pattern] Fallback plan: - If the model fails or returns an error: [Specific fallback behavior] - If accuracy drops below threshold: [Kill switch or graceful degradation plan] --- #### 6. Responsible AI Checklist - [ ] Bias audit completed on training data - [ ] Demographic fairness evaluated (does performance differ by user group?) - [ ] Risk of hallucination/confabulation a