Prompt

Analyze A/B test results with statistical rigor.

<experiment_setup>
- Test name: {what's being tested}
- Variants: {control and treatment descriptions}
- Allocation: {traffic split}
- Duration: {test length}
</experiment_setup>

<metrics>
- Primary: {main success metric}
- Secondary: {additional metrics}
- Guardrails: {metrics that shouldn't degrade}
</metrics>

<sample_data>
- Control: {n, conversions, revenue}
- Treatment: {n, conversions, revenue}
</sample_data>

Analyze:
1. Statistical significance
   - Appropriate test selection
   - P-value calculation
   - Confidence intervals
   - Effect size
   
2. Practical significance
   - Business impact estimation
   - Cost-benefit analysis
   - Long-term projections
   
3. Validity checks
   - Randomization verification
   - Sample ratio mismatch
   - Novelty effects
   - Selection bias
   
4. Recommendation
   - Ship/no-ship decision
   - Rollout strategy
   - Follow-up tests

Consider multiple testing corrections.