Skip to main content

Module experiment

Module experiment 

Source
Expand description

Experiment design & evaluation (Practical Statistics ch 3) — planning and analysing experiments, with uncertainty made explicit.

  • power — power analysis & required sample size.
  • [ab_test] — A/B testing (two-proportion comparison with a lift CI).
  • bandit — multi-armed bandits (ε-greedy / UCB1 / Thompson sampling).

Reuses statistics::distributions::normal; no new silo.

Re-exports§

pub use ab_test::ab_test;
pub use ab_test::AbResult;
pub use bandit::Bandit;
pub use bandit::Policy;
pub use power::power_two_sample;
pub use power::required_sample_size_two_proportion;
pub use power::required_sample_size_two_sample;

Modules§

ab_test
A/B testing (Practical Statistics ch 3) — compare two variants’ conversion rates with a two-proportion z-test and a confidence interval on the lift. The honest output is “B beat A by 2.1pp [0.4pp, 3.8pp], p = 0.01” — effect size with uncertainty, not a bare “B wins”. Reuses the Normal CDF/quantile.
bandit
Multi-armed bandits (Practical Statistics ch 3) — sequential experimentation that adapts: instead of a fixed-split A/B test, allocate more trials to the arms that look better, trading exploration against exploitation. Three classic policies: ε-greedy, UCB1, and Thompson sampling (Beta-Bernoulli). Kernel-class Divergent (the sampling/branch logic).
power
Power analysis & sample-size (Practical Statistics ch 3) — how many observations an experiment needs to detect an effect, and the power it achieves at a given size. Honest experiment planning: state the effect you care about and the confidence you need, get the sample size you must collect. Reuses the Normal quantile/CDF from statistics::distributions.