Expand description
Query-by-committee — query the points where an ensemble disagrees. Disagreement is the signal: where independently-trained models diverge, a human label resolves the most uncertainty.
Two soft measures (over each member’s predicted distribution) and one hard measure (over each member’s single vote), all per-sample; plus a pool ranking.
Functions§
- average_
kl_ disagreement - Average KL disagreement: mean
KL(member ‖ consensus)over the committee. This is the canonical soft-QBC measure — it is large precisely when members are confident but about different classes (genuine disagreement), not merely diffuse. - consensus
- The committee consensus distribution: the mean of the members’ predicted
distributions for one sample.
membersisn_members × n_classes. - consensus_
entropy - Consensus entropy (soft vote entropy): entropy of the consensus distribution. Diffuse consensus ⇒ informative.
- rank_
by_ disagreement - Rank pool indices by average-KL disagreement, most-informative first.
poolisn_samplescommittees, eachn_members × n_classes. - vote_
entropy - Vote entropy over hard votes: entropy of the distribution of committee votes
across
n_classes. Maximal when the committee splits evenly.votesare class indices, one per member.