Shannon Entropy Calculator

What Is Shannon Entropy?

Shannon entropy measures uncertainty in a probability distribution. Higher entropy means outcomes are harder to predict. Zero entropy means one outcome is certain. Related tools: Shannon Diversity Index Calculator and Probability Calculator.

Shannon Entropy Formula

$$H(x) = -\sum_{i=1}^{n} p_i \log_b p_i$$

Each probability $p_i$ must be between 0 and 1, and all probabilities must sum to 1. The logarithm base sets the unit: base 2 gives bits, base $e$ gives nats, and base 10 gives dits.

How to Use This Calculator

Choose how many probabilities you need, enter each value, and select the log base. The tool validates that the probabilities sum to 1 and shows each outcome's contribution to total entropy.

Example

For probabilities 0.5, 0.3, and 0.2 with base 2, entropy is about 1.485 bits. A uniform distribution over three outcomes would be higher; a certain outcome would be 0.

Frequently Asked Questions

Why must probabilities sum to 1?

Entropy is defined for a full probability distribution. Partial totals represent an incomplete set of outcomes.

What happens if one probability is zero?

That term contributes 0 to entropy because $\lim_{p \to 0} p \log p = 0$.

Which log base should I choose?

Use base 2 for information theory and computer science, base $e$ for physics and many statistical models, and base 10 when working with hartleys.

Is Shannon entropy the same as diversity index?

The formula is the same when proportions are used, but diversity tools often start from species counts while this calculator starts from probabilities directly.

Report