Extreme Values

Minimum:

If only one particular message appears, so the probability for this message is 1, the entropy of this information source reaches its minimum. The absolute value results in

(1 log₂ 1) -> 0.

This information source does not provide information, there is no uncertainty about the next message appearing.

If a particular message never appears, the entropy results in

(0 log₂ 0) -> 0.

Such a message also does not increase the information at the receiving side. The entropy and therefore the average code length will not be affected by those messages.

Maximum:

If any message will appear with the same probability, the entropy will get their maximum. In this case a potential receiver has the largest possible uncertainty about the nature of the next message. The gain of information consequently is the largest.

 <   ^   >