The Noisy Channel Model
Assuming that you want to communicate messages over a channel of restricted capacity, optimize (in terms of throughput and accuracy) the communication in the presence of noise in the channel.
A channel’s capacity can be reached by designing an input code that maximizes the mutual information between the input and output over all possible input distributions.
This model can be applied to NLP.