Towards a Theory of Pragmatic Information

The subject generally known as ``information theory'' has nothing to say about how much meaning is conveyed by the information. Accordingly, we fill this gap with the first rigorously justifiable, quantitative definition of ``pragmatic information'' as the amount of information that becomes meaningful because it is used in making a decision. We posit that such information updates a ``state of the world'' random variable, $\omega$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the a priori and updated probability distributions of $\omega$, and the pragmatic information of a message ensemble is defined as the expected value of the pragmatic information values of the ensemble's component messages. We justify these definitions by showing, first, that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $\omega$ under the {\it a priori} and updated probability distributions, and, second, that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value. The resulting pragmatic information formulas have many hoped-for properties, such as non-negativity and additivity for independent decisions and ``pragmatically independent'' messages. We also sketch two applications of these formulas: The first is the single play of a slot machine, a.k.a. a ``one armed bandit'', with an unknown probability of payout; the second being the reformulation of the efficient market hypothesis of financial economics as the claim that the pragmatic information content of all available data about a given security is zero.