I should be working on any of a multitude of things the completion of which are imminently necessary to prevent my being a complete failure in life.
But instead I’m reading about information theory on this blog. (More here.) I did this a lot in undergrad: I’d have a midterm in a linguistics class the next day and suddenly I’d find my girlfriend’s roommate’s Ocean Biology book irresistibly fascinating. For hours.
It’s an interesting thing because the definition of information in the context of mathematical information theory is sort of the reverse of what you’d expect, if I understand the author correctly. Information is divorced entirely from meaning and is purely a quantitative measure. The information density of a signal goes as the inverse of the amount of redundancy, which is roughly like saying the information present in a signal is the amount of randomness in the signal. By this measure, adding noise to a telephone signal increases information. (But not meaning, which is a completely different question.)
So by this definition, the oft-quoted law of thermodynamics that suggests that the universe is spinning down into a “heat death” of a sea of random particles doing random things implies that the quantity of information in the universe is constantly, irrevocably increasing.
Weird.
It makes better sense to me if I think of it in terms of a compression algorithm. Using some known algorithm, you can most likely compress a 10-second telephone voice signal to some extent without information loss. Heck, I can compress a 10-second sine wave right here for you: “10 seconds of a 440 Hz sine wave starting at 0.” There: what’s in quotes uniquely identifies 10 seconds of audio without any loss of information.* But I can’t uniquely specify 10 seconds of white noise without giving you every sample in order. That’s a lot of information.
So it makes sense, but it is a novel and counter-intuitive definition of “information” for me. And it brings to mind perennial musings on structure, meaning, and life.
Before any man-made object ever landed on Mars, James Lovelock predicted that it would be devoid of life because its atmosphere was chemically stable. That is, all chemical reactions that could take place in the atmosphere had already taken place, leaving behind a sea of gases in equilibrium. If there were life present, he reasoned, its chemical processes would probably be coupled to the planet’s chemistry, as it is on Earth. So chemically the planet was random, homogenous, flat, and dead. But apparently also at a local maximum of information quantity?
Autopoietic structures, such as hurricanes on the shallow end, and humans on the deep end, actually reduce the amount of information present, locally. It takes less information to describe a human than to describe equivalent amounts of chemicals randomly mixed in a vat. You know, from this particular point of view.
I guess what I find curious is that I can’t use the word “information” to refer to “interesting structures,” at least not when in the company of information theorists. I thought it was a useful word, but apparently not. Also I think it’s funny to point out how much of interest is missed by a tight specialization like this: how would an information theorist take into account the memory of a human in the accounting of the quantity of information present in our compressible selves as compared to a vat of chemicals? More generally, how would you deal with the behavior of a system in such an accounting? I dunno, maybe the questions don’t have meaning.
The really interesting question is, if I were reduced to a vat of equivalent mixture of chemicals, what color would I be?
Um. Okay. To work!
* Given an appropriate Turing machine to generate the signal from such a description, which admittedly would be more complex than one that would generate random noise. But presumably the two would be roughly equivalent, and at any rate, given a long enough signal you could overcome any difference in the requirements of the machine. Um. I presume.
