Were you able to tell what I was trying to communicate there? Let me express it differently:
Click here for the sentence I've been trying to convey, and its significance. The point is that the same information can be expressed in different ways, and understood (with some work) even if you don't know the the communication system (i.e. code) I'm using.
I started thinking recently about how one would define the concept of information. I don't have a definition yet, but I think one of its crucial properties is that information is independent of the system used to communicate it. The same information can be expressed in a variety of languages, codes, or pictures, and received through lights on a screen, ink arranged on paper, or compressed waves of air. To reach us, information might travel in the form of electrical pulses (as were used in telegraph machines), radio waves, light in fiber-optical channels, or smoke signals. This independence of physical form distinguishes information from other physical quantities. Energy, for example, can come in many forms; but you would react much differently from coming in contact with light energy versus heat energy or kinetic energy.
It takes a certain level of intelligence to conceive of information independently of its form. We humans understand that ink on paper can refer to the same thing as sound waves, whereas to a bacterium these are completely different physical phenomena. It would be interesting to investigate which animals can understand different physical processes as conveying the same message.
One might be tempted to conclude that information exists only in the minds of intelligent beings, with no independent physical meaning. But this is not true: information appears in the laws of physics. The second law of thermodynamics, for example, says that the closed systems become less predictable with time, and more information is therefore required to understand them.
So information is a physical quantity, but exists independently of its many possible forms. This falls far short of a definition, but it may help explain the uniqueness of information among the quantities considered in science.
I think one of its crucial properties is that information is independent of the system used to communicate it
ReplyDeleteI think this has to be extremely carefully defined. If you and I agree on an encoding then it shouldn't matter which system we choose. But if I'm expecting 7-bit ASCII and you send Unicode, then we've got a problem.
I think a big problem here is that you're trying to get at the notion of a message independent of a medium. Removing a message (and its information content) from a channel is a process of stripping away complexity. I can send you a message by smoke signals, and you discard all the random, complicated motions of smoke particles when you decode it.
It's true that a physical signal has no intrinsic meaning without an associated decoding scheme. This creates an interesting distinction between physical information (entropy) and communication information: Physical information is a property of the physical system, but communication information is a property of the system together with the decoding scheme, which, as you say, discards most of the physical information. Physical information/entropy is in fact the limit of how much information can be stored or communicated in a physical system.
ReplyDeleteThanks for highlighting these issues.
I don't really know much about this topic but I thought there was already an agreed upon definition of information via information theory.
ReplyDeleteDoes the fact that you're looking for a different definition mean that you're looking to approach information from a different (more intuitive?) perspective or does it mean you disagree with the current definition?
Also, it seems to me that the physical form of information contains information itself. So I would respond differently to information on a stone tablet to the same information transmitted via radiowaves. The method of communication would itself give me some information about the sender.
Of course, that's not to disprove what you've said as there still exists the message information which exists independent of the physical manifestation. I just thought it was an interesting thing to note.
AP, that's a good question, and one which I plan to address in the near future.
ReplyDeleteShort answer: Information theory defines ways to quantify information. I believe there are other ways to quantify information outside of traditional information theory, and I've just written a paper to that effect (to be posted soon.) But regardless, this is different from having a philosophical definition of what information really is. I don't think information theory goes there, though I may have missed something.
Okay, I see. It'll be interesting to read the paper when you post it.
ReplyDeleteIt's here.
ReplyDelete