Skip to content

Information Theory

Miguel Amezola edited this page Sep 21, 2016 · 15 revisions

Informally, information is some message stored or transmitted using some medium. Messages are formed by arranging symbols in specific patterns. Communications as selection?

Information can be measured and compared using a measurement called entropy. The bit is a measure of surprise?

The message space is the set of all possible messages. Polybius square was a 5 by 5 grid that could represent 25 distinct messages. Sushruta Samhita: Given six different spices, how many possible different tastes can you make? Given n yes or no questions, there are 2^n possible answer sequences. Example: Lord George Murray's Shutter Telegraph.

Signal vs Noise: Do protein sequences contain noise? If so, eliminating it during preprocessing could make the algorithm more efficient.

  • Can we treat the symbols in protein sequences to be signals?
  • What is noise?

Discrete sources.

Capacity is the rate of sending information.

Symbol space.

Information is a selection from a collection of possible symbols.

Clone this wiki locally