-
Notifications
You must be signed in to change notification settings - Fork 0
Information Theory
Informally, information is some message stored or transmitted using some medium. Messages are formed by arranging symbols in specific patterns. Communications as selection?
Information can be measured and compared using a measurement called entropy. The bit is a measure of surprise?
The message space is the set of all possible messages. Polybius square was a 5 by 5 grid that could represent 25 distinct messages. Sushruta Samhita: Given six different spices, how many possible different tastes can you make? Given n yes or no questions, there are 2^n possible answer sequences. Example: Lord George Murray's Shutter Telegraph.
Signal vs Noise: Do protein sequences contain noise? If so, eliminating it during preprocessing could make the algorithm more efficient.
- Can we treat the symbols in protein sequences to be signals?
- What is noise?
Discrete sources.
Capacity is the rate of sending information.
Symbol space.
Information is a selection from a collection of possible symbols.
-
C. E. Shannon: A mathematical theory of communication. Bell System Technical Journal, vol. 27, pp. 379–423 and 623–656, July and October 1948
-
"What Is FASTA Format?" FASTA Format. Zhang Lab, University of Michigan, n.d. Web. 24 Sept. 2016.
-
Rost, B.; J.Liu; R.Nair; K.O. Wrzeszczynski; Y. Ofran (2003). "Automatic prediction of protein function". Cellular and Molecular Life Sciences. 60 (12): 2637–2650. doi:10.1007/s00018-003-3114-8. PMID 14685688.
-
Alpaydin, Ethem. "Introduction to Machine Learning." MIT Press, 04 Dec. 2009. Web. 24 Sept. 2016.
-
Nielsen, Michael. "Neural Networks and Deep Learning." Neural Networks and Deep Learning. N.p., June 2016. Web. 24 Sept. 2016.
-
"Implementing a Neural Network from Scratch in Python – An Introduction." WildML. N.p., 10 Jan. 2016. Web. 24 Sept. 2016.
-
"Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs." WildML. N.p., 08 July 2016. Web. 24 Sept. 2016.
-
Singh, Jagjit. Great Ideas in Information Theory, Language and Cybernetics. New York: Dover Publications, 1966. Print.
-
"Structural Bioinformatics Guide: Protein Structure, Protein Sequence Analysis and Alignment, Homology Modeling." Protein Structure and Structural Bioinformatics. N.p., n.d. Web. 25 Sept. 2016.
-
Kurzynski, M.; Chelminiak, P. Stochastic Dynamics of Proteins and the Action of Biological Molecular Machines. Entropy 2014, 16, 1969-1982.
-
Strait, B J, and T G Dewey. “The Shannon Information Entropy of Protein Sequences.” Biophysical Journal 71.1 (1996): 148–155. Print.
-
Heunen, Christiaan Johan Marie., Mehrnoosh Sadrzadeh, and Edward Grefenstette. Quantum Physics and Linguistics: A Compositional, Diagrammatic Discourse. Oxford, United Kingdom: Oxford UP, 2013. Print.