ÉtudeScience

Computer Science & AI · 1948

A Mathematical Theory of Communication

Claude E. Shannon

Bell Telephone Laboratories

The Bell System Technical Journal, vol. 27, pp. 379–423, 623–656 ·

Cited by 73,000+
View original paper

Opens the version of record via DOI

Shannon founded information theory. He defined information quantitatively in terms of entropy, introduced the 'bit', and proved fundamental limits on how much data a noisy channel can carry — and that error-free communication is possible up to that limit.

Created information theory, the basis of all digital communication and storage.

  • Defined information entropy as the measure of uncertainty in a message source.
  • Established channel capacity — the maximum reliable data rate of a noisy channel.
  • Proved coding theorems showing error-free transmission is achievable below capacity.

A probabilistic and mathematical framework modelling sources and channels as stochastic processes, with theorems deriving entropy, capacity, and the existence of optimal source and channel codes.

Keywords

Computer Science & AI

Highly Accurate Protein Structure Prediction with AlphaFold

Jumper et al. · 2021 · Nature

AlphaFold solved a 50-year grand challenge in biology: predicting a protein's three-dimensional shape from its amino-acid sequence. Its accuracy approached that of experimental methods, transforming structural biology.

Cited by 37,000+Open access

Computer Science & AI

Attention Is All You Need

Vaswani et al. · 2017 · Advances in Neural Information Processing Systems (NeurIPS)

The paper that introduced the Transformer, an architecture built entirely on attention mechanisms with no recurrence or convolution. It became the foundation of modern large language models such as GPT and BERT.

Cited by 180,000+Open access

Computer Science & AI

Mastering the Game of Go with Deep Neural Networks and Tree Search

Silver et al. · 2016 · Nature

AlphaGo became the first program to beat a professional human at the ancient board game Go, long considered a grand challenge for AI because of its astronomical complexity. It combined deep neural networks with Monte Carlo tree search.

Cited by 19,000+Open access

Étude Science indexes and summarises this work; it is not the publisher. The summary above is written by Étude. For the definitive text, figures, and data, please consult the original publication via the link above. Shannon (1948) hold the rights to the original work.