←Articles

AncientML is a series of paper reading notes. The purpose is to review outstanding contributions to machine learning that are valuable to the formation as an academic field.

## A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955 (McCarthy et al., 2006), PDF

• The paper/event that gets credited with the foundation of the field of Artificial Intelligence research.
• The paper is three pages long and the authors include Claude Shannon.
• scale of the proposed project: 2 months, 10 men
• focused on language, abstraction and concepts
• identifies seven areas to improve: Automatic Computers, How Can a Computer be Programmed to Use a Language, Neuron Nets, Theory of the Size of a Calculation, Self-Improvement, Abstractions, Randomness and Creativity
• "the major obstacle is not lack of machine capacity, but our inability to write programs"
• There is Wikipedia article on the Dartmouth workshop.
• 102 pages of Ray Solomonoff's hand written notes including some doodles on page 3.

## The Mathematical Theory of Communication (Shannon et al., 1951), PDF

• Central paper for many fields. 90 pages (skip the part by Weaver).
• The Idea Factory (Gertner, 2012) is a book about Bell Labs around that time.
• Khinchin (1957) is a book that discusses this paper.
• p.49: information is not attached to a particular message but to the amount of freedom of choice
• p.49: "decomposition of choice" is a beautiful requirement for $$H$$, and leads with the other two requirements to a unique form for $$H$$
• p.50: simple example to visualize the connection between probability of a message and information is shown in the figure below
• p.53: origin for terms of the form $$p_i\log{}p_i$$
• p.56: relative entropy, maximum possible compression, redundancy
• p.70: capacity of a noisy channel; includes a max() over all possible information sources

## Bibliography

Jon Gertner. The Idea Factory: Bell Labs and the great age of American innovation. Penguin Press, New York, 2012. ISBN 978-0143122791.

A Khinchin. Mathematical foundations of information theory. Dover Publications, New York, 1957. ISBN 978-0486604343.

John McCarthy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. AI magazine, 27(4):12, 2006.

Claude E Shannon, Warren Weaver, and Arthur W Burks. The Mathematical Theory of Communication. The University of Illinois Press, 1951.

Go Top