AncientML is a series of paper reading notes. The purpose is to review outstanding contributions to machine learning that are valuable to the formation as an academic field.

## A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955 (McCarthy et al., 2006), PDF

- The paper/event that gets credited with the foundation of the field of Artificial Intelligence research.
- The paper is three pages long and the authors include Claude Shannon.
- scale of the proposed project: 2 months, 10 men
- focused on language, abstraction and concepts
- identifies seven areas to improve: Automatic Computers, How Can a Computer be Programmed to Use a Language, Neuron Nets, Theory of the Size of a Calculation, Self-Improvement, Abstractions, Randomness and Creativity
- "the major obstacle is not lack of machine capacity, but our inability to write programs"
- There is Wikipedia article on the Dartmouth workshop.
- 102 pages of Ray Solomonoff's hand written notes including some doodles on page 3.

## The Mathematical Theory of Communication (Shannon et al., 1951), PDF

- Central paper for many fields. 90 pages (skip the part by Weaver).
*The Idea Factory*(Gertner, 2012) is a book about Bell Labs around that time.- Khinchin (1957) is a book that discusses this paper.
- p.49:
*information*is not attached to a particular message but to the amount of freedom of choice - p.49: "decomposition of choice" is a beautiful requirement for \(H\), and leads with the other two requirements to a unique form for \(H\)
- p.50: simple example to visualize the connection between probability of a message and information is shown in the figure below
- p.53: origin for terms of the form \(p_i\log{}p_i\)
- p.56: relative entropy, maximum possible compression, redundancy
- p.70: capacity of a noisy channel; includes a
`max()`

over all possible information sources

## Bibliography

Jon Gertner.
*The Idea Factory: Bell Labs and the great age of American innovation*.
Penguin Press, New York, 2012.
ISBN 978-0143122791. ↩

A Khinchin.
*Mathematical foundations of information theory*.
Dover Publications, New York, 1957.
ISBN 978-0486604343. ↩

John McCarthy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon.
A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955.
*AI magazine*, 27(4):12, 2006. ↩

Claude E Shannon, Warren Weaver, and Arthur W Burks.
The Mathematical Theory of Communication.
*The University of Illinois Press*, 1951. ↩