AncientML is a series of paper reading notes. The purpose is to review outstanding contributions to machine learning that are valuable to the formation as an academic field.
A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955 (McCarthy et al., 2006), PDF
- The paper/event that gets credited with the foundation of the field of Artificial Intelligence research.
- The paper is three pages long and the authors include Claude Shannon.
- scale of the proposed project: 2 months, 10 men
- focused on language, abstraction and concepts
- identifies seven areas to improve: Automatic Computers, How Can a Computer be Programmed to Use a Language, Neuron Nets, Theory of the Size of a Calculation, Self-Improvement, Abstractions, Randomness and Creativity
- "the major obstacle is not lack of machine capacity, but our inability to write programs"
- There is Wikipedia article on the Dartmouth workshop.
- 102 pages of Ray Solomonoff's hand written notes including some doodles on page 3.
The Mathematical Theory of Communication (Shannon et al., 1951), PDF
- Central paper for many fields. 90 pages (skip the part by Weaver).
- The Idea Factory (Gertner, 2012) is a book about Bell Labs around that time.
- Khinchin (1957) is a book that discusses this paper.
- p.49: information is not attached to a particular message but to the amount of freedom of choice
- p.49: "decomposition of choice" is a beautiful requirement for \(H\), and leads with the other two requirements to a unique form for \(H\)
- p.50: simple example to visualize the connection between probability of a message and information is shown in the figure below
- p.53: origin for terms of the form \(p_i\log{}p_i\)
- p.56: relative entropy, maximum possible compression, redundancy
- p.70: capacity of a noisy channel; includes a
max()over all possible information sources
Bibliography
Jon Gertner. The Idea Factory: Bell Labs and the great age of American innovation. Penguin Press, New York, 2012. ISBN 978-0143122791. ↩
A Khinchin. Mathematical foundations of information theory. Dover Publications, New York, 1957. ISBN 978-0486604343. ↩
John McCarthy, Marvin L Minsky, Nathaniel Rochester, and Claude E Shannon. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. AI magazine, 27(4):12, 2006. ↩
Claude E Shannon, Warren Weaver, and Arthur W Burks. The Mathematical Theory of Communication. The University of Illinois Press, 1951. ↩
Go Top