An Intro to AI

A list by Arthur Kamienski · 50 papers

Fifty papers that take you from Turing’s question to today’s reasoning models, in an order meant for a first read. Start with a few short essays, see how neural networks learn, follow language from word vectors to Transformers and large language models, then branch out to images, science and the engineering that makes it all run. Each paper has a theme and a line on why it’s worth your time.

183 citations among these papers

Computing Machinery and Intelligence
1950 · A. M. Turing
Step1
ThemeFoundations
Why read itTuring asks "can machines think?" and proposes the imitation game; still frames the debate.
Deep learning
2015 · Yann LeCun, Yoshua Bengio, Geoffrey E. Hinton
Step2
ThemeFoundations
Why read itThe three Turing Award winners' own overview of deep learning, written just as it took off.
A mathematical theory of communication
1963 · Claude E Shannon, Warren Weaver
Step3
ThemeFoundations
Why read itShannon defines information and entropy and builds the first statistical models of English text; the root of every language model.

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
1958 · F. Rosenblatt
Step4
ThemeFoundations
Why read itThe first learning neural network, and the hype cycle that followed.
Learning representations by back-propagating errors
1986 · David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams
Step5
ThemeFoundations
Why read itBackpropagation: how every neural network since has been trained.
Backpropagation Applied to Handwritten Zip Code Recognition
1989 · Yann LeCun et al.
Step6
ThemeVision
Why read itLeNet's first appearance: a convolutional network trained with backprop to read zip codes.
Long Short-Term Memory
1997 · Sepp Hochreiter, Jürgen Schmidhuber
Step7
ThemeSequences
Why read itGated memory that made recurrent networks work on long sequences; dominant until Transformers.
A Neural Probabilistic Language Model
2001 · Yoshua Bengio, R. Ducharme, Pascal Vincent
Step8
ThemeLanguage
Why read itBengio's neural language model: learned word embeddings and a network predicting the next word, the seed of every LLM.

Efficient Estimation of Word Representations in Vector Space
2013 · Tomas Mikolov et al.
Step9
ThemeLanguage
Why read itword2vec: words as vectors, where meaning becomes geometry (king - man + woman = queen).

Deep Sparse Rectifier Neural Networks
2011 · Xavier Glorot, Antoine Bordes, Yoshua Bengio
Step10
ThemeTraining
Why read itReLU: the simple activation that made deep networks much easier to train. Hinton (Nair & Hinton, 2010) used it first, but Bengio's group explains better why it works.
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
2014 · Nitish Srivastava et al.
Step11
ThemeTraining
Why read itA simple trick against overfitting, and a good intro to regularisation.
ImageNet Classification with Deep Convolutional Neural Networks
2012 · Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
Step12
ThemeVision
Why read itAlexNet: the result that started the deep learning boom.
Adam: A Method for Stochastic Optimization
2014 · Diederik P. Kingma, Jimmy Lei Ba
Step13
ThemeTraining
Why read itThe optimiser almost everyone uses; short and practical.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
2015 · Sergey Ioffe, Christian Szegedy
Step14
ThemeTraining
Why read itNormalise each layer's inputs while training, and deep networks suddenly train faster and more reliably. Its cousin, layer norm, sits inside every Transformer.
Deep Residual Learning for Image Recognition
2016 · Kaiming He et al.
Step15
ThemeVision
Why read itSkip connections let networks go hundreds of layers deep; they are in nearly every model today.

Sequence to Sequence Learning with Neural Networks
2014 · Ilya Sutskever, Oriol Vinyals, Quoc Viet Le
Step16
ThemeLanguage
Why read itEncoder-decoder networks: one recipe for translation, summarisation and more.
Neural Machine Translation by Jointly Learning to Align and Translate
2014 · Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Step17
ThemeLanguage
Why read itIntroduces attention, the idea the Transformer is built on.
Neural Machine Translation of Rare Words with Subword Units
2016 · Rico Sennrich, Barry Haddow, Alexandra Birch
Step18
ThemeLanguage
Why read itByte-pair encoding: split words into frequent pieces so a model can handle any word. The tokenizers of today's LLMs come from here.
Attention Is All You Need
2017 · Ashish Vaswani et al.
Step19
ThemeLanguage
Why read itThe Transformer: the architecture behind every large language model.

Improving Language Understanding by Generative Pre-Training
2018 · Alec Radford et al.
Step20
ThemeLarge language models
Why read itThe original GPT: pre-train a Transformer to predict the next word, then fine-tune; the start of the GPT line.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2018 · Jacob Devlin et al.
Step21
ThemeLanguage
Why read itPre-train once, fine-tune for anything: the template for NLP after 2018.
Language Models are Unsupervised Multitask Learners
2019 · Alec Radford et al.
Step22
ThemeLarge language models
Why read itA bigger GPT starts doing tasks it was never trained for, like translating and answering questions, just by predicting text; and the first model held back for fear of misuse.

The Bitter Lesson
2019 · Rich Sutton
Step23
ThemeFoundations
Why read itSutton's short essay: over 70 years, general methods that scale with compute (search and learning) beat clever human-built knowledge. A two-page read that explains the scaling era.
Language Models are Few-Shot Learners
2020 · Tom B. Brown et al.
Step24
ThemeLarge language models
Why read itGPT-3: scale alone gives models the ability to learn from a few examples in the prompt.
Scaling Laws for Neural Language Models
2020 · Jared Kaplan et al.
Step25
ThemeLarge language models
Why read itPerformance follows smooth power laws in size, data and compute; why labs keep scaling.
Training Compute-Optimal Large Language Models
2022 · Jordan Hoffmann et al.
Step26
ThemeLarge language models
Why read itChinchilla: most models were undertrained, and how to balance size against data.

Human-level control through deep reinforcement learning
2015 · Volodymyr Mnih et al.
Step27
ThemeReinforcement learning
Why read itDQN learns Atari games from pixels: the start of deep RL.
Mastering the game of Go with deep neural networks and tree search
2016 · David Silver et al.
Step28
ThemeReinforcement learning
Why read itAlphaGo: learning plus search beats the world's best Go players.
Deep Reinforcement Learning from Human Preferences
2017 · Paul F. Christiano et al.
Step29
ThemeReinforcement learning
Why read itThe original RLHF: learn a reward from people comparing two behaviours, later used to turn LLMs into assistants.

Training Language Models to Follow Instructions with Human Feedback
2022 · Long Ouyang et al.
Step30
ThemeLarge language models
Why read itInstructGPT and RLHF: how a raw language model becomes an assistant.
Direct Preference Optimization: Your Language Model Is Secretly a Reward Model
2023 · Rafael Rafailov et al.
Step31
ThemeLarge language models
Why read itDPO: RLHF without a reward model or RL, just a classification loss on preferred answers. Now the default way to align open models.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
2022 · Jason Wei et al.
Step32
ThemeLarge language models
Why read itAsking models to show their steps makes them reason better; the root of today's reasoning models.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
2020 · Patrick S. H. Lewis et al.
Step33
ThemeLarge language models
Why read itRAG: look documents up, then generate from them. The idea behind every chat-with-your-docs product.
ReAct: Synergizing Reasoning and Acting in Language Models
2023 · Shunyu Yao et al.
Step34
ThemeLarge language models
Why read itInterleave thinking and tool calls in one loop: the pattern behind today's LLM agents.
Why Language Models Hallucinate
2025 · Adam Tauman Kalai et al.
Step35
ThemeLarge language models
Why read itOpenAI argues hallucinations are what training and benchmarks reward: guessing scores better than saying "I don't know". A short, clear read on a problem everyone hits.

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
2024 · Zhihong Shao et al.
Step36
ThemeLarge language models
Why read itIntroduces GRPO, a cheaper RL method that scores answers against each other instead of training a critic. The method DeepSeek-R1 is built on.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
2025 · DeepSeek-AI
Step37
ThemeLarge language models
Why read itReasoning learned by reinforcement learning on verifiable answers, published openly; how today's reasoning models are made.
s1: Simple test-time scaling
2025 · Niklas Muennighoff et al.
Step38
ThemeLarge language models
Why read itA thousand good examples and forcing the model to keep thinking get close to o1: the cheapest demonstration that thinking longer pays.

Generative Adversarial Nets
2014 · Ian J. Goodfellow et al.
Step39
ThemeGenerative models
Why read itTwo networks competing to generate realistic data; a whole era of image generation.
Denoising Diffusion Probabilistic Models
2020 · Jonathan Ho, Ajay Jain, Pieter Abbeel
Step40
ThemeGenerative models
Why read itDiffusion models: the method behind today's image and video generators.
High-Resolution Image Synthesis with Latent Diffusion Models
2022 · Robin Rombach et al.
Step41
ThemeGenerative models
Why read itStable Diffusion: run diffusion in a compressed space, so image generation fits on a home GPU.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2021 · Alexey Dosovitskiy et al.
Step42
ThemeVision
Why read itVision Transformers: one architecture for both text and images.
Learning Transferable Visual Models From Natural Language Supervision
2021 · Alec Radford et al.
Step43
ThemeMultimodal
Why read itCLIP: images and text in one space, learned from captions on the web.
Highly accurate protein structure prediction with AlphaFold
2021 · John Jumper et al.
Step44
ThemeScience
Why read itAlphaFold 2: deep learning solves protein folding, a 50-year problem in biology, and wins a Nobel prize.

Distilling the Knowledge in a Neural Network
2015 · Geoffrey E. Hinton, Oriol Vinyals, Jeffrey Dean
Step45
ThemeTraining
Why read itTrain a small model to copy a big one's soft predictions: how today's small, fast models are made from frontier ones.
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
2017 · Benoit Jacob et al.
Step46
ThemeTraining
Why read itRun a network in 8-bit integers instead of floats with almost no loss: the root of the quantized models people run on laptops and phones.
LoRA: Low-Rank Adaptation of Large Language Models
2021 · Edward J. Hu et al.
Step47
ThemeTraining
Why read itFine-tune a huge model by training two small matrices per layer: how almost everyone adapts open models on one GPU.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
2022 · Tri Dao et al.
Step48
ThemeTraining
Why read itSame attention, computed with the GPU's memory in mind: much faster, and long contexts become affordable. A lesson that hardware matters as much as maths.
Efficient Memory Management for Large Language Model Serving with PagedAttention
2023 · Woosuk Kwon et al.
Step49
ThemeLarge language models
Why read itvLLM: manage the attention cache like virtual memory pages, serving many more requests per GPU. The engine under most open-model serving.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
2022 · William Fedus, Barret Zoph, Noam Shazeer
Step50
ThemeLarge language models
Why read itMixture-of-experts made simple: each token uses only a few of many experts, so models grow without costing more per token. The design of DeepSeek and most frontier models.

Paper	Year	Step	Theme	Why read it

Computing Machinery and Intelligence A. M. Turing	1950	1	Foundations	Turing asks "can machines think?" and proposes the imitation game; still frames the debate.
Deep learning Yann LeCun, Yoshua Bengio, Geoffrey E. Hinton	2015	2	Foundations	The three Turing Award winners' own overview of deep learning, written just as it took off.
A mathematical theory of communication Claude E Shannon, Warren Weaver	1963	3	Foundations	Shannon defines information and entropy and builds the first statistical models of English text; the root of every language model.

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain F. Rosenblatt	1958	4	Foundations	The first learning neural network, and the hype cycle that followed.
Learning representations by back-propagating errors David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams	1986	5	Foundations	Backpropagation: how every neural network since has been trained.
Backpropagation Applied to Handwritten Zip Code Recognition Yann LeCun et al.	1989	6	Vision	LeNet's first appearance: a convolutional network trained with backprop to read zip codes.
Long Short-Term Memory Sepp Hochreiter, Jürgen Schmidhuber	1997	7	Sequences	Gated memory that made recurrent networks work on long sequences; dominant until Transformers.
A Neural Probabilistic Language Model Yoshua Bengio, R. Ducharme, Pascal Vincent	2001	8	Language	Bengio's neural language model: learned word embeddings and a network predicting the next word, the seed of every LLM.

Efficient Estimation of Word Representations in Vector Space Tomas Mikolov et al.	2013	9	Language	word2vec: words as vectors, where meaning becomes geometry (king - man + woman = queen).

Deep Sparse Rectifier Neural Networks Xavier Glorot, Antoine Bordes, Yoshua Bengio	2011	10	Training	ReLU: the simple activation that made deep networks much easier to train. Hinton (Nair & Hinton, 2010) used it first, but Bengio's group explains better why it works.
Dropout: A Simple Way to Prevent Neural Networks from Overfitting Nitish Srivastava et al.	2014	11	Training	A simple trick against overfitting, and a good intro to regularisation.
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton	2012	12	Vision	AlexNet: the result that started the deep learning boom.
Adam: A Method for Stochastic Optimization Diederik P. Kingma, Jimmy Lei Ba	2014	13	Training	The optimiser almost everyone uses; short and practical.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe, Christian Szegedy	2015	14	Training	Normalise each layer's inputs while training, and deep networks suddenly train faster and more reliably. Its cousin, layer norm, sits inside every Transformer.
Deep Residual Learning for Image Recognition Kaiming He et al.	2016	15	Vision	Skip connections let networks go hundreds of layers deep; they are in nearly every model today.

Sequence to Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc Viet Le	2014	16	Language	Encoder-decoder networks: one recipe for translation, summarisation and more.
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio	2014	17	Language	Introduces attention, the idea the Transformer is built on.
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich, Barry Haddow, Alexandra Birch	2016	18	Language	Byte-pair encoding: split words into frequent pieces so a model can handle any word. The tokenizers of today's LLMs come from here.
Attention Is All You Need Ashish Vaswani et al.	2017	19	Language	The Transformer: the architecture behind every large language model.

Improving Language Understanding by Generative Pre-Training Alec Radford et al.	2018	20	Large language models	The original GPT: pre-train a Transformer to predict the next word, then fine-tune; the start of the GPT line.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin et al.	2018	21	Language	Pre-train once, fine-tune for anything: the template for NLP after 2018.
Language Models are Unsupervised Multitask Learners Alec Radford et al.	2019	22	Large language models	A bigger GPT starts doing tasks it was never trained for, like translating and answering questions, just by predicting text; and the first model held back for fear of misuse.

The Bitter Lesson Rich Sutton	2019	23	Foundations	Sutton's short essay: over 70 years, general methods that scale with compute (search and learning) beat clever human-built knowledge. A two-page read that explains the scaling era.
Language Models are Few-Shot Learners Tom B. Brown et al.	2020	24	Large language models	GPT-3: scale alone gives models the ability to learn from a few examples in the prompt.
Scaling Laws for Neural Language Models Jared Kaplan et al.	2020	25	Large language models	Performance follows smooth power laws in size, data and compute; why labs keep scaling.
Training Compute-Optimal Large Language Models Jordan Hoffmann et al.	2022	26	Large language models	Chinchilla: most models were undertrained, and how to balance size against data.

Human-level control through deep reinforcement learning Volodymyr Mnih et al.	2015	27	Reinforcement learning	DQN learns Atari games from pixels: the start of deep RL.
Mastering the game of Go with deep neural networks and tree search David Silver et al.	2016	28	Reinforcement learning	AlphaGo: learning plus search beats the world's best Go players.
Deep Reinforcement Learning from Human Preferences Paul F. Christiano et al.	2017	29	Reinforcement learning	The original RLHF: learn a reward from people comparing two behaviours, later used to turn LLMs into assistants.

Training Language Models to Follow Instructions with Human Feedback Long Ouyang et al.	2022	30	Large language models	InstructGPT and RLHF: how a raw language model becomes an assistant.
Direct Preference Optimization: Your Language Model Is Secretly a Reward Model Rafael Rafailov et al.	2023	31	Large language models	DPO: RLHF without a reward model or RL, just a classification loss on preferred answers. Now the default way to align open models.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason Wei et al.	2022	32	Large language models	Asking models to show their steps makes them reason better; the root of today's reasoning models.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Patrick S. H. Lewis et al.	2020	33	Large language models	RAG: look documents up, then generate from them. The idea behind every chat-with-your-docs product.
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao et al.	2023	34	Large language models	Interleave thinking and tool calls in one loop: the pattern behind today's LLM agents.
Why Language Models Hallucinate Adam Tauman Kalai et al.	2025	35	Large language models	OpenAI argues hallucinations are what training and benchmarks reward: guessing scores better than saying "I don't know". A short, clear read on a problem everyone hits.

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao et al.	2024	36	Large language models	Introduces GRPO, a cheaper RL method that scores answers against each other instead of training a critic. The method DeepSeek-R1 is built on.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI	2025	37	Large language models	Reasoning learned by reinforcement learning on verifiable answers, published openly; how today's reasoning models are made.
s1: Simple test-time scaling Niklas Muennighoff et al.	2025	38	Large language models	A thousand good examples and forcing the model to keep thinking get close to o1: the cheapest demonstration that thinking longer pays.

Generative Adversarial Nets Ian J. Goodfellow et al.	2014	39	Generative models	Two networks competing to generate realistic data; a whole era of image generation.
Denoising Diffusion Probabilistic Models Jonathan Ho, Ajay Jain, Pieter Abbeel	2020	40	Generative models	Diffusion models: the method behind today's image and video generators.
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach et al.	2022	41	Generative models	Stable Diffusion: run diffusion in a compressed space, so image generation fits on a home GPU.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy et al.	2021	42	Vision	Vision Transformers: one architecture for both text and images.
Learning Transferable Visual Models From Natural Language Supervision Alec Radford et al.	2021	43	Multimodal	CLIP: images and text in one space, learned from captions on the web.
Highly accurate protein structure prediction with AlphaFold John Jumper et al.	2021	44	Science	AlphaFold 2: deep learning solves protein folding, a 50-year problem in biology, and wins a Nobel prize.

Distilling the Knowledge in a Neural Network Geoffrey E. Hinton, Oriol Vinyals, Jeffrey Dean	2015	45	Training	Train a small model to copy a big one's soft predictions: how today's small, fast models are made from frontier ones.
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob et al.	2017	46	Training	Run a network in 8-bit integers instead of floats with almost no loss: the root of the quantized models people run on laptops and phones.
LoRA: Low-Rank Adaptation of Large Language Models Edward J. Hu et al.	2021	47	Training	Fine-tune a huge model by training two small matrices per layer: how almost everyone adapts open models on one GPU.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao et al.	2022	48	Training	Same attention, computed with the GPU's memory in mind: much faster, and long contexts become affordable. A lesson that hardware matters as much as maths.
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon et al.	2023	49	Large language models	vLLM: manage the attention cache like virtual memory pages, serving many more requests per GPU. The engine under most open-model serving.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity William Fedus, Barret Zoph, Noam Shazeer	2022	50	Large language models	Mixture-of-experts made simple: each token uses only a few of many experts, so models grow without costing more per token. The design of DeepSeek and most frontier models.