Enter to search with the selected external provider · Press / anywhere to focus search
Enter opens your selected web provider in a new tab
External jump (enhancement)
Enter = open in new tab
Strengths
- Top AI researchers explain in person
- Build from scratch and gain a deep understanding of the principles
- The code is clear and can be run directly
- The explanation is simple and in-depth, suitable for learners with a certain foundation.
- Free and open to the public, quality comparable to top courses
Best for
- In-depth understanding of neural network and deep learning principles
- Learn to build Transformer models like GPT from scratch
- Understand how LLM works
- Improve the theoretical and practical capabilities of AI/ML
- Understand cutting-edge thinking in the field of AI
Core course recommendations
A few of Karpathy’s most important video lessons.
Scenario
Neural Networks: Zero to Hero Series
Prompt example
This is Karpathy’s most important course series, Build a neural network from scratch to GPT: 1. The spelled-out introduction to neural networks - Build a tiny neural network library from scratch - Understand backpropagation - Duration: approximately 2.5 hours 2.Building makemore -Character level language model - From bigram to MLP - Duration: approximately 1.5 hours 3. Building a GPT from scratch - Complete implementation of Transformer - Understand the attention mechanism - Duration: approximately 2 hours 4. Let's build the GPT Tokenizer - Understand BPE tokenization - Duration: approximately 2 hours GitHub: github.com/karpathy/nn-zero-to-hero
Output / what to expect
After completing this series,
You will truly understand how GPT works,
Rather than just calling the API.
Tips
It is recommended to write the code by hand while reading. Karpathy's code is on GitHub and can be run directly.
Scenario
Key videos for understanding LLM
Prompt example
A few important videos for understanding LLM: 1. "Intro to Large Language Models" (2023) - 1 hour overview of LLM - Suitable for those with technical background - Explain the capabilities, limitations and future of LLM 2. "State of GPT" (Microsoft Build 2023) - GPT training process - How RLHF works - How to use GPT effectively 3. "Let's reproduce GPT-2" (2024) - Complete reproduction of GPT-2 - Contains training process - Duration: approximately 4 hours Suitable for the crowd: - Have Python basics - Understand basic machine learning concepts - Want to deeply understand the principles of LLM
Output / what to expect
These videos provide the most in-depth understanding of LLM,
“Intro to LLMs” is one of the best introductory videos to LLM,
Even professional researchers recommend viewing.
Tips
You can first read "Intro to LLMs" to get the overall concept, and then study the specific implementation in depth.