Primer on GPUs and kernel optimization
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
A primer on GPUs and kernel optimization, first of a three-part series from an RL framework called Jam, posted to Hacker News.
Commenters pressed the author on how generated kernels should be benchmarked and evaluated.
Background
The primer is the first of a three-part series documenting GPU kernel optimization work from a custom RL (reinforcement learning) framework called Jam, and covers GPU architecture. It was submitted to Hacker News by user nomi_uw and also discussed on Reddit's r/hackernews, which described it as covering GPU basics, profilers, distributed computation and kernel optimization. The Hacker News thread centered on how kernels should be benchmarked and evaluated.
How it unfolded
- Nov 9, 2026The primer on GPUs and kernel optimization was published as the first of a three-part series documenting work from the custom RL framework Jam.
- Nov 9, 2026It was posted to Hacker News (nabo.dev), reaching 84 points with 5 comments, and described on Reddit's r/hackernews as covering GPU basics, profilers, distributed computation and kernel optimization.
- Nov 10, 2026A commenter disputed a characterization of the post, saying it is clearly about human-guided optimization with LLMs rather than hillclimbing.
Key terms
- GPU
- A specialized electronic circuit that accelerates digital image processing and computer graphics, and is also used for AI computation.
- RL
- A machine learning approach in which an agent learns actions that maximize a reward signal in a dynamic environment.
Source: Wikipedia
Who’s saying what
- Caution
- Commenter augment_me said it was 'crazy to not see a word about benchmarking/evaluation of the kernels,' calling benchmarking the largest source of error when performant kernels are generated, and asked about hot vs cold cache, bitwise equivalence to a reference, error margins, thermal noise, separate GPUs, cudagraph compilation and A/B B/A interleaving.
- Expert
- Commenter Farmadupe asked whether thresholds should be set on individual ops, such as 99% of l2norms within 1e-4 error of a golden reference kernel, or on task-based evals such as word error no worse than 1% above a reference speech-to-text model.
- Analysts
- augment_me replied that the answer depends on training versus inference and on scale, suggesting end-to-end evaluation on the target task for inference on a 3090 and noting numerics and correctness matter more for training kernels and backwards passes.
- Public
- Commenter kollegekid praised the clarity of the writing and asked whether LLMs were used to write it.
Still unverified
Commenter saagarjha guessed that LLMs helped with the diagrams but that the writing is mostly or completely human; this was stated as a guess.