flash-attention-minimal
AI
ml kernels
SHOW HN2024
230points
39comments
#324of 3,515 in 2024
Top 10%of 2024 by points
How HN took it
DebatedReal support and real criticism
2 positive, 4 neutral or asking, 0 mixed, 2 critical, out of 8 comments about the project. Thread score +0.02.
Execution: 1 for, 1 againstDocs: 0 for, 1 againstIdea: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Neat, clear, minimal implementation
From 3 comments
Fantastic work! Extremely neat and clear implementation! Interesting note on the backward pass - what do you think are the main blockers for a backward pass?
danielhanchen on HNPretty neat implementation. In general, for these sort of exercises (and even if the intention is to go to prod with custom kernels) I lean towards Triton to write the kernels…
treesciencebot on HNThis is fantastic. I am just starting in the ML space (compile from compilers) and I love short kernels that I can use to understand things better with.
zer0zzz on HN
Objections
Line count is misleading given reliance on PyTorch codebase
From 1 comment
My GPU work is not in ML (deep or otherwise); but ... 1. "100 lines of CUDA" + PyTorch; maybe this is useful and maybe it isn't, but counting lines of code on top of a huge…
einpoklum on HN
Synchronous default-stream kernel launches are not ideal GPU use
From 1 comment
My GPU work is not in ML (deep or otherwise); but ... 1. "100 lines of CUDA" + PyTorch; maybe this is useful and maybe it isn't, but counting lines of code on top of a huge…
einpoklum on HN
Requests
Explanations of ML concepts for non-ML readers
From 3 comments
Regarding your comment about how fast the research and industry is moving, would HN readers be interested in relevant one or two paragraph summaries that are basically "explain…
robrenaud on HNThat sounds at least somewhat helpful. Honestly, a gradient for some of this stuff would be nice. Explain to me like I'm: "five", "a high schooler", "a college grad (not…
araes on HNI would love an explanation for software enginners / CS majors who aren't familiar with ML. Last I studied ML was 2016 and that was stuff like decision trees, k nearest…
andoando on HN
Questions
What __syncthreads() does in the kernel
From 1 comment
What does __syncthreads() do here exactly? I'm new to CUDA, could get the overall idea of the FlashAttention paper but not the details.
saiojd on HN
Whether Triton is preferable to CUDA for custom kernels
From 2 comments
Pretty neat implementation. In general, for these sort of exercises (and even if the intention is to go to prod with custom kernels) I lean towards Triton to write the kernels…
treesciencebot on HNAs a person who finds CUDA extremely easy to write and integrate, what does Triton have to offer?
queuebert on HN
Alternatives named
Triton DSL
From 2 comments
Pretty neat implementation. In general, for these sort of exercises (and even if the intention is to go to prod with custom kernels) I lean towards Triton to write the kernels…
treesciencebot on HNThe DSL: https://openai.com/research/triton
p1esk on HN
CUTLASS
From 1 comment
yeah even the official flashattention is moving many implementations from cutlass to triton except for the main mha backward/forward pass
whimsicalism on HN
Halide
From 1 comment
If CPU/GPU execution speed is the goal while simultaneously code golfing the source size, https://halide-lang.org/ might have come in handy.
dcanelhas on HN