GuppyLM
Learning and science
educational llm implementation
SHOW HN2026
How HN took it
33 positive, 25 neutral or asking, 1 mixed, 6 critical, out of 65 comments about the project. Thread score +0.46.
Idea: 18 for, 2 againstExecution: 1 for, 4 againstDocs: 4 for, 0 againstDesign: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Great educational tool for learning how LLMs work
From 3 comments
It's genuinely a great introduction to LLMs. I built my own awhile ago based off Milton's Paradise Lost: https://www.wvrk.org/works/milton
ordinarily on HNThis is a nice idea. A tiny implementation can be way more useful for learning than yet another wrapper around a big model, especially if it keeps the training loop and inference…
rpdaiml on HNBuilding it yourself is always the best test if you really understand how it works.
EmilioOldenziel on HN
Fun personality and humor in the fish responses
From 2 comments
I love these kinds of educational implementations. I want to really praise the (unintentional?) nod to Nagel, by limiting capabilities to representation of a fish, the user is…
AndrewKemendo on HNI... wow, you made an LLM that can actually tell jokes?
gnarlouse on HN
Simple, readable code without abstractions
From 2 comments
The decision to strip out GQA/RoPE/SwiGLU and go vanilla transformer is the right call here — at 9M params those additions add complexity without meaningful gains, and keeping…
jiusanzhou on HNThe constraint-driven approach here is what makes it actually useful as a learning tool. When you're working with ~130 lines of PyTorch, you can't hide behind abstractions —…
areys on HN
Objections
Examples appear copied from training data and the model handles unknown queries poorly
From 2 comments
I like the idea, just that the examples are reproduced from the training data set. How does it handle unknown queries?
zwaps on HNIt mostly doesn't, at 9M it has very limited capacity. The whole idea of this project is to demonstrate how Language Models work.
armanified on HN
Title oversells it; the project does not really demystify anything
From 1 comment
Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier. Then, some criticism. I probably don't get it, but I…
Propelloni on HN
Seems like low-effort AI-generated project
From 1 comment
I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me. There is nothing wrong using AI tools to write code, but nothing here…
rahen on HN
Lowercase-only training data means uppercase input is ignored
From 3 comments
This is probably a consequence of the training data being fully lowercase: You> hello Guppy> hi. did you bring micro pellets. You> HELLO Guppy> i don't know what it means but…
mudkipdev on HNI love this! Seems like it can't understand uppercase letters though
Duplicake on HNUppercase letters were intentionally ignored.
armanified on HN
Requests
Documentation on how the synthetic data was generated
From 3 comments
I am trying to find how the synthetic data was created (looking through the repo) and didn't find it. Maybe I am missing it - Would love to see the prompts and process on that…
CaseFlatline on HNhow did you generate the synthetic data?
fawabc on HN* How creating dataset? I download it but it is commpresed in binary format. * How training. In cloud or in my own dev * How creating a gguf
gdzie-jest-sol on HN
Comparison with microgpt and minGPT
From 1 comment
How does this compare to Andrej Karpathy's microgpt (https://karpathy.github.io/2026/02/12/microgpt/) or minGPT (https://github.com/karpathy/minGPT)?
fg137 on HN
Write-up on how it was built and what was referenced
From 1 comment
This is so cool! I'd love to see a write-up on how made it, and what you referenced because designing neural networks always feel like a maze ;)
kaipereira on HN
Questions
How does it handle longer context and unknown queries?
From 2 comments
how's it handle longer context or does it start hallucinating after like 2 sentences? curious what the ceiling is before the 9M params
kubrador on HNI like the idea, just that the examples are reproduced from the training data set. How does it handle unknown queries?
zwaps on HN
How small can the training data and model get before output stops being coherent?
From 2 comments
How much training data did you end up needing for the fish personality to feel coherent? Curious what the minimum viable dataset looks like for something like this.
martmulx on HNThis is a great way to learn. Most people treat LLMs as black boxes, but building one from scratch — even tiny — forces you to understand attention, tokenization, and loss at a…
LeonTing1010 on HN
Can it be trained through conversation alone?
From 1 comment
Could it be possible to train LLM only through the chat messages without any other data or input? If Guppy doesn't know regular expressions yet, could I teach it to it just by…
bblb on HN
Alternatives named
Karpathy's microgpt and minGPT
From 1 comment
How does this compare to Andrej Karpathy's microgpt (https://karpathy.github.io/2026/02/12/microgpt/) or minGPT (https://github.com/karpathy/minGPT)?
fg137 on HN
bbycroft.net LLM 3D visualizer
From 1 comment
https://bbycroft.net/llm has 3d Visualization of tiny example LLM layers that do a very good job at showing what is going on (https://news.ycombinator.com/item?id=38505211)
totetsu on HN
rasbt LLMs-from-scratch
From 1 comment
Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier. Then, some criticism. I probably don't get it, but I…
Propelloni on HN