[{"data":1,"prerenderedAt":234},["ShallowReactive",2],{"project-48328184":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":41,"post_text":44,"what":45,"said":80,"maker_in_thread":144,"shape":147,"grid":150,"attempts":196,"same_week":197,"maker_shows":42,"maker_hidden":226,"reader_version":227,"card":228},"48328184","Tiny-vLLM","high performance LLM inference engine in C++ and CUDA","title","Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA","https:\u002F\u002Fgithub.com\u002Fjmaczan\u002Ftiny-vllm","github.com","yu3zhou4",1780083507,2026,205,18,280,4366,7,"notable","ai","inference engine","library",[24,25,26,27,28],"llm inference","cuda","c++ inference engine","kv cache","llm from scratch","loved",0.516,{"positive":32,"neutral":33,"mixed":34,"critical":34},10,2,1,14,{"docs":37,"execution":40},{"positive":38,"critical":39},8,0,{"positive":39,"critical":34},[39,42,42,43,42,42,42,39,39,42,42,42,42,43,43,39,39,34,42],5,3,null,{"problem":46,"audience":53,"pricing":44,"motivation":65,"time_to_build":44,"stack":69,"openness":79,"stage":79,"team_size":79},{"value":47,"source":48,"evidence":49},"Teaches how LLM inference works internally so people can rebuild it","maker",[50],{"id":51,"by":11,"text":52},"48328913","README is in my opinion (author here) the most interesting - I wrote it to help others build useful mental model to be able to recreate the project yourself, without need to even…",{"value":54,"source":48,"evidence":55},"Developers learning LLM inference internals",[56,57,61],{"id":51,"by":11,"text":52},{"id":58,"by":59,"text":60},"48331668","janalsncm","Really practical teaching approach. I clicked in to see how safetensors are loaded and just kept reading. Thanks for sharing.",{"id":62,"by":63,"text":64},"48337238","lukemerrick","I am not super familiar with C and CUDA, so I read solely for the README and enjoyed it supremely. The blend of cheerful walking through instructive examples and your…",{"value":66,"source":48,"evidence":67},"Help others build a useful mental model to recreate the project themselves",[68],{"id":51,"by":11,"text":52},[70,75],{"value":71,"source":48,"evidence":72},"C++",[73],{"id":74},"post",{"value":76,"source":48,"evidence":77},"CUDA",[78],{"id":74},"unknown",{"praise":81,"objections":112,"requests":127,"questions":135,"alternatives":136},[82,103],{"value":83,"source":84,"evidence":85},"README is a clear, practical lesson-style walkthrough of inference internals","commenter",[86,87,91,95,99],{"id":58,"by":59,"text":60},{"id":88,"by":89,"text":90},"48363579","paralleliq","Great README. Genuinely one of the clearest walkthrough of inference internals. The KV cache section is worth lingering one as most of the OOM and throughput issues trace back to…",{"id":92,"by":93,"text":94},"48331721","xuanlin314","The lesson-style README is a great approach. Breaking down LLM inference into digestible steps makes the codebase approachable even for people who haven't touched CUDA before.",{"id":96,"by":97,"text":98},"48328953","nazgulsenpai","I love the documentation formatted in lessons. I can't wait to read through it.",{"id":100,"by":101,"text":102},"48333867","tom-wal","I feel like I learned twice as much in 10 minutes reading this than I did reading LLM for Dummies. Thank you",{"value":104,"source":84,"evidence":105},"Approachable for people new to CUDA and LLMs",[106,107,108],{"id":62,"by":63,"text":64},{"id":92,"by":93,"text":94},{"id":109,"by":110,"text":111},"48332029","GoldenJade","Thanks for sharing this. As someone currently researching LLMs, I'm sure I'll be referencing this quite a bit going forward.",[113,120],{"value":114,"source":84,"evidence":115},"README is dense and could use schemas",[116],{"id":117,"by":118,"text":119},"48355439","quanglee","love the details you put into to explain different techniques. it's a bit dense though, some schemas will help i think",{"value":121,"source":84,"evidence":122},"Author skips CUDA API return value checks, against the tiny goal",[123],{"id":124,"by":125,"text":126},"48330003","einpoklum","It seems the author believes checking the return values of CUDA API calls is not \"tiny\" enough :-(",[128],{"value":129,"source":84,"evidence":130},"Wants C or assembly LLM inference implementations",[131],{"id":132,"by":133,"text":134},"48334472","sylware","I am looking at a plain and simple C implemented LLM inference, and\u002For x86_64 assembly implemented, and\u002For AMD GPU RDNA assembly. Anybody?",[],[137],{"value":138,"source":84,"evidence":139},"llama.cpp",[140],{"id":141,"by":142,"text":143},"48329707","juancn","Looks interesting, it reminds me of the first llama.cpp, but better documented.",{"comments":34,"answered":39,"key":145},[146],{"id":51,"text":52},{"commenters":15,"max_depth":43,"maker_replies":34,"first_comment_mins":148,"lifespan_hours":149},61,75,[151,153,154,155,157,161,162,165,169,172,174,175,177,178,180,183,187,189,191],[51,39,39,152,11,34],"project\u002Fpositive\u002Fdocs",[58,39,42,152,59,39],[62,39,42,152,63,39],[117,39,43,156,118,39],"project\u002Fmixed\u002Fdocs",[158,34,42,159,160,39],"48330136","maker\u002Fpositive\u002Fnone","cookiengineer",[88,33,42,152,89,39],[163,43,42,152,164,39],"48329982","dwa3592",[166,43,39,167,168,39],"48330037","other\u002Fneutral\u002Fnone","cyanydeez",[170,43,39,167,171,39],"48333760","nullpoint420",[92,173,42,152,93,39],4,[109,42,42,152,110,39],[100,176,42,152,101,39],6,[96,18,42,152,97,39],[141,38,43,179,142,39],"project\u002Fneutral\u002Fdocs",[132,181,43,182,133,39],9,"project\u002Fneutral\u002Fnone",[184,181,39,185,186,39],"48335775","commenter\u002Fneutral\u002Fnone","irishcoffee",[188,181,39,185,133,39],"48344855",[124,32,34,190,125,39],"project\u002Fcritical\u002Fexecution",[192,193,42,194,195,39],"48337813",11,"project\u002Fpositive\u002Fnone","smy_smy",[],[198,203,207,211,216,221],{"id":199,"name":200,"points":201,"category":202},"48304260","Hallucinate",442,"games",{"id":204,"name":205,"points":206,"category":202},"48308376","Continue? Y\u002FN",386,{"id":208,"name":209,"points":210,"category":202},"48367723","Eyeball",295,{"id":212,"name":213,"points":214,"category":215},"48374552","Edsger",264,"dev_tools",{"id":217,"name":218,"points":219,"category":220},"48344327","Test-Drive-3-Maps",215,"learning_science",{"id":222,"name":223,"points":224,"category":225},"48325624","TV Explorer",198,"consumer_social",false,"v2",{"id":229,"name":5,"line":6,"ai":226,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":230,"top":18,"kw":231,"maker":11,"domain":10,"tier":19,"attempts":39,"strip":232,"url":9,"mix":233,"praise":83,"objection":114,"sub":21},48328184,0.52,[24,25,26,27,28],"0553555005555330015",[32,33,34,34],1794036701822]