[{"data":1,"prerenderedAt":293},["ShallowReactive",2],{"project-49613068":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":43,"post_text":44,"what":45,"said":69,"maker_in_thread":160,"shape":169,"grid":174,"attempts":254,"same_week":255,"maker_shows":40,"maker_hidden":285,"reader_version":286,"card":287},"49613068","LLM Attention Visualization","Thanks for trying it out!","maker","Show HN: LLM Attention Visualization","https:\u002F\u002Fishamf.dev\u002Fp\u002Fllm-attention-visualizer\u002F","ishamf.dev","ifz",1788886781,2026,177,32,343,4366,8,"notable","ai","model visualization","web_app",[24,25,26,27,28],"llm attention","transformer visualization","self attention","attention heads","kv cache","mixed",0.296,{"positive":32,"neutral":33,"mixed":34,"critical":35},9,5,0,3,17,{"idea":38,"design":41},{"positive":39,"critical":40},6,1,{"positive":40,"critical":42},2,[33,40,34,33,40,35,33,34,34,34,34,34,34,34,34,34,40,34,34,35,33,34,33,33,33,35,35,34,33,35,34,33],null,{"problem":46,"audience":58,"pricing":44,"motivation":44,"time_to_build":44,"stack":66,"openness":67,"stage":68,"team_size":67},{"value":47,"source":48,"evidence":49},"Makes the attention mechanism intuitive to learn and explain","commenter",[50,54],{"id":51,"by":52,"text":53},"49617390","MCP123","This is great, thank you. I have to teach this stuff on Friday so perfect timing. It's hard to explain the attention mechanism in a way that becomes intuitive because the…",{"id":55,"by":56,"text":57},"49614910","fuddle","This is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.",{"value":59,"source":48,"evidence":60},"Students and teachers learning how transformer attention works",[61,62],{"id":51,"by":52,"text":53},{"id":63,"by":64,"text":65},"49639477","sshussain270","This is very cool. It's simpler than Bertviz for understanding inference and surface level and a good starting people for new learners as well.",[],"unknown","launched",{"praise":70,"objections":88,"requests":99,"questions":129,"alternatives":148},[71,77],{"value":72,"source":48,"evidence":73},"Clear, intuitive way to understand attention",[74,75,76],{"id":51,"by":52,"text":53},{"id":55,"by":56,"text":57},{"id":63,"by":64,"text":65},{"value":78,"source":48,"evidence":79},"Simple and visualizable",[80,84],{"id":81,"by":82,"text":83},"49621855","talhaanwar","thanks for making it simple and visualizable",{"id":85,"by":86,"text":87},"49614661","itsnasme","I like the visualisation. Pretty cool",[89],{"value":90,"source":48,"evidence":91},"Weighting by value vector magnitude is a simplistic measure of influence",[92,96],{"id":93,"by":94,"text":95},"49613902","sva_","I highly question this simplistic idea of high vector magnitude = high influence.",{"id":97,"by":11,"text":98},"49614075","I don't disagree with that. I did add an entire caveat paragraph there. To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even…",[100,111,116,122],{"value":101,"source":48,"evidence":102},"Pause should pause playback instead of resetting",[103,107],{"id":104,"by":105,"text":106},"49619436","mncharity","UX report. I wished to examine attention state step by step, but I found the animation moved along too fast for that. So I tried pausing... On Chromium\u002Flinux, pressing pause…",{"id":108,"by":109,"text":110},"49671146","Mugshelf","The pause-on-load problem is a real UX trap. Stepping through one token at a time is how people actually learn attention; single-step controls would help more than autoplay.",{"value":112,"source":48,"evidence":113},"Single-step controls",[114,115],{"id":104,"by":105,"text":106},{"id":108,"by":109,"text":110},{"value":117,"source":7,"evidence":118},"Controls to limit which layers are summed",[119],{"id":120,"by":11,"text":121},"49615117","Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns. Right now only simple correlations are visible.",{"value":123,"source":48,"evidence":124},"Per-head color layers instead of summed view",[125],{"id":126,"by":127,"text":128},"49625338","nullbio","Might be cool to try different colors for the different attention heads instead of summing them across all attention heads, and making the backgrounds composed of stacked color…",[130,141],{"value":131,"source":48,"evidence":132},"What is the actual formula",[133,137],{"id":134,"by":135,"text":136},"49614902","stared","I am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted…",{"id":138,"by":139,"text":140},"49620648","asd000hh","How it works?",{"value":142,"source":48,"evidence":143},"Whether later layers get drowned out by earlier ones",[144],{"id":145,"by":146,"text":147},"49614860","wopak","neat, combining info from two phrases is hard to see without such a tool. are you worried later-layer attention gets drowned out by earlier layers just because there are more of…",[149,153],{"value":150,"source":48,"evidence":151},"BertViz",[152],{"id":63,"by":64,"text":65},{"value":154,"source":48,"evidence":155},"Transformer circuits attention QK analysis",[156],{"id":157,"by":158,"text":159},"49614254","smallmancontrov","You get what you pay for. If you want to think harder and get more, https:\u002F\u002Ftransformer-circuits.pub\u002F2025\u002Fattention-qk\u002Findex.html",{"comments":33,"answered":35,"key":161},[162,165,168],{"id":163,"text":164},"49647611","Thanks for trying it out! You can hover \u002F click \u002F tap on any word to focus it. I actually intended for this to be main way to use the visualization, I just added the…",{"id":166,"text":167},"49647637","Oh, I actually didn't know about Bertviz before. That looks interesting as well!",{"id":97,"text":98},{"commenters":170,"max_depth":171,"maker_replies":33,"first_comment_mins":172,"lifespan_hours":173},24,4,51,90,[175,177,179,181,182,183,187,188,190,194,198,201,203,205,208,210,214,216,218,220,223,224,225,228,233,235,238,240,242,245,247,250],[51,34,33,176,52,34],"project\u002Fpositive\u002Fidea",[104,40,40,178,105,34],"project\u002Fcritical\u002Fdesign",[163,40,34,180,11,40],"maker\u002Fneutral\u002Fdesign",[55,42,33,176,56,34],[108,35,40,178,109,34],[184,171,35,185,186,34],"49619594","project\u002Fneutral\u002Fidea","lhk931122",[63,33,33,176,64,34],[166,33,34,189,11,40],"maker\u002Fneutral\u002Fnone",[191,39,34,192,193,34],"49616220","other\u002Fneutral\u002Fnone","ex-aws-dude",[195,39,34,196,197,34],"49616270","commenter\u002Fneutral\u002Fnone","acedTrex",[199,39,34,196,200,34],"49616379","TomatoCo",[202,39,34,196,193,34],"49616842",[204,39,34,196,200,34],"49650170",[206,39,34,196,207,34],"49617243","libraryofbabel",[209,39,34,196,200,34],"49648766",[211,212,34,192,213,34],"49619522",7,"scottcodie",[93,18,40,215,94,34],"project\u002Fcritical\u002Fidea",[97,18,34,217,11,40],"maker\u002Fneutral\u002Fidea",[157,18,34,219,158,34],"commenter\u002Fneutral\u002Fidea",[221,18,35,185,222,34],"49614759","apnabhidu47",[145,32,33,176,146,34],[120,32,34,217,11,40],[85,226,33,227,86,34],10,"project\u002Fpositive\u002Fdesign",[229,230,33,231,232,34],"49656942",11,"project\u002Fpositive\u002Fnone","zephyroszhou",[81,234,33,176,82,34],12,[138,236,35,237,139,34],13,"project\u002Fneutral\u002Fnone",[134,239,35,185,135,34],14,[241,239,34,217,11,40],"49615081",[243,239,33,244,135,34],"49615442","maker\u002Fpositive\u002Fidea",[126,239,35,246,127,34],"project\u002Fneutral\u002Fdesign",[248,239,34,192,249,34],"49615415","visarga",[251,252,33,231,253,34],"49618148",15,"fermlon30000",[],[256,261,266,270,275,280],{"id":257,"name":258,"points":259,"category":260},"49567437","OpenTrailPaper",424,"hardware",{"id":262,"name":263,"points":264,"category":265},"49610059","Copperhead",250,"dev_tools",{"id":267,"name":268,"points":269,"category":265},"49562219","TERMy",225,{"id":271,"name":272,"points":273,"category":274},"49603546","Stuxnet",179,"security_privacy",{"id":276,"name":277,"points":278,"category":279},"49623398","Fast Polynomials",139,"learning_science",{"id":281,"name":282,"points":283,"category":284},"49568162","Jigsaw Haiku",118,"games",false,"v2",{"id":288,"name":5,"line":6,"ai":285,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":289,"top":18,"kw":290,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":291,"url":9,"mix":292,"praise":72,"objection":90,"sub":21},49613068,0.3,[24,25,26,27,28],"51051350000000001003505553305305",[32,33,34,35],1794036709646]