[{"data":1,"prerenderedAt":244},["ShallowReactive",2],{"project-29128998":3},{"id":4,"name":5,"line":5,"line_source":6,"title":7,"url":8,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":42,"post_text":44,"what":45,"said":70,"maker_in_thread":120,"shape":127,"grid":131,"attempts":206,"same_week":207,"maker_shows":43,"maker_hidden":236,"reader_version":237,"card":238},"29128998","How does JAX allocate memory on a TPU? An interactive C++ walkthrough","title","Show HN: How does JAX allocate memory on a TPU? An interactive C++ walkthrough","https:\u002F\u002Fgist.github.com\u002Fshawwn\u002F0e524d4a7a5d8fb152a86616559cc02a","gist.github.com","sillysaurusx",1636193799,2021,78,28,649,2533,26,"notable","learning_science","interactive tutorial","web_app",[23,24,25,26,27],"jax","tpu","memory allocation","xla","c++","mixed",-0.105,{"positive":31,"neutral":32,"mixed":31,"critical":33},2,6,3,13,{"design":36,"execution":38,"docs":40,"performance":41},{"positive":37,"critical":37},0,{"positive":39,"critical":39},1,{"positive":39,"critical":39},{"positive":37,"critical":37},[33,39,33,33,39,37,37,37,37,37,37,37,37,37,37,33,37,43,33,33,37,39,33,33,37,43,37,37],5,null,{"problem":46,"audience":44,"pricing":44,"motivation":44,"time_to_build":44,"stack":52,"openness":69,"stage":69,"team_size":69},{"value":47,"source":48,"evidence":49},"Explains how JAX allocates memory on a TPU","maker",[50],{"id":51},"post",[53,57,61],{"value":54,"source":48,"evidence":55},"C++",[56],{"id":51},{"value":58,"source":48,"evidence":59},"JAX",[60],{"id":51},{"value":62,"source":63,"evidence":64},"XLA","commenter",[65],{"id":66,"by":67,"text":68},"29132588","alevskaya","One thing that's important to note about memory management with XLA, is that inside a compiled program there's no user-exposed \"malloc\"\u002F\"free\". The memory usage and schedule of a…","unknown",{"praise":71,"objections":81,"requests":98,"questions":99,"alternatives":104},[72],{"value":73,"source":63,"evidence":74},"Jax debugging and TPU VM access praised as enjoyable",[75,78],{"id":76,"by":10,"text":77},"29130582","> The downside of Jax is it’s not easy to debug. PyTorch, for better or for worse, will actually run your Python code as you wrote it. Hmm. Jax's ease of debugging was the very…",{"id":79,"by":10,"text":80},"29130757","> So I'm surprised you can run linux userspace on it? For what it's worth, I was equally shocked! Felt like a miracle. That special hardware also turns out to be…",[82,92],{"value":83,"source":63,"evidence":84},"Memory scheduling claims in XLA questioned as unverified",[85,89],{"id":86,"by":87,"text":88},"29134137","tubby12345",">The memory usage and schedule of a given program\u002Fgraph is statically optimized by the compiler Is it really though? The only thing I see is…",{"id":90,"by":87,"text":91},"29135051","I believe you but I'm wondering what that memory scheduling actually looks like (e.g. solving the full MIP?). do you have a code pointer to that scheduling routine?",{"value":93,"source":63,"evidence":94},"PyTorch XLA on TPU performs poorly and recompiles often",[95],{"id":96,"by":10,"text":97},"29129400","A terrible one. It’s literally one of the worst experiences you can have. I say that without an ounce of bias. When tpu SSH first came out, I immediately went to my pytorch buddy…",[],[100],{"value":101,"source":63,"evidence":102},"How does XLA schedule memory inside compiled programs",[103],{"id":90,"by":87,"text":91},[105,113],{"value":106,"source":63,"evidence":107},"PyTorch with XLA backend",[108,112],{"id":109,"by":110,"text":111},"29129254","boibombeiro","Jax uses XLA (a IR that targets multiple platforms, including the Google TPU) as backend. Pytorch also has a XLA backend.",{"id":96,"by":10,"text":97},{"value":114,"source":63,"evidence":115},"Google Colab for TPU\u002FGPU training",[116],{"id":117,"by":118,"text":119},"29131706","Salgat","Plenty of free services like Google Colab exist for Tensorflow\u002FPytorch and offer gpu\u002Ftpu. I pay just $10 a month to train my models at my leisure on it, and the free version is…",{"comments":43,"answered":31,"key":121},[122,125,126],{"id":123,"text":124},"29129234","They're pretty rad. You can even get a few (dozen) for free: https:\u002F\u002Fblog.gpt4.org\u002Fjaxtpu Incidentally, that blog runs on a TPU. You can just SSH into them. Stuff like that is…",{"id":96,"text":97},{"id":76,"text":77},{"commenters":128,"max_depth":32,"maker_replies":43,"first_comment_mins":129,"lifespan_hours":130},16,39,38,[132,134,136,138,140,144,148,151,154,156,159,161,162,165,167,171,175,177,180,183,187,189,192,196,198,199,201,203],[66,37,33,133,67,37],"project\u002Fneutral\u002Fdesign",[86,37,39,135,87,37],"project\u002Fcritical\u002Fexecution",[137,37,33,133,67,37],"29134731",[90,37,33,139,87,37],"project\u002Fneutral\u002Fexecution",[141,39,39,142,143,37],"29129723","project\u002Fcritical\u002Fdocs","WanderPanda",[145,39,37,146,147,37],"29130326","other\u002Fneutral\u002Fnone","p1esk",[149,39,37,146,150,37],"29130480","dekhn",[152,39,37,153,87,37],"29132739","commenter\u002Fneutral\u002Fnone",[155,39,37,153,150,37],"29140213",[157,31,37,146,158,37],"29129154","azalemeth",[123,31,37,160,10,39],"project\u002Fpositive\u002Fexecution",[109,31,37,146,110,37],[163,31,37,146,164,37],"29129384","carbocation",[96,31,37,166,10,39],"other\u002Fcritical\u002Fperformance",[168,31,37,169,170,37],"29132015","other\u002Fneutral\u002Fdesign","bertr4nd",[172,31,33,173,174,37],"29129458","project\u002Fmixed\u002Fperformance","currymj",[76,31,37,176,10,39],"commenter\u002Fcritical\u002Fdocs",[178,31,43,179,174,37],"29132853","maker\u002Fpositive\u002Fdocs",[181,31,33,173,182,37],"29131465","6gvONxR4sf7o",[184,31,33,185,186,37],"29129554","maker\u002Fneutral\u002Fnone","rrss",[188,31,37,185,10,39],"29130255",[190,31,39,191,186,37],"29130938","maker\u002Fcritical\u002Fnone",[193,31,33,194,195,37],"29130368","project\u002Fneutral\u002Fnone","manuel_w",[197,31,33,194,195,37],"29130402",[79,31,37,160,10,39],[200,31,43,160,186,37],"29131101",[117,31,37,202,118,37],"other\u002Fneutral\u002Fpricing",[204,33,37,146,205,37],"29145051","mrlonglong",[],[208,213,218,222,227,232],{"id":209,"name":210,"points":211,"category":212},"29118785","Nom de Bébé",1077,"consumer_social",{"id":214,"name":215,"points":216,"category":217},"29167580","Slingshot Effect",985,"games",{"id":219,"name":220,"points":221,"category":212},"29161585","Stovetop",403,{"id":223,"name":224,"points":225,"category":226},"29165601","You.com",361,"web_saas",{"id":228,"name":229,"points":230,"category":231},"29179429","Cedille",240,"ai",{"id":233,"name":234,"points":235,"category":217},"29106581","Initiative.sh",222,false,"v2",{"id":239,"name":5,"line":5,"ai":236,"cat":19,"year":12,"points":13,"comments":14,"rec":28,"s":240,"top":17,"kw":241,"maker":10,"domain":9,"tier":18,"attempts":37,"strip":242,"url":8,"mix":243,"praise":73,"objection":83,"sub":20},29128998,-0.1,[23,24,25,26,27],"3133100000000003053301330500",[31,32,31,33],1794036612874]