[{"data":1,"prerenderedAt":300},["ShallowReactive",2],{"project-42196381":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":43,"post_text":44,"what":45,"said":68,"maker_in_thread":154,"shape":165,"grid":170,"attempts":262,"same_week":263,"maker_shows":205,"maker_hidden":292,"reader_version":293,"card":294},"42196381","Weave","actually measure engineering productivity","title","Show HN: Weave - actually measure engineering productivity","https:\u002F\u002Fapp.workweave.ai\u002Fwelcome","app.workweave.ai","adchurch",1732124614,2024,22,39,1827,3515,52,"shown","dev_tools","engineering metrics","web_app",[24,25,26,27],"engineering productivity","developer metrics","pull request analytics","engineering output","roasted",-0.556,{"positive":31,"neutral":32,"mixed":31,"critical":33},0,3,16,19,{"docs":36,"idea":37,"execution":39,"performance":41},{"positive":31,"critical":32},{"positive":31,"critical":38},10,{"positive":31,"critical":40},1,{"positive":31,"critical":42},2,[31,31,31,32,31,40,31,40,31,40,40,31,40,31,40,40,31,31,31,40,31,31,40,31,31,32,40,31,40,31,40,40,31,31,40,40,32,40,31],"Hey HN,\n\nWe’re building Weave: an ML-powered tool to measure engineering output, that actually understands engineering output!\n\nWhy? Here’s the thing: almost every eng leader already measures output - either openly or behind closed doors. But they rely on metrics like lines of code (correlation with effort: ~0.3), number of PRs, or story points (slightly better at ~0.35). These metrics are, frankly, terrible proxies for productivity.\n\nWe’ve developed a custom model that analyzes code and its impact directly, with a far better 0.94 correlation. The result? A standardized engineering output metr",{"problem":46,"audience":52,"pricing":56,"motivation":57,"time_to_build":56,"stack":65,"openness":66,"stage":67,"team_size":66},{"value":47,"source":48,"evidence":49},"Existing engineering metrics like LOC, PR counts and story points are poor proxies for productivity","maker",[50],{"id":51},"post",{"value":53,"source":48,"evidence":54},"Engineering leaders and managers",[55],{"id":51},null,{"value":58,"source":48,"evidence":59},"Existing metrics are fundamentally broken; a more accurate output measure is needed",[60,61],{"id":51},{"id":62,"by":63,"text":64},"42196739","adambeecee","Hey HN! I'm one of the co-founders of Weave, and I wanted to jump in here to share a bit more. Building this has been a wild ride. The challenge of measuring engineering output…",[],"unknown","launched",{"praise":69,"objections":74,"requests":119,"questions":134,"alternatives":149},[70],{"value":71,"source":48,"evidence":72},"Maker says startup testers gave fantastic feedback",[73],{"id":51},[75,83,97,108],{"value":76,"source":77,"evidence":78},"Metric can be gamed once known","commenter",[79],{"id":80,"by":81,"text":82},"42197803","henning","As soon as people know how the metric is calculated, they will game that metric and it will cease to be useful.",{"value":84,"source":77,"evidence":85},"Ignores maintenance, bug fixes and outcome or product impact",[86,89,93],{"id":87,"by":81,"text":88},"42206932","In a complex, mature system, a high impact bug could have a very small fix that is highly non-obvious. Your metric assumes that the person shitting out 1000 lines of a new…",{"id":90,"by":91,"text":92},"42200959","captn3m0","A hotfix PR that was 2 lines long, prevents (or managed) an outage incident, but was merged with no title, description, or review because time was of the essence. Some of such…",{"id":94,"by":95,"text":96},"42197889","jaredsohn","If you build something that doesn't solve problems with impact to the business, your real productivity is zero. How does this account for that?…",{"value":98,"source":77,"evidence":99},"Black-box metric with little information on the landing page",[100,104],{"id":101,"by":102,"text":103},"42197855","karmajunkie","I'd like to add that you need way more information on the landing page before I'm going to do much more than let you have my email address (if that.) Right now its a black box…",{"id":105,"by":106,"text":107},"42199296","id00","I worked as a manager in a big tech company that used metrics such as amount of PRs done, amount of PR reviews and etc. At least with those when there were questions based on…",{"value":109,"source":77,"evidence":110},"Effort cannot be inferred from code alone; context switching ignored",[111,115],{"id":112,"by":113,"text":114},"42198215","mg57","Pretty dumb to think you can infer effort from the code itself. You make one \"smart invocation\" to a remote microservice and replace 1000 lines of code! The information for…",{"id":116,"by":117,"text":118},"42198054","jumploops","Curious how these numbers correlate to the estimates of the engineers behind the PRs? For example, the first PR is correlated with ~15 \"hours of work for an expert engineer\"…",[120,128],{"value":121,"source":77,"evidence":122},"More detail on how the correlation numbers were validated",[123,127],{"id":124,"by":125,"text":126},"42197829","rkagerer","How did you come up with those magic correlation numbers? Is this generally just sniffing surface quality and quantity of written code, or is consideration given to how…",{"id":62,"by":63,"text":64},{"value":129,"source":77,"evidence":130},"Explain how the metric handles refactoring and long-term regressions",[131],{"id":132,"by":106,"text":133},"42198568","Let me just ignore my natural distain to the whole thing (as a engineer and a manager) > We’ve developed a custom model that analyzes code and its impact directly... This is a…",[135,142],{"value":136,"source":77,"evidence":137},"How were the 0.94 and 0.3\u002F0.35 correlation figures computed",[138,139],{"id":124,"by":125,"text":126},{"id":140,"by":11,"text":141},"42198642","> How did you come up with those magic correlation numbers? Evaluated on a proprietary data set of manually labelled PRs > Is this generally just sniffing surface quality and…",{"value":143,"source":77,"evidence":144},"Does it account for branches that never reach production",[145],{"id":146,"by":147,"text":148},"42197766","cssanchez","Skeptic here. How can you validate the difference in effort from a startup where growth happens in explosive moments with many rewrites in between vs a refined enterprise…",[150],{"value":151,"source":48,"evidence":152},"Lines of code, PR counts and story points",[153],{"id":51},{"comments":155,"answered":156,"key":157},14,9,[158,161,162],{"id":159,"text":160},"42198741","LOL - I shudder at the idea of a manager making HR decisions based solely on this one metric! To be clear we're not claiming this is 1 number to holistically evaluate an entire…",{"id":140,"text":141},{"id":163,"text":164},"42196755","Our metric is approximately \"hours of work for an expert engineer.\" Here are some example open source PRs and their output metrics calculated by our algorithm:…",{"commenters":166,"max_depth":167,"maker_replies":155,"first_comment_mins":168,"lifespan_hours":169},20,5,41,24,[171,175,177,181,183,185,187,188,190,193,194,196,198,199,201,204,206,209,213,216,219,223,225,227,229,231,233,235,237,239,241,244,246,249,251,252,255,259,260],[172,31,31,173,174,31],"42197944","other\u002Fcritical\u002Fidea","senko",[159,31,31,176,11,40],"maker\u002Fneutral\u002Fidea",[178,31,31,179,180,31],"42200515","other\u002Fneutral\u002Fnone","flashgordon",[124,40,32,182,125,31],"project\u002Fneutral\u002Fdocs",[140,40,31,184,11,40],"project\u002Fneutral\u002Fexecution",[90,40,40,186,91,31],"project\u002Fcritical\u002Fidea",[163,42,31,184,11,40],[189,42,40,186,81,31],"42197826",[191,42,31,192,11,40],"42198549","project\u002Fneutral\u002Fidea",[87,42,40,186,81,31],[116,42,40,195,117,31],"project\u002Fcritical\u002Fexecution",[197,42,31,192,11,40],"42198529",[94,32,40,186,95,31],[200,32,31,192,11,40],"42199744",[202,32,40,186,203,31],"42203378","jrs235",[80,205,40,186,81,31],4,[207,205,31,173,208,31],"42197870","vasco",[210,205,31,211,212,31],"42197945","other\u002Fneutral\u002Fidea","sam_lowry_",[214,205,31,215,11,40],"42198707","project\u002Fmixed\u002Fidea",[217,205,40,186,218,31],"42198034","ryandvm",[220,205,31,221,222,31],"42198084","commenter\u002Fneutral\u002Fnone","dragonwriter",[224,205,31,192,11,40],"42198397",[226,205,40,186,81,31],"42198450",[228,205,31,192,11,40],"42198578",[230,205,31,221,203,31],"42203417",[62,167,32,232,63,31],"maker\u002Fneutral\u002Fnone",[146,167,40,234,147,31],"project\u002Fcritical\u002Fperformance",[236,167,31,192,11,40],"42198504",[101,167,40,238,102,31],"project\u002Fcritical\u002Fdocs",[240,167,31,182,11,40],"42198657",[242,167,40,238,243,31],"42201488","Kimitri",[132,245,40,234,106,31],6,[247,245,31,248,11,40],"42198795","project\u002Fneutral\u002Fperformance",[250,245,31,176,11,40],"42198819",[105,245,40,238,106,31],[253,254,40,186,95,31],"42199192",7,[256,257,32,248,258,31],"42200156",8,"itsdrewmiller",[112,156,40,186,113,31],[261,156,31,192,11,40],"42198476",[],[264,269,274,279,283,287],{"id":265,"name":266,"points":267,"category":268},"42156977","lanparty.house",1117,"consumer_social",{"id":270,"name":271,"points":272,"category":273},"42174829","FastGraphRAG",457,"ai",{"id":275,"name":276,"points":277,"category":278},"42185668","Black Hole Vision",316,"creative",{"id":280,"name":281,"points":282,"category":20},"42173119","Tips.io",291,{"id":284,"name":285,"points":286,"category":20},"42197754","Trailimap",279,{"id":288,"name":289,"points":290,"category":291},"42149044","closing.wtf",273,"finance_crypto",false,"v2",{"id":295,"name":5,"line":6,"ai":292,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":296,"top":18,"kw":297,"maker":11,"domain":10,"tier":19,"attempts":31,"strip":298,"url":9,"mix":299,"praise":71,"objection":76,"sub":21},42196381,-0.56,[24,25,26,27],"000301010110101100010010031010110011310",[31,32,31,33],1793991105552]