[{"data":1,"prerenderedAt":325},["ShallowReactive",2],{"project-28088882":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":34,"aspects":35,"strip":42,"post_text":45,"what":46,"said":71,"maker_in_thread":196,"shape":198,"grid":201,"attempts":284,"same_week":289,"maker_shows":38,"maker_hidden":317,"reader_version":318,"card":319},"28088882","GrowthBook","Open-source A\u002FB testing framework","title","Show HN: Open-source A\u002FB testing framework","https:\u002F\u002Fgithub.com\u002Fgrowthbook\u002Fgrowthbook","github.com","cheeseblubber",1628265204,2021,199,37,236,2533,10,"notable","data","ab testing","web_app",[21,24,25,26],"bayesian statistics","experimentation","open source","curious",-0.108,{"positive":30,"neutral":31,"mixed":32,"critical":33},4,20,0,7,31,{"design":36,"idea":39,"execution":41},{"positive":37,"critical":38},1,6,{"positive":40,"critical":37},2,{"positive":37,"critical":32},[43,32,44,43,43,37,44,44,44,37,44,32,44,44,37,44,37,37,37,32,32,32,43,44,44,44,37,44,44,44,44,44,44,44,32,44,44],5,3,null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":55,"openness":68,"stage":69,"team_size":70},{"value":48,"source":49,"evidence":50},"Running and analysing controlled A\u002FB experiments and deciding on results.","commenter",[51],{"id":52,"by":53,"text":54},"28094015","jrdorn","Ax is for automated optimization using machine learning. You define the parameters and optimization function and it decides the variations, traffic splitting, and everything else…",[56,62],{"value":57,"source":49,"evidence":58},"MongoDB",[59],{"id":60,"by":53,"text":61},"28090315","Hi! One of the authors here. We're using MongoDB to store caches A\u002FB test results (among other things), which are deeply nested JSON objects. MongoDB let us develop features…",{"value":63,"source":49,"evidence":64},"JavaScript SDK bundled with apps (webpack)",[65],{"id":66,"by":53,"text":67},"28091088","Hi, thanks for the questions! 1. We don't store any raw user data. We pull things like mean and standard deviation from data sources, run the statistics, and store the result. 2.…","open_source","launched","unknown",{"praise":72,"objections":88,"requests":120,"questions":152,"alternatives":176},[73,84],{"value":74,"source":49,"evidence":75},"Polished and well executed",[76,80],{"id":77,"by":78,"text":79},"28092184","gingerlime","Wow this looks very polished. (Out of frustration with Optimizely) I created and maintain a couple of A\u002FB test open source projects[0][1] but the statistical analysis was always…",{"id":81,"by":82,"text":83},"28093186","jaggednad","I think you guys really did a good job with something that's hard to do well. Kudos",{"value":85,"source":49,"evidence":86},"Bayesian approach reduces peeking problems",[87],{"id":66,"by":53,"text":67},[89,99,106,113],{"value":90,"source":49,"evidence":91},"MongoDB is not open source (SSPL) and is a poor choice for some users",[92,96],{"id":93,"by":94,"text":95},"28092231","ensignavenger","Hi- thanks for making this and open sourcing it! I would also suggest supporting an alternative to MongoDB. Postgres using jsonb is a great option. I try to always use and…",{"id":97,"by":94,"text":98},"28093485","> PostgreSQL using JSONB however is a way to hurt your head Really? I have used it pretty extensively and like it... I don't do a lot of complex manipulations though, it might be…",{"value":100,"source":49,"evidence":101},"Short tests give untrustworthy results and the tool may encourage poor decisions",[102],{"id":103,"by":104,"text":105},"28091253","Dyac","Interesting, thanks. 1. How do you get around needing session level data instead of aggregate data when working with non parametric KPIs? GA in particular is notorious for…",{"value":107,"source":49,"evidence":108},"Metrics coupled to experimentation is a design concern",[109],{"id":110,"by":111,"text":112},"28093380","ablearcher83","Coupling metrics to experimentation is a huge red flag. You should offload that onto DBT or some other data modeling tools.",{"value":114,"source":49,"evidence":115},"Configuration cannot be stored in the code repository",[116],{"id":117,"by":118,"text":119},"28090281","tehlike","From the looks of it, it doesn't look like the configuration can be stored in the code repository itself. This is one of the key things to do - treating configuration as a code…",[121,133,139,145],{"value":122,"source":49,"evidence":123},"Support PostgreSQL or SQLite instead of MongoDB",[124,128,132],{"id":125,"by":126,"text":127},"28090379","sojournerc","Nice project! PostgreSQL has excellent JSON support these days, including the ability to query nested fields which may be beneficial to your project.",{"id":129,"by":130,"text":131},"28091140","nrjames","Honestly, I'd love to see it just use SQLite as a backend. If it's just storing results, that seems feasible and it would reduce the complexity of the tech stack.",{"id":93,"by":94,"text":95},{"value":134,"source":49,"evidence":135},"Store configuration in code (YAML) for self-hosting",[136],{"id":137,"by":53,"text":138},"28090391","That's on our roadmap. We originally built the tool as a multi-tenant hosted platform so storing configs in a database made the most sense initially. For self hosting, we want to…",{"value":140,"source":49,"evidence":141},"MySQL\u002FMariaDB data source support",[142],{"id":143,"by":53,"text":144},"28091982","We plan to add MySQL\u002FMariaDB support soon which should let you use Matomo data as long as you have raw SQL access. For cloud-hosted Matomo, we would have to use the reporting…",{"value":146,"source":49,"evidence":147},"Mobile SDK support",[148],{"id":149,"by":150,"text":151},"28091481","dreamer7","Would this be useful for A\u002FB testing mobile app features as well?",[153,159,165,169],{"value":154,"source":49,"evidence":155},"Does it store raw user data or query data sources?",[156],{"id":157,"by":104,"text":158},"28090960","Hi. Regular A\u002FB experimenter here. This looks like a great tool! Does your system store the stats, or does it trust the stats to be stored in, eg. GA, and then just allow you to…",{"value":160,"source":49,"evidence":161},"Can it analyse tests run in other tools?",[162],{"id":163,"by":104,"text":164},"28092133","Brilliant, thank you. Can you use the system to analyse results of tests it didn't run? ie. If I run tests using some SAAS that only supports frequentist stats could I use your…",{"value":166,"source":49,"evidence":167},"How does it handle non-parametric KPIs and sampling?",[168],{"id":103,"by":104,"text":105},{"value":170,"source":49,"evidence":171},"How does it compare to Facebook Ax?",[172],{"id":173,"by":174,"text":175},"28093791","dimitry12","How does this compare to Facebook Ax?",[177,181,185,189],{"value":178,"source":49,"evidence":179},"Analytics Toolkit",[180],{"id":77,"by":78,"text":79},{"value":182,"source":49,"evidence":183},"Alephbet",[184],{"id":77,"by":78,"text":79},{"value":186,"source":49,"evidence":187},"Facebook Ax",[188],{"id":173,"by":174,"text":175},{"value":190,"source":49,"evidence":191},"sixpack",[192],{"id":193,"by":194,"text":195},"28098493","JensRantil","This reminds me of https:\u002F\u002Fgithub.com\u002Fsixpack\u002Fsixpack which I've been eyeballing for years.",{"comments":32,"answered":32,"key":197},[],{"commenters":199,"max_depth":38,"maker_replies":32,"first_comment_mins":200,"lifespan_hours":199},22,57,[202,204,207,210,212,213,215,217,218,220,222,224,228,231,235,238,239,240,241,242,246,248,251,253,255,259,261,263,265,269,270,273,275,276,278,281,283],[77,32,43,203,78,32],"project\u002Fpositive\u002Fdesign",[205,32,32,206,53,32],"28092348","commenter\u002Fneutral\u002Fidea",[208,32,44,209,78,32],"28096175","project\u002Fneutral\u002Fidea",[157,37,43,211,104,32],"project\u002Fpositive\u002Fidea",[66,37,43,211,53,32],[103,37,37,214,104,32],"project\u002Fcritical\u002Fidea",[216,37,44,209,53,32],"28091539",[163,37,44,209,104,32],[219,37,44,209,53,32],"28092189",[117,40,37,221,118,32],"project\u002Fcritical\u002Fdesign",[137,40,44,223,53,32],"maker\u002Fneutral\u002Fdesign",[225,40,32,226,227,32],"28092282","other\u002Fneutral\u002Fnone","travisjungroth",[229,44,44,209,230,32],"28094383","contravariant",[232,30,44,233,234,32],"28090070","project\u002Fneutral\u002Fdesign","mooneater",[236,30,37,221,237,32],"28090128","marcinzm",[60,30,44,223,53,32],[125,30,37,221,126,32],[129,30,37,221,130,32],[93,30,37,221,94,32],[243,30,32,244,245,32],"28092912","commenter\u002Fneutral\u002Fdesign","gqewogpdqa",[97,30,32,247,94,32],"commenter\u002Fcritical\u002Fdesign",[249,30,32,247,250,32],"28093496","pphysch",[81,43,43,252,82,32],"maker\u002Fpositive\u002Fexecution",[193,38,44,254,194,32],"project\u002Fneutral\u002Fnone",[256,33,44,257,258,32],"28089767","maker\u002Fneutral\u002Fidea","cardosof",[260,33,44,257,53,32],"28090358",[110,262,37,221,111,32],8,[264,262,44,223,53,32],"28093663",[266,267,44,209,268,32],"28091798",9,"nacs",[143,267,44,223,53,32],[271,267,44,209,272,32],"28093368","XCSme",[274,267,44,223,53,32],"28093732",[149,18,44,209,150,32],[277,18,44,223,53,32],"28091587",[279,18,32,244,280,32],"28095576","bob_roboto",[173,282,44,209,174,32],11,[52,282,44,209,53,32],[285,288],{"id":286,"t":287,"points":282,"comments":40},"27439671",1623180479,{"id":4,"t":12,"points":14,"comments":15},[290,295,300,304,308,312],{"id":291,"name":292,"points":293,"category":294},"28074827","Repo Visualization",283,"dev_tools",{"id":296,"name":297,"points":298,"category":299},"28120247","colab.research.google.com",169,"ai",{"id":301,"name":302,"points":303,"category":294},"28084104","Systemizer",167,{"id":305,"name":306,"points":307,"category":299},"28100346","Stray Robots toolkit",166,{"id":309,"name":310,"points":311,"category":294},"28128445","Lona",165,{"id":313,"name":314,"points":315,"category":316},"28085920","Paper Time",161,"learning_science",false,"v2",{"id":320,"name":5,"line":6,"ai":317,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":321,"top":18,"kw":322,"maker":11,"domain":10,"tier":19,"attempts":40,"strip":323,"url":9,"mix":324,"praise":74,"objection":90,"sub":21},28088882,-0.11,[21,24,25,26],"5035513331303313111000533313333333033",[30,31,32,33],1794036609824]