[{"data":1,"prerenderedAt":318},["ShallowReactive",2],{"project-35629050":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":45,"post_text":47,"what":48,"said":82,"maker_in_thread":188,"shape":199,"grid":203,"attempts":280,"same_week":281,"maker_shows":31,"maker_hidden":310,"reader_version":311,"card":312},"35629050","Checksum","generate and maintain end-to-end tests using AI","title","Show HN: Checksum – generate and maintain end-to-end tests using AI",null,"Bootstrapper909",1681916714,2023,78,32,809,3109,27,"notable","dev_tools","end-to-end testing","web_app",[20,23,24,25,26],"test generation","playwright","cypress","ai testing","roasted",-0.25,{"positive":30,"neutral":30,"mixed":31,"critical":32},4,1,9,18,{"design":35,"performance":37,"idea":38,"docs":41,"execution":42,"privacy":43,"pricing":44},{"positive":36,"critical":31},0,{"positive":36,"critical":31},{"positive":39,"critical":40},2,3,{"positive":31,"critical":31},{"positive":36,"critical":39},{"positive":36,"critical":36},{"positive":36,"critical":31},[40,36,31,36,31,40,36,36,31,36,31,36,40,36,46,36,31,46,40,36,31,36,36,31,31,36,31,46,46,36,40,36],5,"Hey HN!\n\nI’m Gal, co-founder at Checksum (https:\u002F\u002Fchecksum.ai). Checksum is a tool for automatically generating and maintaining end-to-end tests using AI.\n\nI cut my teeth in applied ML in 2016 at a maritime tech company called TSG, based in Israel. When I was there, I worked on a cool product that used machine learning to detect suspicious vehicles. Radar data is pretty tough for humans to parse, but a great fit for AI – and it worked very well for detecting smugglers, terrorist activity, and that sort of thing.\n\nIn 2021, after a few years working in big tech (Lyft, Google), I joined a YC comp",{"problem":49,"audience":55,"pricing":59,"motivation":65,"time_to_build":9,"stack":69,"openness":80,"stage":81,"team_size":80},{"value":50,"source":51,"evidence":52},"Keeping end-to-end tests with good coverage is hard and tedious to build and maintain.","maker",[53],{"id":54},"post",{"value":56,"source":51,"evidence":57},"Medium-sized startups struggling with test coverage, currently design partners.",[58],{"id":54},{"value":60,"source":51,"evidence":61},"No pricing yet; not decided because training costs vary per app.",[62],{"id":63,"by":10,"text":64},"35634716","Yep totally understand. We are an early stage startup and currently 100% focused on improving our models and our product. We don’t have pricing, not because we try to be vague,…",{"value":66,"source":51,"evidence":67},"Founder's pain keeping E2E tests in good state at a prior startup; believed ML could outperform humans here.",[68],{"id":54},[70,76],{"value":71,"source":51,"evidence":72},"Playwright",[73],{"id":74,"by":10,"text":75},"35629819","Our landing page at checksum.ai has a video in the hero section of test. We added some graphics (e.g. the green checkmark), but the steps executed are real tests that we…",{"value":77,"source":51,"evidence":78},"Cypress",[79],{"id":74,"by":10,"text":75},"unknown","beta",{"praise":83,"objections":100,"requests":146,"questions":162,"alternatives":183},[84],{"value":85,"source":86,"evidence":87},"Idea seen as promising and interesting.","commenter",[88,92,96],{"id":89,"by":90,"text":91},"35629935","jtambunt","Congratulations on all the progress you've made! We are all learning as we're building and talking to users. I know for my team, E2E\u002FIntegration testing is our main priority…",{"id":93,"by":94,"text":95},"35632342","execore-1","Awesome idea! Excited to see where this goes",{"id":97,"by":98,"text":99},"35631977","varunjain99","Congrats! Definitely think QA farms can be automated using AI! Can you explain more what part Checksum is using AI? Is it for the identification of user sessions that are good…",[101,108,122,132,139],{"value":102,"source":86,"evidence":103},"Name 'Checksum' suggests crypto\u002Fsecurity rather than testing.",[104],{"id":105,"by":106,"text":107},"35633123","therealrifath","This is dope, a couple of feedback points from a technical person that is a potential customer (I could be wrong on these!): -- I think the name doesn't sell me or even most…",{"value":109,"source":86,"evidence":110},"Unclear how the AI verifies correct behavior and bugs without human oracles.",[111,115,118],{"id":112,"by":113,"text":114},"35635218","satisfice","Most tool companies making claims about their tools show a shocking lack of knowledge about testing. This generally guarantees that their tools are dismissed by serious…",{"id":116,"by":113,"text":117},"35635245","So you are saying that your system needs me to do all the testing (it is infeasible to watch our users, because we test the product before it is released to any users) so it can…",{"id":119,"by":120,"text":121},"35629596","sachuin23","How is the product different from the other test generation tools? How do you check if the are testing the intended behavior. My experience with automated testing solutions has…",{"value":123,"source":86,"evidence":124},"Privacy handling of user session data, especially for HIPAA-type apps.",[125,129],{"id":126,"by":127,"text":128},"35634541","afro88","How do you know what data is sensitive, and how do you anonymize? Thinking of apps that might fall under HIPAA etc",{"id":130,"by":10,"text":131},"35629491","That's a great question! We actually use real user sessions to train our model, so when I use the term coverage our main metric is covering as many user behaviors as possible. We…",{"value":133,"source":86,"evidence":134},"No pricing published on the site.",[135],{"id":136,"by":137,"text":138},"35634527","BillSaysThis","Nothing on pricing on your site. Makes any other question difficult to formulate.",{"value":140,"source":86,"evidence":141},"Performance claims were vague.",[142],{"id":143,"by":144,"text":145},"35632008","ezekg","> Our impact on performance is non-existent as we use battle-tested open source tools used by Fortune 500 companies What does that mean, exactly? Just because it's open source…",[147,155],{"value":148,"source":86,"evidence":149},"Show example generated tests and a demo of setup and passing tests.",[150,154],{"id":151,"by":152,"text":153},"35629691","JohnFriel","This is a really compelling idea – but I'm having a little trouble making the leap from the high level description to what it would mean for my projects in more concrete terms.…",{"id":105,"by":106,"text":107},{"value":156,"source":86,"evidence":157},"Support for mobile apps such as React Native.",[158],{"id":159,"by":160,"text":161},"35630415","mackeyja92","Does it only support web? What about react native mobile apps?",[163,170,178],{"value":164,"source":86,"evidence":165},"How does the model handle edge cases absent from training sessions?",[166],{"id":167,"by":168,"text":169},"35632282","johnhenning","If this is trained on user sessions, how would the model learn to generate tests for edge cases that wouldn’t necessarily show up in the training data?",{"value":171,"source":86,"evidence":172},"How does the system decide a test passed, and how does it handle UI changes?",[173,177],{"id":174,"by":175,"text":176},"35652515","artur_makly","congrats on the launch. 0 - seriously rethink your branding. I can help. 1 - how does the Ai know when the test is successful? — is it a visual comparison? — if so.. is there a…",{"id":112,"by":113,"text":114},{"value":179,"source":86,"evidence":180},"How does it differ from other test generation tools?",[181,182],{"id":119,"by":120,"text":121},{"id":174,"by":175,"text":176},[184],{"value":185,"source":86,"evidence":186},"Meticulous.ai",[187],{"id":174,"by":175,"text":176},{"comments":189,"answered":190,"key":191},11,8,[192,195,198],{"id":193,"text":194},"35633258","Thanks for your feedback. It definitely makes sense and we'll incorporate it!",{"id":196,"text":197},"35632223","That's a fair comment and I guess we are missing and \"AND\" there. 1. We (and others) have tested our tools' impact on memory, CPU, network performance and found only negligible…",{"id":74,"text":75},{"commenters":200,"max_depth":30,"maker_replies":189,"first_comment_mins":201,"lifespan_hours":202},19,23,45,[204,206,208,210,212,214,216,218,222,225,227,229,231,232,234,237,239,244,247,249,251,253,255,258,261,262,264,265,267,270,274,277],[105,36,40,205,106,36],"project\u002Fmixed\u002Fdesign",[193,36,36,207,10,31],"maker\u002Fpositive\u002Fnone",[143,31,31,209,144,36],"project\u002Fcritical\u002Fperformance",[196,31,36,211,10,31],"project\u002Fneutral\u002Fperformance",[112,39,31,213,113,36],"project\u002Fcritical\u002Fidea",[151,40,40,215,152,36],"project\u002Fneutral\u002Fdocs",[74,40,36,217,10,31],"project\u002Fneutral\u002Fexecution",[219,30,36,220,221,36],"35630388","other\u002Fneutral\u002Fnone","johnsillings",[223,46,31,213,224,36],"35629394","8organicbits",[130,46,36,226,10,31],"project\u002Fneutral\u002Fprivacy",[167,46,31,228,168,36],"project\u002Fcritical\u002Fexecution",[230,46,36,217,10,31],"35634490",[126,46,40,226,127,36],[233,46,36,226,10,31],"35634756",[89,235,46,236,90,36],6,"project\u002Fpositive\u002Fidea",[238,235,36,207,10,31],"35632290",[240,241,31,242,243,36],"35631509",7,"project\u002Fcritical\u002Fdocs","hiatus",[245,241,46,246,221,36],"35631736","maker\u002Fpositive\u002Fdocs",[97,190,40,248,98,36],"project\u002Fneutral\u002Fidea",[250,190,36,217,10,31],"35632261",[136,32,31,252,137,36],"project\u002Fcritical\u002Fpricing",[63,32,36,254,10,31],"maker\u002Fneutral\u002Fpricing",[256,32,36,257,137,36],"35634732","commenter\u002Fneutral\u002Fpricing",[174,259,31,260,175,36],10,"project\u002Fcritical\u002Fdesign",[119,189,31,213,120,36],[263,189,36,217,10,31],"35629723",[116,189,31,228,113,36],[93,266,46,236,94,36],12,[268,266,46,207,269,36],"35634287","shyamkumar7",[271,272,36,220,273,36],"35638200",13,"firedup",[159,275,40,276,160,36],14,"project\u002Fneutral\u002Fnone",[278,275,36,279,10,31],"35630485","maker\u002Fneutral\u002Fnone",[],[282,287,292,296,301,306],{"id":283,"name":284,"points":285,"category":286},"35621417","AI Playground",249,"ai",{"id":288,"name":289,"points":290,"category":291},"35572434","WikiBinge",246,"games",{"id":293,"name":294,"points":295,"category":19},"35580959","IPython-GPT",144,{"id":297,"name":298,"points":299,"category":300},"35620314","Tesseract Analytics",113,"data",{"id":302,"name":303,"points":304,"category":305},"35628642","Kratos",88,"security_privacy",{"id":307,"name":308,"points":309,"category":286},"35568799","Debate Devil",84,false,"v2",{"id":313,"name":5,"line":6,"ai":310,"cat":19,"year":12,"points":13,"comments":14,"rec":27,"s":28,"top":17,"kw":314,"maker":10,"domain":315,"tier":18,"attempts":36,"strip":316,"url":-1,"mix":317,"praise":85,"objection":102,"sub":20},35629050,[20,23,24,25,26],"","30101300101030501530100110155030",[30,30,31,32],1793991064044]