[{"data":1,"prerenderedAt":419},["ShallowReactive",2],{"project-28738141":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":46,"post_text":48,"what":49,"said":83,"maker_in_thread":208,"shape":218,"grid":222,"attempts":381,"same_week":382,"maker_shows":47,"maker_hidden":411,"reader_version":412,"card":413},"28738141","SpotML","Managed ML Training on Cheap AWS\u002FGCP Spot Instances","title","Show HN: SpotML – Managed ML Training on Cheap AWS\u002FGCP Spot Instances","https:\u002F\u002Fwww.spotml.io\u002F","spotml.io","vishnukool",1633276637,2021,157,68,309,2533,13,"notable","ai","ml training infrastructure","cli",[24,25,26,27,28],"spot instances","ml training","gpu cost","checkpointing","aws","mixed",0.051,{"positive":32,"neutral":33,"mixed":34,"critical":34},7,11,5,28,{"pricing":37,"idea":39,"performance":40,"privacy":41,"design":42,"docs":43,"execution":45},{"positive":38,"critical":38},0,{"positive":34,"critical":34},{"positive":38,"critical":38},{"positive":38,"critical":38},{"positive":38,"critical":38},{"positive":44,"critical":38},1,{"positive":38,"critical":38},[47,38,38,34,34,34,47,38,47,38,44,44,44,47,38,47,38,38,44,47,38,47,47,38,38,38,47,47,34,47,44,47,47,34,47,47,34,38,38,34],3,null,{"problem":50,"audience":57,"pricing":66,"motivation":75,"time_to_build":48,"stack":79,"openness":80,"stage":81,"team_size":82},{"value":51,"source":52,"evidence":53},"Spot instance training is cheap but interruptions require manual devops to resume","maker",[54],{"id":55,"by":11,"text":56},"28738149","We built a tool called spotML to make training on AWS\u002FGCP cheaper. Spot Instances are 70% cheaper than On-Demand instances but are prone to interruptions. We mitigate the…",{"value":58,"source":59,"evidence":60},"ML engineers and teams training models on AWS or GCP","inferred",[61,62],{"id":55,"by":11,"text":56},{"id":63,"by":64,"text":65},"28738425","talolard","This looks useful, I like the pricing of free\u002F$9.99 a month. No one asked, but it's HN so I'll say it anyway. I think it's a questionable \"vc business\" but a great business for…",{"value":67,"source":68,"evidence":69},"Free tier plus about $9.99 a month, per a commenter's reading of the pricing","commenter",[70,71],{"id":63,"by":64,"text":65},{"id":72,"by":73,"text":74},"28738437","boulos","Disclosure: I used to work for GCP and launched Preemptible VMs. Congrats! Can I suggest charging more? IIUC, your business plan is $10\u002Fmonth for the company, regardless of…",{"value":76,"source":52,"evidence":77},"Their GAN training on expensive instances cost about $1000 per category class",[78],{"id":55,"by":11,"text":56},[],"unknown","beta","team",{"praise":84,"objections":97,"requests":131,"questions":155,"alternatives":175},[85],{"value":86,"source":68,"evidence":87},"Useful for teams and hobbyists paying for spot GPU time",[88,89,93],{"id":63,"by":64,"text":65},{"id":90,"by":91,"text":92},"28738208","cpalumbo","Ha! That's interesting. We use AWS instances for model development and costs are definitely an issue. Sending this to my team. Good luck!",{"id":94,"by":95,"text":96},"28738357","zachthewf","Really excited to try it out. I've had a heck of a time setting up spot instance training on Sagemaker in the past so simplification efforts are much appreciated.",[98,109,124],{"value":99,"source":68,"evidence":100},"Nimbo and Spotty are very similar and were not clearly credited",[101,105],{"id":102,"by":103,"text":104},"28739364","mjaques","I'd like to point out that this seems extremely similar to Nimbo (https:\u002F\u002Fnimbo.sh), to the extent that even some of the terminal messages are exactly the same, and even parts of…",{"id":106,"by":107,"text":108},"28743125","rgj","You should really mention \u002F give attribution \u002F emphasize more that this is a fork of https:\u002F\u002Fspotty.cloud and you took a lot from https:\u002F\u002Fgithub.com\u002Fnimbo-sh\u002Fnimbo as well.",{"value":110,"source":68,"evidence":111},"Native retry on spot instances or managed tools like Metaflow, AWS Batch or GKE may already cover this",[112,116,120],{"id":113,"by":114,"text":115},"28740356","Gatesyp","Read through the homepage, but not entirely sure -- Why not just train on Spot Instances with a retry implemented? I see that SpotML has a configurable fall back to On-Demand…",{"id":117,"by":118,"text":119},"28738947","ignoramous","Neat. Congratulations on the launch, Vishnu! Apart from the fact that it could deploy to both GCP and AWS, what does it do differently than AWS Batch [0]? When we had a similar…",{"id":121,"by":122,"text":123},"28739213","twistedpair","I'm not sure what this provides over what GCP already offers. 4 years ago I switched my co's ML training to use GKE (Google Kubernetes Engine) using a cluster of preemptible…",{"value":125,"source":68,"evidence":126},"Cloud providers could copy this functionality",[127],{"id":128,"by":129,"text":130},"28742829","ssivark","Kudos; this is likely a very useful tool! So useful (and potentially lucrative) that AWS\u002FGCP would likely take over this functionality eventually -- either by building on top of…",[132,143,151],{"value":133,"source":68,"evidence":134},"Support distributed training and multiple instances",[135,139],{"id":136,"by":137,"text":138},"28738353","oakfr","Does it apply to jobs running on multiple instances, e.g using dask?",{"id":140,"by":141,"text":142},"28743491","sandGorgon","do you have an example dockerfile for a ML job ? say xgboost with a CSV data file ... or tensorflow with image data ? unable to figure out how to construct the job itself and how…",{"value":144,"source":68,"evidence":145},"Add examples and documentation",[146,147],{"id":140,"by":141,"text":142},{"id":148,"by":149,"text":150},"28738689","ShamelessC","This looks cool. How much SpotML-specific code is needed to target SpotML? I'm assuming it doesn't just magically detect the training loop in an existing pytorch codebase and…",{"value":152,"source":68,"evidence":153},"Charge more or branch into inference",[154],{"id":72,"by":73,"text":74},[156,161,168],{"value":157,"source":68,"evidence":158},"How it differs from spot retry and Metaflow or AWS Batch",[159,160],{"id":113,"by":114,"text":115},{"id":117,"by":118,"text":119},{"value":162,"source":68,"evidence":163},"How state is saved within the shutdown time limit",[164],{"id":165,"by":166,"text":167},"28740674","JZL003","I'm interested in how they 'hibernate'\u002Fsave the state of the instances within the shutdown time limit. I was also looking into this for myself, there are ways of using docker to…",{"value":169,"source":68,"evidence":170},"Which AWS permissions are needed and how billing works",[171],{"id":172,"by":173,"text":174},"28739736","sknzl","How does this work? From the documentation it's clear that AWS credentials are required. Which permissions are necessary? This leads me to the assumption that the SpotML cli…",[176,180,186,190,194,201],{"value":177,"source":68,"evidence":178},"Nimbo",[179],{"id":102,"by":103,"text":104},{"value":181,"source":52,"evidence":182},"Spotty",[183],{"id":184,"by":11,"text":185},"28739455","Yes we liked the elegance of both the tool and the docs. So it's very much inspired from it. I must also give credit to another great tool https:\u002F\u002Fspotty.cloud\u002F from which this…",{"value":187,"source":68,"evidence":188},"AWS Batch",[189],{"id":117,"by":118,"text":119},{"value":191,"source":68,"evidence":192},"Metaflow",[193],{"id":113,"by":114,"text":115},{"value":195,"source":68,"evidence":196},"Cortex",[197],{"id":198,"by":199,"text":200},"28743383","praveenhm","This is cool project, will try it out. how does it compare to cortex.dev? they use AWS spot instances too.",{"value":202,"source":68,"evidence":203},"Grid.ai",[204],{"id":205,"by":206,"text":207},"28739692","barefeg","How does it compare with grid.ai?",{"comments":209,"answered":18,"key":210},20,[211,214,215],{"id":212,"text":213},"28738578","Thanks for the feedback, yeah at this stage we really just wanted to find out if we can build something useful for the community. Agreed on the pricing suggestion. Also,…",{"id":55,"text":56},{"id":216,"text":217},"28738504","Thank you!",{"commenters":219,"max_depth":220,"maker_replies":209,"first_comment_mins":44,"lifespan_hours":221},33,4,146,[223,225,227,230,233,235,237,239,241,242,244,245,248,249,252,255,257,259,262,263,266,267,269,272,275,278,281,283,285,288,291,293,295,297,299,301,306,309,314,319,322,326,328,331,334,337,339,341,344,346,348,350,352,356,358,360,362,364,368,371,373,375,377],[72,38,47,224,73,38],"project\u002Fmixed\u002Fpricing",[212,38,38,226,11,44],"maker\u002Fneutral\u002Fidea",[228,38,38,229,73,38],"28738654","commenter\u002Fneutral\u002Fperformance",[231,38,38,229,232,38],"28741560","nostrebored",[55,44,38,234,11,44],"project\u002Fneutral\u002Fidea",[90,44,34,236,91,38],"project\u002Fpositive\u002Fidea",[238,44,34,236,137,38],"28738304",[216,44,38,240,11,44],"maker\u002Fneutral\u002Fnone",[94,44,34,236,95,38],[243,44,38,240,11,44],"28738521",[63,44,47,224,64,38],[246,44,38,247,11,44],"28738487","maker\u002Fneutral\u002Fpricing",[117,44,47,234,118,38],[250,44,38,251,11,44],"28739286","maker\u002Fneutral\u002Fexecution",[253,44,38,254,118,38],"28739953","commenter\u002Fneutral\u002Fexecution",[128,44,47,256,129,38],"project\u002Fmixed\u002Fidea",[106,44,44,258,107,38],"maker\u002Fcritical\u002Fidea",[102,260,44,261,103,38],2,"project\u002Fcritical\u002Fidea",[184,260,38,226,11,44],[264,47,44,261,265,38],"28742043","Mehdi2277",[113,220,47,234,114,38],[268,220,38,226,11,44],"28740564",[270,220,38,254,271,38],"28741931","vtuulos",[273,34,47,256,274,38],"28738464","site-packages1",[276,34,38,229,277,38],"28738779","a-dub",[279,34,38,280,11,44],"28738955","maker\u002Fneutral\u002Fperformance",[282,34,38,229,277,38],"28740957",[284,34,38,226,11,44],"28738920",[286,34,44,261,287,38],"28740082","electroly",[165,289,47,290,166,38],6,"project\u002Fneutral\u002Fperformance",[292,289,38,251,11,44],"28740702",[294,289,38,229,166,38],"28740740",[296,289,38,254,166,38],"28740704",[172,32,47,298,173,38],"project\u002Fneutral\u002Fprivacy",[300,32,38,247,11,44],"28740086",[302,303,47,304,305,38],"28740410",8,"project\u002Fneutral\u002Fdesign","tyingq",[307,303,38,308,11,44],"28740454","maker\u002Fneutral\u002Fdesign",[310,311,38,312,313,38],"28741472",9,"other\u002Fneutral\u002Fpricing","freediver",[315,316,38,317,318,38],"28743619",10,"other\u002Fneutral\u002Fnone","TrackerFF",[320,316,38,317,321,38],"28756781","hcarlens",[323,33,47,324,325,38],"28738909","project\u002Fneutral\u002Fpricing","dexter89_kp3",[327,33,38,247,11,44],"28739148",[148,329,47,330,149,38],12,"project\u002Fneutral\u002Fdocs",[332,329,38,333,11,44],"28739109","maker\u002Fneutral\u002Fdocs",[335,329,34,336,149,38],"28739121","maker\u002Fpositive\u002Fdocs",[121,18,38,338,122,38],"commenter\u002Fcritical\u002Fidea",[340,18,47,256,73,38],"28739325",[342,18,44,261,343,38],"28739795","sitkack",[198,345,47,234,199,38],14,[347,345,38,226,11,44],"28811841",[140,349,47,330,141,38],15,[351,349,38,333,11,44],"28811755",[353,354,34,236,355,38],"28738196",16,"eyesasc",[136,357,47,234,137,38],17,[359,357,38,226,11,44],"28738408",[361,357,47,251,73,38],"28738459",[363,357,38,240,11,44],"28738589",[365,366,34,236,367,38],"28744763",18,"ilyakrotov",[205,369,38,370,206,38],19,"other\u002Fneutral\u002Fidea",[372,369,38,226,11,44],"28740041",[374,369,38,317,206,38],"28744068",[376,369,38,317,206,38],"28744076",[378,209,34,379,380,38],"28739950","project\u002Fpositive\u002Fnone","hallqv",[],[383,388,393,398,402,406],{"id":384,"name":385,"points":386,"category":387},"28704873","Haxplore",603,"consumer_social",{"id":389,"name":390,"points":391,"category":392},"28765183","ChessCoach",506,"games",{"id":394,"name":395,"points":396,"category":397},"28758226","Hurl",223,"dev_tools",{"id":399,"name":400,"points":401,"category":397},"28718066","UI for Apache Kafka",158,{"id":403,"name":404,"points":221,"category":405},"28775313","IndiePaper","web_saas",{"id":407,"name":408,"points":409,"category":410},"28725803","PomPom",138,"languages",false,"v2",{"id":414,"name":5,"line":6,"ai":411,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":415,"top":18,"kw":416,"maker":11,"domain":10,"tier":19,"attempts":38,"strip":417,"url":9,"mix":418,"praise":86,"objection":99,"sub":21},28738141,0.05,[24,25,26,27,28],"3005553030111303001303300033531335335005",[32,33,34,34],1794036611862]