[{"data":1,"prerenderedAt":266},["ShallowReactive",2],{"project-22869284":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":40,"post_text":43,"what":44,"said":85,"maker_in_thread":175,"shape":183,"grid":186,"attempts":226,"same_week":227,"maker_shows":31,"maker_hidden":258,"reader_version":259,"card":260},"22869284","Cortex","Open-source alternative to SageMaker for model serving","title","Show HN: Cortex – Open-source alternative to SageMaker for model serving","https:\u002F\u002Fgithub.com\u002Fcortexlabs\u002Fcortex\u002Ftree\u002Fv0.15.1","github.com","calebkaiser",1586887399,2020,65,19,913,3028,31,"notable","ai","model serving","cli",[21,24,25,26],"machine learning deployment","kubernetes","sagemaker alternative","loved",0.219,{"positive":30,"neutral":31,"mixed":32,"critical":32},4,6,0,10,{"idea":35,"design":37,"execution":38,"pricing":39},{"positive":36,"critical":32},1,{"positive":32,"critical":32},{"positive":36,"critical":32},{"positive":32,"critical":32},[41,42,32,41,41,42,32,42,32,41,32,32,32,42,32,42,32,42,32],5,3,null,{"problem":45,"audience":52,"pricing":43,"motivation":60,"time_to_build":43,"stack":65,"openness":82,"stage":83,"team_size":84},{"value":46,"source":47,"evidence":48},"Deploying trained ML models as production APIs without managing Kubernetes and AWS infrastructure","maker",[49],{"id":50,"by":11,"text":51},"22870522","The simplest way to put it is that Kubeflow (whose team we have a ton of respect for) is a tool for helping devops engineers build their ML deployment platform on kubernetes,…",{"value":53,"source":54,"evidence":55},"Developers and data scientists deploying ML models to production","inferred",[56,57],{"id":50,"by":11,"text":51},{"id":58,"by":11,"text":59},"22871632","You're 100% right that Cortex is designed for the production use-case. A lot of our users are running Cortex for \"small\" production use cases, since the Cortex cluster can…",{"value":61,"source":47,"evidence":62},"Make ML model deployment easier than SageMaker and Kubeflow without requiring Kubernetes knowledge",[63,64],{"id":50,"by":11,"text":51},{"id":58,"by":11,"text":59},[66,72,76],{"value":67,"source":47,"evidence":68},"Docker",[69],{"id":70,"by":11,"text":71},"22870000","I'm sure others will comment, but in the meantime, some people have written up their experiences using Cortex in production. I'd point you to AI Dungeon:…",{"value":73,"source":47,"evidence":74},"Kubernetes",[75],{"id":50,"by":11,"text":51},{"value":77,"source":47,"evidence":78},"AWS EC2",[79],{"id":80,"by":11,"text":81},"22872809","In general, Cortex will be significantly cheaper because you're only paying AWS for EC2 (the bulk of the bill) and the other AWS services used (a much smaller portion of the…","open_source","launched","unknown",{"praise":86,"objections":103,"requests":115,"questions":132,"alternatives":155},[87,96],{"value":88,"source":89,"evidence":90},"Cleaner and cheaper alternative to SageMaker","commenter",[91,95],{"id":92,"by":93,"text":94},"22871080","itsderek23","This certainly looks like a cleaner way to deploy an ML model than SageMaker. Couple of questions: * Is this really for more intensive model inference applications that need a…",{"id":80,"by":11,"text":81},{"value":97,"source":89,"evidence":98},"Inspired other projects such as chantilly",[99],{"id":100,"by":101,"text":102},"22870607","Lemaxoxo","Hey there, great work. I used Cortex as an inspiration when designing chantilly: https:\u002F\u002Fgithub.com\u002Fcreme-ml\u002Fchantilly, which is a much less ambitious solution tailored towards…",[104,109],{"value":105,"source":89,"evidence":106},"Likely overkill for small models that do not need a cluster",[107,108],{"id":92,"by":93,"text":94},{"id":58,"by":11,"text":59},{"value":110,"source":47,"evidence":111},"Lacks built-in API authentication; users need AWS API Gateway",[112],{"id":113,"by":11,"text":114},"22871073","Right now, users handle API auth by using AWS API gateway in front of Cortex, but incorporating AWS API Gateway into Cortex to automate this is on our short term roadmap.",[116,123,127],{"value":117,"source":89,"evidence":118},"Managed service offering",[119],{"id":120,"by":121,"text":122},"22887984","cgoltsev","Is it possible to partner with you to offer a managed service for Cortex? We are looking at your solution to offer our clients for deployment.",{"value":124,"source":47,"evidence":125},"Built-in API Gateway integration for authentication",[126],{"id":113,"by":11,"text":114},{"value":128,"source":89,"evidence":129},"Tighter version control integration",[130,131],{"id":92,"by":93,"text":94},{"id":58,"by":11,"text":59},[133,137,144],{"value":134,"source":89,"evidence":135},"Is it suitable for small models without a cluster?",[136],{"id":92,"by":93,"text":94},{"value":138,"source":89,"evidence":139},"Can it be run cheaply for a side project?",[140],{"id":141,"by":142,"text":143},"22872663","dodata","One of the things that has deterred me from SageMaker is how expensive it can be for a side project. Real-time endpoints start at $40-$50 per month, which would be a bit too much…",{"value":145,"source":89,"evidence":146},"How does it compare to Lambda and Kubeflow?",[147,151],{"id":148,"by":149,"text":150},"22872037","cameronfraser","Why would I use this over deploying the model to a lambda function aside from lack of GPU? (not trying to be confrontational, genuinely don't know) Won't lambda functions scale…",{"id":152,"by":153,"text":154},"22869952","_____smurf_____","How is this compared to KubeFlow?",[156,161,165,169],{"value":157,"source":89,"evidence":158},"SageMaker",[159,160],{"id":92,"by":93,"text":94},{"id":141,"by":142,"text":143},{"value":162,"source":89,"evidence":163},"Kubeflow",[164],{"id":152,"by":153,"text":154},{"value":166,"source":89,"evidence":167},"AWS Lambda",[168],{"id":148,"by":149,"text":150},{"value":67,"source":89,"evidence":170},[171],{"id":172,"by":173,"text":174},"22869678","bobosha","Has anyone used Cortex in production? - Could you share your experiences? - why would one choose this over docker for instance?",{"comments":176,"answered":176,"key":177},7,[178,179,180],{"id":58,"text":59},{"id":70,"text":71},{"id":181,"text":182},"22870750","Yep! Cortex deploys load balancers on AWS and manages queueing.",{"commenters":33,"max_depth":31,"maker_replies":176,"first_comment_mins":184,"lifespan_hours":185},30,41,[187,189,191,193,196,197,200,202,204,205,208,209,213,216,218,220,222,224,225],[100,32,41,188,101,32],"project\u002Fpositive\u002Fidea",[92,36,42,190,93,32],"project\u002Fneutral\u002Fdesign",[58,36,32,192,11,36],"maker\u002Fneutral\u002Fnone",[194,36,41,195,93,32],"22871925","maker\u002Fpositive\u002Fnone",[120,36,41,195,121,32],[172,198,42,199,173,32],2,"project\u002Fneutral\u002Fnone",[70,198,32,201,11,36],"project\u002Fneutral\u002Fexecution",[203,198,42,190,173,32],"22870629",[181,198,32,190,11,36],[206,198,41,207,173,32],"22870862","project\u002Fpositive\u002Fexecution",[113,198,32,190,11,36],[210,42,32,211,212,32],"22871386","other\u002Fneutral\u002Fnone","wikibob",[214,42,32,211,215,32],"22871486","ignoramous",[141,30,42,217,142,32],"project\u002Fneutral\u002Fpricing",[80,30,32,219,11,36],"project\u002Fpositive\u002Fpricing",[148,41,42,221,149,32],"project\u002Fneutral\u002Fidea",[223,41,32,221,11,36],"22872163",[152,31,42,221,153,32],[50,31,32,188,11,36],[],[228,233,238,243,248,253],{"id":229,"name":230,"points":231,"category":232},"22867775","Tom Nook's Laptop",552,"games",{"id":234,"name":235,"points":236,"category":237},"22837990","Valora",306,"creative",{"id":239,"name":240,"points":241,"category":242},"22867767","TerminusDB",231,"data",{"id":244,"name":245,"points":246,"category":247},"22858235","DJ3D",212,"consumer_social",{"id":249,"name":250,"points":251,"category":252},"22823320","LearnObit",209,"learning_science",{"id":254,"name":255,"points":256,"category":257},"22848779","Quiver Quant",202,"finance_crypto",false,"v2",{"id":261,"name":5,"line":6,"ai":258,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":262,"top":18,"kw":263,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":264,"url":9,"mix":265,"praise":88,"objection":105,"sub":21},22869284,0.22,[21,24,25,26],"5305530305000303030",[30,31,32,32],1794036589080]