[{"data":1,"prerenderedAt":234},["ShallowReactive",2],{"project-14212211":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":40,"post_text":43,"what":44,"said":58,"maker_in_thread":136,"shape":145,"grid":149,"attempts":196,"same_week":197,"maker_shows":42,"maker_hidden":227,"reader_version":228,"card":229},"14212211","Infreqdb","S3 backed key\u002Fvalue database for infrequent read access","title","Show HN: Infreqdb – S3 backed key\u002Fvalue database for infrequent read access","https:\u002F\u002Fgithub.com\u002Fturbobytes\u002Finfreqdb","github.com","sajal83",1493306919,2017,39,28,912,2242,41,"shown","data","key value database","library",[24,25,26,27,28],"key value store","s3 database","object storage","timeseries storage","infrequent access","curious",0.02,{"positive":32,"neutral":33,"mixed":34,"critical":35},1,4,0,2,7,{"idea":38,"pricing":39},{"positive":32,"critical":32},{"positive":34,"critical":32},[34,34,34,34,41,34,42,34,34,34,42,34,32,34,34,32,34,42,34,42,34],5,3,null,{"problem":43,"audience":43,"pricing":43,"motivation":45,"time_to_build":43,"stack":55,"openness":56,"stage":57,"team_size":57},{"value":46,"source":47,"evidence":48},"Storing timeseries data cheaply on S3 for infrequent reads, with batched writes","maker",[49,52],{"id":50,"by":11,"text":51},"14213080","Yes. I should include this info in the README calcs. For my usecase I intend to do 4 - 10 PUTs per hour, since they are batched. 10 PUTS\u002Fh = 7300 PUTs\u002Fmonth = $0.073\u002Fmonth. The…",{"id":53,"by":11,"text":54},"14213227","Athena looks cool. Didn't know about it. It probably describes what I'm trying to do. > Also why don't you dump the log data into a NoSQL like dynamoDB instead of S3 ? Price.",[],"open_source","unknown",{"praise":59,"objections":68,"requests":79,"questions":84,"alternatives":100},[60],{"value":61,"source":62,"evidence":63},"Useful idea; batching writes enables high-volume timeseries storage","commenter",[64],{"id":65,"by":66,"text":67},"14213386","marknadal","Stuff like this is great! And you said you mentioned you are primarily doing it for timeseries data - then you can easily batch writes to handle high volume throughput too! We…",[69],{"value":70,"source":62,"evidence":71},"Costs dominated by PUT requests; retries can rack up charges",[72,76],{"id":73,"by":74,"text":75},"14212919","spullara","Be careful using S3 for lots of small writes. The price is utterly dominated by the PUT calls which cost $0.01\u002F1000.",{"id":77,"by":74,"text":78},"14215032","Be most careful with retries. Saw a case where the bucket wasn't writable and the code just retried in a loop. You get charged for the API call, successful or not.",[80],{"value":81,"source":62,"evidence":82},"Clarify pricing in README calculations",[83],{"id":50,"by":11,"text":51},[85,92],{"value":86,"source":62,"evidence":87},"How does it differ from AWS Athena?",[88],{"id":89,"by":90,"text":91},"14213185","udkl","How is this different than AWS Athena ? - https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Faws\u002Famazon-athena-interactive-sql-queries-for-data-in-amazon-s3\u002F Also why don't you dump the log data into…",{"value":93,"source":62,"evidence":94},"Why not DynamoDB instead of S3?",[95,96],{"id":89,"by":90,"text":91},{"id":97,"by":98,"text":99},"14213319","eropple","Maybe you should take another look at pricing? DynamoDB is like $0.002\u002FGB\u002Fmonth more expensive than S3 for storage. Requests are more expensive, but you're also not reinventing…",[101,105,109,115,122,129],{"value":102,"source":62,"evidence":103},"AWS Athena",[104],{"id":89,"by":90,"text":91},{"value":106,"source":62,"evidence":107},"DynamoDB",[108],{"id":89,"by":90,"text":91},{"value":110,"source":62,"evidence":111},"MinIO",[112],{"id":113,"by":90,"text":114},"14213399","If your data storage requirement is semi-constant at 1.5TB and is ephemeral, have you looked at just using your own replicated servers ? That would probably turn out to be…",{"value":116,"source":62,"evidence":117},"TrailDB",[118],{"id":119,"by":120,"text":121},"14214196","vtuulos","We use a similar approach with TrailDB. You can make S3 access totally seamless with user-space page fault handling, which is pretty cool:…",{"value":123,"source":62,"evidence":124},"Datomic",[125],{"id":126,"by":127,"text":128},"14213628","derefr","So, a database that's has a file format of write-once content-addressed \"shards\" and is persisted\u002Fdistributed by \"offlining\" those shards into object storage, and then \"onlining\"…",{"value":130,"source":62,"evidence":131},"CouchDB",[132],{"id":133,"by":134,"text":135},"14219090","reubano","> The world needs an open-source, non-JVM-targeted Datomic.) Couchdb?",{"comments":36,"answered":41,"key":137},[138,139,142],{"id":50,"text":51},{"id":140,"text":141},"14213472","That's cool. Last week I tried googling for similar stuff but all I could find was people asking \"How to run postgres on S3\"...",{"id":143,"text":144},"14214402","This is pretty cool. Do you invalidate all pages if the file changes upstream on S3?",{"commenters":146,"max_depth":41,"maker_replies":36,"first_comment_mins":147,"lifespan_hours":148},11,80,25,[150,152,154,155,159,161,163,165,167,170,174,175,177,179,182,184,186,188,189,190,193],[73,34,34,151,74,34],"commenter\u002Fcritical\u002Fpricing",[50,34,34,153,11,32],"maker\u002Fneutral\u002Fpricing",[77,34,34,151,74,34],[156,34,34,157,158,34],"14215363","other\u002Fcritical\u002Fpricing","saurik",[65,32,41,160,66,34],"project\u002Fpositive\u002Fidea",[140,32,34,162,11,32],"other\u002Fneutral\u002Fnone",[119,35,42,164,120,34],"project\u002Fneutral\u002Fidea",[143,35,34,166,11,32],"project\u002Fneutral\u002Fperformance",[168,35,34,169,120,34],"14214533","commenter\u002Fneutral\u002Fnone",[171,35,34,172,173,34],"14214578","other\u002Fneutral\u002Fidea","rakoo",[89,42,42,164,90,34],[53,42,34,176,11,32],"project\u002Fneutral\u002Fpricing",[97,42,32,178,98,34],"project\u002Fcritical\u002Fpricing",[180,42,34,181,11,32],"14213405","commenter\u002Fneutral\u002Fpricing",[183,42,34,169,98,34],"14214075",[113,42,32,185,90,34],"project\u002Fcritical\u002Fidea",[187,42,34,164,11,32],"14220981",[126,33,42,164,127,34],[133,33,34,162,134,34],[191,41,42,164,192,34],"14213148","ddxv",[194,41,34,195,11,32],"14213191","maker\u002Fneutral\u002Fperformance",[],[198,203,208,213,218,223],{"id":199,"name":200,"points":201,"category":202},"14226889","jacquesmattheij.com",1264,"hardware",{"id":204,"name":205,"points":206,"category":207},"14254357","Cone",418,"dev_tools",{"id":209,"name":210,"points":211,"category":212},"14240626","Kryptonite",338,"security_privacy",{"id":214,"name":215,"points":216,"category":217},"14212054","Helio Workstation",305,"creative",{"id":219,"name":220,"points":221,"category":222},"14245538","VPS Comparison",228,"infrastructure",{"id":224,"name":225,"points":226,"category":207},"14210877","Pygit",208,false,"v2",{"id":230,"name":5,"line":6,"ai":227,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":30,"top":18,"kw":231,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":232,"url":9,"mix":233,"praise":61,"objection":70,"sub":21},14212211,[24,25,26,27,28],"000050300030100103030",[32,33,34,35],1794036554845]