[{"data":1,"prerenderedAt":226},["ShallowReactive",2],{"project-13072039":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":43,"post_text":45,"what":46,"said":77,"maker_in_thread":143,"shape":153,"grid":157,"attempts":189,"same_week":190,"maker_shows":44,"maker_hidden":219,"reader_version":220,"card":221},"13072039","ToroDB Stampede","Automagical MongoDB to PostgreSQL, 100x Faster Queries","title","Show HN: ToroDB Stampede: Automagical MongoDB to PostgreSQL, 100x Faster Queries","https:\u002F\u002Fwww.torodb.com\u002Fstampede\u002F","torodb.com","ahachete",1480523337,2016,43,17,817,2264,37,"shown","data","database replication","other",[24,25,26,27,28],"mongodb","postgresql","replication","etl","sql","curious",0.13,{"positive":32,"neutral":33,"mixed":34,"critical":35},4,5,0,1,10,{"design":38,"performance":39,"idea":41,"execution":42},{"positive":34,"critical":34},{"positive":40,"critical":34},2,{"positive":35,"critical":35},{"positive":34,"critical":34},[34,44,34,33,44,44,34,34,33,34,44,33,35,34,44,34,33],3,null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":61,"openness":74,"stage":75,"team_size":76},{"value":48,"source":49,"evidence":50},"Running MongoDB aggregation queries slowly; need relational access to document data without designing schemas by hand","commenter",[51,55,58],{"id":52,"by":53,"text":54},"13074379","gortiz","Hi javiermaestro! All benchmarks have been done on AWS i2.xlarge instances. They have 4 vCPUs, 30GBs of RAM and 800GB local SSDs. It is true that we are not doing an apples to…",{"id":56,"by":11,"text":57},"13072793","Hi kasano. One of the most relevant use cases of ToroDB Stampede is precisely what you say, replacing MongoDB to PostgreSQL ETLs, where you have to design the schema, solve data…",{"id":59,"by":11,"text":60},"13095316","Thanks for the feedback. I agree sometimes myself think the same way too. In this case, it conveys a non-dangerous, powerful message: rather than having to design your DDL, and…",[62,70],{"value":63,"source":64,"evidence":65},"PostgreSQL","maker",[66,69],{"id":67,"by":11,"text":68},"13073402","So basically yes. ToroDB creates columns and tables on the fly. All the added columns are nullable, so the ALTER TABLE ADD COLUMN is an almost free operation in PostgreSQL. Sure,…",{"id":56,"by":11,"text":57},{"value":71,"source":49,"evidence":72},"MongoDB",[73],{"id":52,"by":53,"text":54},"unknown","beta","company",{"praise":78,"objections":93,"requests":108,"questions":116,"alternatives":131},[79,86],{"value":80,"source":49,"evidence":81},"Impressive performance gains over MongoDB",[82],{"id":83,"by":84,"text":85},"13072738","3manuek","Impressive performance, even compared against a Mongo shard with 3 nodes!",{"value":87,"source":49,"evidence":88},"Relational approach to document data is a sensible idea",[89],{"id":90,"by":91,"text":92},"13073266","postila","It's so funny to hear sometimes that noSQL is better that SQL databases and see how much more powerful tools can be built using old good relational engines. Also, this story a…",[94,101],{"value":95,"source":49,"evidence":96},"'Automatic' and 'magic' naming suggests loss of control",[97],{"id":98,"by":99,"text":100},"13089088","stephenr","FYI: myself (and I believe a reasonable number of other software developers) treat the word \"Automatic\" as a red flag. Magic means you don't understand how something works.…",{"value":102,"source":49,"evidence":103},"Benchmark methodology and hardware specs were unclear",[104],{"id":105,"by":106,"text":107},"13073895","javiermaestro","Hi there! Stampede looks nice, congrats! Quick question about the 100x performance claim and the benchmarks in https:\u002F\u002Fwww.8kdata.com\u002Fblog\u002Fannouncing-torodb-stampede-1-0-beta\u002F: -…",[109],{"value":110,"source":49,"evidence":111},"Benchmark with dataset fitting in RAM on a single MongoDB instance",[112,113],{"id":105,"by":106,"text":107},{"id":114,"by":106,"text":115},"13076158","LOL I re-read the article and found the specs. I really read it but somehow managed to skip the paragraph or something :-? Anyway, my point stands. I'd use a single instance in…",[117,124],{"value":118,"source":49,"evidence":119},"How optional fields and sparse tables are handled",[120],{"id":121,"by":122,"text":123},"13073321","davidpardo","How do you deal with optional fields in documents? do you modify the table schema on the run? If there's a larg-ish number of optional fields, but each document has only or a few…",{"value":125,"source":49,"evidence":126},"Whether it can run on an existing database and schema",[127],{"id":128,"by":129,"text":130},"13072741","kasano","One of our engineer's recently floated up the idea of using ToroDB Stampede for replacing our MongoDB -> PostgreSQL ETL, since it's merely a set of Python scripts parsing JSON…",[132],{"value":133,"source":49,"evidence":134},"MongoDB BI Connector",[135,136,139],{"id":90,"by":91,"text":92},{"id":137,"by":11,"text":138},"13073454","Indeed, MongoDB BI connector v1 was based on PostgreSQL. However, they used 9.4's foreign data wrappers and since those could not push down query clauses, the connector itself…",{"id":140,"by":141,"text":142},"13077700","eb0la","I've tried that connector and decided against using it. The problem was you had to define your schema by hand and in mongo sometimes you have legacy data records with different…",{"comments":144,"answered":44,"key":145},6,[146,149,150],{"id":147,"text":148},"13072682","Hi, Álvaro here, from 8Kdata, the company behind ToroDB. Please let us know if you have any questions or comments about ToroDB. We will be happy to answer them :) Enjoy!",{"id":67,"text":68},{"id":151,"text":152},"13078829","So far we have benchmarked situations where dataset > RAM or >> RAM. I think it is an interesting point to also analyze the case when dataset \u003C RAM, to see how efficiently both…",{"commenters":154,"max_depth":33,"maker_replies":144,"first_comment_mins":155,"lifespan_hours":156},9,61,67,[158,160,162,164,166,168,169,171,173,176,177,179,182,184,185,187,188],[147,34,34,159,11,35],"maker\u002Fneutral\u002Fnone",[121,34,44,161,122,34],"project\u002Fneutral\u002Fdesign",[67,34,34,163,11,35],"project\u002Fpositive\u002Fdesign",[105,34,33,165,106,34],"project\u002Fpositive\u002Fperformance",[52,34,44,167,53,34],"project\u002Fneutral\u002Fperformance",[114,34,44,167,106,34],[151,34,34,170,11,35],"maker\u002Fneutral\u002Fperformance",[90,35,34,172,91,34],"other\u002Fpositive\u002Fidea",[174,35,33,175,53,34],"13073420","project\u002Fpositive\u002Fidea",[137,35,34,167,11,35],[140,35,44,178,141,34],"project\u002Fneutral\u002Fexecution",[180,35,33,181,53,34],"13078516","maker\u002Fpositive\u002Fnone",[98,40,35,183,99,34],"project\u002Fcritical\u002Fidea",[59,40,34,175,11,35],[128,44,44,186,129,34],"project\u002Fneutral\u002Fidea",[56,44,34,175,11,35],[83,32,33,165,84,34],[],[191,196,201,205,210,215],{"id":192,"name":193,"points":194,"category":195},"13031492","CSS Reference",1117,"dev_tools",{"id":197,"name":198,"points":199,"category":200},"13030603","Dply",673,"infrastructure",{"id":202,"name":203,"points":204,"category":195},"13053137","Antd",507,{"id":206,"name":207,"points":208,"category":209},"13056678","React Native Express",496,"learning_science",{"id":211,"name":212,"points":213,"category":214},"13065670","Anycomplete",390,"productivity",{"id":216,"name":217,"points":218,"category":200},"13048334","Migadu",336,false,"v2",{"id":222,"name":5,"line":6,"ai":219,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":30,"top":18,"kw":223,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":224,"url":9,"mix":225,"praise":80,"objection":95,"sub":21},13072039,[24,25,26,27,28],"03053300503510305",[32,33,34,35],1794036550141]