[{"data":1,"prerenderedAt":472},["ShallowReactive",2],{"project-12227507":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":43,"post_text":46,"what":47,"said":85,"maker_in_thread":201,"shape":212,"grid":216,"attempts":434,"same_week":435,"maker_shows":18,"maker_hidden":464,"reader_version":465,"card":466},"12227507","asyncpg","1M rows\u002Fs from Postgres to Python","title","Show HN: 1M rows\u002Fs from Postgres to Python","http:\u002F\u002Fmagic.io\u002Fblog\u002Fasyncpg-1m-rows-from-postgres-to-python\u002F","magic.io","1st1",1470335382,2016,296,89,67,2264,3,"legend","languages","database driver","library",[24,25,26,27],"postgresql","asyncio","python driver","database performance","loved",0.126,{"positive":31,"neutral":32,"mixed":33,"critical":33},6,31,1,39,{"performance":36,"idea":38,"docs":40,"execution":41,"design":42},{"positive":37,"critical":33},2,{"positive":37,"critical":39},0,{"positive":39,"critical":39},{"positive":37,"critical":39},{"positive":39,"critical":39},[18,18,33,39,39,18,39,39,18,39,39,39,39,44,39,39,18,39,44,44,18,18,18,18,18,18,45,44,18,39,18,45,18,39,18,18,18,18,18,18],5,4,null,{"problem":46,"audience":48,"pricing":46,"motivation":58,"time_to_build":46,"stack":66,"openness":82,"stage":83,"team_size":84},{"value":49,"source":50,"evidence":51},"Python developers using asyncio with PostgreSQL","inferred",[52,55],{"id":53,"by":11,"text":54},"12228241","asyncpg was designed for asyncio (and async\u002Fawait). So it can be used in any asyncio, Tornado (and soon Twisted) program. Django doesn't support asyncio, it's built for blocking…",{"id":56,"by":11,"text":57},"12228174","We've open sourced asyncpg just a couple of weeks ago, and right now it has a beta status. Not that we're aware of any bugs in it or plan to add a lot of new features. So I guess…",{"value":59,"source":60,"evidence":61},"Solve challenges in developing EdgeDB","commenter",[62],{"id":63,"by":64,"text":65},"12228858","RedCrowbar","> I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the…",[67,74],{"value":68,"source":69,"evidence":70},"Python","maker",[71],{"id":72,"by":11,"text":73},"12229985","I use SublimeText with a Cython highlighter; @redcrowbar uses Atom. I started coding uvloop & asyncpg directly in Cython, basically from scratch. Debugging & refactoring are…",{"value":75,"source":69,"evidence":76},"Cython",[77,81],{"id":78,"by":79,"text":80},"12229970","denfromufa","How do you develop your Cython code? Do you use PyCharm? Did you start off from pure Python code? How easy is debugging and refactoring?",{"id":72,"by":11,"text":73},"open_source","beta","team",{"praise":86,"objections":105,"requests":130,"questions":162,"alternatives":184},[87,94],{"value":88,"source":60,"evidence":89},"Fast setup and easy install on OSX",[90],{"id":91,"by":92,"text":93},"12228476","rambos","Fastest setup of a PG client, to date, on OSX for me. No nuances or other package installers like Fink necessary.",{"value":95,"source":60,"evidence":96},"Strong performance for database fetching",[97,101],{"id":98,"by":99,"text":100},"12229881","userbinator","At 3.7GHz that's roughly 3700 clock cycles per row. With something that low, I'd be really interested to see a breakdown of how many cycles were spent doing what.",{"id":102,"by":103,"text":104},"12228512","zzzeek","the 1M row\u002Fsec part is easy to explain: > asyncpg extensively uses PostgreSQL prepared statements. This is an important optimization feature, as it allows to avoid repeated…",[106,116,123],{"value":107,"source":60,"evidence":108},"New driver may lack stability compared to psycopg2",[109,112],{"id":110,"by":103,"text":111},"12228852","I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the…",{"id":113,"by":114,"text":115},"12228415","Dowwie","Does this library address any of the issues\u002Fconcerns that Mike Bayer discussed in this earlier blog post? http:\u002F\u002Ftechspot.zzzeek.org\u002F2015\u002F02\u002F15\u002Fasynchronous-python-and-databases\u002F",{"value":117,"source":60,"evidence":118},"Benchmark run single-threaded, unfair to Go",[119],{"id":120,"by":121,"text":122},"12228115","jargino","> For fairness, all tests were run in a single-thread (GOMAXPROCS=1 for Go code) in async mode. Isn't this just hobbling Go so you can beat it? Doesn't seem like something done…",{"value":124,"source":60,"evidence":125},"Async adds little speed for typical database workloads",[126,127],{"id":102,"by":103,"text":104},{"id":128,"by":103,"text":129},"12228850","> This simply isn't true for all use cases. Everytime I talk about my blog post, I am super careful to qualify: in this case, \"usually has little to do w\u002F speed\" and other…",[131,144,155],{"value":132,"source":60,"evidence":133},"Synchronous version of the driver",[134,138,141],{"id":135,"by":136,"text":137},"12228953","allendoerfer","Sorry to interrupt: You two can click on the timestamp to answer your posts. Please continue, we are listening carefully. And since I am already here @1st1: How hard would it be…",{"id":139,"by":64,"text":140},"12229287","> I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally…",{"id":142,"by":136,"text":143},"12229074","I think you should open a bounty for it. As questions like \"Can I use this with Django\u002FSQLAlchemy\u002F…?\" showed, people love drop-in replacements that speed up their code,…",{"value":145,"source":60,"evidence":146},"Django support",[147,151],{"id":148,"by":149,"text":150},"12228170","bedros","would this work with django?",{"id":152,"by":153,"text":154},"12229675","wampler","Cool! Wish there would be a Django wrapper, but then realised it is designed for asyncio",{"value":156,"source":60,"evidence":157},"Add command-line psql to benchmarks",[158],{"id":159,"by":160,"text":161},"12232555","xgess","i'd love to see raw, command-line psql added to these benchmarks",[163,170,177],{"value":164,"source":60,"evidence":165},"Does it handle datetimes and escaping automatically?",[166],{"id":167,"by":168,"text":169},"12227969","Veratyr","Does this automatically escape\u002Fconvert things like datetimes in a query or does that need to be done separately? The docs are a little sparse on details. Looks fantastic anyway!",{"value":171,"source":60,"evidence":172},"Does it work with Amazon Redshift?",[173],{"id":174,"by":175,"text":176},"12229672","jamiesonbecker","Would this work\u002Fbe binary-compatible with Amazon Red Shift?",{"value":178,"source":60,"evidence":179},"Would PyPy be supported given Cython?",[180],{"id":181,"by":182,"text":183},"12231973","pxlfkr","Using Cython means you don't\u002Fcan't use Pypy? Was that something you considered, e.g. with psycopg2cffi or a pure-python driver? It seems to be the defacto performant runtime for…",[185,190,194],{"value":186,"source":60,"evidence":187},"psycopg2",[188,189],{"id":102,"by":103,"text":104},{"id":110,"by":103,"text":111},{"value":191,"source":60,"evidence":192},"pg8000",[193],{"id":102,"by":103,"text":104},{"value":195,"source":60,"evidence":196},"epgsql (Erlang)",[197],{"id":198,"by":199,"text":200},"12228320","davidw","Wonder how this (Erlang driver) would stack up in the benchmarks (devel branch): https:\u002F\u002Fgithub.com\u002Fepgsql\u002Fepgsql",{"comments":202,"answered":203,"key":204},19,11,[205,206,209],{"id":56,"text":57},{"id":207,"text":208},"12228132","Python and nodejs applications are usually deployed in a multiprocess configuration (and Go apps without GOMAXPROCS). So in production, applications in all of those languages use…",{"id":210,"text":211},"12232288","> I think it would be beneficial if you would split your project in 2, separating the IO from the rest (like hyper does https:\u002F\u002Fgithub.com\u002FLukasa\u002Fhyper). I'd like to do that, but…",{"commenters":213,"max_depth":203,"maker_replies":202,"first_comment_mins":214,"lifespan_hours":215},42,69,316,[217,219,221,225,227,229,231,235,238,241,244,247,251,252,254,257,258,259,263,265,267,270,272,274,275,277,279,281,283,284,287,289,291,293,295,298,300,302,305,307,309,311,314,318,320,323,325,328,330,333,335,337,340,343,346,349,351,353,356,358,359,360,362,364,368,371,376,379,383,388,389,392,393,396,399,402,404,406,409,410,412,414,416,420,422,424,426,428,430],[98,39,18,218,99,39],"project\u002Fneutral\u002Fperformance",[220,39,18,218,64,39],"12232001",[222,33,18,223,224,39],"12228044","project\u002Fneutral\u002Fidea","tjpaudio",[56,33,39,226,11,33],"maker\u002Fneutral\u002Fdocs",[120,37,33,228,121,39],"project\u002Fcritical\u002Fperformance",[207,37,39,230,11,33],"maker\u002Fneutral\u002Fperformance",[232,37,39,233,234,39],"12228152","commenter\u002Fneutral\u002Fnone","Thaxll",[236,37,39,233,237,39],"12228191","riquito",[239,37,39,233,240,39],"12228796","malux85",[242,37,39,233,243,39],"12228872","_qqea",[245,37,39,233,246,39],"12230602","nhumrich",[248,37,39,249,250,39],"12308971","other\u002Fneutral\u002Fnone","mirekrusin",[113,18,18,223,114,39],[102,18,39,253,103,39],"commenter\u002Fcritical\u002Fidea",[255,18,39,256,64,39],"12228601","commenter\u002Fneutral\u002Fperformance",[110,18,39,253,103,39],[63,18,39,233,64,39],[260,18,18,261,262,39],"12230503","project\u002Fmixed\u002Fidea","sametmax",[210,18,39,264,11,33],"maker\u002Fneutral\u002Fnone",[266,18,39,233,262,39],"12242837",[268,18,39,233,269,39],"12229093","rickycook",[139,18,39,271,64,39],"commenter\u002Fneutral\u002Fdesign",[273,18,39,256,11,33],"12228642",[128,18,39,256,103,39],[276,18,39,233,11,33],"12228869",[278,18,39,233,103,39],"12228884",[280,18,39,233,11,33],"12228900",[282,18,39,233,103,39],"12228923",[135,18,39,233,136,39],[285,18,39,286,11,33],"12229001","maker\u002Fneutral\u002Fdesign",[142,18,44,288,136,39],"maker\u002Fpositive\u002Fidea",[290,18,39,264,11,33],"12229129",[292,18,39,233,103,39],"12229144",[294,18,39,249,136,39],"12229304",[296,18,39,249,297,39],"12232525","infinite8s",[299,18,39,233,11,33],"12228967",[301,18,18,218,64,39],"12228951",[303,18,39,304,103,39],"12229025","commenter\u002Fcritical\u002Fperformance",[306,18,39,256,64,39],"12229087",[308,45,39,264,11,33],"12227914",[167,45,18,310,168,39],"project\u002Fneutral\u002Fdocs",[312,45,44,313,64,39],"12228011","project\u002Fpositive\u002Fexecution",[315,45,44,316,317,39],"12227982","project\u002Fpositive\u002Fperformance","rdtsc",[319,45,39,264,11,33],"12228017",[321,45,18,218,322,39],"12227992","random42",[324,45,18,230,64,39],"12228031",[326,45,18,223,327,39],"12228105","welder",[329,45,18,223,64,39],"12228150",[331,45,39,332,297,39],"12228245","commenter\u002Fneutral\u002Fidea",[334,45,18,223,64,39],"12228291",[336,45,39,249,246,39],"12230591",[338,45,18,218,339,39],"12228121","andrioni",[341,45,39,342,11,33],"12228147","maker\u002Fpositive\u002Fperformance",[344,45,18,226,345,39],"12228176","Keats",[347,45,39,348,64,39],"12228190","other\u002Fneutral\u002Fdocs",[350,45,18,226,345,39],"12228505",[352,45,39,226,11,33],"12228661",[354,45,18,286,355,39],"12228338","glogla",[357,45,18,286,64,39],"12228346",[91,44,44,313,92,39],[181,31,44,316,182,39],[361,31,18,218,64,39],"12232041",[363,31,18,218,246,39],"12232047",[365,366,39,249,367,39],"12228157",7,"spriggan3",[369,366,39,249,370,39],"12229076","also_on_sunday",[372,373,39,374,375,39],"12228069",8,"other\u002Fneutral\u002Fperformance","merb",[377,373,18,378,64,39],"12228134","project\u002Fneutral\u002Fnone",[380,381,39,249,382,39],"12230621",9,"Fiahil",[384,385,44,386,387,39],"12229554",10,"project\u002Fpositive\u002Fidea","thang",[148,203,18,378,149,39],[390,203,18,378,391,39],"12228209","JoelBennett",[53,203,39,378,11,33],[394,203,18,378,395,39],"12228516","mamcx",[397,203,39,398,11,33],"12228543","project\u002Fneutral\u002Fexecution",[400,203,39,249,401,39],"12229416","machbio",[198,403,18,218,199,39],12,[405,403,39,378,11,33],"12228366",[78,407,18,408,79,39],13,"maker\u002Fneutral\u002Fexecution",[72,407,39,408,11,33],[411,407,18,408,79,39],"12230168",[413,407,39,408,11,33],"12230200",[152,415,18,378,153,39],14,[417,418,18,218,419,39],"12231947",15,"amelius",[421,418,18,218,64,39],"12231966",[174,423,18,378,175,39],16,[425,423,39,398,11,33],"12229692",[427,423,18,378,64,39],"12229701",[159,429,18,218,160,39],17,[431,432,39,249,433,39],"12230782",18,"landmark3",[],[436,441,445,450,454,459],{"id":437,"name":438,"points":439,"category":440},"12260794","Generating fantasy maps",1004,"creative",{"id":442,"name":443,"points":444,"category":20},"12260958","Xaddress",415,{"id":446,"name":447,"points":448,"category":449},"12240542","Gping.io",184,"hardware",{"id":451,"name":452,"points":453,"category":20},"12241332","cfilter",181,{"id":455,"name":456,"points":457,"category":458},"12232293","Web Dev with Golang Anti-Textbook",158,"learning_science",{"id":460,"name":461,"points":462,"category":463},"12239096","deep learning models",127,"ai",false,"v2",{"id":467,"name":5,"line":6,"ai":464,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":468,"top":18,"kw":469,"maker":11,"domain":10,"tier":19,"attempts":39,"strip":470,"url":9,"mix":471,"praise":88,"objection":107,"sub":21},12227507,0.13,[24,25,26,27],"3310030030000500305533333345303430333333",[31,32,33,33],1794036546492]