[{"data":1,"prerenderedAt":449},["ShallowReactive",2],{"project-35042316":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":42,"post_text":44,"what":45,"said":81,"maker_in_thread":164,"shape":177,"grid":182,"attempts":409,"same_week":410,"maker_shows":367,"maker_hidden":441,"reader_version":442,"card":443},"35042316","UJRPC","Faster FastAPI with simdjson and io_uring on Linux 5.19","title","Show HN: Faster FastAPI with simdjson and io_uring on Linux 5.19","https:\u002F\u002Fgithub.com\u002Funum-cloud\u002Fujrpc","github.com","ashvardanian",1678117283,2023,290,90,158,3109,6,"legend","infrastructure","rpc server","library",[24,25,26,27,28],"json-rpc","io_uring","simdjson","fastapi alternative","c++ server","curious",-0.222,{"positive":32,"neutral":33,"mixed":34,"critical":35},2,14,0,5,21,{"performance":38,"idea":40,"docs":41},{"positive":32,"critical":39},3,{"positive":34,"critical":34},{"positive":34,"critical":32},[34,34,34,34,34,34,34,34,34,39,39,39,34,34,34,34,34,34,34,39,35,43,34,39,34,34,34,39,34,34,34,34,39,34,43,43,43,39,35,43],1,"A few months ago, I benchmarked FastAPI on an i9 MacBook Pro. I couldn't believe my eyes. A primary REST endpoint to `sum` two integers took 6 milliseconds to evaluate. It is okay if you are targeting a server in another city, but it should be less when your client and server apps are running on the same machine.\n\nFastAPI would have bottleneck-ed the inference of our lightweight UForm neural networks recently trending on HN under the title \"Beating OpenAI CLIP with 100x less data and compute\". (Thank you all for the kind words!) So I wrote another library.\n\nIt has been a while since I have wri",{"problem":46,"audience":52,"pricing":52,"motivation":53,"time_to_build":52,"stack":57,"openness":78,"stage":79,"team_size":80},{"value":47,"source":48,"evidence":49},"Python web frameworks like FastAPI add significant per-request overhead","maker",[50],{"id":51},"post",null,{"value":54,"source":48,"evidence":55},"Maker wanted a fast, usable RPC library after benchmarking FastAPI's overhead",[56],{"id":51},[58,62,65,68,72],{"value":59,"source":48,"evidence":60},"C++",[61],{"id":51},{"value":26,"source":48,"evidence":63},[64],{"id":51},{"value":25,"source":48,"evidence":66},[67],{"id":51},{"value":69,"source":48,"evidence":70},"picohttpparser",[71],{"id":51},{"value":73,"source":48,"evidence":74},"Python bindings via CPython",[75],{"id":76,"by":11,"text":77},"35044407","You are right! For the convenience of Python users, we have to introspect the messages and parse JSON into Python objects. Every member of every dictionary being allocated on…","open_source","prototype","unknown",{"praise":82,"objections":98,"requests":121,"questions":126,"alternatives":144},[83,91],{"value":84,"source":85,"evidence":86},"simdjson is a strong choice for fast JSON parsing","commenter",[87],{"id":88,"by":89,"text":90},"35046029","Genbox","simdjson is awesome (Lemire does a lot of great stuff) and io_uring is a good use of circular buffers to get efficient IO, but I always squirm a little when I see JSON in a…",{"value":92,"source":85,"evidence":93},"Performance claims are impressive relative to FastAPI",[94],{"id":95,"by":96,"text":97},"35043349","dom96","100x faster than FastAPI seems easy. I wonder how it compares to other fast Python libraries like Japronto[1] and non-Python ones too. 1 - https:\u002F\u002Fgithub.com\u002Fsqueaky-pl\u002Fjapronto",[99,106,114],{"value":100,"source":85,"evidence":101},"Python work in handlers would bottleneck the speedup",[102],{"id":103,"by":104,"text":105},"35044940","tarasglek","The net performance of your system is only as fast as the slowest part. So, you optimized your C IO loop and it's really fast when there is not much Python...but soon as you add…",{"value":107,"source":85,"evidence":108},"JSON is a poor format for high-performance or tabular data",[109,113],{"id":110,"by":111,"text":112},"35042709","pbreit","I was just thinking how lousy a data format JSON is for tabular data. Would sending it down in CSV improve things at all?",{"id":88,"by":89,"text":90},{"value":115,"source":85,"evidence":116},"Project is pre-production and lacks docs",[117],{"id":118,"by":119,"text":120},"35042807","anentropic","https:\u002F\u002Fwww.unum.cloud\u002Fujrpc Is this supposed to be just a blank page, which is what I see?",[122],{"value":123,"source":85,"evidence":124},"Documentation portal",[125],{"id":118,"by":119,"text":120},[127,138],{"value":128,"source":85,"evidence":129},"Whether protobuf or Arrow would be better for transport",[130,134],{"id":131,"by":132,"text":133},"35043092","billconan","I'm curious if json + simdjson will outperform protobuf. I'm picking a protocol for my project. I was looking at protobuf, but this post made me think otherwise.",{"id":135,"by":136,"text":137},"35042757","ZeroCool2u","If anything you'd probably want to send it in Arrow[1] format. CSV's don't even preserve data types. [1]: https:\u002F\u002Farrow.apache.org\u002F",{"value":139,"source":85,"evidence":140},"How simdjson handles instruction sets at deploy time",[141],{"id":142,"by":136,"text":143},"35042733","Though I have read a bit about simdjson I'm not super familiar with how it works, so please forgive me if this is a silly question. Since simdjson is where a lot of the…",[145,149,156,160],{"value":146,"source":85,"evidence":147},"gRPC",[148],{"id":131,"by":132,"text":133},{"value":150,"source":85,"evidence":151},"yyjson",[152],{"id":153,"by":154,"text":155},"35042895","Dachande663","How does yyjson[0] compare to simdjson? Their benchmarks suggest it could be a positive. [0] https:\u002F\u002Fgithub.com\u002Fibireme\u002Fyyjson",{"value":157,"source":85,"evidence":158},"Japronto",[159],{"id":95,"by":96,"text":97},{"value":161,"source":85,"evidence":162},"Apache Arrow",[163],{"id":135,"by":136,"text":137},{"comments":165,"answered":166,"key":167},9,4,[168,171,174],{"id":169,"text":170},"35042716","There is a simple way and there is a hard way. Simple - memory-map the whole file with `mmap`. We also suggest using `madvice`, to inform the kernel that you will be accessing…",{"id":172,"text":173},"35043841","Thank you! You are right, FastAPI doesn't promise performance. But on the other hand, they have super usable. We wanted to show, that libraries can be both usable and fast, hence…",{"id":175,"text":176},"35043813","The difference would still be huge. It's exceptionally large even when we compare to the gRPC C++ server implementations.",{"commenters":178,"max_depth":179,"maker_replies":165,"first_comment_mins":180,"lifespan_hours":181},51,8,20,73,[183,187,190,193,196,199,201,203,204,206,209,212,214,216,218,221,224,227,230,233,235,237,240,242,245,248,250,253,255,258,261,263,265,268,271,273,275,278,281,284,286,289,292,296,297,298,300,304,305,307,309,312,313,314,317,320,323,325,328,330,332,335,339,341,343,347,350,352,354,356,358,361,364,366,368,370,373,375,378,381,384,387,390,393,395,397,399,402,404,407],[184,34,34,185,186,34],"35042609","other\u002Fneutral\u002Fnone","eatonphil",[188,34,34,185,189,34],"35042698","aobdev",[191,34,34,192,186,34],"35042729","commenter\u002Fneutral\u002Fnone",[194,34,34,185,195,34],"35045411","MikeDelta",[197,34,34,192,198,34],"35049797","capableweb",[200,34,34,185,195,34],"35053197",[202,34,34,192,198,34],"35054202",[169,34,34,185,11,43],[205,34,34,192,186,34],"35042752",[207,34,34,185,208,34],"35042862","loeg",[210,34,34,185,211,34],"35044512","Sesse__",[213,34,34,185,208,34],"35045101",[215,34,34,185,211,34],"35049175",[217,34,34,192,208,34],"35049450",[219,34,34,185,220,34],"35047709","roseway4",[222,34,34,185,223,34],"35050093","AtlasBarfed",[225,34,34,185,226,34],"35043364","saidinesh5",[228,34,34,192,229,34],"35043696","sl-dolt",[231,34,34,185,232,34],"35045635","tinselcity",[234,34,34,185,226,34],"35047919",[142,43,39,236,136,34],"project\u002Fneutral\u002Fnone",[238,43,39,236,239,34],"35042799","knutzui",[241,43,39,236,136,34],"35042933",[243,43,39,244,208,34],"35042900","project\u002Fneutral\u002Fperformance",[246,43,34,185,247,34],"35043062","rpep",[249,43,34,185,136,34],"35048391",[251,32,39,236,252,34],"35042699","iddan",[254,32,34,185,111,34],"35042736",[256,32,34,185,257,34],"35043287","efdee",[259,32,34,192,260,34],"35045670","whalesalad",[262,32,34,192,257,34],"35083172",[264,32,34,192,111,34],"35045763",[266,32,34,192,267,34],"35043387","lxe",[269,32,34,192,270,34],"35043657","naasking",[272,32,34,192,111,34],"35045785",[274,32,34,192,270,34],"35045945",[276,32,34,185,277,34],"35049026","sirsinsalot",[279,32,34,192,280,34],"35050524","dahfizz",[282,32,34,185,283,34],"35050457","MuffinFlavored",[172,32,34,285,11,43],"maker\u002Fneutral\u002Fnone",[287,32,34,192,288,34],"35044889","dan-robertson",[290,32,34,192,291,34],"35046176","sixo",[293,32,34,294,295,34],"35046535","other\u002Fneutral\u002Fperformance","jeremycarter",[95,39,39,244,96,34],[175,39,34,244,11,43],[299,39,39,244,96,34],"35044769",[301,39,35,302,303,34],"35044126","project\u002Fpositive\u002Fperformance","neonsunset",[76,39,34,244,11,43],[103,166,43,306,104,34],"project\u002Fcritical\u002Fperformance",[308,166,34,185,295,34],"35046580",[310,166,34,185,311,34],"35048030","leni536",[110,35,39,236,111,34],[135,35,39,236,136,34],[315,35,34,185,316,34],"35044178","alchemist1e9",[318,35,34,185,319,34],"35044387","pletnes",[321,35,34,185,322,34],"35047351","adgjlsfhk1",[324,35,34,185,319,34],"35044377",[326,35,34,185,327,34],"35052216","FridgeSeal",[329,35,34,185,111,34],"35045823",[331,35,34,185,322,34],"35047816",[333,35,34,334,11,43],"35042777","project\u002Fneutral\u002Fdesign",[336,35,39,337,338,34],"35046411","project\u002Fneutral\u002Fidea","sally_glance",[340,35,34,185,208,34],"35042915",[342,35,34,192,111,34],"35049731",[344,35,34,345,346,34],"35050202","commenter\u002Fneutral\u002Fperformance","mattewong",[348,35,34,345,349,34],"35051033","hermitcrab",[351,35,34,345,346,34],"35051659",[353,35,34,192,349,34],"35053057",[355,35,34,345,346,34],"35078781",[357,35,34,192,349,34],"35080154",[359,18,34,294,360,34],"35049669","LtWorf",[362,18,34,294,363,34],"35050417","highpost",[365,18,34,192,360,34],"35061218",[153,367,39,244,154,34],7,[369,367,34,244,11,43],"35043157",[371,367,34,294,372,34],"35048686","jammycrisp",[118,179,43,374,119,34],"project\u002Fcritical\u002Fdocs",[376,179,34,377,11,43],"35042847","maker\u002Fneutral\u002Fdocs",[379,179,43,374,380,34],"35042911","ddorian43",[382,179,34,383,11,43],"35043317","project\u002Fneutral\u002Fdocs",[385,179,39,383,386,34],"35047288","telotortium",[388,165,43,306,389,34],"35045476","EVa5I7bHFq9mnYK",[391,165,34,185,392,34],"35046255","jalk",[131,394,39,244,132,34],10,[396,394,34,302,11,43],"35043296",[398,394,35,302,132,34],"35043991",[400,394,39,244,401,34],"35049161","gituliar",[403,394,34,294,303,34],"35044252",[405,394,34,294,406,34],"35046490","sigg3",[88,408,43,306,89,34],11,[],[411,416,421,426,431,436],{"id":412,"name":413,"points":414,"category":415},"35073603","Braggoscope",688,"learning_science",{"id":417,"name":418,"points":419,"category":420},"35032506","Lander",564,"games",{"id":422,"name":423,"points":424,"category":425},"35026902","Llama-dl",343,"ai",{"id":427,"name":428,"points":429,"category":430},"35055308","NESFab",298,"languages",{"id":432,"name":433,"points":434,"category":435},"34999774","Mathesar",281,"data",{"id":437,"name":438,"points":439,"category":440},"35071755","CodeGPT.nvim",228,"dev_tools",false,"v2",{"id":444,"name":5,"line":6,"ai":441,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":445,"top":18,"kw":446,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":447,"url":9,"mix":448,"praise":84,"objection":100,"sub":21},35042316,-0.22,[24,25,26,27,28],"0000000003330000000351030003000030111351",[32,33,34,35],1794036632115]