[{"data":1,"prerenderedAt":350},["ShallowReactive",2],{"project-21547369":3},{"id":4,"name":5,"line":5,"line_source":6,"title":7,"url":8,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":42,"post_text":44,"what":45,"said":87,"maker_in_thread":160,"shape":167,"grid":172,"attempts":310,"same_week":311,"maker_shows":34,"maker_hidden":342,"reader_version":343,"card":344},"21547369","Faster UTF-8 validator","title","Show HN: Faster UTF-8 validator","https:\u002F\u002Fgithub.com\u002Fzwegner\u002Ffaster-utf8-validator","github.com","zwegner",1573842609,2019,122,55,379,2195,18,"notable","languages","utf-8 validation library","library",[23,24,25,26,27],"utf-8 validation","simd","avx2","sse4","fast validator","curious",0.167,{"positive":31,"neutral":32,"mixed":33,"critical":34},3,8,0,1,12,{"execution":37,"performance":38,"idea":40,"design":41},{"positive":33,"critical":34},{"positive":39,"critical":33},2,{"positive":33,"critical":33},{"positive":34,"critical":33},[34,33,31,31,33,33,31,43,33,31,33,33,33,33,33,33,33,33,43,33,31,33,43,33,33,33,33,33,33,31,31,33,33,33,33,31,33,33,33,33],5,null,{"problem":46,"audience":44,"pricing":44,"motivation":57,"time_to_build":44,"stack":64,"openness":84,"stage":85,"team_size":86},{"value":47,"source":48,"evidence":49},"Fast validation of UTF-8 byte input using SIMD instructions.","inferred",[50,54],{"id":51,"by":52,"text":53},"21550453","BeeOnRope","Good stuff. The key to a solid claim, especially for something as bold as \"fastest in the world\", is a complete specification of the inputs to the benchmark. You mention random…",{"id":55,"by":10,"text":56},"21550828","Well, to be fair, my claim was that it was the fastest in the world that I'm aware of, which is a much weaker statement. :) The random UTF-8 in the benchmark was generated from…",{"value":58,"source":59,"evidence":60},"Hacked together while starting UTF-8 support for a text editor the maker is writing.","maker",[61],{"id":62,"by":10,"text":63},"21550862","I don't have any particular plans for that, beyond getting a bit of publicity for it here :). Or really any plans at all, besides adding AVX-512 support. It was just something I…",[65,74,80],{"value":66,"source":59,"evidence":67},"C",[68,71],{"id":69,"by":10,"text":70},"21555193","Ah, thanks for the reply. I understand you better now, and for the most part I agree. Taking out the early exit for ASCII can speed up the frequently-changing case to avoid the…",{"id":72,"by":10,"text":73},"21548726","Well how else are you supposed to nest your preprocessor macros? :P Thanks though! I like taking the time to make my code as clean as I can. Glad others appreciate it!",{"value":75,"source":59,"evidence":76},"AVX2",[77],{"id":78,"by":10,"text":79},"21548468","> this might save megawatts of power world wide That'd sure be neat! I really have no idea, though. > A small nitpick: not sure whether those macros bought anything, I guess the…",{"value":81,"source":59,"evidence":82},"SSE4",[83],{"id":78,"by":10,"text":79},"open_source","launched","unknown",{"praise":88,"objections":112,"requests":124,"questions":138,"alternatives":150},[89,101],{"value":90,"source":91,"evidence":92},"Clean, well-documented C code","commenter",[93,97],{"id":94,"by":95,"text":96},"21549703","eyegor","I'm not sure if the compiler is smart enough to do it for you, but you might be able to squeak a bit more performance by precomputing a partial sum (data + V_LEN - 1) outside the…",{"id":98,"by":99,"text":100},"21548649","SlowRobotAhead","I like the way you write and comment your C code. Very nicely written. Except all those spaces between the # and “define”, no sir I don’t like that :)",{"value":102,"source":91,"evidence":103},"Performance gains over prior validators",[104,108],{"id":105,"by":106,"text":107},"21548259","vardump","Pretty nice. Always nice to see better optimized commonly required algorithms. If this is really faster than previous implementations (no reason to doubt, but don't have time to…",{"id":109,"by":110,"text":111},"21549610","CodesInChaos","Do the instructions you use cause downclocking on Intel CPUs?",[113,117],{"value":114,"source":91,"evidence":115},"Benchmark input distribution was not fully specified",[116],{"id":51,"by":52,"text":53},{"value":118,"source":91,"evidence":119},"Validator fails some invalid UTF-8 test cases from Markus Kuhn's suite",[120],{"id":121,"by":122,"text":123},"21555558","andrewf","Hi! I took a shot at vectorized UTF-8 validation in 2012. I just put this faster validator, and Daniel Lemire's code, into my test harness at…",[125,131],{"value":126,"source":59,"evidence":127},"Add a Makefile",[128],{"id":129,"by":10,"text":130},"21557276","Oh wow, that is most definitely a bug. Thank you very much for reporting that, before this spreads too much. How embarrassing... I was in a bit of a rush to stick this up on HN…",{"value":132,"source":91,"evidence":133},"Benchmark with realistic language corpora",[134,135],{"id":51,"by":52,"text":53},{"id":136,"by":10,"text":137},"21555212","Definitely a good idea. I had already downloaded a Mandarin Wikipedia page to look for test sequences, but hadn't benchmarked them. Getting some more pages there is a great place…",[139,146],{"value":140,"source":91,"evidence":141},"Does the approach differ from the simdjson validator PR?",[142],{"id":143,"by":144,"text":145},"21548523","aqrit","how does this differ from here: https:\u002F\u002Fgithub.com\u002Flemire\u002Fsimdjson\u002Fpull\u002F365",{"value":147,"source":91,"evidence":148},"Do these AVX-512 or AVX2 instructions cause downclocking on Intel CPUs?",[149],{"id":109,"by":110,"text":111},[151,156],{"value":152,"source":91,"evidence":153},"Lemire's fastvalidate-utf-8",[154,155],{"id":55,"by":10,"text":56},{"id":121,"by":122,"text":123},{"value":157,"source":91,"evidence":158},"simdjson UTF-8 validation PR",[159],{"id":143,"by":144,"text":145},{"comments":161,"answered":162,"key":163},13,6,[164,165,166],{"id":129,"text":130},{"id":55,"text":56},{"id":69,"text":70},{"commenters":168,"max_depth":169,"maker_replies":161,"first_comment_mins":170,"lifespan_hours":171},31,7,99,106,[173,175,177,179,180,182,183,186,189,191,193,195,197,200,203,207,210,213,216,218,221,224,227,230,232,233,235,237,240,243,246,248,250,254,257,260,263,266,269,272,275,276,279,282,284,286,288,291,293,294,297,300,302,305,307],[121,33,34,174,122,33],"project\u002Fcritical\u002Fexecution",[129,33,33,176,10,34],"maker\u002Fneutral\u002Fexecution",[51,34,31,178,52,33],"project\u002Fneutral\u002Fperformance",[55,34,33,178,10,34],[181,34,31,178,52,33],"21554741",[69,34,33,178,10,34],[184,34,33,185,52,33],"21560076","commenter\u002Fneutral\u002Fnone",[187,34,33,188,10,34],"21555395","project\u002Fneutral\u002Fnone",[190,34,31,178,52,33],"21554782",[136,34,33,192,10,34],"maker\u002Fneutral\u002Fperformance",[105,39,43,194,106,33],"project\u002Fpositive\u002Fperformance",[78,39,33,196,10,34],"project\u002Fneutral\u002Fdesign",[198,39,31,178,199,33],"21550456","thechao",[201,39,33,202,10,34],"21550872","maker\u002Fpositive\u002Fperformance",[204,39,33,205,206,33],"21548866","other\u002Fneutral\u002Fnone","FpUser",[208,39,33,205,209,33],"21549158","marmada",[211,39,33,205,212,33],"21551738","F-0X",[214,39,33,205,215,33],"21554061","jimsmart",[217,39,33,205,206,33],"21558731",[219,39,33,205,220,33],"21566578","dwild",[222,39,33,205,223,33],"21549324","kbenson",[225,39,33,205,226,33],"21549613","LeoPanthera",[228,39,33,205,229,33],"21551274","adrianN",[231,39,33,205,206,33],"21551454",[94,31,43,194,95,33],[234,31,33,202,10,34],"21549896",[236,31,33,205,106,33],"21550298",[143,238,31,239,144,33],4,"project\u002Fneutral\u002Fidea",[241,238,33,242,10,34],"21548706","maker\u002Fneutral\u002Fidea",[244,238,33,205,245,33],"21549679","ysleepy",[98,43,43,247,99,33],"project\u002Fpositive\u002Fdesign",[72,43,33,249,10,34],"maker\u002Fpositive\u002Fdesign",[251,43,33,252,253,33],"21548865","commenter\u002Fneutral\u002Fdesign","usr1106",[255,43,33,252,256,33],"21550667","kevin_thibedeau",[258,43,33,252,259,33],"21549909","brandmeyer",[261,43,33,252,262,33],"21550301","leetcrew",[264,43,33,252,265,33],"21551605","speps",[267,162,33,205,268,33],"21551994","vmurthy",[270,162,33,205,271,33],"21552381","matheusmoreira",[273,169,31,239,274,33],"21550740","pabs3",[62,169,33,242,10,34],[277,169,31,239,278,33],"21550969","nitwit005",[280,169,33,205,281,33],"21551267","pdkl95",[283,169,33,185,278,33],"21580990",[285,169,33,239,10,34],"21555283",[287,169,33,205,278,33],"21580973",[289,169,33,205,290,33],"21552549","killercup",[292,169,33,205,52,33],"21554951",[109,32,31,178,110,33],[295,32,33,296,95,33],"21549657","other\u002Fneutral\u002Fperformance",[298,32,33,205,299,33],"21550268","oconnor663",[301,32,33,205,299,33],"21558089",[303,32,33,205,304,33],"21554223","littlestymaar",[306,32,33,205,52,33],"21554982",[308,32,33,309,10,34],"21555257","maker\u002Fpositive\u002Fnone",[],[312,317,322,327,332,337],{"id":313,"name":314,"points":315,"category":316},"21559139","Chatwoot",417,"web_saas",{"id":318,"name":319,"points":320,"category":321},"21561613","BrowserGap",253,"security_privacy",{"id":323,"name":324,"points":325,"category":326},"21587935","EasyDB",158,"data",{"id":328,"name":329,"points":330,"category":331},"21558343","2019",144,"ai",{"id":333,"name":334,"points":335,"category":336},"21545291","Scholarref tools",127,"productivity",{"id":338,"name":339,"points":340,"category":341},"21533921","Bamboolib",119,"dev_tools",false,"v2",{"id":345,"name":5,"line":5,"ai":342,"cat":19,"year":12,"points":13,"comments":14,"rec":28,"s":346,"top":17,"kw":347,"maker":10,"domain":9,"tier":18,"attempts":33,"strip":348,"url":8,"mix":349,"praise":90,"objection":114,"sub":20},21547369,0.17,[23,24,25,26,27],"1033003503000000005030500000033000030000",[31,32,33,34],1794036583035]