[{"data":1,"prerenderedAt":364},["ShallowReactive",2],{"project-9485714":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":43,"post_text":46,"what":47,"said":67,"maker_in_thread":160,"shape":169,"grid":174,"attempts":325,"same_week":326,"maker_shows":31,"maker_hidden":356,"reader_version":357,"card":358},"9485714","nvParse","Parsing CSV files with GPU","title","Show HN: Parsing CSV files with GPU","https:\u002F\u002Fgithub.com\u002Fantonmks\u002FnvParse","github.com","antonmks",1430742814,2015,123,60,190,2565,8,"notable","data","csv parsing","library",[21,24,25,26,27],"gpu","cuda","thrust","parallel parsing","roasted",-0.435,{"positive":31,"neutral":32,"mixed":33,"critical":34},2,11,0,16,29,{"execution":37,"performance":39,"idea":41,"docs":42},{"positive":38,"critical":31},1,{"positive":33,"critical":40},13,{"positive":38,"critical":38},{"positive":33,"critical":33},[38,38,38,38,44,38,38,38,33,33,38,33,33,33,33,38,33,38,44,44,33,45,44,44,44,33,38,45,33,33,33,44,38,38,44,38,33,33,33,38],3,5,null,{"problem":46,"audience":46,"pricing":46,"motivation":46,"time_to_build":46,"stack":48,"openness":65,"stage":66,"team_size":66},[49,56,60],{"value":50,"source":51,"evidence":52},"CUDA","maker",[53],{"id":54,"by":11,"text":55},"9490265","Thrust CUDA calls are synchronous to each other. You can add an explicit synchronization call cudaDeviceSynchronize() and there won't be a difference in results.",{"value":57,"source":51,"evidence":58},"Thrust",[59],{"id":54,"by":11,"text":55},{"value":61,"source":62,"evidence":63},"C++","inferred",[64],{"id":54,"by":11,"text":55},"open_source","unknown",{"praise":68,"objections":77,"requests":118,"questions":127,"alternatives":142},[69],{"value":70,"source":71,"evidence":72},"Creative application of GPU to a less typical task","commenter",[73],{"id":74,"by":75,"text":76},"9485885","WhitneyLand","I like your creativity in applying the GPU to a less typical task. Is it the case that the CPU version could be sped up dramatically by using multiple cores and a variation on…",[78,89,101,112],{"value":79,"source":71,"evidence":80},"Benchmark compares single-threaded CPU to top-tier GPU",[81,85],{"id":82,"by":83,"text":84},"9486312","castratikron","I'm skeptical of the 8x speedup for several reasons, the main one being that this particular problem does not fit the paradigm of problems that work well on the GPU; the GPU…",{"id":86,"by":87,"text":88},"9486741","stephen272","You hit the nail on the head. All of that on top of the fact that he is comparing a top of the line GPU against a mediocre i3 processor. GPU to CPU comparisons are always apples…",{"value":90,"source":71,"evidence":91},"Parsing is memory-bound and poorly suited to GPU",[92,93,97],{"id":82,"by":83,"text":84},{"id":94,"by":95,"text":96},"9486700","pcwalton","> Generally the only code that maps well to the GPU is that which contains large for loops and has good spacial locality (e.g. matrix multiplication). You also need high…",{"id":98,"by":99,"text":100},"9489327","paulmd","There's also just the sheer bandwidth factor. A GTX Titan has 288 GB\u002Fs of BW, a Haswell with DDR3 puts out about 35 GB\u002Fs. That's a factor of 8.22x more bandwidth, so there's your…",{"value":102,"source":71,"evidence":103},"Hard-coded to the test dataset and not a general CSV parser",[104,108],{"id":105,"by":106,"text":107},"9488358","andrewguenther","This title is incredibly misleading. * This isn't parsing a CSV, this is a program written to split this exact dataset. (The code is filled with hard coded values) * You're…",{"id":109,"by":110,"text":111},"9487452","kazinator","Though there is no standard definition of CSV, de facto processing it properly requires recognizing quotes, and also escapes of literal quotes using double quoting: ``` this,…",{"value":113,"source":71,"evidence":114},"Benchmark timing may measure kernel launch, not completion",[115],{"id":116,"by":99,"text":117},"9489290","I kinda suspect he might be measuring the time it takes to launch a kernel rather than the time it takes the kernel to complete. Thrust device calls, like those of the underlying…",[119],{"value":120,"source":71,"evidence":121},"Run the GPU and CPU benchmarks in a fair comparison",[122,123],{"id":86,"by":87,"text":88},{"id":124,"by":125,"text":126},"9486165","avarsheny","I don't think CPU and GPU is going to make much difference here. You speed up in GPU run could be because of hot cache of file system. Try running GPU run first and non-GPU after…",[128,135],{"value":129,"source":71,"evidence":130},"How is a 750GB file read in 1.5 seconds?",[131],{"id":132,"by":133,"text":134},"9486037","roel_v","I don't understand the benchmark. How can a 750GB file be read from disk in 1.5 seconds, let alone be parsed? He mentions it's a 2TB drive, so it's not even SSD presumably?",{"value":136,"source":71,"evidence":137},"How fast is parsing cell values like dates on the GPU?",[138],{"id":139,"by":140,"text":141},"9487024","victorNicollet","Very interesting. From my experience, the hard part about parsing CSV isn't to identify the individual cells, but rather parsing those cells afterwards (as numbers, dates, etc).…",[143,149,153],{"value":144,"source":71,"evidence":145},"Thrust multicore OpenMP backend",[146],{"id":147,"by":99,"text":148},"9489447","You can compile Thrust using a device target of OpenMP, so no need to rewrite it in OpenCL.",{"value":150,"source":71,"evidence":151},"Lokad flatfiles",[152],{"id":139,"by":140,"text":141},{"value":154,"source":71,"evidence":155},"PapaParse",[156],{"id":157,"by":158,"text":159},"9487255","mholt","I gotta admit, having written a high-speed, multi-threaded streaming CSV parser for the browser[1], I'm quite impressed by your project. I've done CUDA programming before and…",{"comments":45,"answered":38,"key":161},[162,165,166],{"id":163,"text":164},"9490312","Sorry if you are misled :-) The program does parses selected fields, there are strings-to-binaries procedures. I tested this approach on multiterabyte files, take a look at my…",{"id":54,"text":55},{"id":167,"text":168},"9486243","I used average time for multiple runs, both GPU and CPU benchmarks ran from memory.",{"commenters":170,"max_depth":171,"maker_replies":45,"first_comment_mins":172,"lifespan_hours":173},39,4,36,168,[175,177,179,181,182,185,186,188,191,193,195,198,200,203,205,208,211,214,217,221,223,225,227,230,231,233,235,236,240,242,244,246,249,252,256,259,261,264,266,269,270,271,274,276,279,282,284,287,289,292,294,298,301,304,307,310,313,316,318,321],[105,33,38,176,106,33],"project\u002Fcritical\u002Fexecution",[163,33,33,178,11,38],"maker\u002Fneutral\u002Fnone",[82,38,38,180,83,33],"project\u002Fcritical\u002Fperformance",[94,38,38,180,95,33],[183,38,33,184,83,33],"9486915","commenter\u002Fneutral\u002Fnone",[86,38,38,180,87,33],[147,38,44,187,99,33],"project\u002Fneutral\u002Fexecution",[189,38,38,180,190,33],"9486762","zvrba",[98,38,44,192,99,33],"project\u002Fneutral\u002Fperformance",[109,31,38,194,110,33],"project\u002Fcritical\u002Fidea",[196,31,38,176,197,33],"9487489","motoboi",[199,31,33,184,110,33],"9487578",[201,31,33,184,202,33],"9487558","bostonpete",[204,31,33,184,110,33],"9487612",[206,31,33,184,207,33],"9487914","nolok",[209,31,38,180,210,33],"9487816","haberman",[212,31,33,184,213,33],"9487963","TheLoneWolfling",[215,31,33,184,216,33],"9488111","infogulch",[218,31,33,219,220,33],"9488122","other\u002Fneutral\u002Fnone","wfunction",[222,31,33,219,213,33],"9490717",[224,31,33,219,95,33],"9488800",[226,31,33,219,210,33],"9494243",[228,31,33,219,229,33],"9520060","thesz",[116,44,38,180,99,33],[232,44,33,184,106,33],"9489296",[54,44,33,234,11,38],"maker\u002Fneutral\u002Fperformance",[132,171,38,180,133,33],[237,171,44,238,239,33],"9486077","project\u002Fneutral\u002Fdocs","bbrks",[139,45,44,241,140,33],"project\u002Fneutral\u002Fidea",[243,45,44,241,213,33],"9490730",[245,45,33,184,140,33],"9491447",[157,247,45,248,158,33],6,"project\u002Fpositive\u002Fexecution",[250,247,33,219,251,33],"9488044","pestaa",[253,254,44,192,255,33],"9486190",7,"hannibalhorn",[257,254,33,219,258,33],"9486218","atwebb",[260,254,44,192,140,33],"9486964",[262,254,44,192,263,33],"9487072","bane",[265,254,33,184,140,33],"9487145",[267,254,33,184,268,33],"9488625","maxhou",[124,18,38,180,125,33],[167,18,33,234,11,38],[74,272,45,273,75,33],9,"project\u002Fpositive\u002Fidea",[275,272,33,234,11,38],"9485937",[277,272,33,184,278,33],"9485997","Narishma",[280,272,33,184,281,33],"9486400","sharpneli",[283,272,33,184,83,33],"9486528",[285,272,44,192,286,33],"9485971","hvidgaard",[288,272,33,192,11,38],"9486025",[290,272,38,180,291,33],"9486086","cjg_",[293,272,44,192,286,33],"9524116",[295,296,38,180,297,33],"9490256",10,"rdc12",[299,32,44,241,300,33],"9487492","snissn",[302,303,38,180,268,33],"9488179",12,[305,303,38,180,306,33],"9488235","azurezyq",[308,40,33,219,309,33],"9486738","ape4",[311,40,33,219,312,33],"9486861","dilap",[314,40,33,219,315,33],"9487866","ldng",[317,40,33,219,140,33],"9486943",[319,40,33,219,320,33],"9487764","bitwize",[322,323,38,180,324,33],"9503370",14,"aftbit",[],[327,332,337,342,347,352],{"id":328,"name":329,"points":330,"category":331},"9498874","kiUi",136,"languages",{"id":333,"name":334,"points":335,"category":336},"9477032","zindex",125,"dev_tools",{"id":338,"name":339,"points":340,"category":341},"9491978","My Hacker News",78,"consumer_social",{"id":343,"name":344,"points":345,"category":346},"9476603","PID",73,"learning_science",{"id":348,"name":349,"points":350,"category":351},"9502015","Estimote Indoor Location SDK",71,"infrastructure",{"id":353,"name":354,"points":355,"category":336},"9500823","Onboarded",54,false,"v2",{"id":359,"name":5,"line":6,"ai":356,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":360,"top":18,"kw":361,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":362,"url":9,"mix":363,"praise":70,"objection":79,"sub":21},9485714,-0.43,[21,24,25,26,27],"1111311100100001013305333015000311310001",[31,32,33,34],1793990924023]