[{"data":1,"prerenderedAt":305},["ShallowReactive",2],{"project-14043527":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":42,"post_text":44,"what":45,"said":88,"maker_in_thread":184,"shape":192,"grid":195,"attempts":268,"same_week":269,"maker_shows":39,"maker_hidden":297,"reader_version":298,"card":299},"14043527","CSV Explorer (YC F1)","Explore CSVs with Millions of Rows","title","Show HN: CSV Explorer (YC F1) - Explore CSVs with Millions of Rows","https:\u002F\u002Fwww.csvexplorer.com\u002F","csvexplorer.com","jastr",1491411434,2017,60,34,710,2242,32,"notable","data","csv exploration","web_app",[24,25,26,27],"csv viewer","large csv","explore csv","csv explorer","curious",-0.154,{"positive":31,"neutral":32,"mixed":33,"critical":34},2,7,0,4,13,{"idea":37,"execution":40,"privacy":41},{"positive":38,"critical":39},1,3,{"positive":38,"critical":33},{"positive":33,"critical":38},[33,33,33,33,33,39,33,39,33,33,33,38,33,43,39,33,39,33,43,33,33,38,33,39,33,33,33,39,38,33,39,33,38,33],5,null,{"problem":46,"audience":53,"pricing":44,"motivation":62,"time_to_build":44,"stack":69,"openness":86,"stage":87,"team_size":86},{"value":47,"source":48,"evidence":49},"Opening very large CSV files in Excel is slow or impossible","maker",[50],{"id":51,"by":11,"text":52},"14043533","Hi HN! A few months ago, a college friend reached out because his consulting company had just received a 60 Gb spreadsheet, and they didn't know what to do with it! They actually…",{"value":54,"source":48,"evidence":55},"Non-engineers such as consultants, journalists and lawyers",[56,59],{"id":57,"by":11,"text":58},"14044746","Csvkit is great, and I use it a lot. The target users for CSV Explorer though are mostly non-engineers ie. people who don't use the command line or code.",{"id":60,"by":11,"text":61},"14044773","This project came out of some of my contracting work for consultants, journalists, lawyers - people who don't know what a command line is.",{"value":63,"source":48,"evidence":64},"A college friend's consulting firm received a 60 GB spreadsheet they could not open",[65,66],{"id":51,"by":11,"text":52},{"id":67,"by":11,"text":68},"14044810","Some consultant friends asked for help when they got a 60Gb CSV file from their client! Since then, I've also found that lots of tech companies share CSV internally. I've also…",[70,76,82],{"value":71,"source":48,"evidence":72},"PostgreSQL",[73],{"id":74,"by":11,"text":75},"14045557","Perhaps - I tried a client-side implementation, but I had issues scaling past a few hundred thousand rows. I understand hesitations of uploading data to the cloud, but I now have…",{"value":77,"source":48,"evidence":78},"Redshift",[79],{"id":80,"by":11,"text":81},"14045330","I originally wrote it in entirely client side JS, but it didn't scale nicely past a few hundred thousand rows. For the really big datasets, CSV Explorer loads them into Redshift…",{"value":83,"source":48,"evidence":84},"JavaScript",[85],{"id":80,"by":11,"text":81},"unknown","launched",{"praise":89,"objections":98,"requests":125,"questions":141,"alternatives":153},[90],{"value":91,"source":92,"evidence":93},"Works well with large files and faceted filtering","commenter",[94],{"id":95,"by":96,"text":97},"14044751","danvoell","Worked like a charm. This is really cool. I can see a lot of uses with large files. Especially filtering down \"faceted\" search items.",[99,114],{"value":100,"source":92,"evidence":101},"Uploading sensitive data to a server is a privacy and legal risk",[102,106,110],{"id":103,"by":104,"text":105},"14044762","deadringerr","Cool idea - Having to work with sensitive data, it'd be great to have this functionality without importing\u002Fuploading to the internet.",{"id":107,"by":108,"text":109},"14054072","dromenkoning","This tool would be great were it not for me (rightfully) getting fired and possibly sued by any of my clients if I uploaded even a single file. In fact, what kind of business…",{"id":111,"by":112,"text":113},"14044742","goatlover","Open it with a programming language like R, Julia or Python that supports the data frames. No reason to upload 60 gigs to the web or Excel.",{"value":115,"source":92,"evidence":116},"Could be built fully client-side instead of using a server",[117,121],{"id":118,"by":119,"text":120},"14045259","seanp2k2","Think not of a desktop app, but of doing everything client-side in JS. That way, it's still a web app, but you're not schlepping [sensitive|large] data between front and…",{"id":122,"by":123,"text":124},"14045509","emodendroket","Is there any reason this tool couldn't be 100% client-side?",[126,134],{"value":127,"source":92,"evidence":128},"Offer a desktop or self-hosted version",[129,133],{"id":130,"by":131,"text":132},"14067163","chatmasta","I suggest exposing a config variable in the js client to set the hostname of the Postgres server. This way you can provide a self hosted version of the backend, and customers can…",{"id":103,"by":104,"text":105},{"value":135,"source":92,"evidence":136},"Provide an API for own datasets",[137],{"id":138,"by":139,"text":140},"14044691","patwalls","This is really awesome. What sparked this idea? Not sure if you have this feature, but I think it would be really cool if you could open APIs to your own datasets. This could be…",[142],{"value":143,"source":92,"evidence":144},"What does it do that existing tools cannot",[145,149],{"id":146,"by":147,"text":148},"14044750","chrisweekly","Does this do stuff lnav can't?",{"id":150,"by":151,"text":152},"14044736","hprotagonist","My solution to this problem, so far, has been ipython and pandas. (and maybe jupyter notebook for visualization and sharing). What's the value-add here?",[154,161,169,173,180],{"value":155,"source":92,"evidence":156},"csvkit",[157],{"id":158,"by":159,"text":160},"14044706","xaybey","I usually use csvkit (https:\u002F\u002Fcsvkit.readthedocs.io\u002Fen\u002F1.0.1\u002Findex.html). There are commands to list the columns, filter, browse the data in a somewhat formatted way with less.…",{"value":162,"source":92,"evidence":163},"Pandas and Jupyter",[164,168],{"id":165,"by":166,"text":167},"14045281","a3n","At a former job, our embedded device logs decoded to csv. Some of them were too large for Excel. Pandas handled them without a burp. Pandas in Jupyter (Ipython Notebook) was a…",{"id":150,"by":151,"text":152},{"value":170,"source":92,"evidence":171},"Excel",[172],{"id":158,"by":159,"text":160},{"value":174,"source":92,"evidence":175},"Kibana",[176],{"id":177,"by":178,"text":179},"14045447","lomereiter","Kibana has quite a few rough edges but it takes non-technical users very far in terms of data exploration. Local setup also alleviates privacy concerns. The bummer is…",{"value":181,"source":92,"evidence":182},"lnav",[183],{"id":146,"by":147,"text":148},{"comments":185,"answered":43,"key":186},11,[187,188,191],{"id":57,"text":58},{"id":189,"text":190},"14059624","It sounds like CSV Explorer might work well for them.",{"id":51,"text":52},{"commenters":193,"max_depth":34,"maker_replies":185,"first_comment_mins":38,"lifespan_hours":194},21,70,[196,198,200,201,203,204,207,210,213,215,218,219,221,222,224,226,229,230,232,233,235,237,239,242,244,245,248,250,251,252,254,259,262,264],[158,33,33,197,159,33],"other\u002Fneutral\u002Fnone",[57,33,33,199,11,38],"project\u002Fneutral\u002Fidea",[177,33,33,197,178,33],[202,33,33,197,119,33],"14045219",[165,33,33,197,166,33],[205,33,39,199,206,33],"14049141","donquichotte",[208,33,33,197,209,33],"14050833","infinite8s",[211,33,39,199,212,33],"14058820","jclos",[189,33,33,214,11,38],"project\u002Fpositive\u002Fidea",[216,33,33,197,217,33],"14058451","adamhepner",[51,38,33,199,11,38],[111,38,38,220,112,33],"project\u002Fcritical\u002Fidea",[60,38,33,199,11,38],[95,38,43,223,96,33],"project\u002Fpositive\u002Fexecution",[103,31,39,225,104,33],"project\u002Fneutral\u002Fprivacy",[227,31,33,228,11,38],"14044838","maker\u002Fneutral\u002Fprivacy",[118,31,39,225,119,33],[80,31,33,231,11,38],"project\u002Fneutral\u002Fperformance",[138,39,43,214,139,33],[67,39,33,234,11,38],"maker\u002Fneutral\u002Fidea",[236,39,33,197,119,33],"14045229",[107,34,38,238,108,33],"project\u002Fcritical\u002Fprivacy",[240,43,33,241,11,38],"14051372","project\u002Fneutral\u002Fdocs",[122,243,39,225,123,33],6,[74,243,33,231,11,38],[246,243,33,197,247,33],"14046713","Zaheer",[249,243,33,197,123,33],"14046783",[130,243,39,225,131,33],[150,32,38,220,151,33],[253,32,33,234,11,38],"14044759",[255,256,39,257,258,33],"14043632",8,"project\u002Fneutral\u002Fnone","cocktailpeanuts",[260,256,33,261,11,38],"14043648","maker\u002Fneutral\u002Fnone",[146,263,38,220,147,33],9,[265,266,33,197,267,33],"14046962",10,"pclark",[],[270,275,279,284,288,293],{"id":271,"name":272,"points":273,"category":274},"14032201","React-data sheet",345,"dev_tools",{"id":276,"name":277,"points":278,"category":274},"14023196","Is It Snappy?",293,{"id":280,"name":281,"points":282,"category":283},"13995891","Redux Offline",174,"languages",{"id":285,"name":286,"points":287,"category":274},"14022059","develmail",149,{"id":289,"name":290,"points":291,"category":292},"14007257","Interactive ClojureScript book",134,"learning_science",{"id":294,"name":295,"points":296,"category":292},"14043519","Robotopia",133,false,"v2",{"id":300,"name":5,"line":6,"ai":297,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":301,"top":18,"kw":302,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":303,"url":9,"mix":304,"praise":91,"objection":100,"sub":21},14043527,-0.15,[24,25,26,27],"0000030300010530305001030003103010",[31,32,33,34],1794036554147]