[{"data":1,"prerenderedAt":260},["ShallowReactive",2],{"project-6727586":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":40,"post_text":42,"what":43,"said":87,"maker_in_thread":166,"shape":175,"grid":178,"attempts":221,"same_week":222,"maker_shows":202,"maker_hidden":252,"reader_version":253,"card":254},"6727586","DataNitro","Hadoop in Excel","title","Show HN: Hadoop in Excel","https:\u002F\u002Fdatanitro.com\u002Fhadoop_in_excel.html","datanitro.com","karamazov",1384369922,2013,82,22,296,1739,18,"notable","data","big data analytics","desktop_app",[24,25,26,27],"hadoop excel","mapreduce spreadsheet","big data excel","hadoop analytics","debated",0.067,{"positive":31,"neutral":31,"mixed":32,"critical":33},4,0,3,11,{"idea":36,"performance":38},{"positive":31,"critical":37},1,{"positive":32,"critical":39},2,[32,41,32,32,41,32,33,32,41,33,32,41,33,37,32,37,32,33,32,37,32,32],5,null,{"problem":44,"audience":58,"pricing":42,"motivation":42,"time_to_build":42,"stack":67,"openness":85,"stage":86,"team_size":85},{"value":45,"source":46,"evidence":47},"Lets business users analyze large Hadoop datasets using Excel formulas without writing MapReduce code","inferred",[48,52,55],{"id":49,"by":50,"text":51},"6727815","cs702","Looks like a killer product, because there are a lot of business people who readily know how to write what is essentially functional code in Excel but for whatever reason cannot…",{"id":53,"by":11,"text":54},"6727930","Formulas should be used as columnar operations: you can apply one formula to every element of a column (map) or aggregate an entire column (reduce). (This isn't that much of a…",{"id":56,"by":11,"text":57},"6729676","We let people with Hadoop Clusters pull a small sample of data into Excel, analyze it with Excel formulas, and then run the analysis on the full data set. The last part happens…",{"value":59,"source":60,"evidence":61},"Business users who know Excel but not programming","commenter",[62,63],{"id":49,"by":50,"text":51},{"id":64,"by":65,"text":66},"6728718","pvnick","Did you write your mappers and reducers in java using the hadoop api or does this translate into hiveql or some other higher-level language? Great job btw, this looks super…",[68,75,81],{"value":69,"source":70,"evidence":71},"Java","maker",[72],{"id":73,"by":11,"text":74},"6728967","Thanks! We're working directly in Java right now, but might explore alternatives later. We're also planning to add support for Impala\u002FPresto\u002Fetc.",{"value":76,"source":70,"evidence":77},"Hadoop MapReduce",[78],{"id":79,"by":11,"text":80},"6728351","We are using MapReduce. Latency will depend on your cluster and the query; it's just a regular MapReduce operation from Hadoop's point of view.",{"value":82,"source":46,"evidence":83},"Excel",[84],{"id":53,"by":11,"text":54},"unknown","beta",{"praise":88,"objections":104,"requests":126,"questions":135,"alternatives":157},[89,97],{"value":90,"source":60,"evidence":91},"Useful for business users to get reports without involving data engineers",[92,93],{"id":64,"by":65,"text":66},{"id":94,"by":95,"text":96},"6728202","eigenvalue","I think this would really benefit from a dead simple tool that would allow users to import from csv files into a local Hadoop instance, without having to do anything besides…",{"value":98,"source":60,"evidence":99},"Could suit many Hadoop use cases",[100],{"id":101,"by":102,"text":103},"6728099","staunch","Funny as this sounds it may be in fact exactly perfect for a large subset of Hadoop use-cases. If it works well.",[105,116],{"value":106,"source":60,"evidence":107},"Excel row limit of about one million rows",[108,112],{"id":109,"by":110,"text":111},"6728350","jackmaney","Ummmm...doesn't Excel have a row limit of somewhere around 1 million?",{"id":113,"by":114,"text":115},"6728480","wbsun","Can Excel open a 1-billion-row data file?",{"value":117,"source":60,"evidence":118},"Excel is a poor tool for serious analysis",[119,123],{"id":120,"by":121,"text":122},"6729057","Fomite","While impressive in terms of a technical achievement, Excel is a pretty appalling analysis tool generally. I fear for what it will turn into when you throw this much at it. Big…",{"id":124,"by":121,"text":125},"6729916","Which is why I'm not besmirching your technical achievement as much as...Excel is widely abused by the ignorant, Big Data is widely abused by the ignorant...Hadoop in Excel...",[127,131],{"value":128,"source":70,"evidence":129},"Support for Impala and Presto",[130],{"id":73,"by":11,"text":74},{"value":132,"source":60,"evidence":133},"CSV import into a local Hadoop instance",[134],{"id":94,"by":95,"text":96},[136,143,150],{"value":137,"source":60,"evidence":138},"Latency expectations",[139],{"id":140,"by":141,"text":142},"6728326","monstrado","What are you using on the back-end to perform the queries? Are you using MapReduce? What is the average latency expectations when using the application?",{"value":144,"source":60,"evidence":145},"How it differs from Power Query",[146],{"id":147,"by":148,"text":149},"6728462","prawks","Being pretty naive to the space, I'm assuming the killer differentiator from Microsoft's own Power Query (which looks like it can pull from Hadoop) is that this pulls a subset of…",{"value":151,"source":60,"evidence":152},"How it compares to Datameer",[153],{"id":154,"by":155,"text":156},"6730473","RobGoretsky","I've seen demos of a tool called Datameer which seems to offer very similar functionality (an Excel-like interface for configuring a job on a small set of data, followed by…",[158,162],{"value":159,"source":60,"evidence":160},"Microsoft Power Query",[161],{"id":147,"by":148,"text":149},{"value":163,"source":60,"evidence":164},"Datameer",[165],{"id":154,"by":155,"text":156},{"comments":167,"answered":168,"key":169},9,7,[170,173,174],{"id":171,"text":172},"6727733","Hi, I'm one of the developers. I'd be happy to answer any questions you have on this. If you're in New York, I'd love to meet you in person at our Big Data in Excel meetup this…",{"id":53,"text":54},{"id":73,"text":74},{"commenters":176,"max_depth":31,"maker_replies":167,"first_comment_mins":177,"lifespan_hours":167},12,14,[179,181,183,185,189,190,192,194,195,196,198,200,201,203,205,207,209,211,213,214,216,219],[171,32,32,180,11,37],"other\u002Fneutral\u002Fnone",[49,32,41,182,50,32],"project\u002Fpositive\u002Fidea",[53,32,32,184,11,37],"project\u002Fneutral\u002Fdesign",[186,32,32,187,188,32],"6729612","commenter\u002Fneutral\u002Fnone","ajasmin",[64,37,41,182,65,32],[73,37,32,191,11,37],"project\u002Fneutral\u002Fexecution",[140,39,33,193,141,32],"project\u002Fneutral\u002Fperformance",[79,39,32,193,11,37],[101,33,41,182,102,32],[147,31,33,197,148,32],"project\u002Fneutral\u002Fidea",[199,31,32,197,11,37],"6728547",[94,41,41,182,95,32],[154,202,33,197,155,32],6,[109,168,37,204,110,32],"project\u002Fcritical\u002Fperformance",[206,168,32,193,11,37],"6728554",[113,208,37,204,114,32],8,[210,208,32,193,11,37],"6728552",[212,208,33,193,114,32],"6729474",[56,208,32,193,11,37],[120,167,37,215,121,32],"project\u002Fcritical\u002Fidea",[217,167,32,218,11,37],"6729082","commenter\u002Fneutral\u002Fidea",[124,167,32,220,121,32],"commenter\u002Fcritical\u002Fidea",[],[223,228,233,238,243,248],{"id":224,"name":225,"points":226,"category":227},"6691316","StackDock",387,"infrastructure",{"id":229,"name":230,"points":231,"category":232},"6719582","thisplusthat",148,"ai",{"id":234,"name":235,"points":236,"category":237},"6721474","giveit100.com",147,"productivity",{"id":239,"name":240,"points":241,"category":242},"6696654","Rather",98,"consumer_social",{"id":244,"name":245,"points":246,"category":247},"6689214","Shippo",95,"web_saas",{"id":249,"name":250,"points":251,"category":237},"6725995","ReRead",55,false,"v2",{"id":255,"name":5,"line":6,"ai":252,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":256,"top":18,"kw":257,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":258,"url":9,"mix":259,"praise":90,"objection":106,"sub":21},6727586,0.07,[24,25,26,27],"0500503053053101030100",[31,31,32,33],1794036511995]