[{"data":1,"prerenderedAt":319},["ShallowReactive",2],{"project-14636699":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":43,"post_text":45,"what":46,"said":79,"maker_in_thread":160,"shape":171,"grid":175,"attempts":280,"same_week":281,"maker_shows":310,"maker_hidden":311,"reader_version":312,"card":313},"14636699","HyperLogLog","An improved version of HyperLogLog","title","Show HN: An improved version of HyperLogLog","https:\u002F\u002Fgithub.com\u002Faxiomhq\u002Fhyperloglog.git","github.com","seiflotfy",1498484569,2017,194,44,173,2242,8,"notable","languages","probabilistic data structure library","library",[24,25,26,27],"hyperloglog","cardinality estimation","count distinct","go library","curious",0.055,{"positive":31,"neutral":32,"mixed":33,"critical":34},2,15,0,3,20,{"design":37,"idea":38,"performance":40,"execution":41,"docs":42},{"positive":33,"critical":33},{"positive":39,"critical":33},1,{"positive":33,"critical":33},{"positive":33,"critical":31},{"positive":33,"critical":39},[34,33,34,33,33,44,44,33,34,34,33,34,39,39,34,33,33,33,33,33,33,33,33,33,34,34,34,33,39,34,34,33,34,33,34,33,34,34,33,33],5,null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":58,"openness":77,"stage":78,"team_size":78},{"value":48,"source":49,"evidence":50},"Estimating distinct element counts in large streams using less memory","maker",[51,54],{"id":52,"by":11,"text":53},"14637055","Author here, the main 3 things that make this stand out is: 1) 4-bit register instead of 5 (HLL) and 6 (HLL++), but most implementations use 1 byte registers out of convenience…",{"id":55,"by":56,"text":57},"14637042","StreamBright","This is absolutely amazing. Seems like a great tradeoff of having smaller space usage and better accurancy on average at the same time. It is also worth to read the related white…",[59,71],{"value":60,"source":61,"evidence":62},"Go","inferred",[63,67],{"id":64,"by":65,"text":66},"14637735","chimeracoder","This is awesome! We make heavy use of HyperLogLogs in our monitoring systems[0], which are also written in Go and currently use Clark Duvall's library, which this library is…",{"id":68,"by":69,"text":70},"14637048","tmaly","why use metro hash instead of xxhash ?",{"value":72,"source":49,"evidence":73},"MetroHash",[74],{"id":75,"by":11,"text":76},"14637059","In my test metro hash gave me higher throughput, must be something with the xxhash implementation in Go","open_source","unknown",{"praise":80,"objections":90,"requests":108,"questions":130,"alternatives":142},[81,86],{"value":82,"source":83,"evidence":84},"Smaller space usage with good accuracy","commenter",[85],{"id":55,"by":56,"text":57},{"value":87,"source":83,"evidence":88},"Useful for monitoring systems that use HyperLogLogs",[89],{"id":64,"by":65,"text":66},[91,98],{"value":92,"source":83,"evidence":93},"TailCut can make results order-dependent when values repeat",[94],{"id":95,"by":96,"text":97},"14645096","oertl","The tailcut approach breaks one important property of cardinality estimation algorithms: Adding the same value multiple times should change the state of the sketch only once when…",{"value":99,"source":83,"evidence":100},"Space claims lack a clear comparison table",[101,105],{"id":102,"by":103,"text":104},"14637041","lorenzhs","I don't understand what the result table measures. What are the numbers and what is the \"Exact\" column?",{"id":106,"by":103,"text":107},"14638004","That doesn't match up with the tagline \"An improved version of HyperLogLog for the count-distinct problem, approximating the number of distinct elements in a multiset using…",[109,116,123],{"value":110,"source":83,"evidence":111},"Implementations in Python or Java",[112],{"id":113,"by":114,"text":115},"14678347","JensRantil","I'm looking for implementations of this in other languages. Specifically in Python or Java. Please hook me up if you know of any such implementation.",{"value":117,"source":83,"evidence":118},"Intersection support for two HLL instances",[119],{"id":120,"by":121,"text":122},"14640835","buremba","Do you guys know any way to calculate the intersection of two HLL instances? The algorithm allows merging two sets but it's not that easy to take the intersection of them. A + B…",{"value":124,"source":83,"evidence":125},"Comprehensive benchmark across Go implementations",[126],{"id":127,"by":128,"text":129},"14637624","helper","There really are a lot of hyperloglog implementations in go. I'd be interested in seeing a comprehensive comparison between implementations for speed, accuracy and space…",[131,138],{"value":132,"source":83,"evidence":133},"How does TailCut compare with Ertl's estimator",[134],{"id":135,"by":136,"text":137},"14637171","jbapple","How does the Xiao et al. paper (which introduces TailCut, \"Better with Fewer Bits: Improving the Performance of Cardinality Estimation of Large Data Streams\",…",{"value":139,"source":83,"evidence":140},"Why MetroHash rather than xxHash",[141],{"id":68,"by":69,"text":70},[143,150,154],{"value":144,"source":83,"evidence":145},"Redis HyperLogLog",[146],{"id":147,"by":148,"text":149},"14637226","antirez","Somewhat similar to Redis implementation in many ways: Redis also uses loglog-beta, sparse representations, and dense representations in packed format without wasting 1 byte.…",{"value":151,"source":83,"evidence":152},"Clark Duvall's Go HyperLogLog library",[153],{"id":64,"by":65,"text":66},{"value":155,"source":83,"evidence":156},"Cuckoo filter",[157],{"id":158,"by":11,"text":159},"14637917","https:\u002F\u002Fgithub.com\u002Fseiflotfy\u002Fcuckoofilter but this will require you to know how many max unique values per status your will have.",{"comments":161,"answered":162,"key":163},13,6,[164,167,170],{"id":165,"text":166},"14637242","Its heavily inspired by the rowdies implementation, fact is I looked back then how you did loglog-beta in redis and did a repo for it. Now I just added TailCut to it. Do you mind…",{"id":168,"text":169},"14637540","If there is a version annotation it is very easy to convert the old HyperLogLog to a 4 bit register representation. I will send you an email ASAP.",{"id":52,"text":53},{"commenters":172,"max_depth":162,"maker_replies":161,"first_comment_mins":173,"lifespan_hours":174},18,45,337,[176,178,180,182,184,186,187,189,192,195,197,199,202,205,207,209,212,214,217,221,222,225,227,230,233,236,239,241,243,245,247,249,252,255,256,258,260,263,265,268,270,272,274,278],[147,33,34,177,148,33],"project\u002Fneutral\u002Fdesign",[165,33,33,179,11,39],"maker\u002Fneutral\u002Fidea",[181,33,34,177,148,33],"14637337",[168,33,33,183,11,39],"maker\u002Fneutral\u002Fdesign",[52,39,33,185,11,39],"project\u002Fpositive\u002Fidea",[55,31,44,185,56,33],[64,34,44,188,65,33],"project\u002Fpositive\u002Fnone",[190,34,33,191,11,39],"14637796","maker\u002Fneutral\u002Fnone",[135,193,34,194,136,33],4,"project\u002Fneutral\u002Fidea",[196,193,33,194,11,39],"14637187",[198,193,34,194,136,33],"14637283",[200,193,33,201,11,39],"14637380","maker\u002Fpositive\u002Fexecution",[203,193,34,204,136,33],"14637420","project\u002Fneutral\u002Fperformance",[95,193,39,206,96,33],"project\u002Fcritical\u002Fexecution",[208,193,39,206,136,33],"14645682",[210,193,34,211,96,33],"14645940","project\u002Fneutral\u002Fexecution",[213,193,33,191,11,39],"14648563",[215,193,33,216,96,33],"14652030","commenter\u002Fneutral\u002Fnone",[218,44,33,219,220,33],"14637903","other\u002Fneutral\u002Fnone","pjungwir",[158,44,33,219,11,39],[223,44,33,219,224,33],"14637933","teej",[226,44,33,219,220,33],"14638205",[228,44,33,219,229,33],"14637949","E6300",[231,44,33,219,232,33],"14638043","contravariant",[234,44,33,219,235,33],"14638597","zintinio5",[237,44,33,219,238,33],"14736601","vogtb",[102,162,34,240,103,33],"project\u002Fneutral\u002Fdocs",[242,162,34,240,56,33],"14637053",[244,162,34,240,103,33],"14637737",[246,162,33,204,11,39],"14637803",[106,162,39,248,103,33],"project\u002Fcritical\u002Fdocs",[250,162,33,251,11,39],"14638210","maker\u002Fneutral\u002Fdocs",[113,253,34,254,114,33],7,"project\u002Fneutral\u002Fnone",[120,18,34,194,121,33],[257,18,33,179,11,39],"14640967",[259,18,34,179,121,33],"14643235",[261,18,33,262,235,33],"14657455","other\u002Fneutral\u002Fidea",[127,264,34,204,128,33],9,[266,264,33,267,11,39],"14637666","maker\u002Fneutral\u002Fperformance",[269,264,34,191,114,33],"14678342",[68,271,34,204,69,33],10,[75,271,33,273,11,39],"project\u002Fpositive\u002Fperformance",[275,271,33,276,277,33],"14638064","other\u002Fneutral\u002Fperformance","dom0",[279,271,33,276,56,33],"14637078",[],[282,287,292,297,301,305],{"id":283,"name":284,"points":285,"category":286},"14644539","GreenPiThumb",537,"hardware",{"id":288,"name":289,"points":290,"category":291},"14625795","Insect",354,"dev_tools",{"id":293,"name":294,"points":295,"category":296},"14652217","Hackerhunt",335,"productivity",{"id":298,"name":299,"points":300,"category":291},"14612440","Sourcetrail",328,{"id":302,"name":303,"points":304,"category":291},"14614304","pack3d",260,{"id":306,"name":307,"points":308,"category":309},"14610882","Crown",193,"games",12,false,"v2",{"id":314,"name":5,"line":6,"ai":311,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":315,"top":18,"kw":316,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":317,"url":9,"mix":318,"praise":82,"objection":92,"sub":21},14636699,0.06,[24,25,26,27],"3030055033031130000000003330133030303300",[31,32,33,34],1793990960682]