[{"data":1,"prerenderedAt":311},["ShallowReactive",2],{"project-6297225":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":43,"post_text":45,"what":46,"said":72,"maker_in_thread":165,"shape":167,"grid":171,"attempts":274,"same_week":275,"maker_shows":32,"maker_hidden":304,"reader_version":305,"card":306},"6297225","Ferret","A fast substring search engine in Go","title","Show HN: We built a fast substring search engine in Go","http:\u002F\u002Fwww.tamber.com\u002Fposts\u002Fferret.html","tamber.com","alexirobbins",1377797961,2013,145,41,139,1739,8,"notable","dev_tools","search library","library",[24,25,26,27,28],"substring search","go search engine","autocomplete","suffix search","in-memory search","curious",-0.24,{"positive":32,"neutral":33,"mixed":34,"critical":18},3,11,0,22,{"idea":37,"performance":39,"pricing":41,"execution":42},{"positive":38,"critical":38},2,{"positive":34,"critical":40},1,{"positive":34,"critical":32},{"positive":40,"critical":38},[32,40,44,34,40,44,34,34,34,34,34,34,40,40,34,34,32,32,32,32,32,34,34,34,34,34,34,34,34,34,34,44,32,32,32,40,32,40,40,40],5,null,{"problem":47,"audience":45,"pricing":45,"motivation":58,"time_to_build":45,"stack":63,"openness":70,"stage":71,"team_size":70},{"value":48,"source":49,"evidence":50},"Slow low-cost autocomplete search over a small dictionary of artist names","commenter",[51,55],{"id":52,"by":53,"text":54},"6299449","argusdusty","EC2 medium instances come with 480GB space. Our database currently takes roughly 60GB. Ferret isn't running on this entire database, though - it's for auto-complete searches over…",{"id":56,"by":53,"text":57},"6298072","Author here: They are unrelated. We did take the time to examine writing our own full search engine based on Lucene, like the aforementioned search library. The reasons why we…",{"value":59,"source":49,"evidence":60},"Replace slow MongoDB prefix search that ran out of performance for autocomplete; Lucene judged too bloated",[61,62],{"id":56,"by":53,"text":57},{"id":52,"by":53,"text":54},[64],{"value":65,"source":66,"evidence":67},"Go","maker",[68],{"id":69},"post","unknown","launched",{"praise":73,"objections":92,"requests":123,"questions":130,"alternatives":141},[74,85],{"value":75,"source":49,"evidence":76},"Go is a good fit for a small, efficient search library",[77,81],{"id":78,"by":79,"text":80},"6298577","fspeech","Very cool. Considering that the core functionality you want is done in less than 500 LOC (https:\u002F\u002Fgithub.com\u002Fargusdusty\u002FFerret\u002Fblob\u002Fmaster\u002Fferret.go) it seems a great way to go…",{"id":82,"by":83,"text":84},"6297942","drsintoma","nice. I've been waiting to see some full-text search engine in Go for a while to compete with the omnipresent Lucene. Things like Go's low memory footprint could give it an…",{"value":86,"source":49,"evidence":87},"Suffix arrays used in practice is interesting",[88],{"id":89,"by":90,"text":91},"6297827","danieldk","It's nice to see suffix arrays being used in the real world. For anyone who is interested in suffix arrays, the following to papers are recommended: * Suffix arrays: A new method…",[93,100,112],{"value":94,"source":49,"evidence":95},"Lucene or Elasticsearch would have been a simpler choice",[96],{"id":97,"by":98,"text":99},"6300348","diego","This is cool. However, I wonder why the author didn't switch to a Lucene-based server such as Elasticsearch or Solr, for which you can find decent Go clients (or use the REST api…",{"value":101,"source":49,"evidence":102},"Hosting cost and engineering time are not worth it for a small startup",[103,107,111],{"id":104,"by":105,"text":106},"6298629","rorrr2","$150\u002Fmonth can buy you a very very good dedicated server. http:\u002F\u002Fwww.hetzner.de\u002Fen\u002Fhosting\u002Fprodukte_rootserver\u002Fex10",{"id":108,"by":109,"text":110},"6300095","thedufer","What are you valuing your time at? That sounds like shaky logic at best, especially considering how much work it sounds like this took.",{"id":97,"by":98,"text":99},{"value":113,"source":49,"evidence":114},"Lacks phonetic or spelling-agnostic matching",[115,119],{"id":116,"by":117,"text":118},"6299813","damian2000","I'd say having spelling agnostic search would be crucial. I searched yesterday on google play music for an artist I'd heard of that sounded something like \"n door\" so I typed…",{"id":120,"by":121,"text":122},"6299741","DrJosiah","There are two methods that are more typically known in the context of substring search that may help you get something even better than your prefix + Levenshtein distance 1…",[124],{"value":125,"source":49,"evidence":126},"Add phonetic matching module",[127],{"id":128,"by":53,"text":129},"6299880","This is a good idea :) I'll probably add in a module for phonetic matching when I get the time.",[131],{"value":132,"source":49,"evidence":133},"Why not use a trie or Boyer-Moore instead",[134,138],{"id":135,"by":136,"text":137},"6299478","fleitz","4MB? Wouldn't boyer-moore, or a trie be very efficient for this?",{"id":139,"by":109,"text":140},"6300279","I don't understand why the trie would take quadratic memory. I would expect it to be high-constant linear. Can you explain?",[142,149,161],{"value":143,"source":49,"evidence":144},"Lucene, Elasticsearch, Solr",[145,146],{"id":97,"by":98,"text":99},{"id":147,"by":90,"text":148},"6298258","Things like Go's low memory footprint could give it an important advantage in this area. You know that there is a C++ version of Lucene? http:\u002F\u002Fclucene.sourceforge.net\u002F There's…",{"value":150,"source":49,"evidence":151},"CLucene, Xapian, Sphinx, Apache Lucy",[152,153,157],{"id":147,"by":90,"text":148},{"id":154,"by":155,"text":156},"6300631","nkurz","I hadn't realized that CLucene was still active, thanks! Apache Lucy is another option that might appeal if you consider Lucene bloated. It's written in C, and while there aren't…",{"id":158,"by":159,"text":160},"6301978","vidarh","As for other non-Java Lucene alternatives, Sphinx works great, is compact and the C++ code is fairly easy to pick up enough of to make modifications if needed:…",{"value":162,"source":49,"evidence":163},"Metaphone and Double Metaphone for phonetic matching",[164],{"id":120,"by":121,"text":122},{"comments":34,"answered":34,"key":166},[],{"commenters":168,"max_depth":18,"maker_replies":34,"first_comment_mins":169,"lifespan_hours":170},25,76,24,[172,174,176,178,181,183,185,189,192,195,197,200,202,205,208,211,213,215,216,218,219,221,224,226,228,229,230,231,234,237,240,244,246,251,254,256,261,264,267,269,272],[120,34,32,173,121,34],"project\u002Fneutral\u002Fidea",[116,34,40,175,117,34],"project\u002Fcritical\u002Fperformance",[128,34,44,177,53,34],"maker\u002Fpositive\u002Fidea",[179,34,34,180,121,34],"6300142","commenter\u002Fneutral\u002Fnone",[97,40,40,182,98,34],"project\u002Fcritical\u002Fidea",[89,38,44,184,90,34],"project\u002Fpositive\u002Fidea",[186,38,34,187,188,34],"6297933","other\u002Fneutral\u002Fnone","ramayac",[190,38,34,187,191,34],"6298069","Scaevolus",[193,38,34,180,194,34],"6299443","LukeShu",[196,38,34,180,191,34],"6299551",[198,38,34,180,199,34],"6303006","fnl",[201,32,34,187,191,34],"6298026",[203,204,40,182,109,34],"6300107",4,[206,44,40,207,136,34],"6298345","project\u002Fcritical\u002Fpricing",[209,44,34,180,210,34],"6298569","pjscott",[212,44,34,180,136,34],"6299411",[52,44,32,214,53,34],"project\u002Fneutral\u002Fperformance",[135,44,32,173,136,34],[217,44,32,214,109,34],"6300265",[139,44,32,214,109,34],[220,44,32,214,53,34],"6300316",[222,44,34,223,109,34],"6300400","commenter\u002Fneutral\u002Fperformance",[225,44,34,223,53,34],"6300429",[82,227,34,187,83,34],6,[147,227,34,187,90,34],[154,227,34,187,155,34],[158,227,34,187,159,34],[232,227,34,187,233,34],"6298454","mumrah",[235,227,34,187,236,34],"6299261","moondowner",[238,227,34,187,239,34],"6299221","gwu78",[241,242,34,187,243,34],"6298680",7,"adcuz",[78,18,44,245,79,34],"project\u002Fpositive\u002Fexecution",[247,248,32,249,250,34],"6297800",9,"project\u002Fneutral\u002Fnone","jph",[252,248,32,249,253,34],"6298053","kalv",[56,248,32,255,53,34],"maker\u002Fneutral\u002Fidea",[257,258,40,259,260,34],"6298990",10,"project\u002Fcritical\u002Fexecution","6thSigma",[262,258,32,263,53,34],"6299103","maker\u002Fneutral\u002Fexecution",[265,33,40,259,266,34],"6298832","oron",[104,268,40,207,105,34],12,[270,268,32,271,53,34],"6298784","maker\u002Fneutral\u002Fpricing",[108,268,40,273,109,34],"maker\u002Fcritical\u002Fpricing",[],[276,281,286,291,296,300],{"id":277,"name":278,"points":279,"category":280},"6310317","Hire Programmers Leaving Syria",155,"web_saas",{"id":282,"name":283,"points":284,"category":285},"6320509","devos",150,"languages",{"id":287,"name":288,"points":289,"category":290},"6302825","Gingko",140,"productivity",{"id":292,"name":293,"points":294,"category":295},"6301478","Musicfellas",114,"consumer_social",{"id":297,"name":298,"points":299,"category":20},"6322875","road.li",88,{"id":301,"name":302,"points":303,"category":20},"6305069","Cleaver",82,false,"v2",{"id":307,"name":5,"line":6,"ai":304,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":30,"top":18,"kw":308,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":309,"url":9,"mix":310,"praise":75,"objection":94,"sub":21},6297225,[24,25,26,27,28],"3150150000001100333330000000000533313111",[32,33,34,18],1793990897314]