[{"data":1,"prerenderedAt":384},["ShallowReactive",2],{"project-41088273":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":43,"post_text":44,"what":45,"said":67,"maker_in_thread":181,"shape":192,"grid":196,"attempts":348,"same_week":349,"maker_shows":41,"maker_hidden":377,"reader_version":378,"card":379},"41088273","Semantic Grep","A Word2Vec-powered search tool","title","Show HN: Semantic Grep – A Word2Vec-powered search tool","https:\u002F\u002Fgithub.com\u002Farunsupe\u002Fsemantic-grep","github.com","arunsupe",1722103378,2024,356,57,141,3515,5,"legend","dev_tools","text search cli","cli",[24,25,26,27],"semantic search","word2vec","grep alternative","word embeddings","curious",0.07,{"positive":18,"neutral":31,"mixed":32,"critical":33},20,2,4,31,{"performance":36,"idea":38,"design":40,"execution":42},{"positive":37,"critical":37},0,{"positive":39,"critical":39},3,{"positive":41,"critical":37},1,{"positive":37,"critical":41},[39,37,37,37,18,37,39,41,37,41,39,39,39,39,39,37,37,37,39,39,37,39,39,18,39,39,37,39,18,39,37,37,39,37,41,37,37,18,39,18],"Much improved new version. Search for words similar to the query. For example, \"death\" will find \"death\", \"dying\", \"dead\", \"killing\"... Incredibly useful for exploring large text datasets where exact matches are too restrictive.",{"problem":46,"audience":52,"pricing":52,"motivation":52,"time_to_build":52,"stack":53,"openness":66,"stage":66,"team_size":66},{"value":47,"source":48,"evidence":49},"Exact-match grep is too restrictive for exploring large text datasets","maker",[50],{"id":51},"post",null,[54,62],{"value":55,"source":56,"evidence":57},"Go","inferred",[58],{"id":59,"by":60,"text":61},"41091710","microtonal","Some small tips from superficially reading the code: https:\u002F\u002Fgithub.com\u002Farunsupe\u002Fsemantic-grep\u002Fblob\u002Fb7dcc82a7cbab4fc4fdc1d4ef86092a071e48455\u002Fsgrep.go#L96 You can read the vector…",{"value":63,"source":48,"evidence":64},"Word2Vec",[65],{"id":51},"unknown",{"praise":68,"objections":89,"requests":119,"questions":136,"alternatives":152},[69],{"value":70,"source":71,"evidence":72},"Clever, useful idea","commenter",[73,77,81,85],{"id":74,"by":75,"text":76},"41089251","onli","That's totally clever and sound really useful. And it's one of those ideas where you go \"Why didn't I think of that\" when stumbling over the materials, word2vec in this case.",{"id":78,"by":79,"text":80},"41089516","randcraw","This would be really useful if it could take a descriptive phrase or a compound phrase (like SQL 'select X and Y and Z') and match against the semantic cluster(s) that the query…",{"id":82,"by":83,"text":84},"41090794","sam_perez","This tool seems really cool, want to play with it for sure. Just in general, semantic search across text seems like a much better ux for many applications. Wish it was more…",{"id":86,"by":87,"text":88},"41092515","tgw43279w","I really like how simple the implementation is!",[90,101,111],{"value":91,"source":71,"evidence":92},"Slow compared to grep",[93,97],{"id":94,"by":95,"text":96},"41090833","SloopJon","I take a Show HN to be a solicitation for feedback. It took me a few tries to get this to run. I tried passing -model_path, but it complained that it couldn't find config.json. I…",{"id":98,"by":99,"text":100},"41089355","pgroves","How fast is it?",{"value":102,"source":71,"evidence":103},"Word2vec cannot handle phrases or negation",[104,108],{"id":105,"by":106,"text":107},"41092513","throw156754228","The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even…",{"id":109,"by":11,"text":110},"41092569","This implementation does not because the query has to be a word. One way to extend it to phrases is to average the vectors of each word in the phrase. Another way is to have a…",{"value":112,"source":71,"evidence":113},"Setup and configuration were hard to get working",[114,118],{"id":115,"by":116,"text":117},"41089523","samatman","This is a good idea. I'm going to offer some unsolicited feedback here: The configuration thing is unclear to me. I think that \"current directory\" means \"same directory as the…",{"id":94,"by":95,"text":96},[120,127,131],{"value":121,"source":71,"evidence":122},"Multi-CPU parallelism",[123],{"id":124,"by":125,"text":126},"41091523","_flux","I wonder if it would be possible to easily add support for multiple CPUs? It seems to be taking at most 150% CPU, so on my workstation it could be (assuming high parallellism) 10…",{"value":128,"source":71,"evidence":129},"Performance optimizations such as BLAS or SIMD",[130],{"id":59,"by":60,"text":61},{"value":132,"source":71,"evidence":133},"Support for phrases and multiword queries",[134,135],{"id":78,"by":79,"text":80},{"id":109,"by":11,"text":110},[137,145],{"value":138,"source":71,"evidence":139},"How fast is it on large corpora and does it use a GPU",[140,144],{"id":141,"by":142,"text":143},"41089158","fbdab103","I might have a work use case for which this would be perfect. Having no experience with word2vec, some reference performance numbers would be great. If I have one million PDF…",{"id":98,"by":99,"text":100},{"value":146,"source":71,"evidence":147},"Can it search filenames",[148],{"id":149,"by":150,"text":151},"41089139","gunalx","Really cool. Often just want to fuzzy search for a word, and this would be useful. Can it do filenames as well ? Or do I need to pipe something like LS first.",[153,160,167,174],{"value":154,"source":71,"evidence":155},"Semgrep",[156],{"id":157,"by":158,"text":159},"41091190","fzeindl","Fyi, there is already a widely used tool along with a company called semgrep, which stems from semantic grep: https:\u002F\u002Fsemgrep.dev\u002F.",{"value":161,"source":71,"evidence":162},"Semantra",[163],{"id":164,"by":165,"text":166},"41093012","ttpphd","You might be interested in Semantra: https:\u002F\u002Fgithub.com\u002Ffreedmand\u002Fsemantra",{"value":168,"source":71,"evidence":169},"fltr",[170],{"id":171,"by":172,"text":173},"41091513","molli","Similar: https:\u002F\u002Fgithub.com\u002Fmoritztng\u002Ffltr Like grep but for natural language questions. Based on Mistral LLMs.",{"value":175,"source":71,"evidence":176},"Sentence-BERT embeddings",[177],{"id":178,"by":179,"text":180},"41092798","djoldman","Definitely a limitation of word2vec as applied here. Something like SBERT addresses this.",{"comments":182,"answered":183,"key":184},7,6,[185,186,189],{"id":109,"text":110},{"id":187,"text":188},"41091727","The model in Google drive is the official model from Google and will work. Haven't tried the huggingface model, but, looks very different. Unlikely to work.",{"id":190,"text":191},"41092395","I would think translation to other languages would be trivial. The model is just a map of word to vector. Every word is converted to it's vector representation; then the query…",{"commenters":193,"max_depth":183,"maker_replies":182,"first_comment_mins":194,"lifespan_hours":195},43,89,108,[197,199,203,207,209,212,214,216,220,222,224,226,228,231,232,236,238,240,241,242,245,247,251,254,256,257,261,264,266,268,270,274,275,277,279,281,284,287,289,292,295,300,304,305,307,309,311,315,318,322,326,328,331,334,337,340,344],[59,37,39,198,60,37],"maker\u002Fneutral\u002Fperformance",[200,37,37,201,202,37],"41092778","commenter\u002Fneutral\u002Fperformance","neonsunset",[204,37,37,205,206,37],"41093097","commenter\u002Fneutral\u002Fnone","bitexploder",[208,37,37,205,202,37],"41093540",[210,37,37,205,211,37],"41094342","e12e",[213,37,37,205,202,37],"41094515",[74,41,18,215,75,37],"project\u002Fpositive\u002Fidea",[217,41,37,218,219,37],"41090096","other\u002Fneutral\u002Fidea","ignoramous",[115,32,39,221,116,37],"project\u002Fmixed\u002Fdesign",[94,32,41,223,95,37],"project\u002Fcritical\u002Fexecution",[105,39,41,225,106,37],"project\u002Fcritical\u002Fidea",[109,39,37,227,11,41],"project\u002Fneutral\u002Fidea",[229,39,39,227,230,37],"41095437","vunderba",[178,39,41,225,179,37],[233,39,39,234,235,37],"41093560","project\u002Fmixed\u002Fidea","VHRanger",[124,33,39,237,125,37],"project\u002Fneutral\u002Fperformance",[187,33,37,239,11,41],"project\u002Fneutral\u002Fnone",[78,18,39,227,79,37],[157,183,39,227,158,37],[243,183,39,239,244,37],"41091292","icholy",[246,183,37,205,158,37],"41091368",[248,183,37,249,250,37],"41091375","other\u002Fneutral\u002Fnone","tracerbulletx",[252,183,37,249,253,37],"41092020","throwaway290",[255,183,37,249,206,37],"41093125",[171,182,37,218,172,37],[258,259,39,227,260,37],"41089498",8,"drdeca",[262,259,37,249,263,37],"41089866","throwawaydummy",[265,259,39,239,219,37],"41090084",[267,259,37,249,263,37],"41090155",[269,259,37,249,219,37],"41093763",[271,272,39,227,273,37],"41092057",9,"rldjbpin",[164,272,39,239,165,37],[82,276,18,215,83,37],10,[149,278,39,239,150,37],11,[141,280,39,237,142,37],12,[282,280,39,237,283,37],"41089250","9dev",[285,280,37,249,286,37],"41089675","michaelmior",[288,280,37,249,283,37],"41091786",[290,280,39,237,291,37],"41089337","SkyPuncher",[293,280,39,237,294,37],"41089513","kristopolous",[296,297,18,298,299,37],"41089679",13,"project\u002Fpositive\u002Fnone","rasengan0",[301,302,39,239,303,37],"41092316",14,"infruset",[190,302,37,239,11,41],[306,302,37,239,11,41],"41096489",[308,302,37,239,11,41],"41126469",[98,310,39,237,99,37],15,[312,313,39,239,314,37],"41098551",16,"piyushtechsavy",[316,313,37,317,11,41],"41116736","maker\u002Fneutral\u002Fnone",[319,320,41,225,321,37],"41097451",17,"synergy20",[323,324,37,249,325,37],"41088760",18,"sitkack",[327,324,37,249,11,41],"41088967",[329,324,37,249,330,37],"41089004","threatofrain",[86,332,18,333,87,37],19,"project\u002Fpositive\u002Fdesign",[335,31,39,227,336,37],"41095111","shubham13596",[338,31,39,227,339,37],"41096376","mmcwilliams",[341,342,37,249,343,37],"41089875",21,"low_tech_punk",[345,346,18,215,347,37],"41090961",22,"danielmarkbruce",[],[350,354,358,363,368,372],{"id":351,"name":352,"points":353,"category":20},"41068719","Haystack",546,{"id":355,"name":356,"points":357,"category":20},"41070428","Wat",393,{"id":359,"name":360,"points":361,"category":362},"41086620","Opslane",319,"infrastructure",{"id":364,"name":365,"points":366,"category":367},"41105130","embedding.io",305,"ai",{"id":369,"name":370,"points":371,"category":20},"41080189","gg",205,{"id":373,"name":374,"points":375,"category":376},"41095839","statsbomb-3d-viz",182,"creative",false,"v2",{"id":380,"name":5,"line":6,"ai":377,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":29,"top":18,"kw":381,"maker":11,"domain":10,"tier":19,"attempts":37,"strip":382,"url":9,"mix":383,"praise":70,"objection":91,"sub":21},41088273,[24,25,26,27],"3000503101333330003303353303530030100535",[18,31,32,33],1793991096717]