[{"data":1,"prerenderedAt":584},["ShallowReactive",2],{"project-44878151":3},{"id":4,"name":5,"line":5,"line_source":6,"title":7,"url":8,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":46,"post_text":49,"what":50,"said":89,"maker_in_thread":192,"shape":197,"grid":201,"attempts":546,"same_week":547,"maker_shows":48,"maker_hidden":576,"reader_version":577,"card":578},"44878151","Building a web search engine from scratch with 3B neural embeddings","title","Show HN: Building a web search engine from scratch with 3B neural embeddings","https:\u002F\u002Fblog.wilsonl.in\u002Fsearch-engine\u002F","blog.wilsonl.in","wilsonzlin",1755014560,2025,699,118,39,3837,2,"legend","web_saas","search engine","web_app",[23,24,25,26],"web search engine","neural embeddings","vector search","semantic search","mixed",0.406,{"positive":30,"neutral":31,"mixed":17,"critical":32},29,12,11,54,{"idea":35,"pricing":37,"docs":40,"execution":41,"design":44,"performance":45},{"positive":36,"critical":17},7,{"positive":38,"critical":39},5,0,{"positive":36,"critical":39},{"positive":42,"critical":43},1,8,{"positive":42,"critical":39},{"positive":39,"critical":42},[38,39,38,47,38,47,38,39,39,39,39,39,47,47,17,38,38,39,39,39,39,42,39,47,39,39,42,39,39,39,47,42,38,47,48,38,38,38,38,38],3,4,null,{"problem":51,"audience":49,"pricing":65,"motivation":49,"time_to_build":77,"stack":85,"openness":86,"stage":87,"team_size":88},{"value":52,"source":53,"evidence":54},"Search results that are hard to get from major engines, using neural embeddings over crawled web pages","inferred",[55,57,61],{"id":56},"post",{"id":58,"by":59,"text":60},"44882069","poly2it","Very cool project! Just out of interest, I sent a query I've had difficulties getting good results for with major engines: \"what are some good options for high-resolution…",{"id":62,"by":63,"text":64},"44888301","rkagerer","Tried this search: What is an sbert embedding? Google still gave me a better result: https:\u002F\u002Ftowardsdatascience.com\u002Fsbert-deb3d4aef8a4\u002F Nevertheless this project looks great and…",{"value":66,"source":67,"evidence":68},"Maker's build cost was under $1000 and running cost around 50k, per commenters; no pricing stated","commenter",[69,73],{"id":70,"by":71,"text":72},"44880128","1gn15","This is incredibly, incredibly cool. Creating a search engine that beats Google in quality in just 2 months and less than a thousand dollars. Really great idea about the…",{"id":74,"by":75,"text":76},"44883732","dcreater","This wasn't even in the realm of what I thought is possible for a single person to do. Incredible work! It doesn't seem that far in diatance from a commercial search engine?…",{"value":78,"source":67,"evidence":79},"About two months, per the write-up as summarised by a commenter",[80,81],{"id":70,"by":71,"text":72},{"id":82,"by":83,"text":84},"44882586","de6u99er","Thank you for sharing! This is one of the coolest articles I have seen in a while on HN. I did some searches and I think the search results looked very useful so far. I…",[],"unknown","launched","solo",{"praise":90,"objections":112,"requests":131,"questions":152,"alternatives":175},[91,103],{"value":92,"source":67,"evidence":93},"Excellent, detailed write-up",[94,95,99],{"id":82,"by":83,"text":84},{"id":96,"by":97,"text":98},"44885418","gbrindisi","This must be the best technical article I read on HN in months!",{"id":100,"by":101,"text":102},"44898880","siquick","Brilliant write up - learnt a lot.",{"value":104,"source":67,"evidence":105},"Impressive solo effort producing results that rival major engines",[106,107,108],{"id":70,"by":71,"text":72},{"id":74,"by":75,"text":76},{"id":109,"by":110,"text":111},"44881255","tmelm","Incredibly cool. What a write-up. What an engineer.",[113,120,124],{"value":114,"source":67,"evidence":115},"Vector-only search misses exact keyword matches",[116],{"id":117,"by":118,"text":119},"44882913","jvia","A vector-only search engine will fail for a lot of common use cases where the keywords do matter. I tried searching for `garbanzo bean stew` and got totally irrelevant bean…",{"value":121,"source":67,"evidence":122},"Ranking favours meta-pages over specialist sources",[123],{"id":58,"by":59,"text":60},{"value":125,"source":67,"evidence":126},"Embeddings are open to SEO and prompt-injection style manipulation",[127],{"id":128,"by":129,"text":130},"44886787","zipy124","The author claims \"It should be far less susceptible to keyword spam and SEO tactics.\" however anyone with a cursory knowledge of the limitations of embeddings\u002FLLM's knows the…",[132,140],{"value":133,"source":67,"evidence":134},"Open source the code",[135,139],{"id":136,"by":137,"text":138},"44879815","Imustaskforhelp","This is really really cool. I had earlier wanted to entirely run my searches on it and though that seems possible, I feel like it would be sadly a little bit more waste of time…",{"id":82,"by":83,"text":84},{"value":141,"source":67,"evidence":142},"Add Common Crawl or other open crawl data",[143,147,148],{"id":144,"by":145,"text":146},"44879617","ccgreg","At the end, the author thinks about adding Common Crawl data. Our ranking information, generated from our web graph, would probably be a big help in picking which pages to crawl.…",{"id":136,"by":137,"text":138},{"id":149,"by":150,"text":151},"44886994","jll29","Not sure where you are based, but if you were in the EU and had no commercial intentions, you might want to consider adding the crawls from OpenWebSearch.eu, an EU-funded…",[153,168],{"value":154,"source":67,"evidence":155},"Does the demo still work, given CORS errors and no results reported",[156,160,164],{"id":157,"by":158,"text":159},"44881394","Flux159","Getting a CORS error from the API - is the demo at https:\u002F\u002Fsearch.wilsonl.in\u002F working for anyone else?",{"id":161,"by":162,"text":163},"44881339","rkunnamp","I couldn't get the search working (there was some cors error) . But what a feat and writeup. Wonder Stuck!",{"id":165,"by":166,"text":167},"44933815","iamcreasy","Is search.wilconl.in still up? I do not get any search result.",{"value":169,"source":67,"evidence":170},"How much did it cost",[171],{"id":172,"by":173,"text":174},"44882097","sciencesama","how much did it cost ?",[176,181,188],{"value":177,"source":67,"evidence":178},"Google",[179,180],{"id":62,"by":63,"text":64},{"id":74,"by":75,"text":76},{"value":182,"source":67,"evidence":183},"Kagi",[184],{"id":185,"by":186,"text":187},"44886396","profstasiak","please, I want to pay for this. 10x better than Kagi which I stopped paying for",{"value":189,"source":67,"evidence":190},"YaCy",[191],{"id":70,"by":71,"text":72},{"comments":42,"answered":42,"key":193},[194],{"id":195,"text":196},"44886547","I'm not sure what were the exact limits, but I definitely recall running into server errors with S3 and the OCI equivalent service — not technically 429s but enough to…",{"commenters":198,"max_depth":43,"maker_replies":42,"first_comment_mins":199,"lifespan_hours":200},92,42,165,[202,206,210,213,216,219,221,223,225,229,232,234,237,239,243,244,246,250,253,256,259,263,265,267,269,272,274,276,278,280,283,286,288,291,293,296,299,302,305,307,309,311,314,317,320,323,326,331,333,337,340,342,345,347,350,353,356,359,362,365,368,371,374,378,381,384,386,390,392,396,401,405,409,412,414,417,419,422,426,429,432,434,436,438,441,444,447,450,452,455,458,461,463,466,468,472,474,476,481,485,488,490,494,498,501,505,509,511,515,518,521,525,529,533,537,539,541,544],[203,39,38,204,205,39],"44880668","maker\u002Fpositive\u002Fidea","lysecret",[207,39,39,208,209,39],"44881811","other\u002Fneutral\u002Fnone","throwawaylaptop",[211,39,39,208,212,39],"44881865","mv4",[214,39,39,208,215,39],"44884450","johnthescott",[217,39,39,208,218,39],"44884576","what",[220,39,39,208,209,39],"44906183",[144,42,38,222,145,39],"project\u002Fpositive\u002Fpricing",[136,17,38,224,137,39],"project\u002Fpositive\u002Fidea",[226,47,38,227,228,39],"44885470","maker\u002Fpositive\u002Fdocs","demarq",[230,47,42,231,228,39],"44886075","project\u002Fcritical\u002Fexecution",[195,47,39,233,10,42],"project\u002Fneutral\u002Fexecution",[235,47,38,236,228,39],"44887386","maker\u002Fpositive\u002Fexecution",[149,48,47,238,150,39],"project\u002Fneutral\u002Fidea",[240,48,38,241,242,39],"44891044","maker\u002Fpositive\u002Fnone","bjornsing",[74,38,38,222,75,39],[58,245,42,231,59,39],6,[247,245,39,248,249,39],"44882752","commenter\u002Fneutral\u002Fnone","ricardobeat",[251,245,39,248,252,39],"44886431","creesch",[254,245,39,248,255,39],"44891728","jacobr1",[257,36,38,222,258,39],"44882636","divineg",[260,36,39,261,262,39],"44882836","other\u002Fneutral\u002Fidea","noosphr",[264,36,39,208,145,39],"44884589",[266,36,39,208,262,39],"44884930",[268,36,39,208,145,39],"44885163",[270,36,39,208,271,39],"44894545","nickpsecurity",[273,36,39,208,145,39],"44894989",[275,36,39,208,271,39],"44904840",[277,36,39,208,145,39],"44948606",[279,36,39,208,271,39],"44951293",[281,36,39,208,282,39],"44888176","kiririn",[284,36,39,208,285,39],"44890698","dieortin",[287,36,39,208,271,39],"44890197",[289,36,39,208,290,39],"44884040","echelon",[292,36,39,208,215,39],"44884457",[294,36,39,208,295,39],"44886907","gunalx",[297,43,47,238,298,39],"44883058","ww520",[300,43,47,238,301,39],"44885362","lelanthran",[303,43,39,248,304,39],"44888532","supriyo-biswas",[306,43,39,248,301,39],"44896846",[82,308,38,204,83,39],9,[117,310,42,231,118,39],10,[312,310,47,238,313,39],"44882954","cyanydeez",[315,310,47,233,316,39],"44883607","osigurdson",[318,310,42,231,319,39],"44885811","ospider",[321,32,47,233,322,39],"44898479","leobg",[324,31,38,222,325,39],"44883722","ryanlime",[327,328,38,329,330,39],"44880176",13,"maker\u002Fpositive\u002Fdesign","AndrewKemendo",[70,332,38,224,71,39],14,[334,335,39,208,336,39],"44883022",15,"arkmm",[338,335,39,208,339,39],"44883701","cedws",[341,335,39,208,215,39],"44884439",[343,335,39,208,344,39],"44885184","dannyw",[346,335,39,208,339,39],"44885390",[348,335,39,208,349,39],"44885634","mattigames",[351,335,39,208,352,39],"44893158","j33zusjuice",[354,335,39,208,355,39],"44886239","reasonableklout",[357,335,39,208,358,39],"44886437","gkbrk",[360,335,39,208,361,39],"44887682","dweinus",[363,335,39,208,364,39],"44892893","dymk",[366,335,39,208,367,39],"44886469","trhway",[369,335,39,248,370,39],"44889164","dpoloncsak",[372,335,39,248,373,39],"44885741","anothernewdude",[375,335,39,376,377,39],"44885871","commenter\u002Fneutral\u002Fpricing","magicalhippo",[128,379,42,380,129,39],16,"project\u002Fcritical\u002Fidea",[382,379,39,248,383,39],"44887246","binarymax",[385,379,39,248,129,39],"44898602",[387,379,39,388,389,39],"44887827","commenter\u002Fneutral\u002Fidea","yorwba",[391,379,42,380,129,39],"44898645",[393,394,38,241,395,39],"44883303",17,"mfrye0",[397,398,47,399,400,39],"44879501",18,"maker\u002Fneutral\u002Fnone","randomcatuser",[402,403,39,208,404,39],"44879683",19,"jobswithgptcom",[406,407,39,208,408,39],"44879062",20,"giancarlostoro",[410,407,39,208,411,39],"44879178","thr0w",[413,407,39,208,408,39],"44879287",[415,407,39,208,416,39],"44879231","andai",[418,407,39,208,408,39],"44879297",[420,407,39,208,421,39],"44879505","entropie",[423,407,42,424,425,39],"44888272","project\u002Fcritical\u002Fperformance","kdfjgbdfkjgb",[427,407,39,208,428,39],"44879313","xnx",[430,407,39,208,431,39],"44879615","ASalazarMX",[433,407,39,208,137,39],"44879854",[435,407,39,208,431,39],"44880119",[437,407,39,208,137,39],"44880911",[439,407,39,208,440,39],"44881383","mwcz",[442,407,39,208,443,39],"44880156","reactordev",[445,407,39,208,446,39],"44880338","h2zizzle",[448,407,39,208,449,39],"44881127","masfuerte",[451,407,39,208,389,39],"44879547",[453,407,39,261,454,39],"44879822","msgodel",[456,407,39,208,457,39],"44880634","mike_hearn",[459,407,47,238,460,39],"44881361","cosmic_cheese",[157,462,42,231,158,39],21,[464,462,42,231,465,39],"44881427","bstsb",[467,462,42,231,158,39],"44881442",[469,470,38,227,471,39],"44882477",22,"dangoodmanUT",[473,470,38,227,471,39],"44882688",[475,470,39,208,471,39],"44882901",[477,478,47,479,480,39],"44888180",23,"project\u002Fneutral\u002Fperformance","deepsquirrelnet",[482,483,47,238,484,39],"44881640",24,"mmargenot",[62,486,47,487,63,39],25,"project\u002Fmixed\u002Fperformance",[165,489,42,231,166,39],26,[491,492,38,224,493,39],"44889089",27,"indoorcomic",[495,496,38,227,497,39],"44883885",28,"seriocomic",[499,30,38,241,500,39],"44881793","a_spy",[502,503,38,241,504,39],"44881112",30,"A_Stefan",[506,507,38,241,508,39],"44881292",31,"voiper1",[96,510,38,227,97,39],32,[512,513,38,227,514,39],"44883234",33,"costco",[161,516,47,517,162,39],34,"project\u002Fmixed\u002Fexecution",[172,519,47,520,173,39],35,"project\u002Fneutral\u002Fpricing",[522,523,38,241,524,39],"44915358",36,"mips_avatar",[526,527,38,224,528,39],"44896117",37,"msephton",[530,531,38,224,532,39],"44895874",38,"sdotdev",[534,15,47,535,536,39],"44878786","maker\u002Fneutral\u002Fpricing","abraxas",[100,538,38,227,101,39],40,[185,540,38,222,186,39],41,[542,199,38,241,543,39],"44888796","matteohorvath",[109,545,38,241,110,39],43,[],[548,553,558,562,567,571],{"id":549,"name":550,"points":551,"category":552},"44846281","horizon",788,"creative",{"id":554,"name":555,"points":556,"category":557},"44855157","Engineering.fyi",470,"dev_tools",{"id":559,"name":560,"points":561,"category":557},"44878650","Omnara",310,{"id":563,"name":564,"points":565,"category":566},"44856935","Bolt",261,"languages",{"id":568,"name":569,"points":570,"category":552},"44870137","Tilf",214,{"id":572,"name":573,"points":574,"category":575},"44882441","Gore",148,"games",false,"v2",{"id":579,"name":5,"line":5,"ai":576,"cat":19,"year":12,"points":13,"comments":14,"rec":27,"s":580,"top":17,"kw":581,"maker":10,"domain":9,"tier":18,"attempts":39,"strip":582,"url":8,"mix":583,"praise":92,"objection":114,"sub":20},44878151,0.41,[23,24,25,26],"5053535000003325500001030010003153455555",[30,31,17,32],1794036680147]