[{"data":1,"prerenderedAt":959},["ShallowReactive",2],{"project-46294144":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":44,"post_text":47,"what":48,"said":74,"maker_in_thread":165,"shape":176,"grid":181,"attempts":920,"same_week":921,"maker_shows":43,"maker_hidden":951,"reader_version":952,"card":953},"46294144","fuzzycanary","Stop AI scrapers from hammering your self-hosted blog (using porn)","title","Show HN: Stop AI scrapers from hammering your self-hosted blog (using porn)","https:\u002F\u002Fgithub.com\u002Fvivienhenz24\u002Ffuzzy-canary","github.com","misterchocolat",1765917758,2025,373,277,129,3837,4,"legend","security_privacy","anti-scraping defense","library",[24,25,26,27],"ai scrapers","anti scraping","self-hosted blog","honeypot links","roasted",-0.373,{"positive":31,"neutral":32,"mixed":33,"critical":34},6,13,2,23,44,{"idea":37,"execution":39,"design":41,"privacy":42},{"positive":31,"critical":38},14,{"positive":40,"critical":18},0,{"positive":40,"critical":33},{"positive":40,"critical":43},1,[45,40,40,40,40,40,40,46,40,40,40,40,40,40,40,40,40,40,40,46,46,45,43,43,40,40,43,43,40,18,33,40,43,46,46,43,33,33,40,43],5,3,"Alright so if you run a self-hosted blog, you've probably noticed AI companies scraping it for training data. And not just a little (RIP to your server bill).\n\nThere isn't much you can do about it without cloudflare. These companies ignore robots.txt, and you're competing with teams with more resources than you. It's you vs the MJs of programming, you're not going to win.\n\nBut there is a solution. Now I'm not going to say it's a great solution...but a solution is a solution. If your website contains content that will trigger their scraper's safeguards, it will get dropped from their data pipel",{"problem":49,"audience":55,"pricing":56,"motivation":62,"time_to_build":55,"stack":66,"openness":71,"stage":72,"team_size":73},{"value":50,"source":51,"evidence":52},"AI companies scrape self-hosted blogs, driving up server costs and ignoring robots.txt","maker",[53],{"id":54},"post",null,{"value":57,"source":51,"evidence":58},"free",[59],{"id":60,"by":11,"text":61},"46296767","yes but here it's free, whereas this (https:\u002F\u002Fwebdecoy.com\u002F) is at least 59$ a month",{"value":63,"source":51,"evidence":64},"Stop AI scrapers from hammering self-hosted blogs",[65],{"id":54},[67],{"value":68,"source":51,"evidence":69},"npm package",[70],{"id":54},"open_source","launched","unknown",{"praise":75,"objections":88,"requests":125,"questions":126,"alternatives":138},[76],{"value":77,"source":78,"evidence":79},"Creative, entertaining idea","commenter",[80,84],{"id":81,"by":82,"text":83},"46318752","kstrauser","I love the insanity of this idea. Not saying it's a good idea, but it's a very highly entertaining one, and I like that! I've also had enormous luck with Anubis. AI scrapers…",{"id":85,"by":86,"text":87},"46320208","samename","This is a very creative hack to a common, growing problem. Well done! Also, I like that you acknowledge it's a bad idea: that gives you more freedom to experiment and iterate.",[89,96,103,114],{"value":90,"source":78,"evidence":91},"Cloaking different content to search engines can get sites banned",[92],{"id":93,"by":94,"text":95},"46325828","jakub_g","> checks user agents and won't show the links to legitimate search engines, so Google and Bing won't see them. Serving different contents to search engines is called \"cloaking\"…",{"value":97,"source":78,"evidence":98},"Cloudflare already offers free bot mitigation",[99],{"id":100,"by":101,"text":102},"46319088","JohnMakin","Cloudflare offers bot mitigation for free, and pretty generous WAF rules that makes mitigations like this seem a little overblown to me",{"value":104,"source":78,"evidence":105},"User agents are easily spoofed; verify by IP or reverse DNS",[106,110],{"id":107,"by":108,"text":109},"46322743","onion2k","So fuzzycanary also checks user agents and won't show the links to legitimate search engines, so Google and Bing won't see them. Unscrupulous AI scrapers will not be using a…",{"id":111,"by":112,"text":113},"46324434","dewey","> user agents and won't show the links to legitimate search engines, so Google and Bing won't see them Worth noting that in general if you do any \"is this Google or not\" you…",{"value":115,"source":78,"evidence":116},"Could get the blog flagged as porn by corporate proxies",[117,121],{"id":118,"by":119,"text":120},"46326896","MayeulC","Ah, I wonder if corporate proxies will end up flagging your blog as porn, if you protect it this way?",{"id":122,"by":123,"text":124},"46318733","wazoox","Isn't there a risk to get your blog blocked in corporate environment though? If it's a technical blog that would be unfortunate.",[],[127,134],{"value":128,"source":78,"evidence":129},"Do scrapers actually stop after seeing the links?",[130],{"id":131,"by":132,"text":133},"46319435","owl57","> scrapers can ingest them and say \"nope we won't scrape there again in the future\" Do all the AI scrapers actually do that?",{"value":135,"source":78,"evidence":136},"How to avoid links showing for static site generators",[137],{"id":54},[139,143,151,158],{"value":140,"source":78,"evidence":141},"Anubis",[142],{"id":81,"by":82,"text":83},{"value":144,"source":78,"evidence":145},"Cloudflare",[146,150],{"id":147,"by":148,"text":149},"46321302","eek2121","Disclosure, I've not run a website since my health issues began, however, Cloudflare has an AI firewall, Cloudflare is super cheap (also: unsure if the AI firewall is on the free…",{"id":100,"by":101,"text":102},{"value":152,"source":78,"evidence":153},"webdecoy",[154],{"id":155,"by":156,"text":157},"46296481","cport1","That's a pretty hilarious idea, but in all serious you could use something like https:\u002F\u002Fwebdecoy.com\u002F",{"value":159,"source":78,"evidence":160},"zip bomb protection",[161],{"id":162,"by":163,"text":164},"46320219","montroser","Reminds me of poisoning bot responses with zip bombs of sorts: https:\u002F\u002Fidiallo.com\u002Fblog\u002Fzipbomb-protection",{"comments":18,"answered":33,"key":166},[167,170,173],{"id":168,"text":169},"46311134","hey! thanks for that read suggestion that's indeed a pretty funny captcha strat. Yup the links show up if you use the Lynx web browser. As for AI scrapers impersonating googlebot…",{"id":171,"text":172},"46318736","the parent container uses display: none, so a screen reader will skip the links",{"id":174,"text":175},"46325861","didn't know that thanks for pointing it out, i'll remove that feature",{"commenters":177,"max_depth":178,"maker_replies":18,"first_comment_mins":179,"lifespan_hours":180},155,9,199,340,[182,184,188,190,193,195,198,200,202,204,206,209,211,214,217,220,223,226,228,230,232,235,238,241,244,247,249,251,253,255,258,261,264,268,270,272,274,277,280,283,286,290,293,296,299,303,306,309,311,314,317,320,323,325,328,331,334,337,340,343,346,348,351,355,357,360,363,365,367,370,372,374,376,378,381,384,387,389,391,393,396,398,400,402,405,408,410,413,416,419,422,424,427,430,432,434,436,438,440,443,445,447,450,453,456,459,462,464,467,469,471,473,476,478,481,484,486,488,490,492,494,497,499,502,505,508,511,513,516,519,522,525,527,529,532,534,536,540,542,544,546,549,552,554,557,559,561,564,568,571,574,576,578,580,583,586,587,592,594,596,600,604,605,608,611,613,616,619,621,624,627,630,633,635,638,640,643,645,648,652,654,656,660,662,666,668,672,675,679,681,683,685,688,691,694,696,698,700,702,704,707,709,713,717,719,723,725,728,730,735,739,742,746,749,751,753,755,757,760,762,764,766,768,771,773,777,779,781,785,789,793,795,799,801,803,805,810,814,816,818,822,826,828,831,836,839,842,844,847,851,853,857,861,864,866,868,872,874,878,880,882,885,887,890,893,896,899,901,905,909,913,917],[81,40,45,183,82,40],"project\u002Fpositive\u002Fidea",[185,40,40,186,187,40],"46318940","other\u002Fneutral\u002Fnone","n1xis10t",[189,40,40,186,82,40],"46319518",[191,40,40,186,192,40],"46320009","ziml77",[194,40,40,186,187,40],"46320061",[196,40,40,186,197,40],"46321438","xena",[199,40,40,186,187,40],"46321527",[201,40,40,186,197,40],"46321673",[203,40,40,186,187,40],"46321747",[205,40,40,186,82,40],"46322341",[207,40,40,186,208,40],"46324467","63stack",[210,40,40,186,197,40],"46325361",[212,40,40,186,213,40],"46322629","tommica",[215,40,40,186,216,40],"46322792","gettingoverit",[218,40,40,186,219,40],"46324316","mrweasel",[221,40,40,186,222,40],"46324398","Yizahi",[224,40,40,186,225,40],"46324051","ramonga",[227,40,40,186,187,40],"46327031",[229,40,40,186,187,40],"46319076",[231,40,40,186,82,40],"46319577",[233,40,40,234,187,40],"46319614","commenter\u002Fneutral\u002Fnone",[236,40,40,186,237,40],"46323444","miki123211",[239,40,40,186,240,40],"46323795","PeterStuer",[242,40,40,234,243,40],"46326133","throw10920",[245,40,40,234,246,40],"46328193","zhengyi13",[248,40,40,234,82,40],"46329352",[250,40,40,234,243,40],"46334067",[252,40,40,186,219,40],"46324302",[254,40,40,186,187,40],"46327000",[256,40,40,186,257,40],"46327588","kelvinjps10",[259,40,40,186,260,40],"46328092","lelanthran",[262,40,40,186,263,40],"46336066","rurban",[265,40,40,266,267,40],"46320421","other\u002Fpositive\u002Fnone","anonymous908213",[269,40,40,266,82,40],"46320491",[271,40,40,186,187,40],"46320696",[273,40,40,266,82,40],"46320824",[275,40,40,186,276,40],"46324200","Imustaskforhelp",[278,40,40,186,279,40],"46326230","D-Machine",[281,40,40,186,282,40],"46322124","teeray",[284,40,40,186,285,40],"46338953","GoblinSlayer",[287,40,40,288,289,40],"46323983","other\u002Fcritical\u002Fperformance","PunchyHamster",[291,40,40,288,292,40],"46325157","timpera",[294,40,40,266,295,40],"46324148","m4rtink",[297,40,40,186,298,40],"46324296","acheong08",[300,40,40,301,302,40],"46325061","other\u002Fcritical\u002Fnone","prmoustache",[304,40,40,186,305,40],"46325651","stefanka",[307,40,40,186,308,40],"46326071","easton",[310,40,40,186,302,40],"46329461",[312,40,40,301,313,40],"46327159","brettermeier",[315,40,40,266,316,40],"46330920","opem",[318,40,40,186,319,40],"46321542","buu700",[321,40,40,186,322,40],"46327633","amypetrik8",[324,40,45,183,260,40],"46328177",[326,40,43,327,285,40],"46339326","project\u002Fcritical\u002Fidea",[329,43,40,186,330,40],"46326827","zackmorris",[332,43,40,186,333,40],"46328353","dvfjsdhgfv",[335,43,40,186,336,40],"46329119","smegger001",[338,43,40,186,339,40],"46329553","ATechGuy",[341,43,40,186,342,40],"46333605","conrs",[344,43,40,186,345,40],"46342923","venturecruelty",[347,33,40,186,163,40],"46320077",[349,33,40,350,187,40],"46320127","other\u002Fneutral\u002Fperformance",[352,33,40,353,354,40],"46320497","other\u002Fneutral\u002Fidea","hollowturtle",[356,33,40,186,187,40],"46320686",[358,33,40,353,359,40],"46322004","vivzkestrel",[361,33,40,234,362,40],"46322450","chii",[364,33,40,234,359,40],"46323029",[366,33,40,234,354,40],"46323615",[368,33,40,369,359,40],"46324249","commenter\u002Fneutral\u002Fidea",[371,33,40,369,362,40],"46324945",[373,33,40,234,359,40],"46325918",[375,33,40,234,302,40],"46325095",[377,33,40,234,359,40],"46325908",[379,33,40,380,208,40],"46330508","commenter\u002Fneutral\u002Fperformance",[382,33,40,380,383,40],"46326412","dirkc",[385,33,40,380,386,40],"46327854","integralid",[388,33,40,234,289,40],"46324017",[390,33,40,234,359,40],"46323038",[392,33,40,234,386,40],"46328021",[394,33,40,234,395,40],"46327290","mook",[397,33,40,234,260,40],"46328235",[399,33,40,186,345,40],"46342934",[401,33,40,353,289,40],"46324022",[403,33,40,369,404,40],"46325525","jrm4",[406,33,40,186,407,40],"46326278","lowwave",[409,33,40,186,345,40],"46342929",[411,33,40,353,412,40],"46326329","nkrisc",[414,33,40,186,415,40],"46327251","encom",[417,46,40,186,418,40],"46319207","thethingundone",[420,46,40,234,421,40],"46319314","sandblast",[423,46,40,234,418,40],"46319677",[425,46,40,234,426,40],"46319329","danpalmer",[428,46,40,234,429,40],"46319562","crote",[431,46,40,353,187,40],"46319608",[433,46,40,234,418,40],"46319691",[435,46,40,234,426,40],"46320521",[437,46,40,234,418,40],"46321364",[439,46,40,234,426,40],"46321996",[441,46,40,186,442,40],"46319423","andrepd",[444,46,40,186,418,40],"46319746",[446,46,40,234,187,40],"46319780",[448,46,40,186,449,40],"46320364","bobbiechen",[451,46,40,186,452,40],"46323097","happymellon",[454,46,40,186,455,40],"46324119","wooger",[457,46,40,234,458,40],"46320545","reallyhuh",[460,46,40,186,461,40],"46320973","giantrobot",[463,46,40,186,187,40],"46321017",[465,46,40,186,466,40],"46319771","sethops1",[468,46,40,186,187,40],"46319793",[470,46,40,186,461,40],"46320867",[472,46,40,186,285,40],"46339589",[474,46,40,353,475,40],"46322551","thisislife2",[477,46,40,234,452,40],"46323058",[479,46,40,186,480,40],"46323498","chrismorgan",[482,46,40,186,483,40],"46321296","tokioyoyo",[485,46,40,234,418,40],"46321351",[487,46,40,234,483,40],"46321390",[489,46,40,353,187,40],"46321361",[491,46,40,234,483,40],"46321408",[493,46,40,234,187,40],"46321566",[495,46,40,186,496,40],"46323010","makapuf",[498,46,40,186,289,40],"46324003",[500,46,40,186,501,40],"46325345","nephihaha",[503,46,40,186,504,40],"46322646","idiotsecant",[506,46,40,186,507,40],"46324077","symbogra",[509,46,40,353,510,40],"46327530","tmnvix",[512,46,40,186,219,40],"46324323",[514,46,40,186,515,40],"46324823","GaryBluto",[517,46,40,186,518,40],"46325531","csomar",[520,46,40,186,521,40],"46331618","stevage",[523,18,46,524,187,40],"46307364","project\u002Fneutral\u002Fidea",[168,18,40,526,11,43],"maker\u002Fneutral\u002Fnone",[528,18,40,234,187,40],"46317732",[530,18,40,234,531,40],"46319488","throawayonthe",[533,18,40,234,187,40],"46319543",[535,18,40,234,132,40],"46321633",[537,18,46,538,539,40],"46319052","project\u002Fneutral\u002Fexecution","kuylar",[541,18,40,234,187,40],"46319090",[543,18,40,186,132,40],"46319366",[545,18,40,234,187,40],"46319453",[547,18,40,234,548,40],"46321829","ProllyInfamous",[550,18,40,234,551,40],"46321979","netsharc",[553,18,40,186,548,40],"46322037",[555,45,45,183,556,40],"46322233","cookiengineer",[558,45,40,234,187,40],"46322258",[560,45,40,234,556,40],"46322460",[562,31,40,186,563,40],"46323946","voodooEntity",[565,566,40,186,567,40],"46321131",7,"asphero",[569,566,40,186,570,40],"46323714","rhdunn",[572,566,40,186,573,40],"46325206","socialcommenter",[575,566,40,186,187,40],"46327193",[577,566,40,186,573,40],"46327564",[579,566,40,186,187,40],"46329399",[107,581,43,582,108,40],8,"project\u002Fcritical\u002Fexecution",[584,581,43,582,585,40],"46322788","bakugo",[111,178,43,582,112,40],[588,589,43,590,591,40],"46319582",10,"project\u002Fcritical\u002Fdesign","xg15",[593,589,40,186,82,40],"46319607",[595,589,40,234,187,40],"46319739",[597,598,40,186,599,40],"46325176",11,"santiagobasulto",[601,602,43,582,603,40],"46326661",12,"darepublic",[147,32,43,327,148,40],[606,32,40,234,607,40],"46321338","brigandish",[609,32,40,186,610,40],"46321507","ddtaylor",[612,32,40,186,548,40],"46322179",[614,32,40,186,615,40],"46333977","rglynn",[617,32,40,186,618,40],"46334724","Rastonbury",[620,32,40,186,548,40],"46337316",[622,32,40,234,623,40],"46323548","pjc50",[625,32,40,186,626,40],"46323671","nottorp",[628,32,40,186,629,40],"46325775","vaylian",[631,38,40,186,632,40],"46321741","nkurz",[634,38,40,186,187,40],"46321753",[636,38,40,186,637,40],"46321914","dspillett",[639,38,40,234,632,40],"46322022",[641,38,40,186,642,40],"46326165","s0laster",[122,644,43,327,123,40],15,[646,644,46,524,647,40],"46324201","jeroenhd",[649,650,43,327,651,40],"46318908",16,"reconnecting",[653,650,40,234,302,40],"46325192",[655,650,40,234,651,40],"46339140",[657,658,40,234,659,40],"46322016",17,"temporallobe",[661,658,40,234,626,40],"46323678",[663,664,40,186,665,40],"46320222",18,"globalnode",[667,664,40,234,187,40],"46320260",[669,670,40,186,671,40],"46324045",19,"bytehowl",[673,670,40,186,674,40],"46326224","icepush",[676,677,43,327,678,40],"46320451",20,"efilife",[680,677,40,234,187,40],"46320735",[682,677,43,327,339,40],"46320813",[684,677,40,234,678,40],"46320874",[686,677,40,186,687,40],"46327772","andersmurphy",[689,677,40,186,690,40],"46320925","phyzome",[692,677,40,186,693,40],"46322860","cyphar",[695,677,40,234,678,40],"46323055",[697,677,40,234,690,40],"46325144",[699,677,40,234,693,40],"46325566",[701,677,40,234,339,40],"46440144",[703,677,40,234,289,40],"46324029",[705,677,40,234,706,40],"46327688","6031769",[85,708,45,183,86,40],21,[710,711,46,524,712,40],"46331257",22,"mannanj",[714,34,46,715,716,40],"46318699","project\u002Fneutral\u002Fdesign","yjftsjthsd-h",[171,34,40,718,11,43],"maker\u002Fneutral\u002Fdesign",[720,721,43,327,722,40],"46322279",24,"true_religion",[131,724,46,524,132,40],25,[726,724,40,234,727,40],"46320073","amarant",[118,729,46,524,119,40],26,[731,732,46,733,734,40],"46334633",27,"project\u002Fmixed\u002Fidea","rl3",[736,737,46,524,738,40],"46327066",28,"jt2190",[740,737,40,234,741,40],"46327416","charlie-83",[743,744,40,186,745,40],"46318515",29,"username223",[747,744,40,186,748,40],"46319705","yupyupyups",[750,744,40,186,745,40],"46321551",[752,744,40,186,187,40],"46321625",[754,744,40,186,748,40],"46322871",[756,744,40,186,748,40],"46322855",[758,744,40,186,759,40],"46320600","whynotmaybe",[761,744,40,186,501,40],"46325391",[93,763,43,327,94,40],30,[174,763,40,765,11,43],"maker\u002Fneutral\u002Fidea",[767,763,40,234,687,40],"46327488",[769,770,43,327,163,40],"46320180",31,[772,770,40,234,187,40],"46320210",[774,775,46,733,776,40],"46324362",32,"drclegg",[162,778,40,186,163,40],33,[780,778,40,186,302,40],"46325215",[782,783,40,186,784,40],"46323688",34,"docheinestages",[786,787,46,524,788,40],"46323184",35,"megamix",[790,791,40,186,792,40],"46335484",36,"654wak654",[794,791,40,186,792,40],"46335498",[796,797,40,186,798,40],"46319515",37,"taurath",[800,797,40,186,187,40],"46319557",[802,797,40,186,429,40],"46319576",[804,797,40,234,187,40],"46319620",[806,807,46,808,809,40],"46326083",38,"project\u002Fneutral\u002Fnone","xgulfie",[811,812,46,808,813,40],"46322303",39,"admiralrohan",[155,815,46,524,156,40],40,[60,815,40,817,11,43],"maker\u002Fneutral\u002Fpricing",[819,820,46,808,821,40],"46324861",41,"shadowangel",[823,824,40,234,825,40],"46319389",42,"MisterTea",[827,824,40,186,187,40],"46319440",[829,824,40,186,830,40],"46319975","kylecazar",[832,833,43,834,835,40],"46320946",43,"project\u002Fcritical\u002Fprivacy","inetknght",[837,35,40,186,838,40],"46324938","cuku0078",[840,35,40,186,841,40],"46324947","FelipeCortez",[843,35,40,186,838,40],"46326855",[845,846,45,183,197,40],"46321458",45,[848,849,43,327,850,40],"46320638",46,"valenceidra",[852,849,40,234,187,40],"46320708",[854,849,43,855,856,40],"46326778","project\u002Fcritical\u002Fnone","rpigab",[858,859,43,590,860,40],"46324189",47,"kislotnik",[862,859,40,234,863,40],"46324275","brazukadev",[100,865,43,327,101,40],48,[867,865,45,183,187,40],"46319107",[869,865,40,870,871,40],"46319432","commenter\u002Fneutral\u002Fpricing","conception",[873,865,40,870,187,40],"46319475",[875,865,40,876,877,40],"46319583","other\u002Fneutral\u002Fpricing","Terr_",[879,865,40,234,101,40],"46319672",[881,865,40,234,187,40],"46319752",[883,865,40,884,678,40],"46320903","commenter\u002Fneutral\u002Fprivacy",[886,865,40,876,339,40],"46320835",[888,865,40,870,889,40],"46321268","gilrain",[891,865,40,186,892,40],"46322593","nospice",[894,865,40,186,895,40],"46323811","cakealert",[897,865,40,186,898,40],"46326749","oidar",[900,865,40,186,892,40],"46327686",[902,903,40,186,904,40],"46326679",49,"wcarss",[906,907,43,327,908,40],"46331240",50,"rogerwong",[910,911,43,855,912,40],"46327052",51,"geldedus",[914,915,43,327,916,40],"46324322",52,"_jsmh",[918,915,43,327,919,40],"46324642","lblume",[],[922,927,932,937,942,946],{"id":923,"name":924,"points":925,"category":926},"46290349","Picknplace.js",455,"languages",{"id":928,"name":929,"points":930,"category":931},"46234734","bidicalc",252,"dev_tools",{"id":933,"name":934,"points":935,"category":936},"46234186","Sim",240,"ai",{"id":938,"name":939,"points":940,"category":941},"46249538","uvm32",196,"infrastructure",{"id":943,"name":944,"points":945,"category":931},"46276002","Sqlit",190,{"id":947,"name":948,"points":949,"category":950},"46264657","UDP-7777",104,"productivity",false,"v2",{"id":954,"name":5,"line":6,"ai":951,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":955,"top":18,"kw":956,"maker":11,"domain":10,"tier":19,"attempts":40,"strip":957,"url":9,"mix":958,"praise":77,"objection":90,"sub":21},46294144,-0.37,[24,25,26,27],"5000000300000000000335110011042013312201",[31,32,33,34],1794036687542]