[{"data":1,"prerenderedAt":544},["ShallowReactive",2],{"project-21215484":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":47,"post_text":49,"what":50,"said":101,"maker_in_thread":220,"shape":223,"grid":228,"attempts":507,"same_week":508,"maker_shows":43,"maker_hidden":536,"reader_version":537,"card":538},"21215484","Spider Pro","easy and cheap way to scrape the internet","title","Show HN: Spider Pro – easy and cheap way to scrape the internet","https:\u002F\u002Ftryspider.com\u002F","tryspider.com","hyperyolo",1570720706,2019,391,103,60,2195,3,"legend","dev_tools","web scraper","browser_extension",[24,25,26,27],"web scraping","data extraction","browser extension","no-code scraper","curious",-0.032,{"positive":31,"neutral":32,"mixed":33,"critical":34},10,25,2,11,48,{"idea":37,"docs":38,"execution":40,"pricing":42,"design":44,"privacy":45,"performance":46},{"positive":33,"critical":33},{"positive":39,"critical":39},0,{"positive":39,"critical":41},4,{"positive":43,"critical":41},1,{"positive":18,"critical":39},{"positive":39,"critical":39},{"positive":43,"critical":43},[18,18,18,39,39,39,39,18,18,39,39,39,48,39,39,39,18,18,39,18,43,18,48,18,39,43,43,18,18,33,18,43,48,48,43,39,43,39,48,48],5,null,{"problem":51,"audience":63,"pricing":68,"motivation":49,"time_to_build":49,"stack":79,"openness":98,"stage":99,"team_size":100},{"value":52,"source":53,"evidence":54},"Scraping web data without programming","commenter",[55,59],{"id":56,"by":57,"text":58},"21217558","penagwin","Disclaimer: I have nothing to do with this project, I'm also a programmer isn't part of it's target demographic. But: Are you suggesting that if somebody wanted to setup a basic…",{"id":60,"by":61,"text":62},"21217111","ianmobbs","Since this is designed for non-technical users and only scrapes content that's already been displayed to the user, I can't imagine many folks would use it in such a way that they…",{"value":64,"source":53,"evidence":65},"Non-technical users who need web data",[66,67],{"id":60,"by":61,"text":62},{"id":56,"by":57,"text":58},{"value":69,"source":53,"evidence":70},"One-time payment, pay once and use forever",[71,75],{"id":72,"by":73,"text":74},"21216449","ithkuil","EDIT: snippet from the website to help answer this question without requiring you to click. I didn't intend this to be a rude answer, I don't think it deserves downvotes. \" + No…",{"id":76,"by":77,"text":78},"21216990","NoNotTheDuo","On the homepage, right above the Purchase and Trial buttons is \"Pay Once and Use It Forever\"",[80,91],{"value":81,"source":53,"evidence":82},"Chrome extension",[83,87],{"id":84,"by":85,"text":86},"21216392","seanwilson","Any comments on using Gumroad as your payment processor? I went with Paddle for a Chrome extension - it seems more flexible (e.g. you could do team subscription plans with custom…",{"id":88,"by":89,"text":90},"21216107","joshmn","This is scary good and easy to use. Clever idea with the Chrome extension too.",{"value":92,"source":93,"evidence":94},"Firefox extension","maker",[95],{"id":96,"by":11,"text":97},"21221081","yup, firefox is supported! Though not distributed through the webstore - I'm planning to move it to the store sometimes soon.","paid","launched","solo",{"praise":102,"objections":118,"requests":163,"questions":175,"alternatives":194},[103,111],{"value":104,"source":53,"evidence":105},"Easy to use with clean UI",[106,107],{"id":88,"by":89,"text":90},{"id":108,"by":109,"text":110},"21215869","porker","This has a nice UI. It reminds me of Kantu (now https:\u002F\u002Fui.vision\u002F) which I've used with varying degrees of success. That works by recording Selenium scripts; is Spider Pro…",{"value":112,"source":53,"evidence":113},"Clever browser-extension approach",[114,115],{"id":88,"by":89,"text":90},{"id":116,"by":57,"text":117},"21216936","Detection is always a cat and mouse game. Using an extension in a real browser (instead of electron\u002Fheadless chrome) is probably one of the hardest to detect because it requires…",[119,134,145,156],{"value":120,"source":53,"evidence":121},"Pricing unclear or not cheapest versus free tools",[122,126,130],{"id":123,"by":124,"text":125},"21216428","visualphoenix","This is lacking clarity in the payment schedule... Is it One-time? Monthly? Yearly?",{"id":127,"by":128,"text":129},"21216674","chirau","How is it the cheapest if it is not free? There are plenty of free scrapping tools out there",{"id":131,"by":132,"text":133},"21216838","impatientduck","Nonsense - beautiful soup is free. Don't pay for something you can program yourself.",{"value":135,"source":53,"evidence":136},"Not automated enough for large-scale scraping",[137,141],{"id":138,"by":139,"text":140},"21216850","chrismarlow9","Go watch the demo video. From what I can see it's not automated enough to do tb\u002Fweek. That said I bet you could hook it into scrapy and automate it. Probably will be slow though",{"id":142,"by":143,"text":144},"21216693","gravypod","How is this at all sustainable as a business model? My company is in charge of scraping a few TB\u002Fweek. Would we be allowed to use your service?",{"value":146,"source":53,"evidence":147},"Contact page broken",[148,152],{"id":149,"by":150,"text":151},"21217444","wumms","Error info: contact us at the bottom shows page not found.",{"id":153,"by":154,"text":155},"21220748","xurias","Unfortunate that the contact us page doesn't work. I'm interested, but I wanted to ask if a FF extension is planned, since that's my primary browser.",{"value":157,"source":53,"evidence":158},"Cannot re-scrape a site without starting over",[159],{"id":160,"by":161,"text":162},"21216824","turtlebits","Seems to work well, but it appears that you can't re-scrape a site without starting from scratch, it's a one time deal.",[164,171],{"value":165,"source":53,"evidence":166},"Safari extension support",[167],{"id":168,"by":169,"text":170},"21220391","Eric_WVGG","So is a Safari extension in development?",{"value":172,"source":53,"evidence":173},"Firefox store distribution",[174],{"id":96,"by":11,"text":97},[176,183],{"value":177,"source":53,"evidence":178},"How updates work for extensions distributed via Gumroad",[179],{"id":180,"by":181,"text":182},"21217520","atarian","Since the extensions are being distributed via Gumroad, how would updates work?",{"value":184,"source":53,"evidence":185},"Whether it works on sites like Zillow, Facebook, LinkedIn",[186,190],{"id":187,"by":188,"text":189},"21216623","mehhh","I wonder how Spider Pro does with Facebook, Linkedin, Whitepages and others that try their best to block scraping but still have an introductory free to view webpage...",{"id":191,"by":192,"text":193},"21215820","uptown","Interesting that their demo screen cast shows Zillow. Zillow is fairly aggressive in applying anti-scraping defensive measures.",[195,202,206,213],{"value":196,"source":53,"evidence":197},"webscraper.io",[198],{"id":199,"by":200,"text":201},"21218468","ikeboy","webscraper.io is free and has more functionality. I've used it for a quick and easy way to scrape data off multiple pages. Probably has a slightly higher learning curve but once…",{"value":203,"source":53,"evidence":204},"Beautiful Soup",[205],{"id":131,"by":132,"text":133},{"value":207,"source":53,"evidence":208},"Kimono Labs",[209],{"id":210,"by":211,"text":212},"21216661","bvm","Ah it reminds me of Kimono Labs. I miss that product, it was fantastic.",{"value":214,"source":53,"evidence":215},"Diffbot",[216],{"id":217,"by":218,"text":219},"21217518","basch","https:\u002F\u002Fwww.diffbot.com\u002F is still well regarded, no?",{"comments":43,"answered":39,"key":221},[222],{"id":96,"text":97},{"commenters":224,"max_depth":225,"maker_replies":43,"first_comment_mins":226,"lifespan_hours":227},82,8,24,44,[229,233,237,241,243,246,247,248,251,254,257,260,263,265,268,271,274,277,280,283,286,290,293,295,297,300,302,305,308,311,313,317,320,323,326,328,331,334,337,340,342,344,348,351,354,356,358,361,364,366,369,372,375,376,378,381,383,385,388,391,392,395,399,401,403,404,406,409,414,417,418,420,422,423,425,427,430,433,437,440,441,443,446,449,454,456,460,464,466,470,472,475,478,480,482,484,488,491,493,495,499,502,505],[230,39,48,231,232,39],"21219931","maker\u002Fpositive\u002Fidea","seeekr",[234,39,43,235,236,39],"21220228","project\u002Fcritical\u002Fidea","meritt",[238,39,39,239,240,39],"21221437","commenter\u002Fneutral\u002Fnone","Mathnerd314",[191,43,18,242,192,39],"project\u002Fneutral\u002Fnone",[244,43,39,239,245,39],"21215855","mahesh_rm",[187,43,18,242,188,39],[60,43,18,242,61,39],[249,43,39,250,192,39],"21216624","other\u002Fneutral\u002Fnone",[252,43,39,250,253,39],"21217095","invisiblerobot",[255,43,39,250,256,39],"21217148","walrus01",[258,43,39,250,259,39],"21217370","core-questions",[261,43,39,250,262,39],"21217556","luckylion",[264,43,39,250,256,39],"21217781",[266,43,39,250,267,39],"21219901","ozi",[269,43,39,250,270,39],"21217162","samstave",[272,43,39,250,273,39],"21217363","not_a_cop75",[275,43,39,250,276,39],"21217429","hk__2",[278,43,39,250,279,39],"21218052","antsar",[281,43,39,250,282,39],"21221018","mycall",[284,43,18,242,285,39],"21217215","bdcravens",[287,43,18,288,289,39],"21217877","project\u002Fneutral\u002Fdocs","hbcondo714",[291,43,39,250,292,39],"21218022","dang",[294,43,39,250,256,39],"21218074",[296,43,39,250,236,39],"21218372",[298,43,39,250,299,39],"21218503","chucksmash",[301,43,39,250,292,39],"21227329",[303,43,39,250,304,39],"21218984","nostrademons",[306,43,39,250,307,39],"21220153","tmb",[309,43,39,250,310,39],"21219117","sebst",[312,43,39,250,256,39],"21221473",[314,33,48,315,316,39],"21215891","maker\u002Fpositive\u002Fnone","johnwheeler",[318,33,39,239,319,39],"21216204","shujito",[321,33,48,315,322,39],"21216296","meddlepal",[324,33,39,239,325,39],"21216551","blunte",[327,33,39,250,270,39],"21217100",[329,33,39,250,330,39],"21220164","bob1029",[332,33,39,239,333,39],"21220457","rajacombinator",[335,33,39,239,336,39],"21220692","mariomariomario",[338,33,39,239,339,39],"21220817","madelyn",[341,33,39,239,330,39],"21221140",[343,33,39,239,333,39],"21221323",[345,33,18,346,347,39],"21220827","project\u002Fneutral\u002Fexecution","Open-Sourcery",[349,33,39,239,350,39],"21221049","mantap",[352,33,39,239,353,39],"21217522","ergothus",[210,18,18,355,211,39],"project\u002Fneutral\u002Fidea",[217,18,39,357,218,39],"commenter\u002Fneutral\u002Fidea",[359,18,39,239,360,39],"21217645","soared",[362,18,39,250,363,39],"21220224","MattyMc",[365,18,18,355,289,39],"21217903",[367,18,18,355,368,39],"21217960","amitport",[370,18,18,355,371,39],"21218437","tigerBL00D",[373,18,39,250,374,39],"21218778","btbuildem",[199,41,43,235,200,39],[84,48,18,377,85,39],"maker\u002Fneutral\u002Fpricing",[153,379,18,380,154,39],6,"project\u002Fmixed\u002Fexecution",[96,379,39,382,11,43],"maker\u002Fneutral\u002Fexecution",[384,379,48,315,154,39],"21222545",[88,386,48,387,89,39],7,"project\u002Fpositive\u002Fdesign",[389,386,18,242,390,39],"21216830","mmcwilliams",[116,386,39,239,57,39],[393,386,18,394,390,39],"21218290","project\u002Fneutral\u002Fprivacy",[396,386,39,397,398,39],"21220025","commenter\u002Fneutral\u002Fprivacy","SquareWheel",[400,386,39,397,390,39],"21220483",[402,386,39,397,57,39],"21221494",[168,386,18,242,169,39],[160,225,43,405,161,39],"project\u002Fcritical\u002Fexecution",[407,225,43,405,408,39],"21216947","tmikaeld",[410,411,43,412,413,39],"21232195",9,"project\u002Fcritical\u002Fpricing","Sephr",[415,31,18,355,416,39],"21219028","riantogo",[108,34,48,387,109,39],[123,419,43,412,124,39],12,[72,419,18,421,73,39],"project\u002Fneutral\u002Fpricing",[142,419,18,421,143,39],[424,419,18,346,73,39],"21216845",[138,419,43,426,139,39],"project\u002Fcritical\u002Fperformance",[428,419,18,355,429,39],"21216853","evandena",[431,419,18,355,432,39],"21216861","thinkloop",[434,419,18,435,436,39],"21216879","project\u002Fmixed\u002Fidea","elldoubleyew",[438,419,18,355,439,39],"21216887","primitivesuave",[76,419,18,421,77,39],[149,442,43,405,150,39],13,[444,442,43,405,445,39],"21217852","esnard",[447,442,39,357,448,39],"21218266","y4mi",[450,451,48,452,453,39],"21216175",14,"project\u002Fpositive\u002Fidea","ct520",[180,455,18,346,181,39],15,[457,458,39,250,459,39],"21222035",16,"s3nnyy",[461,462,48,387,463,39],"21217830",17,"neovive",[127,465,43,412,128,39],18,[467,465,39,468,469,39],"21217097","commenter\u002Fcritical\u002Fpricing","bluetidepro",[471,465,18,355,408,39],"21217182",[473,465,39,239,474,39],"21217626","CobrastanJorji",[476,465,39,239,477,39],"21217906","bauerd",[479,465,39,239,408,39],"21218537",[481,465,39,250,292,39],"21217988",[131,483,43,412,132,39],19,[485,483,39,486,487,39],"21216885","commenter\u002Fneutral\u002Fpricing","Cyberdog",[489,483,39,239,490,39],"21216997","sillydinosaur",[492,483,39,239,487,39],"21217045",[494,483,39,239,57,39],"21217582",[496,483,48,497,498,39],"21218469","project\u002Fpositive\u002Fperformance","slig",[500,483,18,421,501,39],"21217026","nirvdrum",[503,483,39,239,504,39],"21217284","JustFinishedBSG",[56,483,48,506,57,39],"project\u002Fpositive\u002Fpricing",[],[509,514,518,522,526,531],{"id":510,"name":511,"points":512,"category":513},"21231208","Bel",1288,"languages",{"id":515,"name":516,"points":517,"category":20},"21270424","Pipedream",245,{"id":519,"name":520,"points":521,"category":513},"21234891","dial",190,{"id":523,"name":524,"points":525,"category":20},"21268324","reloading",187,{"id":527,"name":528,"points":529,"category":530},"21253850","Alt-tab-macos",156,"productivity",{"id":532,"name":533,"points":534,"category":535},"21239930","Slouch Stoppah",153,"hardware",false,"v2",{"id":539,"name":5,"line":6,"ai":536,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":540,"top":18,"kw":541,"maker":11,"domain":10,"tier":19,"attempts":39,"strip":542,"url":9,"mix":543,"praise":104,"objection":120,"sub":21},21215484,-0.03,[24,25,26,27],"3330000330005000330313530113323155101055",[31,32,33,34],1794036581771]