[{"data":1,"prerenderedAt":470},["ShallowReactive",2],{"project-21417965":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":45,"post_text":46,"what":47,"said":85,"maker_in_thread":240,"shape":246,"grid":251,"attempts":433,"same_week":434,"maker_shows":294,"maker_hidden":462,"reader_version":463,"card":464},"21417965","Simplescraper","Turn any website into an API (for those who miss Kimono)","title","Show HN: Turn any website into an API (for those who miss Kimono)","https:\u002F\u002Fsimplescraper.io\u002F","simplescraper.io","welanes",1572610778,2019,332,74,86,2195,4,"legend","dev_tools","web scraping","web_app",[21,24,25,26,27],"api","kimono alternative","data extraction","json csv","curious",-0.215,{"positive":31,"neutral":32,"mixed":33,"critical":34},3,21,1,12,37,{"idea":37,"execution":39,"pricing":40,"docs":42,"privacy":43,"design":44},{"positive":33,"critical":38},5,{"positive":33,"critical":18},{"positive":41,"critical":41},0,{"positive":41,"critical":41},{"positive":41,"critical":33},{"positive":41,"critical":41},[31,41,33,41,41,31,31,38,31,33,38,31,33,41,41,33,31,41,33,31,41,31,41,33,41,31,41,33,33,31,31,31,33,38,31,41,31,31,31,33],null,{"problem":48,"audience":55,"pricing":64,"motivation":70,"time_to_build":46,"stack":76,"openness":83,"stage":84,"team_size":83},{"value":49,"source":50,"evidence":51},"Getting data from websites without writing a scraper.","maker",[52],{"id":53,"by":11,"text":54},"21418374","Hey HN, I posted this in a comment thread the other day and (to my surprise) it got a positive reception so added a few more updates and decided to post it proper. The idea is to…",{"value":56,"source":57,"evidence":58},"People who need scraped web data as JSON, CSV or an API without coding.","inferred",[59,60],{"id":53,"by":11,"text":54},{"id":61,"by":62,"text":63},"21418246","save_ferris","What is it about this service as a business model that prevents it from taking off? I’ve known at least two YC startups that tried to build businesses around this idea. I think…",{"value":65,"source":50,"evidence":66},"Free for now; planned starting price about $25 for 4000 scraping credits, 200k API calls and storage.",[67],{"id":68,"by":11,"text":69},"21426138","Right now it's free and will be until it's stable. Starting price will be about $25 for 4000 scraping credits, 200k API calls and data storage. This will likely change as I have…",{"value":71,"source":50,"evidence":72},"Bring back the Kimono workflow after Kimono shut down.",[73,75],{"id":74},"post",{"id":53,"by":11,"text":54},[77],{"value":78,"source":50,"evidence":79},"Puppeteer",[80],{"id":81,"by":11,"text":82},"21418690","Yeah, that's offered although it's currently free. No particular tricks to avoid detection. It's Puppeteer under the hood with a few customizations which works well on the…","unknown","beta",{"praise":86,"objections":105,"requests":146,"questions":183,"alternatives":205},[87,98],{"value":88,"source":89,"evidence":90},"Kimono-style point-and-click UI with checkmarks feels effortless.","commenter",[91,95],{"id":92,"by":93,"text":94},"21419554","phsource","This is very cool! I love how you brought back the original Kimono UI with the checkmark and Xs for adding and removing data tags. We built WrapAPI (https:\u002F\u002Fwrapapi.com) back in…",{"id":96,"by":11,"text":97},"21421615","Thanks! Yeah the checkmark confirmation just feels effortless. Haven't got it perfected yet, but soon. Really appreciate the insights. You're right that much depends on mapping…",{"value":99,"source":89,"evidence":100},"Fixed the coinmarketcap demo video quickly.",[101],{"id":102,"by":103,"text":104},"21419348","uberswe","Nice, that looks much better!",[106,114,125,136],{"value":107,"source":89,"evidence":108},"Generic scraping tools are hard to monetize; developers can roll their own.",[109,110],{"id":92,"by":93,"text":94},{"id":111,"by":112,"text":113},"21418783","omarhaneef","I would guess: 1. Narrow target Your market is people who need scraped data to input into some kind of app\u002Fprogram\u002Fcode, but don't have the resources\u002Fskills\u002Ftime to use scrapy or…",{"value":115,"source":89,"evidence":116},"Scraping sites may violate terms of service, e.g. coinmarketcap.",[117,121],{"id":118,"by":119,"text":120},"21421047","treve","Also interesting that this main example is also a violation of coinmarketcap's terms. They have a paid API.",{"id":122,"by":123,"text":124},"21424841","sh87","I think so too. From their terms [1] > You agree that you will not: > Copy, modify or create derivative works of the Service or any Content; > Copy, manipulate or aggregate any…",{"value":126,"source":89,"evidence":127},"Selection breaks on pages without classes, and the 'turn into API' name oversells it.",[128,132],{"id":129,"by":130,"text":131},"21426832","MildlySerious","I just gave this a shot on the ISO website to get a list of country codes[1], but it seems the selection algorithm breaks down when there's no specific classes applied to…",{"id":133,"by":134,"text":135},"21424125","beagle3","I don't feel it is right to describe it as \"turns a website into an API\", rather \"gives scraped data through an API\". \"Turn website into an API\", for me, evokes the image that I…",{"value":137,"source":89,"evidence":138},"Authenticated pages not supported in cloud recipes.",[139,143],{"id":140,"by":141,"text":142},"21424422","darkstar999","It doesn't look like it. I got an error trying to scrape my HN upvotes url.",{"id":144,"by":11,"text":145},"21424333","Yes - you're able to save data behind a login using the point and click functionality as it extracts whatever data is loaded in your browser (\"local scraping\"). And no - if you…",[147,162,169,176],{"value":148,"source":89,"evidence":149},"Firefox version.",[150,154,158],{"id":151,"by":152,"text":153},"21419522","nannal",">would appreciate any feedback Any option for a firefox build?",{"id":155,"by":156,"text":157},"21418713","ainiriand","Hi, is it possible to make it compatible with firefox?",{"id":159,"by":160,"text":161},"21456392","cfan01","Firefox add in please.",{"value":163,"source":89,"evidence":164},"Click scripting and load-more button support.",[165],{"id":166,"by":167,"text":168},"21425318","maroonblazer","I like this. Please consider adding the ability to script clicks on elements, e.g. buttons. I manage a site where we load a subset of articles on initial page load and then have…",{"value":170,"source":89,"evidence":171},"RSS feed output.",[172],{"id":173,"by":174,"text":175},"21419800","earth2mars","if you can add RSS feed response that would be great",{"value":177,"source":89,"evidence":178},"Integration with n8n workflows.",[179],{"id":180,"by":181,"text":182},"21423368","monkeydust","Looks good, could this be integrated into n8n.io to be used to drive a workflow?",[184,191,198],{"value":185,"source":89,"evidence":186},"Why does one page scrape cost 2 credits?",[187],{"id":188,"by":189,"text":190},"21419551","giarc","Why is one page scrape 2 credits? Why not just 1?",{"value":192,"source":89,"evidence":193},"Does it work with authenticated pages?",[194],{"id":195,"by":196,"text":197},"21423513","holeyness","Does this work with authenticated pages?",{"value":199,"source":89,"evidence":200},"How does it get around anti-scraping tech?",[201],{"id":202,"by":203,"text":204},"21418475","bko","When I saw this service last week, I think you had a section about a paid service where you do the scraping on a server and send the results. Do you offer that? How do you get…",[206,214,220,227,233],{"value":207,"source":89,"evidence":208},"Kimono",[209,210],{"id":92,"by":93,"text":94},{"id":211,"by":212,"text":213},"21419366","ntaylor","Kimono was cool, nice to see another option. I still have a Kimono t-shirt in a drawer somewhere.",{"value":78,"source":89,"evidence":215},[216],{"id":217,"by":218,"text":219},"21420953","nightnight","OT: or just use puppeteer, not really hard, for free and you can rule the world",{"value":221,"source":89,"evidence":222},"Apify",[223],{"id":224,"by":225,"text":226},"21443916","jancurn","Yes, it can! See https:\u002F\u002Fapify.com\u002Fmarketplace Disclaimer: I'm a co-founder of Apify :)",{"value":228,"source":89,"evidence":229},"Plaid",[230],{"id":231,"by":93,"text":232},"21424101","Plaid, Yodlee, and others abstracts away extracting data from various banks and financial services providers, so they're providing a solution built on top of the same data…",{"value":234,"source":89,"evidence":235},"ParseHub",[236],{"id":237,"by":238,"text":239},"21419562","tsergiu","I'm the founder of parsehub. We are doing well and are independently owned. I think there are 3 things that contribute to this: 1. It is very easy to make a prototype that looks…",{"comments":34,"answered":241,"key":242},7,[243,244,245],{"id":96,"text":97},{"id":53,"text":54},{"id":81,"text":82},{"commenters":247,"max_depth":248,"maker_replies":34,"first_comment_mins":249,"lifespan_hours":250},51,6,45,207,[252,254,258,260,264,267,269,271,274,276,278,280,281,285,287,289,293,296,298,301,303,306,308,310,313,315,317,319,320,324,325,328,331,333,334,337,338,341,344,346,349,351,353,354,357,360,362,363,367,369,372,373,378,381,386,389,391,393,395,397,398,399,403,405,409,410,412,415,419,422,424,426,428,431],[92,41,31,253,93,41],"project\u002Fmixed\u002Fidea",[255,41,41,256,257,41],"21421333","commenter\u002Fneutral\u002Fnone","xmly",[96,41,41,259,11,33],"maker\u002Fpositive\u002Fnone",[261,41,33,262,263,41],"21421875","project\u002Fcritical\u002Fexecution","bravura",[265,41,41,256,266,41],"21424011","MetalGuru",[231,41,41,268,93,41],"commenter\u002Fneutral\u002Fidea",[270,41,41,256,266,41],"21424875",[272,41,41,256,273,41],"21428555","detaro",[53,33,41,275,11,33],"project\u002Fneutral\u002Fidea",[202,33,31,277,203,41],"project\u002Fneutral\u002Fnone",[81,33,41,279,11,33],"project\u002Fneutral\u002Fperformance",[151,33,31,277,152,41],[282,33,38,283,284,41],"21420849","project\u002Fpositive\u002Fnone","lucasverra",[286,33,41,277,11,33],"21424347",[188,33,31,288,189,41],"project\u002Fneutral\u002Fpricing",[290,33,31,291,292,41],"21429889","project\u002Fneutral\u002Fdocs","rapind",[133,294,33,295,134,41],2,"project\u002Fcritical\u002Fidea",[297,31,33,262,103,41],"21418746",[299,31,41,300,11,33],"21419141","project\u002Fneutral\u002Fexecution",[102,31,38,302,103,41],"project\u002Fpositive\u002Fexecution",[304,31,31,300,305,41],"21419633","m00dy",[307,31,41,300,11,33],"21422053",[118,31,33,309,119,41],"project\u002Fcritical\u002Fnone",[311,31,41,256,312,41],"21423312","chirau",[314,31,41,256,119,41],"21426058",[316,31,41,256,312,41],"21426182",[318,31,41,256,119,41],"21426248",[122,31,33,309,123,41],[321,31,41,322,323,41],"21425703","other\u002Fneutral\u002Fnone","1996",[61,18,31,275,62,41],[326,18,41,322,327,41],"21418520","jlokier",[329,18,41,256,330,41],"21418792","pezo1919",[332,18,41,256,327,41],"21419149",[111,18,33,295,112,41],[335,18,33,295,336,41],"21419196","tixocloud",[237,18,31,275,238,41],[339,18,31,275,340,41],"21420022","swalsh",[342,18,41,256,343,41],"21421097","colinchartier",[345,18,41,256,238,41],"21422730",[347,18,31,275,348,41],"21425475","chiefalchemist",[350,18,41,256,238,41],"21425559",[352,18,41,256,348,41],"21452174",[224,18,33,295,225,41],[355,18,41,322,356,41],"21420950","slowenough",[358,18,41,322,359,41],"21422051","hk__2",[361,18,41,322,356,41],"21428661",[155,38,31,277,156,41],[364,38,31,365,366,41],"21420772","project\u002Fneutral\u002Fprivacy","seniorThrowaway",[368,38,41,277,11,33],"21421678",[370,248,31,275,371,41],"21423316","mikikian",[129,241,33,262,130,41],[374,375,33,376,377,41],"21421398",8,"project\u002Fcritical\u002Fprivacy","nopcode",[379,375,41,322,380,41],"21436845","ilrwbwrkhv",[382,383,31,384,385,41],"21437995",9,"project\u002Fneutral\u002Fdesign","flingo",[387,383,41,256,388,41],"21439266","petr-nagy",[390,383,31,384,385,41],"21496256",[166,392,31,300,167,41],10,[394,392,41,277,11,33],"21426161",[195,396,31,277,196,41],11,[144,396,41,365,11,33],[140,396,33,262,141,41],[400,34,38,401,402,41],"21423487","project\u002Fpositive\u002Fidea","mrskitch",[404,34,41,259,11,33],"21426218",[406,407,31,288,408,41],"21425266",13,"joelvalleroy",[68,407,41,288,11,33],[211,411,41,322,212,41],14,[413,411,41,322,414,41],"21419457","kitd",[416,417,31,291,418,41],"21421554",15,"matz1",[420,417,41,421,11,33],"21421750","maker\u002Fneutral\u002Fdocs",[180,423,31,275,181,41],16,[159,425,31,277,160,41],17,[173,427,31,275,174,41],18,[429,427,41,322,430,41],"21421014","SweeToxin",[217,432,33,295,218,41],19,[],[435,439,444,449,453,457],{"id":436,"name":437,"points":438,"category":20},"21439918","Tailwind Builder",295,{"id":440,"name":441,"points":442,"category":443},"21461701","microvm",145,"infrastructure",{"id":445,"name":446,"points":447,"category":448},"21418882","HR code",124,"languages",{"id":450,"name":451,"points":452,"category":443},"21423612","kubeadm-playbook",122,{"id":454,"name":455,"points":456,"category":448},"21427363","Domino",116,{"id":458,"name":459,"points":460,"category":461},"21462993","multi-object-tracker",105,"ai",false,"v2",{"id":465,"name":5,"line":6,"ai":462,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":466,"top":18,"kw":467,"maker":11,"domain":10,"tier":19,"attempts":41,"strip":468,"url":9,"mix":469,"praise":88,"objection":107,"sub":21},21417965,-0.21,[21,24,25,26,27],"3010033531531001301303010301133315303331",[31,32,33,34],1794036582550]