CloudScrape
Data
web scraping platform
SHOW HN2015
How HN took it
3 positive, 12 neutral or asking, 3 mixed, 12 critical, out of 30 comments about the project. Thread score -0.31.
Privacy: 0 for, 4 againstPricing: 1 for, 1 againstPerformance: 0 for, 1 againstDocs: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Tooling and interface are intuitive and feature rich even on free tier
From 3 comments
Nice timing, as just this week I had my first exposure to the Cloudscrape platform. Credit where credit is due, I found the platform refreshingly feature rich even at lower tier…
zer0defex on HNThis can become prohibitively expensive even for a small web scraping job that you're required to do everyday. Say scraping a news site for articles. The tooling and intuitivity…
kennethologist on HNLooks good. I've used WebScraper.io (running in Chrome on AWS EC2 instances) before for many projects. It's quite poerwful and free. http://webscraper.io
BinaryBird on HN
Objections
Ethical concerns about automatic IP rotation and CAPTCHA solving enabling abuse
From 3 comments
Probably a super dumb question, but isn't this fairly unethical? The "automatic IP rotation" feature isn't there for no reason.
danielmiessler on HN- Auto-resolve CAPTCHA’s - Automatic IP rotation That's just wrong. Is there a website where we can blacklist IP addresses of such violators ?
chdir on HNI hope you guys have a good abuse team to handle the incoming reports.
xxdesmus on HN
Pricing seen as expensive for small recurring scraping jobs
From 2 comments
This can become prohibitively expensive even for a small web scraping job that you're required to do everyday. Say scraping a news site for articles. The tooling and intuitivity…
kennethologist on HNI found it really really expensive regarding to what it's offering. May be I'm missing the point here.
Xspirits on HN
Hard to start using the product and no way to delete an account
From 1 comment
Signed up and then was forwarded to a support-request page, is there a how-to guide to start experiencing it with the free account? I then wanted to back out, of course there is…
ausjke on HN
Requests
A cheaper version without CAPTCHA solving
From 1 comment
This can become prohibitively expensive even for a small web scraping job that you're required to do everyday. Say scraping a news site for articles. The tooling and intuitivity…
kennethologist on HN
Clearer differentiation from other platforms and focus on niche use cases
From 1 comment
Nice timing, as just this week I had my first exposure to the Cloudscrape platform. Credit where credit is due, I found the platform refreshingly feature rich even at lower tier…
zer0defex on HN
Questions
Does it respect robots.txt
From 2 comments
I'm wondering about use cases for this (?) Also, do they respect robots.txt? And how are they going to avoid being blocked by websites that don't want to be scraped? (I guess it…
amelius on HNIf you disobey robots.txt it's unethical and rude.
chinathrow on HN
Are there open source alternatives
From 1 comment
Are there open source alternatives to services like this and Diffbot? Boilerpipe and similar libraries are ok but targeted at article extraction.
lsh on HN
Alternatives named
PhantomJS Cloud
From 1 comment
This can become prohibitively expensive even for a small web scraping job that you're required to do everyday. Say scraping a news site for articles. The tooling and intuitivity…
kennethologist on HN
import.io
From 1 comment
Hmm, another cool web scraper is import.io I use to think web scraping was limited to obscure proprietary companies like connotate so I'm glad to see more of these tools becoming…
mojaam on HN
Diffbot
From 2 comments
Diffbot offers a series of awesome automatic APIs for data extraction -- no setting up manual rules, just provide a URL you want to extract data from and they'll visually process…
juanescobarcom on HNAre there open source alternatives to services like this and Diffbot? Boilerpipe and similar libraries are ok but targeted at article extraction.
lsh on HN
WebScraper.io
From 1 comment
Looks good. I've used WebScraper.io (running in Chrome on AWS EC2 instances) before for many projects. It's quite poerwful and free. http://webscraper.io
BinaryBird on HN