Page.REST
Web and SaaS
web scraping api
SHOW HN2017
How HN took it
2 positive, 10 neutral or asking, 0 mixed, 5 critical, out of 17 comments about the project. Thread score -0.24.
Idea: 0 for, 2 againstPricing: 0 for, 2 againstExecution: 0 for, 1 againstPerformance: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Cheap free alternative and useful idea
From 2 comments
For absolutely 0-cost you can use http://proc.link which will return oembed info for ANY url. General example: http://api.proc.link/oembed?url=http%3A%2F%2Fpage.rest Youtube:…
xytop on HNI did something similar a loooong time ago. Granted not as sexy. https://github.com/keyle/json-anything
keyle on HN
Objections
Users must know element selectors, reducing value over an HTML parser
From 2 comments
If I have to know the elements' selectors, why should I prefer this service over using a HTML parser?
ganessh on HNYeah, It kind of defeats the point/value prop of an API.
AznHisoka on HN
Abuse risk without clear API limits
From 1 comment
Someone could really abuse this service, I don't see any mention of API limits.
cinooo on HN
$5 price and payment flow seem off-putting
From 2 comments
I would make the 5$ price and token validity much larger, like "4rem" or something, I was looking at the CC input field and thinking "seriously? how much will you charge?"
kowdermeister on HNLooks interesting. I wonder what kind of market this app might serve. For larger apps, I would worry about support. 5 dollars per year tells me that the developer is doing this…
geetfun on HN
Requests
Support for scraping sites with anti-bot protection and high request rates
From 1 comment
How do you handle sites that have scraper prevention? Such as captcha, IP throttling, etc. I'd pay $100-$500 per-month for a service that could reliably scrape some particularly…
throwaway12s34 on HN
Support for other response formats such as protobuf
From 1 comment
Consider offering responses other formats, possibly protobuf. Then you can sell it as a solution for limited IoT devices where a full HTML parser is too much. If your service…
icebraining on HN
Await for JS frameworks to render before scraping
From 1 comment
Is there a await for JS Frontends to initialise before the scrape via a selector (i.e. Angular/Ember etc)?
andrevv_6 on HN
Questions
How the server cost is covered and whether it is reliable
From 2 comments
Very neat. But the question is, how they pay the cost of the server? I mean, do you think a web service without any API keys, subscriptions, etc. is reliable?
afshinmeh on HNWhat levels of traffic does your server see on a normal day? has it ever gone down from a spike?
have_faith on HN
Whether it handles scraper prevention like captchas
From 1 comment
How do you handle sites that have scraper prevention? Such as captcha, IP throttling, etc. I'd pay $100-$500 per-month for a service that could reliably scrape some particularly…
throwaway12s34 on HN
Alternatives named
proc.link oembed API
From 1 comment
For absolutely 0-cost you can use http://proc.link which will return oembed info for ANY url. General example: http://api.proc.link/oembed?url=http%3A%2F%2Fpage.rest Youtube:…
xytop on HN
Yahoo YQL
From 1 comment
Not unlike YQL - https://developer.yahoo.com/yql/
squiggy22 on HN
wrapapi.com
From 1 comment
Uhm this is interesting. Reminds me of: https://wrapapi.com
Luyanda on HN
embed.ly
From 1 comment
Idea was to make a profitable service out of this. But I'm bad interpreneur :) I wanted to make an analog of http://embed.ly/code but lost interest in the middle. I can code…
xytop on HN
link.fish
From 1 comment
If you are interested the same is possible with https://link.fish/hn-early-access/ which has a Zapier integration. It allows to send an URL and it then returns its data. However,…
janober on HN