still-good-food on HackerShow

Learning and science

recipe scraper

SHOW HN2016

BBC Good Food Scraper in Go

Shown by sleepychu on Jun 23, 2016. Click the card to turn it over.

92points

23comments

#414of 2,264 in 2016

Top 19%of 2016 by points

How HN took it

CuriousMostly questions and neutral comments

1 positive, 3 neutral or asking, 0 mixed, 2 critical, out of 6 comments about the project. Thread score -0.06.

Idea: 1 for, 2 against

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

goquery is a great scraping library
From 3 comments
- Goquery is pretty great - used it last semester to write the crawler for a side-project of mine. https://github.com/prakhar1989/bekanjoos
  krat0sprakhar on HN
- It really is a great package and works extremely well in the context of scraping. It's what I opted to use for an API server [1] I wrote that handles the RFC Instant Answer [2]…
  imwally on HN
- goquery is really neat, does a lot of the lifting for you!
  sleepychu on HN
Useful Go scraping starting point
From 1 comment
- Really cool. I have some scraping projects I'd like to do in Go, so this is a great starting point.
  callmeed on HN

Many people have already scraped the same site
From 2 comments
- It is, but haven't a 1000+ people already scraped it? (myself included!)
  Jaruzel on HN
- Me too!
  sambeau on HN
BBC Good Food is not closing, so scraping is unnecessary
From 1 comment
- BBC Good Food is not going anywhere. So you can stop the scraping... It's BBC Food (different name, different website) the one who is going to close (even the recipes will remain…
  ryck on HN

Share how much the AWS bill runs to
From 1 comment
- bekanjoos looks nice! I was thinking today about making a more generic "when this thing on that website changes" notify me sort of thing. It looks like you're running on AWS.…
  sleepychu on HN

Is the tool open source?
From 1 comment
- Have you open-sources this package somewhere? As an amateur web security researcher this project looks really interesting.
  LunaSea on HN

Internet Archive Wayback Machine
From 2 comments
- It seems with all these website scrapers, we have forgotten about the Internet Archive. http://archive.org/web/
  imjacobclark on HN
- As awesome as the Internet Archive's Wayback Machine is, it is still under central control. Worse, as they (reasonably) abide to robots.txt rules, the BBC could easily block…
  anc84 on HN
ArchiveBot
From 1 comment
- And the fantastic ArchiveBot.. http://www.archiveteam.org/index.php?title=ArchiveBot
  johtso on HN