MechanicalSoup
Dev tools
web automation library
SHOW HN2014
180points
27comments
#122of 2,272 in 2014
Top 6%of 2014 by points
How HN took it
MixedNo clear lean
5 positive, 5 neutral or asking, 2 mixed, 1 critical, out of 13 comments about the project. Thread score +0.25.
Idea: 3 for, 0 againstExecution: 0 for, 1 againstDocs: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Fills a gap left by stagnant Python mechanize
From 3 comments
Mechanize is a core part in quite a few of my projects lately and the fact that it hasn't been modified in over 2 years has been very worrisome. There are lots of edge cases out…
wc- on HNThis is fantastic. I've used python mechanize in some very large projects and it was very frustrating - their lack of documentation and, well, the fact that it's complete…
Deusdies on HNthey're for different use cases. mechanicalsoup is more like zombie.js ( http://zombie.labnotes.org/ ) than selenium, chromedriver, or some other webdriver (…
auvrw on HN
Simple API and lightweight compared to Selenium
From 2 comments
Selenium seems very heavy-weight to me (granted I have only used selenium server). If you don't need to interpret javascript after a page loads then you might be able to use…
wc- on HNI personally find Capybara[0] to be the happy medium for web scraping, if Python isn't a hard requirement. It has a simple API, like MechanicalSoup, but it can also easily be…
lazerwalker on HN
Objections
Does not execute JavaScript
From 2 comments
they're for different use cases. mechanicalsoup is more like zombie.js ( http://zombie.labnotes.org/ ) than selenium, chromedriver, or some other webdriver (…
auvrw on HNDoes it have a javascript engine? Because we had to abandon BeautifulSoup/Mechanize over this a couple years ago and switch to HTMLUnit (Java).
goorpyguy on HN
Very little code so far, unclear roadmap
From 2 comments
There's not a lot to it so far (a single class, three tests). I wonder if the author has a road map for the project.
jdnier on HNHave any documentation on a roadmap for things as they go forward? Would love to send some PR's your way :)
rhgraysonii on HN
Requests
Publish a roadmap and documentation
From 1 comment
Have any documentation on a roadmap for things as they go forward? Would love to send some PR's your way :)
rhgraysonii on HN
Add JavaScript engine support
From 1 comment
Does it have a javascript engine? Because we had to abandon BeautifulSoup/Mechanize over this a couple years ago and switch to HTMLUnit (Java).
goorpyguy on HN
Questions
How does it compare to Scrapy or CasperJS
From 2 comments
How does MechanicalSoup (or RoboBrowse, for that matter, this is the first I've heard of either) compare to Scrapy?: http://scrapy.org/
webmaven on HNWould that be an equivalent to Casper.js ?
volent on HN
Why use it over Selenium
From 1 comment
Sell me MechanicalSoup over Selenium.
diminoten on HN
Alternatives named
RoboBrowser
From 1 comment
An alternative to this is RoboBrowse [1] which is also based on requests + BeautifulSoup4 and seems a lot more mature. [1] http://robobrowser.readthedocs.org/en/latest/readme.html
flexd on HN
Python mechanize
From 1 comment
Mechanize is a core part in quite a few of my projects lately and the fact that it hasn't been modified in over 2 years has been very worrisome. There are lots of edge cases out…
wc- on HN
Selenium
From 1 comment
Sell me MechanicalSoup over Selenium.
diminoten on HN
Capybara
From 1 comment
I personally find Capybara[0] to be the happy medium for web scraping, if Python isn't a hard requirement. It has a simple API, like MechanicalSoup, but it can also easily be…
lazerwalker on HN
Scrapy
From 1 comment
How does MechanicalSoup (or RoboBrowse, for that matter, this is the first I've heard of either) compare to Scrapy?: http://scrapy.org/
webmaven on HN