[{"data":1,"prerenderedAt":301},["ShallowReactive",2],{"project-8192287":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":44,"post_text":45,"what":46,"said":88,"maker_in_thread":155,"shape":166,"grid":171,"attempts":263,"same_week":264,"maker_shows":34,"maker_hidden":293,"reader_version":294,"card":295},"8192287","Scraperjs","A versatile web scraper","title","Show HN: Scraperjs – A versatile web scraper","https:\u002F\u002Fgithub.com\u002Fruipgil\u002Fscraperjs","github.com","ruipgil",1408363802,2014,192,36,108,2272,5,"notable","languages","scraping library","library",[24,25,26,27,28],"web scraper","nodejs","phantomjs","cheerio","javascript scraping","loved",0.231,{"positive":32,"neutral":33,"mixed":34,"critical":34},6,8,1,16,{"docs":37,"idea":39,"performance":41,"design":42,"execution":43},{"positive":38,"critical":34},0,{"positive":40,"critical":38},3,{"positive":34,"critical":38},{"positive":34,"critical":38},{"positive":38,"critical":38},[34,38,40,40,40,38,18,38,40,38,38,38,18,38,38,38,38,18,18,38,38,38,38,38,38,38,38,18,38,18,40,40,40,40,38,40],null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":59,"openness":87,"stage":87,"team_size":87},{"value":48,"source":49,"evidence":50},"Extracting data from web pages with JavaScript","inferred",[51,55],{"id":52,"by":53,"text":54},"8193527","findjashua","I think you'd need CasperJS when you need to perform browser actions (login, click on a particular button, fill a form etc). But if you just want to scrape content (eg episode…",{"id":56,"by":57,"text":58},"8193522","jasode","For example, go to http:\u002F\u002Fwww.imdb.com On the right, you'll notice that under the sidebar \"Opening This Week\" is a movie titled \"Love Is Strange\". With that in mind, press Ctrl+U…",[60,72,80],{"value":61,"source":62,"evidence":63},"Node.js","commenter",[64,68],{"id":65,"by":66,"text":67},"8193435","tsenkov","I guess the biggest difference is, that Casper isn't a NodeJS module. Interaction with node (using the archive of npm) becomes hard. I am releasing something similar in a couple…",{"id":69,"by":70,"text":71},"8194057","martin-adams","The latter probably runs the script as though you are within the context of a web page (so full Ajax\u002FJS support). I assume the Simple version might be completely written in…",{"value":73,"source":62,"evidence":74},"PhantomJS",[75,79],{"id":76,"by":77,"text":78},"8192528","thibauts","As far as I can see it has not much in common with casperjs, apart from the fact that it can use phantomjs.",{"id":69,"by":70,"text":71},{"value":81,"source":62,"evidence":82},"Cheerio",[83],{"id":84,"by":85,"text":86},"8192720","jdc0589","For instances where you don't need a full featured browser to get the data you need, Scraperjs using the Cheerio backend should be WAY faster than casper\u002Fphantom. I've not used…","unknown",{"praise":89,"objections":101,"requests":116,"questions":123,"alternatives":135},[90,97],{"value":91,"source":62,"evidence":92},"Router aspect is a nice idea",[93],{"id":94,"by":95,"text":96},"8192604","bshimmin","I really like the router aspect of this. That's a nice idea and not (to the best of my limited memory) one I can recall seeing in any other scraper.",{"value":98,"source":62,"evidence":99},"Cheerio backend is much faster than PhantomJS for static pages",[100],{"id":84,"by":85,"text":86},[102,109],{"value":103,"source":62,"evidence":104},"Install instructions unclear",[105],{"id":106,"by":107,"text":108},"8193497","brianzelip","It's unclear to me how to actually run this. Only executing the two commands listed under the Installing section does not run it - I had to `cd` into the scraperjs dir, then `npm…",{"value":110,"source":62,"evidence":111},"Promises usage disliked",[112],{"id":113,"by":114,"text":115},"8193289","woah","Looks pretty good, shame about the promises.",[117],{"value":118,"source":62,"evidence":119},"Comparison with CasperJS in docs",[120],{"id":121,"by":57,"text":122},"8192519","It would be helpful if the documentation compared how Scraperjs is different from, or better than, CasperJS for scraping. CasperJS is the older and more well-known wrapper around…",[124,128],{"value":125,"source":62,"evidence":126},"How to use it in a project and whether to install per project",[127],{"id":106,"by":107,"text":108},{"value":129,"source":62,"evidence":130},"Use cases for DynamicScraper",[131],{"id":132,"by":133,"text":134},"8193458","mr5iff","I don't quite get the point of the DynamicScraper... Any real use cases for that?",[136,141,148],{"value":137,"source":62,"evidence":138},"CasperJS",[139,140],{"id":121,"by":57,"text":122},{"id":76,"by":77,"text":78},{"value":142,"source":62,"evidence":143},"Scrapy",[144],{"id":145,"by":146,"text":147},"8193602","cridenour","I think Scrapy is a better Python scraping tool.",{"value":149,"source":62,"evidence":150},"Mechanize",[151],{"id":152,"by":153,"text":154},"8192882","wc-","Check out Mechanize: https:\u002F\u002Fgithub.com\u002Fsparklemotion\u002Fmechanize I haven't used the ruby version but I am pretty happy with the python port of it. It's lighter and faster than…",{"comments":40,"answered":40,"key":156},[157,160,163],{"id":158,"text":159},"8193829","Scraperjs is supposed to be used as npm package. So, if you do \"npm install \u003Cpackage-name>\", you download the latest version of package to the same folder as the closest…",{"id":161,"text":162},"8192904","For web scraping purposes ScraperJS and ScraperJS would probably use the same lines of code, however ScraperJS has most of the tools you need for web scraping, something that…",{"id":164,"text":165},"8193855","On the first question, yes, it's possible and easily done with the Router.",{"commenters":167,"max_depth":168,"maker_replies":40,"first_comment_mins":169,"lifespan_hours":170},27,4,85,22,[172,174,176,178,180,182,184,186,188,189,194,196,199,203,206,209,212,213,216,219,222,225,228,231,234,235,237,240,243,247,250,252,253,255,256,260],[106,38,34,173,107,38],"project\u002Fcritical\u002Fdocs",[158,38,38,175,11,34],"maker\u002Fneutral\u002Fdocs",[121,34,40,177,57,38],"project\u002Fneutral\u002Fdocs",[76,34,40,179,77,38],"project\u002Fneutral\u002Fidea",[181,34,40,179,57,38],"8192540",[161,34,38,183,11,34],"maker\u002Fpositive\u002Fidea",[84,34,18,185,85,38],"project\u002Fpositive\u002Fperformance",[65,34,38,187,66,38],"commenter\u002Fneutral\u002Fidea",[52,34,40,179,53,38],[190,191,38,192,193,38],"8193597",2,"other\u002Fneutral\u002Fnone","halcyondaze",[145,191,38,195,146,38],"other\u002Fcritical\u002Fidea",[197,191,38,192,198,38],"8195189","baldfat",[200,40,18,201,202,38],"8193259","project\u002Fpositive\u002Fidea","justboxing",[204,40,38,192,205,38],"8193330","cwbrandsma",[207,40,38,192,208,38],"8193376","misterbwong",[210,40,38,211,202,38],"8194007","commenter\u002Fpositive\u002Fnone",[164,40,38,183,11,34],[214,40,18,215,202,38],"8194002","maker\u002Fpositive\u002Fnone",[217,40,18,201,218,38],"8195630","guiomie",[220,168,38,192,221,38],"8192560","jdrock",[223,18,38,192,224,38],"8192610","andrejewski",[226,32,38,192,227,38],"8192823","pibefision",[229,32,38,192,230,38],"8192870","jwarren",[232,32,38,192,233,38],"8192872","joeyspn",[152,32,38,192,153,38],[236,32,38,192,53,38],"8193554",[238,32,38,192,239,38],"8193993","riffraff",[241,242,18,201,230,38],"8192868",7,[244,33,38,245,246,38],"8196679","other\u002Fpositive\u002Fnone","roux_rc",[94,248,18,249,95,38],9,"project\u002Fpositive\u002Fdesign",[132,251,40,179,133,38],10,[56,251,40,179,57,38],[254,251,40,179,239,38],"8193959",[69,251,40,179,70,38],[257,258,38,192,259,38],"8194428",11,"novaleaf",[113,261,40,262,114,38],12,"project\u002Fmixed\u002Fexecution",[],[265,270,275,279,284,288],{"id":266,"name":267,"points":268,"category":269},"8187791","GIFyoutube",577,"creative",{"id":271,"name":272,"points":273,"category":274},"8178660","Tuff.gb",260,"games",{"id":276,"name":277,"points":278,"category":20},"8192801","Satellizer",221,{"id":280,"name":281,"points":282,"category":283},"8178940","Ad Detector",204,"security_privacy",{"id":285,"name":286,"points":287,"category":20},"8176672","Way.js",162,{"id":289,"name":290,"points":291,"category":292},"8183014","Practical Guide to Mobile Apps",131,"learning_science",false,"v2",{"id":296,"name":5,"line":6,"ai":293,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":297,"top":18,"kw":298,"maker":11,"domain":10,"tier":19,"attempts":38,"strip":299,"url":9,"mix":300,"praise":91,"objection":103,"sub":21},8192287,0.23,[24,25,26,27,28],"103330503000500005500000000505333303",[32,33,34,34],1794036521540]