[{"data":1,"prerenderedAt":261},["ShallowReactive",2],{"project-18714461":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":40,"post_text":43,"what":44,"said":70,"maker_in_thread":158,"shape":166,"grid":170,"attempts":223,"same_week":224,"maker_shows":159,"maker_hidden":253,"reader_version":254,"card":255},"18714461","CoolQLCool","Turn Websites into GraphQL Accessible APIs","title","Show HN: CoolQLCool – Turn Websites into GraphQL Accessible APIs","https:\u002F\u002Fcoolql.cool","coolql.cool","gavino",1545212533,2018,124,22,342,2155,16,"notable","dev_tools","web scraping api","web_app",[24,25,26,27],"graphql","web scraping","html to api","cheerio","loved",0.276,{"positive":31,"neutral":32,"mixed":33,"critical":33},4,8,0,12,{"idea":36,"docs":38,"execution":39},{"positive":37,"critical":33},1,{"positive":33,"critical":33},{"positive":37,"critical":33},[33,33,33,41,41,33,33,41,42,42,33,41,33,41,41,33,33,41,41,33,42,42],3,5,null,{"problem":45,"audience":43,"pricing":43,"motivation":43,"time_to_build":43,"stack":53,"openness":69,"stage":69,"team_size":69},{"value":46,"source":47,"evidence":48},"Websites return HTML without an API, making data hard to consume","commenter",[49],{"id":50,"by":51,"text":52},"18715881","ralusek","1.) You have a website with data you'd like to consume. 2.) That website doesn't expose an api, but returns statically rendered html. 3.) You don't like parsing statically…",[54,61,65],{"value":55,"source":56,"evidence":57},"NextJS","maker",[58],{"id":59,"by":11,"text":60},"18718185","Sure! The backend is actually pretty straight forward, it's a NextJS app deployed on Now with a few added endpoints to handle the incoming GraphQL queries. Then for actually…",{"value":62,"source":56,"evidence":63},"Now deployment",[64],{"id":59,"by":11,"text":60},{"value":66,"source":56,"evidence":67},"GraphQL schema builder",[68],{"id":59,"by":11,"text":60},"unknown",{"praise":71,"objections":94,"requests":109,"questions":124,"alternatives":136},[72,87],{"value":73,"source":47,"evidence":74},"Great name and project",[75,79,83],{"id":76,"by":77,"text":78},"18715187","VMG","Awesome name, awesome project!",{"id":80,"by":81,"text":82},"18718057","conceptpad","Great project! I can imagine this may greatly improve web certain classes of scraping. @gavino I'm curious what tooling and architecture you used to put this together?",{"id":84,"by":85,"text":86},"18715312","powerslacker","Give this man an internet.",{"value":88,"source":47,"evidence":89},"waitForSelector feature is cool",[90],{"id":91,"by":92,"text":93},"18715343","lachenmayer","I've previously written a very similar project called \"graphql-scraper\" (which is arguably a far less cool name...), you can check it out at…",[95,102],{"value":96,"source":47,"evidence":97},"Unclear how it handles throttling or captchas",[98],{"id":99,"by":100,"text":101},"18717485","nurettin","Not sure what to make of this. How does it handle throttling or captchas?",{"value":103,"source":47,"evidence":104},"Page not loading in Materialistic Android reader",[105],{"id":106,"by":107,"text":108},"18716696","acct1771","Page isn't loading in Materialistic (HN reader on Android\u002FF-Droid repo), but: do you have this exact verbiage on it? It's very concise!",[110,117],{"value":111,"source":47,"evidence":112},"Help selecting table td by nth selector",[113],{"id":114,"by":115,"text":116},"18720701","pdxandi","I've been looking for something like this! I'm trying to play around with it but can't seem to get the selector right. How do I grab a table `td` by its nth selector (tried…",{"value":118,"source":47,"evidence":119},"Swap cheerio for a remote browser",[120],{"id":121,"by":122,"text":123},"18714875","bryanrasmussen","Are you planning to build anything on top of this - service,company? I was thinking it would be a good way to build an api for some projects I've been thinking of working on,…",[125,132],{"value":126,"source":47,"evidence":127},"What are the use cases?",[128],{"id":129,"by":130,"text":131},"18715732","jarjar12","Sorry a dumb question. What are the use cases ? Thx",{"value":133,"source":47,"evidence":134},"How does it handle throttling or captchas?",[135],{"id":99,"by":100,"text":101},[137,141,151],{"value":138,"source":47,"evidence":139},"graphql-scraper",[140],{"id":91,"by":92,"text":93},{"value":142,"source":47,"evidence":143},"GDOM",[144,148],{"id":145,"by":146,"text":147},"18714976","maio","Nice! There is also similar project GDOM - https:\u002F\u002Fgithub.com\u002Fsyrusakbary\u002Fgdom written in Python.",{"id":149,"by":11,"text":150},"18718084","I remember seeing GDOM a while back when I first started this project, but forgot to write it down as a source of inspiration. I'm gonna add all of these as alternatives, because…",{"value":152,"source":47,"evidence":153},"Yahoo Pipes",[154],{"id":155,"by":156,"text":157},"18722280","pnevares","You're probably thinking about Yahoo Pipes: https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FYahoo!_Pipes",{"comments":31,"answered":159,"key":160},2,[161,162,165],{"id":149,"text":150},{"id":163,"text":164},"18718064","Nah, I don't really plan on turning it into a company. I'd gladly accept any PR to swap out cheerio, I haven't touched that part in close to a year :D",{"id":59,"text":60},{"commenters":167,"max_depth":41,"maker_replies":31,"first_comment_mins":168,"lifespan_hours":169},19,91,33,[171,173,175,179,181,182,184,188,190,192,195,197,201,203,204,208,210,213,215,216,217,222],[91,33,33,172,92,33],"commenter\u002Fneutral\u002Fidea",[149,33,33,174,11,37],"maker\u002Fneutral\u002Fidea",[176,33,33,177,178,33],"18719101","commenter\u002Fpositive\u002Fnone","syrusakbary",[145,37,41,180,146,33],"project\u002Fneutral\u002Fidea",[121,159,41,174,122,33],[163,159,33,183,11,37],"maker\u002Fneutral\u002Fnone",[185,41,33,186,187,33],"18715198","other\u002Fneutral\u002Fnone","canadev",[114,31,41,189,115,33],"project\u002Fneutral\u002Fdocs",[76,42,42,191,77,33],"project\u002Fpositive\u002Fidea",[80,193,42,194,81,33],6,"maker\u002Fpositive\u002Fexecution",[59,193,33,196,11,37],"maker\u002Fneutral\u002Fexecution",[198,199,41,180,200,33],"18717717",7,"simonhamp",[155,199,33,202,156,33],"other\u002Fneutral\u002Fidea",[99,32,41,180,100,33],[205,206,41,180,207,33],"18714843",9,"halfjew22",[209,206,33,183,11,37],"18718067",[211,206,33,186,212,33],"18728007","atomi",[129,214,41,180,130,33],10,[50,214,41,180,51,33],[106,214,33,186,107,33],[218,219,42,220,221,33],"18716050",11,"project\u002Fpositive\u002Fnone","hokumguru",[84,34,42,220,85,33],[],[225,230,234,239,243,248],{"id":226,"name":227,"points":228,"category":229},"18697601","remove.bg",640,"creative",{"id":231,"name":232,"points":233,"category":229},"18701772","Fancy fonts (Beautiful Dingbats)",247,{"id":235,"name":236,"points":237,"category":238},"18673928","Minimal Google Analytics Snippet",171,"web_saas",{"id":240,"name":241,"points":242,"category":238},"18710094","DPAGE",161,{"id":244,"name":245,"points":246,"category":247},"18705661","Goldpinger",158,"infrastructure",{"id":249,"name":250,"points":251,"category":252},"18704436","Bubblin",138,"productivity",false,"v2",{"id":256,"name":5,"line":6,"ai":253,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":257,"top":18,"kw":258,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":259,"url":9,"mix":260,"praise":73,"objection":96,"sub":21},18714461,0.28,[24,25,26,27],"0003300355030330033055",[31,32,33,33],1793990983473]