[{"data":1,"prerenderedAt":413},["ShallowReactive",2],{"project-14272133":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":45,"post_text":48,"what":49,"said":91,"maker_in_thread":209,"shape":221,"grid":224,"attempts":377,"same_week":378,"maker_shows":40,"maker_hidden":405,"reader_version":406,"card":407},"14272133","Pocket Stream Archive","A personal Way-Back Machine","title","Show HN: Pocket Stream Archive – A personal Way-Back Machine","https:\u002F\u002Fgithub.com\u002Fpirate\u002Fpocket-archive-stream","github.com","nikisweeting",1493976169,2017,217,68,140,2242,7,"notable","productivity","web archiving","cli",[24,25,26,27,28],"web archive","pocket export","headless chrome","link rot","wayback machine","curious",0.048,{"positive":32,"neutral":33,"mixed":34,"critical":35},10,26,1,6,43,{"idea":38,"execution":41,"docs":43},{"positive":39,"critical":40},8,2,{"positive":34,"critical":42},4,{"positive":44,"critical":44},0,[46,46,44,46,46,46,34,46,46,46,34,34,46,44,34,34,34,47,46,46,46,47,47,46,46,46,47,46,47,46,44,44,46,44,46,47,44,46,47,47],3,5,null,{"problem":50,"audience":48,"pricing":48,"motivation":61,"time_to_build":48,"stack":68,"openness":89,"stage":90,"team_size":90},{"value":51,"source":52,"evidence":53},"Saved articles and pages disappear or fail to render when offline or after sites go down","maker",[54,57],{"id":55,"by":11,"text":56},"14272902","Slowing down the inevitable tide of https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLink_rot. When I cite blog posts or want to share sites that have gone down, I can swap out the links for my…",{"id":58,"by":59,"text":60},"14272879","jl6","Screenshotting or PDFing of a website is an increasingly important archiving tool, to supplement wget. I've come across a lot of websites that won't render any content if not…",{"value":62,"source":52,"evidence":63},"Slow link rot and keep copies of cited pages; headless Chrome made it practical",[64,65],{"id":55,"by":11,"text":56},{"id":66,"by":11,"text":67},"14272665","You're welcome! I've been wanting to build this for ages but headless chrome finally inspired me to actually finish it.",[69,76,83],{"value":70,"source":71,"evidence":72},"Python","inferred",[73],{"id":74,"by":11,"text":75},"14273850","I included a sample of the expected pocket list format in the repo: https:\u002F\u002Fgithub.com\u002Fpirate\u002Fpocket-archive-stream\u002Fblob\u002Fmaster\u002Fexample_ril_export.html And a comment next to the…",{"value":77,"source":52,"evidence":78},"Headless Chrome",[79,82],{"id":80,"by":11,"text":81},"14272603","59 or later, --headless is a brand new feature. apt-get install google-chrome-canary. https:\u002F\u002Fdevelopers.google.com\u002Fweb\u002Fupdates\u002F2017\u002F04\u002Fheadless-chrome",{"id":66,"by":11,"text":67},{"value":84,"source":52,"evidence":85},"wget",[86],{"id":87,"by":11,"text":88},"14273281","I opted not to download images using wget. I figured if I needed in-article images the PDF+screenshot would be enough.","open_source","unknown",{"praise":92,"objections":120,"requests":139,"questions":164,"alternatives":173},[93,109],{"value":94,"source":95,"evidence":96},"Useful personal archiving tool that fills a real need","commenter",[97,101,105],{"id":98,"by":99,"text":100},"14272445","anotheryou","yes! thank you so much. Needed this badly.",{"id":102,"by":103,"text":104},"14272463","anc84","Now if only Chromium could learn to write WARC archives, then it would be on par! :) Great project!",{"id":106,"by":107,"text":108},"14272797","rcarmo","This is great. All it needs is a Docker container and I'd be running it now (need to take some time aside this weekend to do that).",{"value":110,"source":95,"evidence":111},"Suggestions to support Pinboard and Instapaper input",[112,116],{"id":113,"by":114,"text":115},"14272647","ents","Would be cool to see this for Instapaper or Pinboard",{"id":117,"by":118,"text":119},"14274293","unicornporn","If it would be connected to Pinboard.in as an alternative to Pocket I would be screaming with joy. :-)",[121,132],{"value":122,"source":95,"evidence":123},"PDF output breaks on sites with broken print layouts or fixed-page assumptions",[124,128],{"id":125,"by":126,"text":127},"14273624","driverdan","PDFs are really not suitable for archiving websites since they're designed around pages and the web does not have pages. A better option is to render a page with JS turned on and…",{"id":129,"by":130,"text":131},"14274449","frik","But what do you do when a website has a broken media query, essentially destroying the print layout? Then a PDF is useless. Well, I took a screenshot, better than nothing.",{"value":133,"source":95,"evidence":134},"Images are not downloaded",[135],{"id":136,"by":137,"text":138},"14273270","antman","The demo does not have images. Maybe try wget -nc -np -E -H -k -K -p -U 'Mozilla\u002F5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko\u002F20070802 SeaMonkey\u002F1.1.4' -e robots=off",[140,147,154],{"value":141,"source":95,"evidence":142},"Pinboard support",[143,144],{"id":117,"by":118,"text":119},{"id":145,"by":11,"text":146},"14274530","If you can get me a sample pinboard export to look at, I'll whip up a regex that makes it work.",{"value":148,"source":95,"evidence":149},"WARC output option",[150,151],{"id":102,"by":103,"text":104},{"id":152,"by":11,"text":153},"14272945","I would be very on-board with adding a warc exporting option. I also hate how Chrome tosses all history older than 3 months. Running an archiving proxy hooked up to archive.py…",{"value":155,"source":95,"evidence":156},"Tag column or folders for finding articles",[157,161],{"id":158,"by":159,"text":160},"14272728","djhworld","This is cool. Could you add an option to either add tagging, or separate the tagged items into folders? e.g. \"programming\u002F\", \"docker\u002F\" etc, I often find myself digging through my…",{"id":162,"by":11,"text":163},"14272757","I like having the sites by timestamp because they're guaranteed to be unique, and it makes traversing them easy. I'd be happy to add a tag column to the index though, which you…",[165],{"value":166,"source":95,"evidence":167},"Which Chrome version is needed for headless PDF export",[168,172],{"id":169,"by":170,"text":171},"14272573","avian","What version of Google Chrome do you need for the PDF export to work? I tried it on 58.0.3029.96 (Linux) and this does nothing (no error messages, it just quits without writing…",{"id":80,"by":11,"text":81},[174,181,188,195,202],{"value":175,"source":95,"evidence":176},"Firefox ScrapBook",[177],{"id":178,"by":179,"text":180},"14274555","vijucat","I still use Firefox ScrapBook for this: https:\u002F\u002Faddons.mozilla.org\u002Fen-US\u002Ffirefox\u002Faddon\u002Fscrapbook\u002F Just for articles, mind you, not entire websites.",{"value":182,"source":95,"evidence":183},"Zotero",[184],{"id":185,"by":186,"text":187},"14275157","throw98987","Use zotero and you have your own personal Pocket with snapshots. In addition, you can add tags, organize stuff into folders, etc. https:\u002F\u002Fwww.zotero.org\u002F",{"value":189,"source":95,"evidence":190},"wallabag",[191],{"id":192,"by":193,"text":194},"14277142","ticoombs","I've been running wallabag[1] for my own pocket instance. It's been running perfectly for a couple years. Also has a pocket import feature. [1] https:\u002F\u002Fwallabag.org\u002Fen",{"value":196,"source":95,"evidence":197},"wpull and grab-site",[198],{"id":199,"by":200,"text":201},"14280720","toomuchtodo","Highly recommend switching to wpull (https:\u002F\u002Fgithub.com\u002Fchfoo\u002Fwpull), which was built as a wget replacement. It's what grab-site uses, which is a successor to ArchiveTeam's…",{"value":203,"source":95,"evidence":204},"Archive.org",[205],{"id":206,"by":207,"text":208},"14272988","bicubic","Why is archive.org or one of the other centralized web archives not suitable for that? They don't index the content you want to retain?",{"comments":210,"answered":18,"key":211},19,[212,215,218],{"id":213,"text":214},"14272916","I couldn't agree more. I wish more sites would load without needing multiple seconds of JS execution and AJAX. One of my TODOs is to get full-page screenshots working as well.",{"id":216,"text":217},"14273323","Speaking of audio, one of my highest priority TODO's for this project is to use youtube-dl for nicely archiving youtube & soundcloud links.",{"id":219,"text":220},"14274149","I was doing this before with the chrome --dump-dom flag but the output I was getting was garbage, and no more useful than the simpler wget download. PDF turned out to produce…",{"commenters":222,"max_depth":35,"maker_replies":210,"first_comment_mins":223,"lifespan_hours":222},39,81,[225,227,231,232,234,238,242,245,248,251,252,254,258,259,261,264,267,269,270,271,272,276,277,279,281,283,286,287,288,289,291,294,295,297,299,302,305,306,307,309,312,314,316,318,320,323,326,329,331,334,335,337,340,345,347,349,352,353,354,356,358,360,362,366,367,370,374,376],[178,44,46,226,179,44],"project\u002Fneutral\u002Fidea",[228,44,44,229,230,44],"14275346","commenter\u002Fneutral\u002Fnone","cJ0th",[58,34,46,226,59,44],[213,34,44,233,11,34],"maker\u002Fneutral\u002Fnone",[235,34,44,236,237,44],"14273134","other\u002Fneutral\u002Fnone","AznHisoka",[239,34,46,240,241,44],"14274971","project\u002Fneutral\u002Fexecution","zulln",[243,34,46,240,244,44],"14275489","tomcooks",[246,34,44,229,247,44],"14275889","Drdrdrq",[249,34,46,226,250,44],"14273194","jccalhoun",[216,34,44,240,11,34],[125,34,34,253,126,44],"project\u002Fcritical\u002Fexecution",[255,34,46,256,257,44],"14273644","project\u002Fmixed\u002Fexecution","TazeTSchnitzel",[219,34,44,240,11,34],[260,34,46,240,241,44],"14274947",[262,34,46,226,263,44],"14281502","ianbicking",[265,34,46,226,266,44],"14274280","robzyb",[268,34,44,240,11,34],"14274566",[129,34,34,253,130,44],[169,40,34,253,170,44],[80,40,44,240,11,34],[273,40,46,274,275,44],"14274529","project\u002Fneutral\u002Fdocs","edibleEnergy",[199,46,46,240,200,44],[278,46,44,229,11,34],"14283180",[185,42,34,280,186,44],"project\u002Fcritical\u002Fidea",[282,42,44,226,11,34],"14275245",[284,42,46,226,285,44],"14276289","bovine3dom",[192,47,34,280,193,44],[136,35,34,253,137,44],[87,35,44,240,11,34],[117,18,47,290,118,44],"project\u002Fpositive\u002Fidea",[292,18,46,240,293,44],"14274325","joshstrange",[145,18,44,240,11,34],[296,18,46,274,118,44],"14275177",[298,18,46,240,118,44],"14277677",[300,18,46,226,301,44],"14274717","8ig8",[303,18,46,240,304,44],"14277699","snackai",[158,39,47,290,159,44],[162,39,44,226,11,34],[102,308,47,290,103,44],9,[310,308,46,240,311,44],"14272547","motdiem",[313,308,44,240,11,34],"14272651",[315,308,46,226,107,44],"14272807",[317,308,44,290,11,34],"14272825",[319,308,46,226,107,44],"14272886",[321,308,47,322,107,44],"14272810","maker\u002Fpositive\u002Fnone",[324,308,46,240,325,44],"14273449","MrRadar",[327,308,46,274,328,44],"14276643","zmix",[330,308,47,290,107,44],"14272803",[332,308,47,290,333,44],"14272896","throwaway7767",[152,308,44,226,11,34],[336,308,46,226,130,44],"14274488",[338,32,44,236,339,44],"14278055","arkenflame",[341,342,44,343,344,44],"14274145",11,"other\u002Fcritical\u002Fnone","fiatjaf",[346,342,44,229,11,34],"14274172",[348,342,44,229,344,44],"14275347",[350,351,46,226,207,44],"14272877",12,[55,351,44,226,11,34],[206,351,46,226,207,44],[355,351,44,226,11,34],"14273170",[357,351,46,226,103,44],"14273450",[113,359,47,290,114,44],13,[361,359,44,240,11,34],"14272659",[363,359,46,364,365,44],"14273770","maker\u002Fneutral\u002Fdocs","no_news_is",[74,359,44,364,11,34],[106,368,47,369,107,44],14,"project\u002Fpositive\u002Fexecution",[371,372,47,290,373,44],"14274704",15,"burnbabyburn",[98,375,47,290,99,44],16,[66,375,44,233,11,34],[],[379,383,388,393,397,401],{"id":380,"name":381,"points":382,"category":20},"14286133","100M Books",289,{"id":384,"name":385,"points":386,"category":387},"14309143","Insomnia REST Client",218,"dev_tools",{"id":389,"name":390,"points":391,"category":392},"14264201","Interface Lovers",155,"learning_science",{"id":394,"name":395,"points":396,"category":20},"14279515","Flow Dashboard",128,{"id":398,"name":399,"points":400,"category":20},"14307754","Ora.pm",122,{"id":402,"name":403,"points":404,"category":20},"14279653","expense manager",107,false,"v2",{"id":408,"name":5,"line":6,"ai":405,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":409,"top":18,"kw":410,"maker":11,"domain":10,"tier":19,"attempts":44,"strip":411,"url":9,"mix":412,"praise":94,"objection":122,"sub":21},14272133,0.05,[24,25,26,27,28],"3303331333113011153335533353530030350355",[32,33,34,35],1794036555104]