[{"data":1,"prerenderedAt":513},["ShallowReactive",2],{"project-21855986":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":44,"post_text":45,"what":46,"said":71,"maker_in_thread":202,"shape":211,"grid":216,"attempts":475,"same_week":476,"maker_shows":333,"maker_hidden":505,"reader_version":506,"card":507},"21855986","22120","Local Node.js app to save everything you browse and serve it offline","title","Show HN: Local Node.js app to save everything you browse and serve it offline","https:\u002F\u002Fgithub.com\u002Fdosyago\u002F22120","github.com","archivist1",1577004160,2019,406,97,52,2195,3,"legend","infrastructure","self-hosted archiving","desktop_app",[24,25,26,27],"offline browsing","web archive","chrome devtools","self archiving","curious",0.086,{"positive":31,"neutral":32,"mixed":33,"critical":34},9,20,2,5,36,{"execution":37,"idea":39,"privacy":40,"design":41,"docs":43},{"positive":38,"critical":33},0,{"positive":31,"critical":33},{"positive":38,"critical":38},{"positive":38,"critical":42},1,{"positive":38,"critical":38},[38,38,38,38,38,18,18,18,38,38,38,34,18,38,18,34,18,18,38,34,18,34,42,18,33,34,18,34,42,38,18,18,18,34,38,38,18,38,38,38],null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":53,"openness":68,"stage":69,"team_size":70},{"value":48,"source":49,"evidence":50},"Saving everything browsed so it can be read offline later","inferred",[51],{"id":52},"post",[54,58],{"value":55,"source":49,"evidence":56},"Node.js",[57],{"id":52},{"value":59,"source":60,"evidence":61},"Chrome DevTools protocol","maker",[62,65],{"id":63,"by":11,"text":64},"21857289","> What are the security implications of running in remote debugging mode? Great question. First up, as long as you don't put --remote-debugging-address=0.0.0.0 you are only…",{"id":66,"by":11,"text":67},"21861572","Thank you very much for the big compliment! I feel very happy to hear it. A lot of people in this thread talked about proxies, as in \"why did you not implement a proxy\" or \"I…","open_source","launched","unknown",{"praise":72,"objections":96,"requests":125,"questions":152,"alternatives":169},[73,89],{"value":74,"source":75,"evidence":76},"Clever idea using browser debug protocol to archive everything","commenter",[77,81,85],{"id":78,"by":79,"text":80},"21861371","jchook","Really brilliant implementation concept. I love how it uses the browsers debug port to save literally everything. I have often dreamed of “a Google for everything I’ve seen…",{"id":82,"by":83,"text":84},"21857130","EastSmith","Like 20 years ago I've used a program called Teleport Pro, to do something similar. I would dial up with my phone modem when the internet access was cheap (during the night), it…",{"id":86,"by":87,"text":88},"21856493","jdmoreira","Great idea! I like the concept. One of the things I miss most about the old web was how trivial it was to local mirror any website. It was great!",{"value":90,"source":75,"evidence":91},"Useful tool for forensic or personal archiving",[92],{"id":93,"by":94,"text":95},"21856805","mikece","Between this project and the others mentioned in the discussion these are excellent resources for anyone needing to have a forensic record of their how they assembled evidence…",[97,104,111,118],{"value":98,"source":75,"evidence":99},"Saving every page would use large storage over time",[100],{"id":101,"by":102,"text":103},"21858917","ksec","I remember I tried something similar a long time ago but decided it wasn't worth it. 2MB per page at 100 page a day, 200MB \u002F day. That is 73GB per year. May be once a year I get…",{"value":105,"source":75,"evidence":106},"Keeping whole cache in memory and rewriting JSON will not scale",[107],{"id":108,"by":109,"text":110},"21857609","thefreeman","Congrats on getting your project to the front page of HN. With that said I think you are going to need to change your approach if you want this project to be usable as more than…",{"value":112,"source":75,"evidence":113},"Remote debugging mode raises security concerns",[114],{"id":115,"by":116,"text":117},"21856707","aloer","what are the security implications of permanently running chrome in remote debug mode? a bit more than half a year ago I started playing around with this and was surprised how on…",{"value":119,"source":75,"evidence":120},"Only works in Chrome-based browsers",[121],{"id":122,"by":123,"text":124},"21869044","jan6","I love how there's only a single browser or two in the entire world, lol (safari I've got bo clue about) that's while assuming chrome and firefox's debugging streams would be…",[126,134,145],{"value":127,"source":75,"evidence":128},"Use a proper database or per-URL disk storage instead of one in-memory cache",[129,130],{"id":108,"by":109,"text":110},{"id":131,"by":132,"text":133},"21856410","grizzles","You could store the data in a git repo per domain, so that implicit de-duplication happens on re-visits & for shared resources. You could have a raw dir (the files you receive…",{"value":135,"source":75,"evidence":136},"Provide a proxy-based or simpler way to capture without enabling debugger",[137,141],{"id":138,"by":139,"text":140},"21858136","calpaterson","Nice job, I think this is promising but there has got to be a better way than having people enable their debugger. Is there any reason you can't just copy the contents of each…",{"id":142,"by":143,"text":144},"21856220","mauricesvay","Why not use a proxy?",{"value":146,"source":75,"evidence":147},"Mobile support",[148],{"id":149,"by":150,"text":151},"21856893","sneak","> Storage is so dirt cheap today that there is zero reason why we shouldn't have reliable historic website state for everything we have ever looked at I agree entirely, but I do…",[153,161],{"value":154,"source":75,"evidence":155},"What are the security implications of running Chrome in remote debug mode",[156,157],{"id":115,"by":116,"text":117},{"id":158,"by":159,"text":160},"21859228","arkadiyt","> - fetch('http:\u002F\u002Flocalhost:9222\u002Fjson') which errors or is opaque because it is non CORS, or What about DNS rebinding attacks?",{"value":162,"source":75,"evidence":163},"Why use DevTools instead of an HTTP proxy",[164,168],{"id":165,"by":166,"text":167},"21856145","Eikon","I’m curious of why did you go on the path of using chrome debugging functionality instead of implementing an HTTP proxy which would provides the benefit of behind…",{"id":142,"by":143,"text":144},[170,177,184,191,198],{"value":171,"source":75,"evidence":172},"Chronicler",[173],{"id":174,"by":175,"text":176},"21856512","CGamesPlay","Bump for my related project: https:\u002F\u002Fgithub.com\u002FCGamesPlay\u002Fchronicler I'm actually in the process of rewriting this. I like your approach of using DevTools to manage the…",{"value":178,"source":75,"evidence":179},"SingleFile",[180],{"id":181,"by":182,"text":183},"21856905","gildas","For your information, SingleFile can run on Firefox for Android [1]. [1] https:\u002F\u002Fgithub.com\u002Fgildas-lormeau\u002FSingleFile",{"value":185,"source":75,"evidence":186},"wwwoffle",[187],{"id":188,"by":189,"text":190},"21856572","lixtra","Related wwwoffle: http:\u002F\u002Fwww.gedanken.org.uk\u002Fsoftware\u002Fwwwoffle\u002F",{"value":192,"source":75,"evidence":193},"ReadableWebProxy",[194],{"id":195,"by":196,"text":197},"21856391","fake-name","Obligatory bump for my project ReadableWebProxy (https:\u002F\u002Fgithub.com\u002Ffake-name\u002FReadableWebProxy) that was originally intended to do this. At this point, it does a GIANT pile of…",{"value":199,"source":75,"evidence":200},"Teleport Pro",[201],{"id":82,"by":83,"text":84},{"comments":31,"answered":18,"key":203},[204,207,210],{"id":205,"text":206},"21857928","My future plan was to cache responses on disk and just keep cached keys in memory: https:\u002F\u002Fgithub.com\u002Fdosyago\u002F22120#future",{"id":208,"text":209},"21858331","I like this idea, especially using git to version the store. With automatic commits, you could roll back to a particular date to see the page versions then. A personal…",{"id":63,"text":64},{"commenters":212,"max_depth":213,"maker_replies":31,"first_comment_mins":214,"lifespan_hours":215},70,7,47,268,[217,221,224,227,230,232,234,237,240,243,246,248,250,252,254,257,260,262,266,269,272,274,276,278,281,283,287,288,291,293,295,296,299,302,304,307,310,313,316,317,319,322,324,326,327,329,332,334,337,338,341,344,347,350,353,357,362,365,368,372,373,375,378,379,383,385,390,393,395,396,400,402,404,408,412,415,417,420,422,425,427,430,433,435,437,440,441,444,446,450,452,454,458,461,464,467,469,472],[218,38,38,219,220,38],"21856739","commenter\u002Fneutral\u002Fnone","ShorsHammer",[222,38,38,219,223,38],"21856929","VvR-Ox",[225,38,38,219,226,38],"21856996","ImJasonH",[228,38,38,219,229,38],"21857086","YPCrumble",[231,38,38,219,220,38],"21857135",[233,38,38,219,229,38],"21857149",[235,38,38,219,236,38],"21857190","oefrha",[238,38,38,219,239,38],"21858461","thinkingemote",[241,38,38,219,242,38],"21857381","kickscondor",[244,38,38,219,245,38],"21858174","detaro",[247,38,38,219,175,38],"21858043",[249,38,38,219,139,38],"21858130",[108,42,18,251,109,38],"project\u002Fmixed\u002Fexecution",[205,42,38,253,11,42],"project\u002Fneutral\u002Fexecution",[255,42,38,219,256,38],"21858394","dunham",[258,42,38,219,259,38],"21860104","breatheoften",[131,33,18,261,132,38],"project\u002Fneutral\u002Fidea",[263,33,34,264,265,38],"21856429","project\u002Fpositive\u002Fidea","wurst_case",[267,33,42,268,236,38],"21856695","project\u002Fcritical\u002Fexecution",[270,33,38,219,271,38],"21857113","fit2rule",[273,33,38,219,236,38],"21857174",[275,33,38,219,271,38],"21857202",[277,33,38,219,236,38],"21857234",[279,33,38,219,280,38],"21857255","Hello71",[282,33,38,219,271,38],"21857363",[284,33,38,285,286,38],"21858603","other\u002Fneutral\u002Fnone","hunter2_",[208,33,38,219,11,42],[289,33,34,264,290,38],"21858629","solarkraft",[115,18,18,292,116,38],"project\u002Fneutral\u002Fprivacy",[149,18,18,294,150,38],"project\u002Fneutral\u002Fdesign",[181,18,38,285,182,38],[297,18,38,298,150,38],"21856959","other\u002Fneutral\u002Fprivacy",[300,18,38,298,301,38],"21859068","skinnymuch",[303,18,38,298,150,38],"21859750",[305,18,38,285,306,38],"21860320","wildfire",[308,18,18,261,309,38],"21856917","alistproducer2",[311,18,38,219,312,38],"21934805","highmastdon",[314,18,34,264,315,38],"21857778","stazz1",[63,18,38,292,11,42],[318,18,18,292,236,38],"21857391",[320,18,38,321,109,38],"21857505","commenter\u002Fneutral\u002Fprivacy",[323,18,38,321,236,38],"21857530",[325,18,38,292,11,42],"21857656",[158,18,18,292,159,38],[328,18,38,292,11,42],"21861277",[330,18,38,321,331,38],"21858097","HerpDerpLerp",[82,333,34,264,83,38],4,[335,333,38,219,336,38],"21857326","101008",[165,34,18,294,166,38],[339,34,18,294,340,38],"21856160","gear54rus",[342,34,18,292,343,38],"21856268","jeroenhd",[345,34,34,264,346,38],"21856345","stevekemp",[348,34,18,261,349,38],"21856679","prox",[351,34,38,285,352,38],"21856704","randrandrand",[354,34,18,355,356,38],"21856656","project\u002Fneutral\u002Fnone","e12e",[358,359,42,360,361,38],"21856138",6,"project\u002Fcritical\u002Fdesign","wanderingstan",[363,359,38,364,11,42],"21856218","maker\u002Fneutral\u002Fdesign",[366,359,18,261,367,38],"21856598","crooked-v",[369,359,42,370,371,38],"21857580","project\u002Fcritical\u002Fidea","daurnimator",[86,213,34,264,87,38],[101,374,42,370,102,38],8,[376,374,18,261,377,38],"21860350","laurent123456",[93,31,34,264,94,38],[380,31,18,381,382,38],"21856836","maker\u002Fneutral\u002Fnone","ragerino",[384,31,38,355,11,42],"21882275",[386,387,18,388,389,38],"21859434",10,"project\u002Fneutral\u002Fdocs","jimbob45",[391,387,38,219,392,38],"21859485","hombre_fatal",[78,394,34,264,79,38],11,[66,394,38,294,11,42],[397,398,18,261,399,38],"21858777",12,"olah_1",[122,401,42,268,123,38],13,[174,403,38,285,175,38],14,[405,406,38,285,407,38],"21859213",15,"it",[409,410,38,285,411,38],"21857038",16,"jimktrains2",[413,410,18,261,414,38],"21857428","myself248",[416,410,38,219,411,38],"21857452",[418,419,38,355,11,42],"21858857",17,[138,421,18,251,139,38],18,[423,421,18,253,424,38],"21858196","jahewson",[142,426,18,261,143,38],19,[428,426,34,264,429,38],"21856353","number6",[431,426,38,219,432,38],"21856858","supermatt",[434,426,38,219,429,38],"21857367",[436,426,38,219,236,38],"21858248",[438,426,38,321,439,38],"21857719","jiofih",[188,32,38,285,189,38],[442,32,38,298,443,38],"21856725","zo1",[445,32,38,298,432,38],"21856846",[447,448,18,261,449,38],"21857154",21,"dustingetz",[195,451,38,285,196,38],22,[453,451,38,285,349,38],"21856687",[455,451,38,456,457,38],"21857091","other\u002Fneutral\u002Fdocs","hk__2",[459,451,38,456,460,38],"21857642","moneywoes",[462,451,38,456,463,38],"21858365","EternalAugust",[465,451,38,456,466,38],"21859558","off_by_one",[468,451,38,456,457,38],"21860172",[470,451,38,285,471,38],"21857045","rounce",[473,451,38,456,474,38],"21858415","wy35",[],[477,482,486,491,495,500],{"id":478,"name":479,"points":480,"category":481},"21858226","Angeldust",716,"games",{"id":483,"name":484,"points":14,"category":485},"21874772","Gameslab","hardware",{"id":487,"name":488,"points":489,"category":490},"21880446","Cats of Jasnah",367,"learning_science",{"id":492,"name":493,"points":494,"category":490},"21851495","Mandarin Tone Trainer",237,{"id":496,"name":497,"points":498,"category":499},"21841368","gc",192,"languages",{"id":501,"name":502,"points":503,"category":504},"21843382","Fellowage",186,"web_saas",false,"v2",{"id":508,"name":5,"line":6,"ai":505,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":509,"top":18,"kw":510,"maker":11,"domain":10,"tier":19,"attempts":38,"strip":511,"url":9,"mix":512,"praise":74,"objection":98,"sub":21},21855986,0.09,[24,25,26,27],"0000033300053035330535132535103335003000",[31,32,33,34],1794036584271]