[{"data":1,"prerenderedAt":253},["ShallowReactive",2],{"project-7926005":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":40,"post_text":42,"what":43,"said":83,"maker_in_thread":162,"shape":172,"grid":176,"attempts":215,"same_week":216,"maker_shows":33,"maker_hidden":245,"reader_version":246,"card":247},"7926005","Scholar Ninja","An open distributed search engine for science","title","Show HN: An open distributed search engine for science","http:\u002F\u002Fjuretriglav.si\u002Fan-open-distributed-search-engine-for-science\u002F","juretriglav.si","juretriglav",1403383118,2014,98,20,278,2272,13,"notable","learning_science","academic search engine","web_app",[24,25,26,27,28],"p2p search","scientific papers","distributed search","open access","browser extension","curious",0.175,{"positive":32,"neutral":33,"mixed":34,"critical":35},3,4,0,1,8,{"idea":38,"execution":39},{"positive":32,"critical":35},{"positive":34,"critical":34},[41,32,32,34,34,41,34,34,41,35,34,34,34,34,34,34,32,34,32,34],5,null,{"problem":44,"audience":54,"pricing":42,"motivation":63,"time_to_build":42,"stack":70,"openness":80,"stage":81,"team_size":82},{"value":45,"source":46,"evidence":47},"Finding scientific papers via a non-centralised search index built from pages users read.","maker",[48,51],{"id":49,"by":11,"text":50},"7926934","The P2P network approach here is important for two reasons, one is that I do not have fulltext access to journals and researchers using this extension do, and second, I do not…",{"id":52,"by":11,"text":53},"7928416","I'm not really familiar with the term OAI harvesting, could you elaborate? With regards to indexing, it looks like we're going to partner with ContentMine (Peter, Richard et al.)…",{"value":55,"source":56,"evidence":57},"Researchers and science readers.","inferred",[58,59],{"id":49,"by":11,"text":50},{"id":60,"by":61,"text":62},"7948461","petermurrayrust","This is really great and is fully complementary to our Content Mine (contentmine.org). Its' very similar to what I proposed as \"the World Wide Molecular Matrix\" (WWMM) about 10…",{"value":64,"source":46,"evidence":65},"Build a fun, distributed, non-centralised alternative to centralised scholarly search.",[66,69],{"id":67,"by":11,"text":68},"7927200","Thanks for the kind words! Whether or not I'll carry on with this project depends to a large degree upon how well it is received within the community and how many master hackers…",{"id":49,"by":11,"text":50},[71,75],{"value":72,"source":46,"evidence":73},"WebRTC",[74],{"id":49,"by":11,"text":50},{"value":76,"source":46,"evidence":77},"JavaScript browser extension",[78,79],{"id":52,"by":11,"text":53},{"id":67,"by":11,"text":68},"open_source","prototype","unknown",{"praise":84,"objections":102,"requests":121,"questions":137,"alternatives":146},[85,94],{"value":86,"source":87,"evidence":88},"Distributed design guarantees non-centrality and is a valuable alternative to Google-like services.","commenter",[89,93],{"id":90,"by":91,"text":92},"7927154","higherpurpose","I hope you carry on with this project. If there's any search engine that can beat Google (long into the future) it's a P2P one. Speaking of the devil, are you aware you can't…",{"id":60,"by":61,"text":62},{"value":95,"source":87,"evidence":96},"Complementary to ContentMine's scraping work.",[97,101],{"id":98,"by":99,"text":100},"7926329","Blahah","Jure, your projects never cease to impress me. Really looking forward to talking in depth at OKfest. This idea is so close to what we've been doing that it's a real shame we…",{"id":60,"by":61,"text":62},[103,117],{"value":104,"source":87,"evidence":105},"Scraping publisher pages may violate publisher rights, including sui generis database rights.",[106,110,114],{"id":107,"by":108,"text":109},"7926251","yid","It seems like nothing like this currently exists in a centralized, non-distributed way. Why add the complexity of a p2p network into an unproven concept? Is it purely to save on…",{"id":111,"by":112,"text":113},"7928838","gioele","> This is not crawling, only scraping from an already rendered page. They really can't complain. You may want to have a look at…",{"id":115,"by":112,"text":116},"7931402","The problem is that the phrase \"This is not crawling, only scraping\" is to be taken with a grain of salt. If you find a page with a lot of information and you \"just scrape it\",…",{"value":118,"source":87,"evidence":119},"Unclear whether a P2P approach is needed for an unproven concept.",[120],{"id":107,"by":108,"text":109},[122,126],{"value":123,"source":87,"evidence":124},"Index open-access full texts such as CC-BY content.",[125],{"id":98,"by":99,"text":100},{"value":127,"source":87,"evidence":128},"Use preprint sources via OAI harvesting or Common Crawl.",[129,133],{"id":130,"by":131,"text":132},"7927356","nl","Why not index preprints, which are generally available via OAI harvesting? I'm not following the field closely at the moment, but I'm pretty sure PLOS at least has an OAI…",{"id":134,"by":135,"text":136},"7927254","hershel","What about http:\u002F\u002Fcommoncrawl.org\u002F? Why not use it?",[138,142],{"value":139,"source":87,"evidence":140},"Why never index full texts?",[141],{"id":98,"by":99,"text":100},{"value":143,"source":87,"evidence":144},"Is P2P needed for cost savings?",[145],{"id":107,"by":108,"text":109},[147,154,158],{"value":148,"source":87,"evidence":149},"Zotero translators",[150],{"id":151,"by":152,"text":153},"7926342","kanzure","> I've been building a library of ScraperJSON definitions for academic publisher sites, and I've converged on some formats that work for a majority of publishers with no…",{"value":155,"source":87,"evidence":156},"Mendeley",[157],{"id":60,"by":61,"text":62},{"value":159,"source":87,"evidence":160},"Common Crawl",[161],{"id":134,"by":135,"text":136},{"comments":36,"answered":163,"key":164},6,[165,168,171],{"id":166,"text":167},"7928461","Thanks for the highly relevant links. It does make sense to try and outsource the maintenance of scraper rules, especially if there are projects focusing solely on that part.",{"id":169,"text":170},"7926967","Thanks Richard! Open Knowledge festival is going to be off the hook :) I realized you and your team (Peter et al.) have been working on a project in a similar space, and was…",{"id":67,"text":68},{"commenters":173,"max_depth":163,"maker_replies":36,"first_comment_mins":174,"lifespan_hours":175},10,115,107,[177,179,181,183,185,187,189,191,195,197,199,201,203,205,206,209,210,211,212,213],[98,34,41,178,99,34],"maker\u002Fpositive\u002Fidea",[151,34,32,180,152,34],"project\u002Fneutral\u002Fexecution",[182,34,32,180,99,34],"7926359",[166,34,34,184,11,35],"project\u002Fpositive\u002Fexecution",[169,34,34,186,11,35],"project\u002Fneutral\u002Fidea",[90,35,41,188,91,34],"project\u002Fpositive\u002Fidea",[67,35,34,190,11,35],"project\u002Fneutral\u002Fnone",[192,35,34,193,194,34],"7927832","commenter\u002Fneutral\u002Fnone","gorhill",[60,196,41,188,61,34],2,[107,32,35,198,108,34],"project\u002Fcritical\u002Fidea",[200,32,34,193,99,34],"7926302",[111,32,34,202,112,34],"commenter\u002Fcritical\u002Fnone",[204,32,34,190,11,35],"7928963",[115,32,34,202,112,34],[207,32,34,208,11,35],"7933857","maker\u002Fneutral\u002Fnone",[49,32,34,186,11,35],[130,33,32,180,131,34],[52,33,34,180,11,35],[134,41,32,186,135,34],[214,41,34,186,11,35],"7927262",[],[217,222,227,232,237,241],{"id":218,"name":219,"points":220,"category":221},"7937258","Fast.js",323,"languages",{"id":223,"name":224,"points":225,"category":226},"7944540","Hashcash",89,"security_privacy",{"id":228,"name":229,"points":230,"category":231},"7940030","Seed-DB funding visualization",85,"data",{"id":233,"name":234,"points":235,"category":236},"7937476","Checkr.io",79,"web_saas",{"id":238,"name":239,"points":240,"category":236},"7917408","BillForward",73,{"id":242,"name":243,"points":244,"category":231},"7931659","SeeYourVisitors",69,false,"v2",{"id":248,"name":5,"line":6,"ai":245,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":249,"top":18,"kw":250,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":251,"url":9,"mix":252,"praise":86,"objection":104,"sub":21},7926005,0.18,[24,25,26,27,28],"53300500510000003030",[32,33,34,35],1794036519115]