[{"data":1,"prerenderedAt":314},["ShallowReactive",2],{"project-14130280":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":43,"post_text":46,"what":47,"said":90,"maker_in_thread":188,"shape":196,"grid":200,"attempts":275,"same_week":276,"maker_shows":38,"maker_hidden":306,"reader_version":307,"card":308},"14130280","TorrentPeek","Distributed search engines using BitTorrent and SQLite","title","Show HN: TorrentPeek – Distributed search engines using BitTorrent and SQLite","https:\u002F\u002Fgithub.com\u002Flmatteis\u002Ftorrent-peek\u002Fblob\u002Fmaster\u002FREADME.md","github.com","sktrdie",1492428773,2017,184,33,191,2242,9,"notable","data","distributed search engine","other",[24,25,26,27,28],"bittorrent","sqlite","distributed search","sqltorrent","full text search","debated",-0.044,{"positive":32,"neutral":33,"mixed":34,"critical":32},4,7,0,15,{"execution":37,"idea":39,"performance":41,"docs":42},{"positive":34,"critical":38},2,{"positive":40,"critical":34},1,{"positive":38,"critical":38},{"positive":34,"critical":34},[34,34,34,40,34,34,34,44,34,34,44,34,45,44,34,34,45,34,44,34,45,34,40,34,40,34,45,44,44,34,40,34,44],3,5,null,{"problem":46,"audience":46,"pricing":46,"motivation":48,"time_to_build":46,"stack":55,"openness":87,"stage":88,"team_size":89},{"value":49,"source":50,"evidence":51},"Experiment to test real-world feasibility of querying torrent-hosted SQLite databases","maker",[52],{"id":53,"by":11,"text":54},"14133442","Yes sorry about that. The project is still an experiment, mostly used to get real world numbers on the feasibility of the idea. I intend to maintain it and develop it much…",[56,64,73,80],{"value":57,"source":58,"evidence":59},"Node.js","commenter",[60],{"id":61,"by":62,"text":63},"14132854","skrowl","How does it only work on macOS if it's based on Node? Seems like they could have gotten it to work on Windows and Linux.",{"value":65,"source":50,"evidence":66},"SQLite",[67,70],{"id":68,"by":11,"text":69},"14134201","Exactly. The database syntax is the same. You're still writing SQL queries against the SQLite engine. I'm just replacing disk-access with network-piece-access. TorrentPeek goes…",{"id":71,"by":11,"text":72},"14131331","Author here. Yes the SQLite engine returns an SQLITE_OK value when the query is complete, just like if it were a database file on disk. The insight here is that we can tell…",{"value":74,"source":58,"evidence":75},"libtorrent",[76],{"id":77,"by":78,"text":79},"14133195","misterdata","It needs a few libraries (libtorrent, sqlite) which seem to be packaged as binaries for macOS only - build them anew for your platform and it will probably run.",{"value":81,"source":58,"evidence":82},"C",[83],{"id":84,"by":85,"text":86},"14133001","zulln","C exists for basically all plattforms.","open_source","prototype","unknown",{"praise":91,"objections":114,"requests":138,"questions":151,"alternatives":169},[92,103],{"value":93,"source":58,"evidence":94},"Clever idea of using SQLite VFS over torrent pieces",[95,99],{"id":96,"by":97,"text":98},"14132481","halflings","Cool idea! Surprised that SQLite can know exactly which pieces it will need to answer any query!",{"id":100,"by":101,"text":102},"14133953","Groxx","Huh. So it's basically just a torrent-VFS, and sqlite already plays nicely? Neat application though! Seems like it works really well :)",{"value":104,"source":58,"evidence":105},"Good fit for package managers and distributed datasets",[106,110],{"id":107,"by":108,"text":109},"14131428","gravypod","Do you think something like this could be used for some form of a distributed blob store that you can query? Things like package managers could use this effectively.",{"id":111,"by":112,"text":113},"14134422","zzzcpan","It's a perfect idea for package managers. And they could expand on it, like forcing clients to download a certain number of random pieces depending on the amount of seeds to make…",[115,120,131],{"value":116,"source":58,"evidence":117},"Only works on macOS because binaries are macOS-only",[118,119],{"id":61,"by":62,"text":63},{"id":77,"by":78,"text":79},{"value":121,"source":58,"evidence":122},"Repository contains clutter and no clear source",[123,127],{"id":124,"by":125,"text":126},"14132637","JD557","The repository seems to contain a lot of garbage (such as an `a.out` file and a `reddit-browser` folder). This look like a really cool project. Could you please clean up the repo…",{"id":128,"by":129,"text":130},"14135879","formula_ninguna","where is the source code?",{"value":132,"source":58,"evidence":133},"Latency and download cost of partial index fetches are unclear",[134],{"id":135,"by":136,"text":137},"14174342","paxcoder","Is 640KiB not the whole index? Because partial download would mean having to request new pieces based on the previously-downloaded ones, adding latency to the mix. Assuming just…",[139,143,147],{"value":140,"source":58,"evidence":141},"Provide Windows and Linux binaries",[142],{"id":61,"by":62,"text":63},{"value":144,"source":58,"evidence":145},"Clean up the repository",[146],{"id":124,"by":125,"text":126},{"value":148,"source":58,"evidence":149},"Publish real-world timing measurements",[150],{"id":135,"by":136,"text":137},[152,160],{"value":153,"source":50,"evidence":154},"How do you know when a query result list is complete?",[155,159],{"id":156,"by":157,"text":158},"14131140","degenerate","Is there indication when the result list for a given query is 'complete'? Otherwise if the search result don't look like what the person wanted, how long should they wait to see…",{"id":71,"by":11,"text":72},{"value":161,"source":50,"evidence":162},"Does a search require downloading the index first?",[163,166],{"id":164,"by":101,"text":165},"14133819","Neat, both this and sqltorrent. I'll have to read up on this in more detail... Does this mean you (essentially) need to download the index before doing any searches? Or is there…",{"id":167,"by":11,"text":168},"14133888","No you don't need to download the indexes beforehand. The download always starts from the first piece\u002Fpage (0). From there it essentially performs binary search and only…",[170,181],{"value":171,"source":58,"evidence":172},"ZeroNet",[173,177],{"id":174,"by":175,"text":176},"14137192","Sunset","This is interesting and reminds me of Zeronet.",{"id":178,"by":179,"text":180},"14131826","ZainRiz","This is really cool. From the readme it sounds like there's a community of people working on this problem. How much of this is stuff you built from scratch versus libraries you…",{"value":182,"source":58,"evidence":183},"YaCy",[184],{"id":185,"by":186,"text":187},"14132581","tombert","This reminds me a lot of the (I think) very underrated YaCy project: http:\u002F\u002Fyacy.net\u002Fen\u002Findex.html I know there are a ton of technical reasons why this is difficult, but…",{"comments":189,"answered":18,"key":190},11,[191,192,195],{"id":68,"text":69},{"id":193,"text":194},"14133485","The main reason that there are no binaries for Windows\u002FLinux it's because I don't have these machines to compile the project against. I will eventually create binaries for all…",{"id":71,"text":72},{"commenters":197,"max_depth":45,"maker_replies":189,"first_comment_mins":198,"lifespan_hours":199},21,183,128,[201,205,207,209,211,213,214,216,218,221,222,223,226,227,228,232,236,238,241,244,246,248,250,251,252,257,259,260,262,265,268,270,272],[202,34,34,203,204,34],"14133703","commenter\u002Fpositive\u002Fidea","MR4D",[68,34,34,206,11,40],"project\u002Fneutral\u002Fidea",[174,34,34,208,175,34],"other\u002Fneutral\u002Fidea",[61,40,40,210,62,34],"project\u002Fcritical\u002Fexecution",[84,40,34,212,85,34],"commenter\u002Fneutral\u002Fexecution",[77,40,34,212,78,34],[193,40,34,215,11,40],"maker\u002Fneutral\u002Fexecution",[156,38,44,217,157,34],"project\u002Fneutral\u002Fexecution",[219,38,34,212,220,34],"14131247","ORioN63",[71,38,34,215,11,40],[107,44,44,206,108,34],[224,44,34,225,11,40],"14133465","project\u002Fpositive\u002Fidea",[111,44,45,225,112,34],[185,32,44,206,186,34],[229,32,34,230,231,34],"14132834","commenter\u002Fneutral\u002Fidea","feadog",[233,32,34,234,235,34],"14133939","other\u002Fcritical\u002Fidea","j3097736",[178,45,45,237,179,34],"project\u002Fpositive\u002Fnone",[239,45,34,240,11,40],"14133503","maker\u002Fneutral\u002Fnone",[164,242,44,243,101,34],6,"project\u002Fneutral\u002Fperformance",[167,242,34,245,11,40],"maker\u002Fneutral\u002Fperformance",[100,242,45,247,101,34],"project\u002Fpositive\u002Fperformance",[249,242,34,225,11,40],"14134163",[124,33,40,210,125,34],[53,33,34,215,11,40],[253,254,40,255,256,34],"14133475",8,"project\u002Fcritical\u002Fperformance","chx",[258,254,34,245,11,40],"14133558",[96,18,45,247,97,34],[261,18,44,243,78,34],"14133220",[263,18,44,243,264,34],"14133247","tangent128",[266,18,34,267,11,40],"14133524","maker\u002Fpositive\u002Fperformance",[135,18,40,269,136,34],"maker\u002Fcritical\u002Fperformance",[271,18,34,245,11,40],"14133534",[128,273,44,274,129,34],10,"project\u002Fneutral\u002Fdocs",[],[277,282,287,292,296,301],{"id":278,"name":279,"points":280,"category":281},"14135045","Zuck.js",339,"languages",{"id":283,"name":284,"points":285,"category":286},"14135781","Audioshop",285,"creative",{"id":288,"name":289,"points":290,"category":291},"14124285","Hack the user cursor",267,"games",{"id":293,"name":294,"points":295,"category":291},"14147585","Dance of the Fairies",261,{"id":297,"name":298,"points":299,"category":300},"14137563","Visual Studio Code",236,"dev_tools",{"id":302,"name":303,"points":304,"category":305},"14122418","UrlRoulette",180,"consumer_social",false,"v2",{"id":309,"name":5,"line":6,"ai":306,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":310,"top":18,"kw":311,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":312,"url":9,"mix":313,"praise":93,"objection":116,"sub":21},14130280,-0.04,[24,25,26,27,28],"000100030030530050305010105330103",[32,33,34,32],1793990957806]