[{"data":1,"prerenderedAt":170},["ShallowReactive",2],{"project-32940451":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":37,"post_text":39,"what":40,"said":64,"maker_in_thread":106,"shape":115,"grid":117,"attempts":134,"same_week":135,"maker_shows":15,"maker_hidden":163,"reader_version":164,"card":165},"32940451","0xFast Stream","0xFast Stream – 100x Faster Ethereum Block Downloads","title","Show HN: I Built 0xFast Stream – 100x Faster Ethereum Block Downloads","https:\u002F\u002Fmanishrjain.com\u002Flaunching-0xfast-stream","manishrjain.com","mrjn",1663861150,2022,13,8,1831,2570,72,"shown","data","blockchain data api","api",[24,25,26,27],"ethereum blocks","web3 data","blockchain download","ethereum api","quiet",null,{"positive":31,"neutral":32,"mixed":33,"critical":33},1,3,0,4,{"docs":36},{"positive":33,"critical":33},[38,32,33,33,32,33,32,33],5,"Getting web3 data out of Ethereum nodes or node services is excruciatingly slow (at least days if not weeks). I had wanted a way to just download the blocks quickly since I started exploring the web3 space. Took me a bunch of trial and errors to get to a way where I can parse all the blocks since genesis within 12 hours. But, instead of moving on to the next thing, I put this service together over the last few days, so everyone can benefit.",{"problem":41,"audience":29,"pricing":29,"motivation":47,"time_to_build":51,"stack":55,"openness":62,"stage":63,"team_size":62},{"value":42,"source":43,"evidence":44},"Downloading Ethereum block data from nodes is very slow","maker",[45],{"id":46},"post",{"value":48,"source":43,"evidence":49},"Maker wanted quick access to block data while exploring web3 and shares it with others",[50],{"id":46},{"value":52,"source":43,"evidence":53},"A few days for the service, after 12 hours of parsing experiments",[54],{"id":46},[56],{"value":57,"source":43,"evidence":58},"Geth",[59],{"id":60,"by":11,"text":61},"32944527","Glad it's useful! :-) > btw did you verify (PoW) blocks using the nonce? I've validated the data and checksummed while exporting from Geth. But, perhaps can validate data when…","unknown","launched",{"praise":65,"objections":74,"requests":75,"questions":86,"alternatives":105},[66],{"value":67,"source":68,"evidence":69},"Useful and fast for downloading blocks","commenter",[70],{"id":71,"by":72,"text":73},"32944077","high_byte","wow can't believe just yesterday I thought how cool it would be to have that - and today you made it! :D downloading... btw did you verify (PoW) blocks using the nonce? just to…",[],[76,82],{"value":77,"source":68,"evidence":78},"Use binary encoding instead of JSON to reduce size",[79],{"id":80,"by":72,"text":81},"32944131","also if you could use binary encoding rather than json you could probably cut the size by half. I already started download, but perhaps it's worth noting in your website to use…",{"value":83,"source":68,"evidence":84},"Document gzip\u002F--compressed option",[85],{"id":80,"by":72,"text":81},[87,91,98],{"value":88,"source":68,"evidence":89},"Are blocks verified using the PoW nonce?",[90],{"id":71,"by":72,"text":73},{"value":92,"source":68,"evidence":93},"Is it a full node or archive node?",[94],{"id":95,"by":96,"text":97},"32943147","kevinsimper","Are you exposing full node or archive node? Because Archive node is several terabytes :)",{"value":99,"source":68,"evidence":100},"How is compute cost managed?",[101],{"id":102,"by":103,"text":104},"32941673","tarunmuvvala","Curious how you managed compute cost. Are you running a node locally?",[],{"comments":34,"answered":32,"key":107},[108,111,112],{"id":109,"text":110},"32944465","Oh, --compressed is is a great point, thanks. Updating the blog. I wanted to stick to JSON -- it's just the easiest for everyone to work with. And it compresses well, so all good.",{"id":60,"text":61},{"id":113,"text":114},"32943467","It doesn't have the states yet. So, equivalent of full nodes -- block, txns, receipts, logs (no traces, states). But, could be extended to include those, if needed.",{"commenters":34,"max_depth":32,"maker_replies":34,"first_comment_mins":116,"lifespan_hours":38},84,[118,120,122,124,126,128,129,131],[71,33,38,119,72,33],"project\u002Fpositive\u002Fnone",[80,33,32,121,72,33],"project\u002Fneutral\u002Fdocs",[109,33,33,123,11,31],"maker\u002Fneutral\u002Fdesign",[60,33,33,125,11,31],"maker\u002Fneutral\u002Fnone",[95,31,32,127,96,33],"project\u002Fneutral\u002Fnone",[113,31,33,127,11,31],[102,130,32,127,103,33],2,[132,130,33,133,11,31],"32942161","project\u002Fneutral\u002Fpricing",[],[136,141,145,150,154,158],{"id":137,"name":138,"points":139,"category":140},"32993130","Quazel",787,"learning_science",{"id":142,"name":143,"points":144,"category":20},"32943224","Open Prompts",279,{"id":146,"name":147,"points":148,"category":149},"32939407","Dub",255,"dev_tools",{"id":151,"name":152,"points":153,"category":149},"32941613","Ezno",175,{"id":155,"name":156,"points":157,"category":149},"32935506","Rocketry",168,{"id":159,"name":160,"points":161,"category":162},"32976832","muted.io guitar chords",157,"languages",false,"v2",{"id":166,"name":5,"line":6,"ai":163,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":29,"top":18,"kw":167,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":168,"url":9,"mix":169,"praise":67,"objection":-1,"sub":21},32940451,[24,25,26,27],"53003030",[31,32,33,33],1794036625397]