[{"data":1,"prerenderedAt":296},["ShallowReactive",2],{"project-33094720":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":41,"post_text":43,"what":44,"said":68,"maker_in_thread":154,"shape":161,"grid":165,"attempts":259,"same_week":260,"maker_shows":32,"maker_hidden":288,"reader_version":289,"card":290},"33094720","kafka-worker","Kafka 0.8.0 on Cloudflare Workers","title","Show HN: Kafka 0.8.0 on Cloudflare Workers","https:\u002F\u002Fgithub.com\u002Fmaxwellpeterson\u002Fkafka-worker","github.com","maxwellpeterson",1664973765,2022,148,37,406,2570,16,"notable","infrastructure","message broker","other",[24,25,26,21],"kafka","cloudflare workers","durable objects","loved",0.155,{"positive":30,"neutral":31,"mixed":32,"critical":32},5,11,1,18,{"design":35,"pricing":37,"idea":38,"performance":40},{"positive":32,"critical":36},0,{"positive":36,"critical":36},{"positive":39,"critical":32},2,{"positive":36,"critical":36},[30,36,42,42,36,42,42,42,42,42,36,30,30,42,30,36,36,42,36,36,36,36,36,36,36,36,36,30,36,36,36,36,42,42,42,36,32],3,null,{"problem":43,"audience":43,"pricing":43,"motivation":45,"time_to_build":43,"stack":55,"openness":66,"stage":67,"team_size":66},{"value":46,"source":47,"evidence":48},"Proof of concept; wanted the simplest starting point","maker",[49,52],{"id":50,"by":11,"text":51},"33108068","Producers (and all clients) can only see and interact with the gateway worker. DOs aren’t accessible from the outside world, and all communication with them happens through the…",{"id":53,"by":11,"text":54},"33096870","Yep, one of the challenges here is that KafkaJS requires Kafka 0.10+, which is true for several other popular clients. For this broker implementation to be somewhat practical, it…",[56,62],{"value":57,"source":58,"evidence":59},"Cloudflare Workers","inferred",[60],{"id":61},"post",{"value":63,"source":58,"evidence":64},"Cloudflare Durable Objects",[65],{"id":61},"unknown","prototype",{"praise":69,"objections":93,"requests":109,"questions":121,"alternatives":142},[70,82],{"value":71,"source":72,"evidence":73},"Cool demonstration of distributed systems built on Durable Objects","commenter",[74,78],{"id":75,"by":76,"text":77},"33096025","kentonv","This is really cool as a demonstration of how distributed systems can be built on top of Durable Objects. In particular I was really happy to see this: > What about replication?…",{"id":79,"by":80,"text":81},"33096763","kabircf","This is so neat, great demonstration... good work Max!",{"value":83,"source":72,"evidence":84},"Neat proof of concept",[85,89],{"id":86,"by":87,"text":88},"33117574","EdwardDiego","Ah yep, so it's effectively a proxy in front of the broker(s) - like Pulsar's proxy (that Kafka really needs to copy IMO). And kudos mate, it's a very cool way to prove the…",{"id":90,"by":91,"text":92},"33097501","teknopurge","this is cool. edge and decentralized applications are going to be a topic of focus (IMO) in the next 5 years.",[94,98],{"value":95,"source":47,"evidence":96},"Not practical for serious use; throughput may be too low",[97],{"id":50,"by":11,"text":51},{"value":99,"source":72,"evidence":100},"Durable Objects limit transactions per object and cross-object transactions are hard",[101,105],{"id":102,"by":103,"text":104},"33103550","ec109685","The concurrency with Durable Objects when it comes to mutating state is effectively 1: https:\u002F\u002Fdevelopers.cloudflare.com\u002Fworkers\u002Flearning\u002Fusing-durable-objects\u002F Parallel requests…",{"id":106,"by":107,"text":108},"33100739","bastawhiz","DOs are arguably the wrong primitive if you're looking to do transactions across more than one of them. They're essentially partitions, which is how this project uses them. If…",[110,114],{"value":111,"source":72,"evidence":112},"Support newer Kafka protocol versions (0.10+) for client compatibility",[113],{"id":53,"by":11,"text":54},{"value":115,"source":72,"evidence":116},"Implement Elasticsearch-style search on Durable Objects",[117],{"id":118,"by":119,"text":120},"33096185","k__","Now, do Elasticsearch, please.",[122,132],{"value":123,"source":72,"evidence":124},"How is replication and leader election handled?",[125,129],{"id":126,"by":127,"text":128},"33103468","cjdoc29","I feel like this is a dumb question. But how exactly does Durable Objects help you avoid replication and leadership election? I've read the high level overview of Durable…",{"id":130,"by":87,"text":131},"33104968","So every topic has a (Kafka) replication factor of 1, but how do producers find their partition leader if the DO fails? From my understanding, all the DOs are effectively the…",{"value":133,"source":72,"evidence":134},"What can this be used for?",[135,139],{"id":136,"by":137,"text":138},"33096805","m00dy","What can I do with this ?",{"id":140,"by":137,"text":141},"33096798","what are the advantages of having this design ?",[143,147],{"value":144,"source":72,"evidence":145},"Pulsar proxy",[146],{"id":86,"by":87,"text":88},{"value":148,"source":72,"evidence":149},"BigQuery",[150],{"id":151,"by":152,"text":153},"33099206","greatNespresso","That's super cool ! Also, with D1 coming, I can't wait to see an alternative to BigQuery built on CF workers!",{"comments":30,"answered":42,"key":155},[156,159,160],{"id":157,"text":158},"33101663","Thanks Kenton!!",{"id":50,"text":51},{"id":53,"text":54},{"commenters":162,"max_depth":30,"maker_replies":30,"first_comment_mins":163,"lifespan_hours":164},24,107,40,[166,168,172,176,178,179,181,182,186,188,189,190,192,195,196,199,201,204,207,210,212,213,215,216,220,223,226,228,230,233,237,240,242,244,247,250,254],[75,36,30,167,76,36],"project\u002Fpositive\u002Fdesign",[169,36,36,170,171,36],"33096049","commenter\u002Fpositive\u002Fnone","sammy2255",[173,36,42,174,175,36],"33096532","project\u002Fmixed\u002Fpricing","klabb3",[136,36,42,177,137,36],"project\u002Fneutral\u002Fidea",[157,36,36,170,11,32],[126,36,42,180,127,36],"project\u002Fneutral\u002Fdesign",[102,36,42,180,103,36],[183,36,42,184,185,36],"33104222","project\u002Fneutral\u002Fperformance","MuffinFlavored",[187,36,42,184,76,36],"33114025",[130,36,42,180,87,36],[50,36,36,180,11,32],[86,36,30,191,87,36],"project\u002Fpositive\u002Fidea",[193,32,30,191,194,36],"33096218","nevon",[140,32,42,177,137,36],[197,32,30,198,194,36],"33097821","project\u002Fpositive\u002Fnone",[53,32,36,200,11,32],"project\u002Fneutral\u002Fdocs",[202,32,36,203,185,36],"33098044","other\u002Fneutral\u002Fnone",[205,39,42,180,206,36],"33096198","losfair",[208,39,36,209,107,36],"33097654","commenter\u002Fneutral\u002Fdesign",[211,39,36,209,206,36],"33097802",[106,39,36,209,107,36],[214,39,36,209,76,36],"33102383",[151,42,36,203,152,36],[217,218,36,203,219,36],"33096779",4,"darkwater",[221,218,36,203,222,36],"33096846","rocmcd",[224,218,36,225,11,32],"33097004","maker\u002Fneutral\u002Fdocs",[227,218,36,203,219,36],"33097080",[79,30,30,229,80,36],"maker\u002Fpositive\u002Fnone",[90,231,36,232,91,36],6,"other\u002Fpositive\u002Fidea",[234,235,36,203,236,36],"33098613",7,"tommek4077",[238,235,36,203,239,36],"33098907","ignoramous",[241,235,36,203,11,32],"33098956",[118,243,42,177,119,36],8,[245,243,42,180,246,36],"33098966","danmcs",[248,243,42,180,249,36],"33101619","yazaddaruvala",[251,252,36,203,253,36],"33100217",9,"xani_",[255,256,32,257,258,36],"33097592",10,"project\u002Fcritical\u002Fidea","citizenpaul",[],[261,266,271,275,280,284],{"id":262,"name":263,"points":264,"category":265},"33049046","SigNoz",510,"dev_tools",{"id":267,"name":268,"points":269,"category":270},"33068017","Javalin",377,"languages",{"id":272,"name":273,"points":274,"category":270},"33035580","uFuzzy.js",251,{"id":276,"name":277,"points":278,"category":279},"33038117","GhostlyStock",190,"ai",{"id":281,"name":282,"points":283,"category":265},"33094627","Sharing",189,{"id":285,"name":286,"points":287,"category":265},"33089131","Async UI",173,false,"v2",{"id":291,"name":5,"line":6,"ai":288,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":292,"top":18,"kw":293,"maker":11,"domain":10,"tier":19,"attempts":36,"strip":294,"url":9,"mix":295,"praise":71,"objection":95,"sub":21},33094720,0.16,[24,25,26,21],"5033033333055350030000000005000033301",[30,31,32,32],1793991052612]