[{"data":1,"prerenderedAt":279},["ShallowReactive",2],{"project-49421806":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":43,"post_text":46,"what":47,"said":84,"maker_in_thread":161,"shape":171,"grid":175,"attempts":241,"same_week":242,"maker_shows":34,"maker_hidden":271,"reader_version":272,"card":273},"49421806","PicoMQ","Durable Streams over HTTP, on object storage","title","Show HN: PicoMQ – Durable Streams over HTTP, on object storage","https:\u002F\u002Fpicomq.com\u002F","picomq.com","adesh_nalpet",1787587697,2026,159,31,397,4366,10,"notable","data","streaming message broker","api",[24,25,26,27,28],"durable streams","object storage","message queue","http streaming","s3","loved",0.346,{"positive":32,"neutral":33,"mixed":34,"critical":35},6,7,1,0,14,{"design":38,"performance":39,"idea":40,"pricing":42},{"positive":34,"critical":35},{"positive":35,"critical":35},{"positive":41,"critical":35},2,{"positive":35,"critical":35},[44,35,45,45,35,35,45,35,45,35,44,35,44,35,35,35,44,35,35,35,45,44,35,45,35,45,35,44,35,45,35],5,3,"PicoMQ is a Rust server for Durable Streams, built on Object Store. Cheap, URL-addressable, granular streams (create\u002Fappend\u002Fread\u002Flong-poll\u002FSSE), with Pico Protocol or Durable Streams Protocol as the facade.\n\nS3Stream is the stream storage primitive, used in AutoMQ, shipped as a Rust library. Coordination is a command log in Postgres.",{"problem":48,"audience":60,"pricing":60,"motivation":61,"time_to_build":60,"stack":66,"openness":81,"stage":82,"team_size":83},{"value":49,"source":50,"evidence":51},"Cheap, granular, URL-addressable durable streams without Kafka's complexity","maker",[52,54,57],{"id":53},"post",{"id":55,"by":11,"text":56},"49433848","Precisely this! I might even add a section in the docs, “Not a replacement for Kafka,” under the FAQ. Where Kafka starts to fall short is routing. If you want to access the data…",{"id":58,"by":11,"text":59},"49424785","I'm glad you asked, and yes, PicoMQ does have some Kafka-like semantics. However, Kafka is great at being a huge pipe, so you'd create topics like tables. PicoMQ, on the other…",null,{"value":62,"source":50,"evidence":63},"Cheaper, simpler streams than Kafka, built on object storage",[64,65],{"id":53},{"id":55,"by":11,"text":56},[67,71,77],{"value":68,"source":50,"evidence":69},"Rust",[70],{"id":53},{"value":72,"source":50,"evidence":73},"Postgres (metadata command log)",[74],{"id":75,"by":11,"text":76},"49426747","On the same boat, and honestly, it’s not getting the attention it deserves. Given Databricks acquired ElectricSQL primarily for PGlite, the Durable Streams project is likely…",{"value":78,"source":50,"evidence":79},"S3Stream",[80],{"id":53},"open_source","beta","unknown",{"praise":85,"objections":108,"requests":117,"questions":122,"alternatives":134},[86,98],{"value":87,"source":88,"evidence":89},"Single binary with embedded admin dashboard","commenter",[90,94],{"id":91,"by":92,"text":93},"49448177","Farrynet","Awesome work on this, The \"one binary\" approach with an embedded admin dashboard makes the barrier to entry delightfully low. Since you mentioned nodes are stateless and you have…",{"id":95,"by":96,"text":97},"49428721","raiyanr","PicoMQ looks worth experimenting with during your prototype phase, especially because it is designed to run as a single binary and supports a relatively simple HTTP-based model",{"value":99,"source":88,"evidence":100},"Opinionated scope rather than Kafka compatibility",[101,105],{"id":102,"by":103,"text":104},"49433077","jdcasale","Fwiw the only two times I've used kafka in my career have been for traffic on the order of GBs\u002Fsec. And the folks I know who have relied on streaming pipes for genuinely realtime…",{"id":106,"by":103,"text":107},"49449654","Also (in case this isn't obvious) I'm 100% a fan of the decision to _not_ tightly couple to the kafka wire protocol. Kafka's apis are full of landmines and gotchas, and you can…",[109],{"value":110,"source":88,"evidence":111},"Write latency is high (~250 ms ACK) due to object storage",[112,116],{"id":113,"by":114,"text":115},"49424585","KaiserPro","Can you help an old man understand? This sounds like a kafka-like streaming system, but backed onto s3-objects? Doesn't this mean that write performance is going to be bad?",{"id":58,"by":11,"text":59},[118],{"value":119,"source":50,"evidence":120},"Disk\u002FEBS-staged WAL for low-latency ACKs",[121],{"id":55,"by":11,"text":56},[123,127],{"value":124,"source":88,"evidence":125},"How does cold start and WAL recovery affect scaling to zero?",[126],{"id":91,"by":92,"text":93},{"value":128,"source":88,"evidence":129},"How does it compare to S2?",[130],{"id":131,"by":132,"text":133},"49425798","gpsmsn","Love the landing page design! I'm excited to give it a whirl this evening. Initially, I started to compare PicoMQ to https:\u002F\u002Fgithub.com\u002Fs2-streamstore\u002Fs2. If you're familiar with…",[135,143,150,154],{"value":136,"source":88,"evidence":137},"S2 \u002F S2-Lite",[138,139],{"id":131,"by":132,"text":133},{"id":140,"by":141,"text":142},"49431939","dzonga","only if it could be run serverless. but nice product overall - just like s2 lite.",{"value":144,"source":88,"evidence":145},"Ursula",[146],{"id":147,"by":148,"text":149},"49426457","Jonovono","I’ve been obsessed with everything durable lately and this is good timing as electric just got acquired so the state of their durable streams project is unknown. Have you looked…",{"value":151,"source":50,"evidence":152},"AutoMQ and WarpStream",[153],{"id":55,"by":11,"text":56},{"value":155,"source":88,"evidence":156},"SlateDB",[157],{"id":158,"by":159,"text":160},"49429952","srinivaskarre","I've been recently exploring the object store based data stores such as turbopuffer, slateDB. have tons of questions revolving in my mind about latency and relaibility. I'll try…",{"comments":162,"answered":32,"key":163},15,[164,167,170],{"id":165,"text":166},"49426247","Thank you! I'd say pretty close. But S2 is not open source, and the OSS version (https:\u002F\u002Fgithub.com\u002Fs2-streamstore\u002Fs2) is S2-Lite, which is single-node only and uses SlateDB.…",{"id":168,"text":169},"49424900","You nailed it! With some of the similar products I've seen, they either inherit the Kafka protocol and hence the KRaft and other complexities, or go the other way with…",{"id":58,"text":59},{"commenters":172,"max_depth":32,"maker_replies":162,"first_comment_mins":173,"lifespan_hours":174},13,189,50,[176,178,180,182,185,187,188,191,193,195,198,200,201,202,204,206,207,210,213,216,218,219,222,224,227,230,232,234,235,237,239],[131,35,44,177,132,35],"project\u002Fpositive\u002Fdesign",[165,35,35,179,11,34],"project\u002Fneutral\u002Fidea",[113,34,45,181,114,35],"project\u002Fneutral\u002Fperformance",[183,34,45,181,184,35],"49424722","Shakahs",[168,34,35,186,11,34],"commenter\u002Fneutral\u002Fnone",[58,34,35,181,11,34],[189,34,45,181,190,35],"49427294","atombender",[192,34,35,181,11,34],"49427386",[194,34,45,181,190,35],"49430267",[196,34,35,197,11,34],"49433510","project\u002Fneutral\u002Fdocs",[102,34,44,199,103,35],"project\u002Fpositive\u002Fidea",[55,34,35,179,11,34],[106,34,44,199,103,35],[203,34,35,179,11,34],"49452971",[147,41,35,205,148,35],"other\u002Fneutral\u002Fnone",[75,41,35,179,11,34],[208,41,44,209,148,35],"49426793","maker\u002Fpositive\u002Fnone",[211,41,35,212,11,34],"49426999","maker\u002Fneutral\u002Fnone",[214,41,35,205,215,35],"49427761","alasano",[217,41,35,205,11,34],"49427879",[91,45,45,181,92,35],[158,220,44,221,159,35],4,"project\u002Fpositive\u002Fnone",[223,220,35,179,11,34],"49434037",[225,44,45,179,226,35],"49424942","soleveloper",[228,44,35,229,11,34],"49425109","project\u002Fneutral\u002Fpricing",[231,44,45,229,226,35],"49425209",[233,44,35,229,11,34],"49425540",[95,32,44,221,96,35],[236,32,35,197,11,34],"49428798",[140,33,45,238,141,35],"project\u002Fmixed\u002Fnone",[240,33,35,179,11,34],"49433602",[],[243,248,253,257,261,266],{"id":244,"name":245,"points":246,"category":247},"49373456","simedw.com",598,"creative",{"id":249,"name":250,"points":251,"category":252},"49378768","Huzzah",384,"dev_tools",{"id":254,"name":255,"points":256,"category":247},"49419351","rack-02",240,{"id":258,"name":259,"points":260,"category":20},"49437049","LatticeDB",190,{"id":262,"name":263,"points":264,"category":265},"49410814","Thoreau BASIC",156,"languages",{"id":267,"name":268,"points":269,"category":270},"49435675","CarWatch",146,"ai",false,"v2",{"id":274,"name":5,"line":6,"ai":271,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":275,"top":18,"kw":276,"maker":11,"domain":10,"tier":19,"attempts":35,"strip":277,"url":9,"mix":278,"praise":87,"objection":110,"sub":21},49421806,0.35,[24,25,26,27,28],"5033003030505000500035030305030",[32,33,34,35],1793991164589]