[{"data":1,"prerenderedAt":666},["ShallowReactive",2],{"project-40837610":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":48,"post_text":49,"what":50,"said":96,"maker_in_thread":244,"shape":257,"grid":261,"attempts":629,"same_week":630,"maker_shows":39,"maker_hidden":658,"reader_version":659,"card":660},"40837610","SmoothMQ","Drop-in SQS replacement based on SQLite","title","Show HN: Drop-in SQS replacement based on SQLite","https:\u002F\u002Fgithub.com\u002Fpoundifdef\u002FSmoothMQ","github.com","memset",1719760268,2024,656,156,34,3515,1,"legend","infrastructure","message queue","cli",[24,21,25,26,27],"sqs compatible","sqlite queue","self-hosted queue","go queue","mixed",0.118,{"positive":31,"neutral":32,"mixed":33,"critical":34},19,28,2,15,64,{"idea":37,"execution":40,"pricing":42,"performance":45,"design":46,"docs":47},{"positive":38,"critical":39},9,5,{"positive":41,"critical":18},3,{"positive":43,"critical":44},0,4,{"positive":43,"critical":18},{"positive":33,"critical":18},{"positive":43,"critical":18},[39,43,43,43,39,39,39,43,41,18,18,43,41,39,39,39,33,39,41,18,41,18,18,41,18,39,41,39,18,44,33,33,41,41,39,18,18,39,43,41],"Hi! I wanted to share an open source API-compatible replacement for SQS. It's written in Go, distributes as a single binary, and uses SQLite for underlying storage.\n\nI wrote this because I wanted a queue with all the bells and whistles - searching, scheduling into the future, observability, and rate limiting - all the things that many modern task queue systems have.\n\nBut I didn't want to rewrite my app, which was already using SQS. And I was frustrated that many of the best solutions out there (BullMQ, Oban, Sidekiq) were language-specific.\n\nSo I made an SQS-compatible replacement. All you hav",{"problem":51,"audience":57,"pricing":69,"motivation":79,"time_to_build":83,"stack":84,"openness":93,"stage":94,"team_size":95},{"value":52,"source":53,"evidence":54},"Need a self-hostable, SQS-API-compatible queue with search, scheduling, observability and rate limiting without rewriting apps.","maker",[55],{"id":56},"post",{"value":58,"source":59,"evidence":60},"Developers already using SQS who want a self-hostable or cheaper replacement.","inferred",[61,62,66],{"id":56},{"id":63,"by":64,"text":65},"40838206","stuaxo","Good, we need open implementations of all the AWS stuff. I swear they reimplement stuff we have just so there are more places to bill us.",{"id":67,"by":11,"text":68},"40838223","Curious: if you were going to switch, how would you want to run this? Would you want to deploy it to your own EC2 instances, or would you want a hosted solution (just as SQS…",{"value":70,"source":53,"evidence":71},"Open source under AGPL; a hosted paid version is planned but not built.",[72,75],{"id":73,"by":11,"text":74},"40838104","Truthfully, I don't know yet - I haven't even built a paid\u002Fhosted version at all. It is related to my existing business in the sense that it deals with realtime data. But I…",{"id":76,"by":77,"text":78},"40838032","jerrygenser","> In terms of monetization, the goal is to just have a hosted queue system. I believe this can be cheaper than SQS without sacrificing performance. Just as Backblaze and Minio…",{"value":80,"source":53,"evidence":81},"Wanted a queue with modern features while keeping existing SQS-based code unchanged.",[82],{"id":56},null,[85,89],{"value":86,"source":53,"evidence":87},"Go",[88],{"id":56},{"value":90,"source":53,"evidence":91},"SQLite",[92],{"id":56},"open_source","beta","solo",{"praise":97,"objections":128,"requests":157,"questions":192,"alternatives":215},[98,106,117],{"value":99,"source":100,"evidence":101},"Clean, readable Go code and easy to follow.","commenter",[102],{"id":103,"by":104,"text":105},"40841750","dav43","Tangential but I looked at the codebase - because I like to imagine I can code (I can’t) - and for a laymen, I could follow the code. Makes me think quite positively of go lang…",{"value":107,"source":100,"evidence":108},"Useful for testing as an alternative to localstack or mocks.",[109,113],{"id":110,"by":111,"text":112},"40838082","hrisen","Keeping questions from scale & benchmarks aside, this is a cool thing for functional\u002Funit testing module that uses SQS, instead of dumb mocks.",{"id":114,"by":115,"text":116},"40838096","okr","A testcontainer could be useful. I use always this adobe-mock from s3 for my test. Hmm.",{"value":118,"source":100,"evidence":119},"Fits the trend of simple SQLite-based self-hosted tools.",[120,124],{"id":121,"by":122,"text":123},"40838598","skeeter2020","I love that we're seeing a lot of projects apply the KISS principal and leverage (or take inspiration) from SQLite, like this, PocketBase and even DuckDB. An entire generation of…",{"id":125,"by":126,"text":127},"40838606","davidthewatson","The whole idea here is superlative. +1 for k8s, kubernetes, cloud native, self-hosted, edge-enabled at low cost, no cost. I ran rq and minio for years on k8s, but been watching…",[129,136,143,150],{"value":130,"source":100,"evidence":131},"SQLite runs on one server, so reliability is weaker than SQS until distribution exists.",[132],{"id":133,"by":134,"text":135},"40840916","PanMan","Correct me if I’m wrong but: SQLite sounds like: runs on one server. While that will work most of the time, it won’t work 100% of the time. I don’t know the specifics, but I’m…",{"value":137,"source":100,"evidence":138},"AGPL license limits adoption.",[139],{"id":140,"by":141,"text":142},"40838475","Onavo","What's the point of AGPL though? The enterprises with the budget for self hosting this sort of software usually have requirements and scale beyond what SQLite can offer out of…",{"value":144,"source":100,"evidence":145},"SQS is already very cheap, so a cheaper hosted version is a weak pitch.",[146],{"id":147,"by":148,"text":149},"40838440","slotrans","Not sure about the goal of providing a hosted service cheaper than SQS. SQS is already one of the cheapest services on Earth. It's pretty hard to spend more than a few bucks a…",{"value":151,"source":100,"evidence":152},"Disabled foreign keys and TODOs raise reliability concerns.",[153],{"id":154,"by":155,"text":156},"40840044","koito17","I may be asking a naive question, but what is the rationale behind disabling foreign key support and using them anyway in the database schema? See…",[158,168,175,182],{"value":159,"source":100,"evidence":160},"Feature comparison table against SQS.",[161,165],{"id":162,"by":163,"text":164},"40838832","mannyv","So I assume it does the back-end as well? I never cared to figure out what parts of SQS are clients-side and server side, but - does SmoothMQ support long polling, batch…",{"id":166,"by":11,"text":167},"40838956","Everything here is on the backend. The client does very little except make api calls. It implements many of these features so far (ie, visibility timeouts) and there are some…",{"value":169,"source":100,"evidence":170},"Better docs and a small example application.",[171],{"id":172,"by":173,"text":174},"40838862","deskr","Perhaps do a small example application. ``` go get github.com\u002Fpoundifdef\u002FSmoothMQ\u002Fmodels go: github.com\u002Fpoundifdef\u002FSmoothMQ@v0.0.0-20240630162953-46f8b2266d60 requires go >=…",{"value":176,"source":100,"evidence":177},"Live stats in the UI.",[178],{"id":179,"by":180,"text":181},"40838150","systemz","Very cool. I needed something like this. Looking at short video in readme - I suspect adding live stats similar to sidekiq would make UI look more dynamic and allow quick…",{"value":183,"source":100,"evidence":184},"Support for rate limiting and Sidekiq-like features.",[185,189],{"id":186,"by":187,"text":188},"40840024","pqdbr","How would one use this to replace Sidekiq for instance? Rate limiting is something I miss on Sidekiq (only available on the premium plan that I can't afford) and the gems that…",{"id":190,"by":11,"text":191},"40840815","Now you sound like an investor :) My own vision is to take a queue that is relatively dumb and make it smarter. I want it to be able to, for example, allow you to rate limit…",[193,203,208],{"value":194,"source":100,"evidence":195},"How will distributed mode work?",[196,200],{"id":197,"by":198,"text":199},"40838079","sgarland","This looks great! How were you planning on tackling distributed?",{"id":201,"by":11,"text":202},"40838171","This is a great question and something I've been thinking about a lot. Big picture: Each queue node can operate (mostly) independently, and this is good. As a consumer, I don't…",{"value":204,"source":100,"evidence":205},"Does it support long polling, visibility timeouts and FIFO queues?",[206,207],{"id":162,"by":163,"text":164},{"id":166,"by":11,"text":167},{"value":209,"source":100,"evidence":210},"Is it a drop-in for Redis Streams?",[211],{"id":212,"by":213,"text":214},"40884753","lovestaco","I was thinking of using Redis Stream instead of SQS, what would be better Redis or SmoothMQ?",[216,226,233,240],{"value":217,"source":100,"evidence":218},"LocalStack",[219,223],{"id":220,"by":221,"text":222},"40840183","john-tells-all","Why not use LocalStack? It has SQS and a lot of AWS services for testing\u002Fdevelopment. Well documented, open source. https:\u002F\u002Fdocs.localstack.cloud\u002Foverview\u002F",{"id":224,"by":11,"text":225},"40838119","I was not aware of testcontainer until you just mentioned it! I have a very basic Dockerfile for deploying to fly.io. What\u002Fhow would I get started? It looks like localstack is a…",{"value":227,"source":100,"evidence":228},"ElasticMQ",[229],{"id":230,"by":231,"text":232},"40838372","4RealFreedom","The goals are a little different but I think it's worth pointing out ElasticMQ. I use it simulate sqs in a docker environment. https:\u002F\u002Fgithub.com\u002Fsoftwaremill\u002Felasticmq",{"value":234,"source":100,"evidence":235},"RabbitMQ",[236],{"id":237,"by":238,"text":239},"40838384","chromanoid","Could you elaborate why you didn't choose e.g. RabbitMQ? I mean you talk about AMQP and such, it seems to me, that a client-side abstraction would be much more efficient in…",{"value":241,"source":100,"evidence":242},"Redis Streams",[243],{"id":212,"by":213,"text":214},{"comments":245,"answered":246,"key":247},26,8,[248,251,254],{"id":249,"text":250},"40838793","Really interesting. Question: when it comes to running these software on k8s, do you prefer to manage and host yourself, or do you use managed solutions on top of your own infra?…",{"id":252,"text":253},"40838218","Thank you! Yes, you should definitely use this instead of localstack :)",{"id":255,"text":256},"40841809","Thank you for the kind words! I'll be sure to work on making it as complicated as I can :)",{"commenters":258,"max_depth":246,"maker_replies":245,"first_comment_mins":259,"lifespan_hours":260},82,50,338,[262,264,266,269,272,274,277,281,284,286,288,291,293,295,297,298,300,303,305,306,310,313,316,319,321,322,326,329,332,335,337,339,341,345,348,351,354,357,359,361,364,367,370,371,373,376,379,382,384,387,390,392,395,398,403,405,407,410,412,414,417,420,423,424,425,428,432,435,438,442,445,448,450,452,455,457,460,463,466,469,471,475,478,480,483,487,489,492,495,497,499,501,504,507,509,510,511,514,517,520,525,529,532,534,535,537,540,543,545,547,550,555,558,563,565,567,568,570,572,573,578,581,583,585,587,589,592,595,596,598,601,606,610,612,614,617,621,624,626],[125,43,39,263,126,43],"project\u002Fpositive\u002Fidea",[249,43,43,265,11,18],"maker\u002Fneutral\u002Fnone",[267,43,43,268,126,43],"40839163","other\u002Fneutral\u002Fnone",[270,43,43,268,271,43],"40839183","cpill",[273,43,43,268,126,43],"40849451",[275,43,43,268,276,43],"40847668","kmacdough",[278,43,43,279,280,43],"40839276","commenter\u002Fneutral\u002Fnone","pfix",[282,43,43,279,283,43],"40841879","philsnow",[285,43,43,279,126,43],"40849222",[287,43,43,279,126,43],"40849199",[289,43,43,279,290,43],"40842456","raihansaputra",[292,43,43,268,126,43],"40849172",[294,43,43,268,290,43],"40872164",[296,43,43,268,126,43],"40962014",[110,18,39,263,111,43],[252,18,43,299,11,18],"project\u002Fpositive\u002Fnone",[301,18,43,279,302,43],"40839353","thrauat",[103,33,39,304,104,43],"project\u002Fpositive\u002Fexecution",[255,33,43,279,11,18],[307,33,39,308,309,43],"40842106","maker\u002Fpositive\u002Fexecution","x-complexity",[311,33,39,304,312,43],"40842674","silisili",[314,33,43,268,315,43],"40842906","bilekas",[317,33,43,268,318,43],"40843454","ayuhito",[76,41,41,320,77,43],"maker\u002Fneutral\u002Fpricing",[73,41,43,320,11,18],[323,41,43,324,325,43],"40842845","other\u002Fneutral\u002Fpricing","Kwpolska",[327,41,43,268,328,43],"40844643","fragmede",[330,41,43,268,331,43],"40846346","pc86",[333,41,43,279,334,43],"40846607","samatman",[140,41,41,336,141,43],"project\u002Fneutral\u002Fnone",[338,41,43,279,77,43],"40840531",[340,41,43,320,11,18],"40840559",[342,41,18,343,344,43],"40840869","project\u002Fcritical\u002Fnone","andrewstuart",[346,41,43,279,347,43],"40841431","ctippett",[349,41,43,279,350,43],"40841632","Aeolun",[352,41,18,353,344,43],"40841813","project\u002Fcritical\u002Fidea",[355,41,43,279,356,43],"40841815","meiraleal",[358,41,43,279,331,43],"40846378",[360,41,43,279,356,43],"40848471",[362,41,43,279,363,43],"40850613","ahachete",[365,41,43,279,366,43],"40842387","bigstrat2003",[368,41,43,268,369,43],"40843675","lupusreal",[230,44,43,268,231,43],[372,44,43,268,11,18],"40838450",[374,44,41,336,375,43],"40839015","julius",[377,44,43,268,378,43],"40843843","cies",[380,39,39,263,381,43],"40838269","yimpolo",[383,39,43,336,11,18],"40838294",[385,39,39,299,386,43],"40844829","xrd",[388,389,43,268,363,43],"40841227",6,[391,389,43,268,11,18],"40841541",[393,389,43,268,394,43],"40841784","selcuka",[396,389,39,299,397,43],"40842460","bensmoif",[399,400,41,401,402,43],"40838725",7,"project\u002Fneutral\u002Fexecution","philippta",[404,400,43,279,11,18],"40838771",[406,400,41,401,402,43],"40838805",[408,400,41,401,409,43],"40840503","pqb",[133,246,18,411,134,43],"project\u002Fcritical\u002Fperformance",[413,246,43,336,11,18],"40840947",[415,246,41,336,416,43],"40841637","22c",[418,246,39,263,419,43],"40841867","parhamn",[421,246,39,263,422,43],"40845056","jpc0",[63,38,39,263,64,43],[67,38,43,265,11,18],[426,38,18,353,427,43],"40838376","stackskipton",[429,38,41,430,431,43],"40838959","project\u002Fneutral\u002Fidea","jpalomaki",[433,38,43,434,427,43],"40839090","commenter\u002Fneutral\u002Fidea",[436,38,41,336,437,43],"40838493","nilamo",[439,38,18,440,441,43],"40839874","project\u002Fcritical\u002Fpricing","paulddraper",[443,38,43,444,11,18],"40839980","maker\u002Fneutral\u002Fidea",[446,38,41,430,447,43],"40840736","threecheese",[449,38,43,265,11,18],"40840778",[451,38,41,430,447,43],"40955085",[453,38,18,440,454,43],"40840732","mlhpdx",[220,456,18,353,221,43],10,[458,456,43,279,459,43],"40840258","crazysim",[461,456,18,353,462,43],"40842262","mtndew4brkfst",[464,456,43,279,465,43],"40843517","ku1ik",[467,456,43,279,468,43],"40846167","gkbrk",[470,456,18,343,163,43],"40858619",[472,456,41,473,474,43],"40840363","project\u002Fneutral\u002Fperformance","mihaitodor",[476,456,39,477,344,43],"40840855","project\u002Fpositive\u002Fdesign",[154,479,41,401,155,43],11,[481,479,43,482,11,18],"40840114","maker\u002Fneutral\u002Fexecution",[484,479,18,485,486,43],"40840385","project\u002Fcritical\u002Fexecution","plasma",[237,488,41,430,238,43],12,[490,488,39,263,491,43],"40842365","westurner",[493,488,43,494,238,43],"40851748","commenter\u002Fcritical\u002Fidea",[496,488,43,279,491,43],"40856787",[498,488,43,494,238,43],"40858486",[500,488,43,279,491,43],"40859552",[502,488,43,503,238,43],"40861442","commenter\u002Fneutral\u002Fpricing",[505,506,41,430,344,43],"40840839",13,[162,508,41,336,163,43],14,[166,508,43,265,11,18],[179,34,39,477,180,43],[512,34,43,513,11,18],"40838199","maker\u002Fneutral\u002Fdesign",[172,515,18,516,173,43],16,"project\u002Fcritical\u002Fdocs",[518,515,43,519,11,18],"40838974","maker\u002Fneutral\u002Fdocs",[521,522,43,523,524,43],"40840223",17,"other\u002Fneutral\u002Fidea","neuroscisoft",[526,527,41,430,528,43],"40839342",18,"encoderer",[530,31,39,299,531,43],"40838385","rmbyrro",[197,533,41,430,198,43],20,[201,533,43,444,11,18],[536,533,41,430,198,43],"40838497",[538,533,18,353,539,43],"40839253","flaminHotSpeedo",[541,533,41,430,542,43],"40846859","avinassh",[544,533,43,444,11,18],"40846895",[546,533,41,430,542,43],"40847115",[548,549,41,430,447,43],"40840683",21,[551,552,18,553,554,43],"40845111",22,"project\u002Fcritical\u002Fdesign","michaelcampbell",[556,552,43,557,11,18],"40845253","maker\u002Fpositive\u002Fdesign",[559,560,41,561,562,43],"40838115",23,"project\u002Fmixed\u002Fdesign","fbdab103",[564,560,43,265,11,18],"40838147",[114,566,41,336,115,43],24,[224,566,43,519,11,18],[569,566,43,279,115,43],"40848932",[186,571,41,430,187,43],25,[212,245,41,430,213,43],[574,575,39,576,577,43],"40841872",27,"maker\u002Fpositive\u002Fnone","leetrout",[579,32,39,263,580,43],"40838357","12_throw_away",[582,32,43,519,11,18],"40838383",[584,32,43,268,427,43],"40838393",[147,586,18,440,148,43],29,[588,586,43,320,11,18],"40838510",[590,586,18,440,591,43],"40839645","GordonS",[593,586,41,594,447,43],"40840773","project\u002Fmixed\u002Fidea",[190,586,43,444,11,18],[597,586,39,263,454,43],"40840915",[599,586,41,600,134,43],"40840898","project\u002Fneutral\u002Fpricing",[602,603,41,604,605,43],"40843354",30,"maker\u002Fneutral\u002Fperformance","TheAnkurTyagi",[607,608,39,576,609,43],"40839282",31,"xchip",[121,611,43,523,122,43],32,[613,611,43,473,11,18],"40838761",[615,611,43,268,616,43],"40839071","jmspring",[618,619,43,268,620,43],"40837943",33,"tjoff",[622,619,43,268,623,43],"40838027","cwbrandsma",[625,619,43,268,531,43],"40838374",[627,16,41,430,628,43],"40842654","dracarys18",[],[631,636,641,646,650,654],{"id":632,"name":633,"points":634,"category":635},"40843867","Pikimov",1150,"creative",{"id":637,"name":638,"points":639,"category":640},"40832146","Docmost",551,"productivity",{"id":642,"name":643,"points":644,"category":645},"40869877","Xcapture-BPF",447,"dev_tools",{"id":647,"name":648,"points":649,"category":645},"40847699","Doggo",400,{"id":651,"name":652,"points":653,"category":640},"40846242","Edna",331,{"id":655,"name":656,"points":657,"category":645},"40857009","Pretzel",269,false,"v2",{"id":661,"name":5,"line":6,"ai":658,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":662,"top":18,"kw":663,"maker":11,"domain":10,"tier":19,"attempts":43,"strip":664,"url":9,"mix":665,"praise":99,"objection":130,"sub":21},40837610,0.12,[24,21,25,26,27],"5000555031103555253131131535142233511503",[31,32,33,34],1794036657027]