[{"data":1,"prerenderedAt":394},["ShallowReactive",2],{"project-12342781":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":45,"post_text":47,"what":48,"said":71,"maker_in_thread":201,"shape":210,"grid":214,"attempts":356,"same_week":357,"maker_shows":41,"maker_hidden":386,"reader_version":387,"card":388},"12342781","Workq","Job Server in Go","title","Show HN: Workq – Job Server in Go","https:\u002F\u002Fgithub.com\u002Fiamduo\u002Fworkq","github.com","iamduo",1471951112,2016,185,59,152,2264,7,"notable","dev_tools","job queue server","library",[24,25,26,27,28],"job queue","task queue","go job server","background jobs","beanstalkd alternative","curious",-0.065,{"positive":32,"neutral":33,"mixed":34,"critical":35},5,15,0,6,26,{"idea":38,"design":40,"docs":42,"performance":43,"execution":44},{"positive":39,"critical":32},2,{"positive":41,"critical":34},1,{"positive":34,"critical":34},{"positive":34,"critical":34},{"positive":34,"critical":41},[41,46,34,34,46,34,46,41,41,46,46,34,32,34,46,34,34,34,32,46,34,46,34,32,46,41,34,34,34,46,34,41,34,46,41,34,46,46,32,46],3,null,{"problem":49,"audience":47,"pricing":47,"motivation":47,"time_to_build":47,"stack":59,"openness":68,"stage":69,"team_size":70},{"value":50,"source":51,"evidence":52},"Running background jobs that clients submit and workers process and report results for","maker",[53,56],{"id":54,"by":11,"text":55},"12343534","A job in Workq doesn't directly target a system command or anything. The short answer is a job could be thought of as a function call. The name of the job is the function, and…",{"id":57,"by":11,"text":58},"12350874","Workq is similar to beanstalkd and was modeled after many of its concepts especially TTR and reserve (https:\u002F\u002Fgithub.com\u002Fiamduo\u002Fworkq#credits). The one feature which may not be…",[60],{"value":61,"source":51,"evidence":62},"Go",[63,65],{"id":64},"post",{"id":66,"by":11,"text":67},"12343884","I can definitely agree on the HTTP tooling! As for the HTTP2 portion, the server details would be abstracted out especially with HTTP2 support in Go 1.6. At the time I looked,…","open_source","beta","unknown",{"praise":72,"objections":88,"requests":107,"questions":138,"alternatives":160},[73],{"value":74,"source":75,"evidence":76},"Simple design and clean text protocol","commenter",[77,81,84],{"id":78,"by":79,"text":80},"12343023","jalfresi","This appears to be similar to beanstalkd but with some nice additions (scheduled time for job to run, jab max existence time). One of the things I awlays wanted beanstalkd to…",{"id":82,"by":79,"text":83},"12343972","I quite like text based protocols like this, beanstalkd and memcache. Very straight forward to write clients. Each interaction is stateless, and writing clients is trivial. Thats…",{"id":85,"by":86,"text":87},"12343405","igtztorrero","I like it",[89,100],{"value":90,"source":75,"evidence":91},"In-memory only with no failover or durable persistence yet",[92,96],{"id":93,"by":94,"text":95},"12343822","spriggan3","> In-memory only for now, disk backed durability is on the roadmap. > Job payload & results are limited to 1 MiB each. > Workq servers are standalone and do not speak to each…",{"id":97,"by":98,"text":99},"12343857","bryanlarsen","failover",{"value":101,"source":75,"evidence":102},"Not clear why to choose it over beanstalkd",[103],{"id":104,"by":105,"text":106},"12350480","fasteo","It seems to offer the same feature set as beanstalkd. We have been using beanstalkd for years to process gazillions of jobs without an issue and I guess many readers are in the…",[108,118,128],{"value":109,"source":75,"evidence":110},"Disk-backed durability and pluggable storage",[111,115],{"id":112,"by":113,"text":114},"12343081","caleblloyd","Very nice work, I love the simplicity. With a reliable persistence layer, this could be a viable option for a production job queue. What are the plans for persistence? Persist to…",{"id":116,"by":11,"text":117},"12343205","Thanks. Simplicity was the main goal and I took many passes eliminating cruft. The initial plans for persistence is to disk, in the form of a Command Log. Similar to VoltDB's…",{"value":119,"source":75,"evidence":120},"Graceful restarts without downtime",[121,125],{"id":122,"by":123,"text":124},"12343458","andrewfromx","What about graceful restarts? Like can I kill -USR2 the pid of this and have it stop listening on that port, launch another version of itself (new binary) that does…",{"id":126,"by":11,"text":127},"12343565","A big yes to this! Signal handling will be covered in a future update. This is critical to Workq, being that it is intended to run as a standalone. A zero downtime deployment is…",{"value":129,"source":75,"evidence":130},"Job progress streaming",[131,135],{"id":132,"by":133,"text":134},"12343859","amelius","Can it stream data from jobs? For example, to monitor progress?",{"id":136,"by":133,"text":137},"12344602","I would use it to show the progress to a user :) For example: 68% done Or: ETA: 15 minutes",[139,149],{"value":140,"source":75,"evidence":141},"How job dependencies and workflows work",[142,146],{"id":143,"by":144,"text":145},"12344960","alooPotato","Would it be possible to model dependencies between jobs using this? I.e. only run job X if job Y succeeds? We're building an in house CI system (we have some weird requirements…",{"id":147,"by":11,"text":148},"12345197","It is possible, but explicitly through workers. You can have a worker block on the \"result\" command (it allows for a wait-timeout) and wait for the successfully completion of Job…",{"value":150,"source":75,"evidence":151},"How multi-parameter jobs are specified",[152,156],{"id":153,"by":154,"text":155},"12343354","chrishacken","How would you specify a job with multiple parameters? You use \"ping\" as the example. If I wanted to run \"ping -c 20 10.10.10.10\", how would I accomplish that? It's not…",{"id":157,"by":158,"text":159},"12343566","noselasd","This is more or less a message queue, one side (client) sends a message, the other side (worker) receive and act on it. But structured around the common use case of someone…",[161,166,177,181,188,195],{"value":162,"source":75,"evidence":163},"beanstalkd",[164,165],{"id":78,"by":79,"text":80},{"id":104,"by":105,"text":106},{"value":167,"source":75,"evidence":168},"AMQP \u002F RabbitMQ",[169,173],{"id":170,"by":171,"text":172},"12344062","AdamN","Workq would be far better if it followed the AMQP (https:\u002F\u002Fwww.amqp.org\u002F) or SQS (https:\u002F\u002Faws.amazon.com\u002Fsqs\u002F) specs and terminology. There's already a large body of work on…",{"id":174,"by":175,"text":176},"12345192","tbarbugli","Why did you decide to build the queuing\u002Fmessaging part? There are plenty of solid options out there (eg. RabbitMQ)",{"value":178,"source":75,"evidence":179},"Amazon SQS",[180],{"id":170,"by":171,"text":172},{"value":182,"source":75,"evidence":183},"NATS",[184],{"id":185,"by":186,"text":187},"12349544","stevewilhelm","May I suggest NATS Queuing as an alternative. [1] [1] http:\u002F\u002Fnats.io\u002Fdocumentation\u002Ftutorials\u002Fnats-queueing\u002F",{"value":189,"source":75,"evidence":190},"NSQ",[191],{"id":192,"by":193,"text":194},"12348457","hendler","related: go message queue at http:\u002F\u002Fnsq.io\u002F",{"value":196,"source":75,"evidence":197},"Disque",[198],{"id":199,"by":98,"text":200},"12343560","The synchronous job processing is something we had to add to our beanstalkd client, but none of the other enhancements are useful to us. The biggest limitation of beanstalkd IMO…",{"comments":202,"answered":203,"key":204},14,9,[205,206,209],{"id":54,"text":55},{"id":207,"text":208},"12343603","Can you clarify the missing robustness features?",{"id":147,"text":148},{"commenters":211,"max_depth":35,"maker_replies":202,"first_comment_mins":212,"lifespan_hours":213},30,32,31,[215,217,220,224,228,231,235,237,239,240,243,244,246,247,250,252,253,254,257,259,261,263,267,269,271,272,275,277,278,281,283,287,290,293,294,296,300,303,307,310,313,315,317,320,322,324,326,328,329,333,334,335,339,341,343,345,346,349,353],[170,34,41,216,171,34],"project\u002Fcritical\u002Fidea",[218,34,46,219,171,34],"12344085","project\u002Fneutral\u002Fdesign",[221,34,34,222,223,34],"12345123","other\u002Fneutral\u002Fdesign","gtaylor",[225,34,34,226,227,34],"12344205","other\u002Fneutral\u002Fidea","sluukkonen",[229,34,34,226,230,34],"12345224","billhathaway",[232,41,34,233,234,34],"12344894","other\u002Fneutral\u002Fnone","brianolson",[153,39,46,236,154,34],"project\u002Fneutral\u002Fdocs",[54,39,34,238,11,41],"maker\u002Fneutral\u002Fdocs",[157,39,46,236,158,34],[241,39,34,242,154,34],"12343611","commenter\u002Fneutral\u002Fdocs",[199,46,41,216,98,34],[207,46,34,245,11,41],"maker\u002Fneutral\u002Fnone",[97,46,41,216,98,34],[143,248,46,249,144,34],4,"project\u002Fneutral\u002Fidea",[147,248,34,251,11,41],"maker\u002Fneutral\u002Fidea",[122,32,46,249,123,34],[126,32,34,251,11,41],[255,32,32,256,123,34],"12354284","maker\u002Fpositive\u002Fnone",[78,35,32,258,79,34],"project\u002Fpositive\u002Fidea",[260,35,34,251,11,41],"12343170",[262,35,46,249,79,34],"12343714",[264,35,34,265,266,34],"12344089","commenter\u002Fneutral\u002Fidea","snovv_crash",[268,35,34,265,79,34],"12344496",[270,35,34,251,11,41],"12350904",[185,18,34,233,186,34],[273,18,34,274,105,34],"12353330","other\u002Fcritical\u002Fidea",[112,276,32,258,113,34],8,[116,276,34,251,11,41],[279,203,46,249,280,34],"12344066","iamd3vil",[282,203,34,251,11,41],"12344381",[284,285,46,219,286,34],"12343174",10,"karmakaze",[288,285,34,289,11,41],"12343308","maker\u002Fneutral\u002Fdesign",[291,285,34,292,286,34],"12343376","commenter\u002Fneutral\u002Fdesign",[66,285,34,289,11,41],[82,285,32,295,79,34],"project\u002Fpositive\u002Fdesign",[297,285,46,298,299,34],"12344677","project\u002Fneutral\u002Fperformance","timtadh",[93,301,41,302,94,34],11,"project\u002Fcritical\u002Fexecution",[304,301,34,305,306,34],"12343847","commenter\u002Fneutral\u002Fexecution","saghul",[308,301,34,309,98,34],"12343881","commenter\u002Fcritical\u002Fexecution",[311,301,34,265,312,34],"12343971","eitland",[314,301,34,265,98,34],"12344408",[316,301,34,265,312,34],"12345067",[318,301,34,319,11,41],"12344002","maker\u002Fneutral\u002Fexecution",[321,301,46,319,98,34],"12344418",[323,301,34,265,79,34],"12344527",[325,301,34,265,98,34],"12344623",[104,327,41,216,105,34],12,[57,327,34,251,11,41],[330,331,46,249,332,34],"12343838",13,"deforciant",[192,202,34,233,193,34],[174,33,41,216,175,34],[336,337,34,233,338,34],"12344536",16,"sfrailsdev",[340,337,34,233,11,41],"12344570",[132,342,46,249,133,34],17,[344,342,34,251,11,41],"12344050",[136,342,46,249,133,34],[85,347,32,348,86,34],18,"project\u002Fpositive\u002Fnone",[350,351,46,249,352,34],"12342907",19,"ilackarms",[354,351,46,249,355,34],"12342976","cdnsteve",[],[358,362,367,372,376,381],{"id":359,"name":360,"points":361,"category":20},"12355102","Carbide",628,{"id":363,"name":364,"points":365,"category":366},"12319661","posio",220,"games",{"id":368,"name":369,"points":370,"category":371},"12312196","WebGL Fire Simulation",143,"creative",{"id":373,"name":374,"points":375,"category":20},"12320461","Replify",112,{"id":377,"name":378,"points":379,"category":380},"12311930","save-as-ebook",106,"productivity",{"id":382,"name":383,"points":384,"category":385},"12331915","Cryptic",93,"security_privacy",false,"v2",{"id":389,"name":5,"line":6,"ai":386,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":390,"top":18,"kw":391,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":392,"url":9,"mix":393,"praise":74,"objection":90,"sub":21},12342781,-0.06,[24,25,26,27,28],"1300303113305030005303053100030103103353",[32,33,34,35],1794036546959]