Workq
Dev tools
job queue server
SHOW HN2016
185points
59comments
#152of 2,264 in 2016
Top 7%of 2016 by points
How HN took it
CuriousMostly questions and neutral comments
5 positive, 15 neutral or asking, 0 mixed, 6 critical, out of 26 comments about the project. Thread score -0.07.
Idea: 2 for, 5 againstDesign: 1 for, 0 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Simple design and clean text protocol
From 3 comments
This appears to be similar to beanstalkd but with some nice additions (scheduled time for job to run, jab max existence time). One of the things I awlays wanted beanstalkd to…
jalfresi on HNI quite like text based protocols like this, beanstalkd and memcache. Very straight forward to write clients. Each interaction is stateless, and writing clients is trivial. Thats…
jalfresi on HNI like it
igtztorrero on HN
Objections
In-memory only with no failover or durable persistence yet
From 2 comments
> In-memory only for now, disk backed durability is on the roadmap. > Job payload & results are limited to 1 MiB each. > Workq servers are standalone and do not speak to each…
spriggan3 on HNfailover
bryanlarsen on HN
Not clear why to choose it over beanstalkd
From 1 comment
It seems to offer the same feature set as beanstalkd. We have been using beanstalkd for years to process gazillions of jobs without an issue and I guess many readers are in the…
fasteo on HN
Requests
Disk-backed durability and pluggable storage
From 2 comments
Very nice work, I love the simplicity. With a reliable persistence layer, this could be a viable option for a production job queue. What are the plans for persistence? Persist to…
caleblloyd on HNThanks. Simplicity was the main goal and I took many passes eliminating cruft. The initial plans for persistence is to disk, in the form of a Command Log. Similar to VoltDB's…
iamduo on HN
Graceful restarts without downtime
From 2 comments
What about graceful restarts? Like can I kill -USR2 the pid of this and have it stop listening on that port, launch another version of itself (new binary) that does…
andrewfromx on HNA big yes to this! Signal handling will be covered in a future update. This is critical to Workq, being that it is intended to run as a standalone. A zero downtime deployment is…
iamduo on HN
Job progress streaming
From 2 comments
Can it stream data from jobs? For example, to monitor progress?
amelius on HNI would use it to show the progress to a user :) For example: 68% done Or: ETA: 15 minutes
amelius on HN
Questions
How job dependencies and workflows work
From 2 comments
Would it be possible to model dependencies between jobs using this? I.e. only run job X if job Y succeeds? We're building an in house CI system (we have some weird requirements…
alooPotato on HNIt is possible, but explicitly through workers. You can have a worker block on the "result" command (it allows for a wait-timeout) and wait for the successfully completion of Job…
iamduo on HN
How multi-parameter jobs are specified
From 2 comments
How would you specify a job with multiple parameters? You use "ping" as the example. If I wanted to run "ping -c 20 10.10.10.10", how would I accomplish that? It's not…
chrishacken on HNThis is more or less a message queue, one side (client) sends a message, the other side (worker) receive and act on it. But structured around the common use case of someone…
noselasd on HN
Alternatives named
beanstalkd
From 2 comments
This appears to be similar to beanstalkd but with some nice additions (scheduled time for job to run, jab max existence time). One of the things I awlays wanted beanstalkd to…
jalfresi on HNIt seems to offer the same feature set as beanstalkd. We have been using beanstalkd for years to process gazillions of jobs without an issue and I guess many readers are in the…
fasteo on HN
AMQP / RabbitMQ
From 2 comments
Workq would be far better if it followed the AMQP (https://www.amqp.org/) or SQS (https://aws.amazon.com/sqs/) specs and terminology. There's already a large body of work on…
AdamN on HNWhy did you decide to build the queuing/messaging part? There are plenty of solid options out there (eg. RabbitMQ)
tbarbugli on HN
Amazon SQS
From 1 comment
Workq would be far better if it followed the AMQP (https://www.amqp.org/) or SQS (https://aws.amazon.com/sqs/) specs and terminology. There's already a large body of work on…
AdamN on HN
NATS
From 1 comment
May I suggest NATS Queuing as an alternative. [1] [1] http://nats.io/documentation/tutorials/nats-queueing/
stevewilhelm on HN
NSQ
From 1 comment
related: go message queue at http://nsq.io/
hendler on HN
Disque
From 1 comment
The synchronous job processing is something we had to add to our beanstalkd client, but none of the other enhancements are useful to us. The biggest limitation of beanstalkd IMO…
bryanlarsen on HN