There’s a small café in New York City called Ost. I ordered a café au lait at the counter and was told to wait at a table. A few seconds later, a waitress arrived with my order. It got me thinking about queues.
Redis is a wonderful tool for designing queues. By providing the right abstractions, it makes it possible to build a very simple queue with a few lines of code. Probably the most famous Redis-based queue is Resque. One small contender is Ost.
When you push items to a queue, you expect the workers to process them without problems. A worker may face an unexpected situation and crash, and that’s when you want the item to be preserved. That’s what makes the queue reliable.
One of the many commands Redis provides was conceived for
helping in the design of reliable queues: it’s called BRPOPLPUSH.
The idea is that instead of just POP’ing an item from a
queue, you can also keep a backup. When the worker finishes without
errors, the backup is recycled. When the worker dies, the queue still
has a copy of the troubled item.
The short answer: it uses BRPOPLPUSH, which means that
the queues it manages are reliable. There’s no long answer.
You can find more information about it at GitHub, along with a command line tool for running Ost workers.
⊙