soveran

Ost café

There's a small café in New York City called Ost. I ordered a café au lait at the counter and was told to wait at a table. A few seconds later, a waitress arrived with my order. It got me thinking about queues.

Redis based queues

Redis is a wonderful tool for designing queues. By providing the right abstractions, it makes it possible to build a very simple queue with a few lines of code. Probably the most famous Redis-based queue is Resque. One small contender is Ost.

That strange Redis command

Ost café

When you push items to a queue, you expect the workers to process them without problems. A worker may face an unexpected situation and crash, and that's when you want the item to be preserved. That's what makes the queue reliable.

One of the many commands Redis provides was conceived for helping in the design of reliable queues: it's called BRPOPLPUSH. The idea is that instead of just POP'ing an item from a queue, you can also keep a backup. When the worker finishes without errors, the backup is recycled. When the worker dies, the queue still has a copy of the troubled item.

What makes Ost different?

The short answer: it uses BRPOPLPUSH, which means that the queues it manages are reliable. There's no long answer.

You can find more information about it at GitHub, along with a command line tool for running Ost workers.