I need to set up a job/message queue with the option to set a delay for the task so that it s not picked up immediately by a free worker, but after a certain time (can vary from task to task). I looked into a couple of linux queue solutions (rabbitmq, gearman, memcacheq), but none of them seem to offer this feature out of the box.
Any ideas on how I could achieve this?
Thanks!
I ve used BeanstalkD to great effect, using the delay option on inserting a new job to wait several seconds till the item becomes available to be reserved.
If you are doing longer-term delays (more than say 30 seconds), or the jobs are somewhat important to perform (abeit later), then it also has a binary logging system so that any daemon crash would still have a record of the job. That said, I ve put hundreds of thousands of live jobs through Beanstalkd instances and the workers that I wrote were always more problematical than the server.
You could use an AMQP broker (such as RabbitMQ) and I have an "agent" (e.g. a python process built using pyton-amqplib) that sits on an exchange an intercepts specific messages (specific routing_key); once a timer has elapsed, send back the message on the exchange with a different routing_key.
I realize this means "translating/mapping" routing keys but it works. Working with RabbitMQ and python-amqplib is very straightforward.
I have a situation where a basic message, such as wm_paint, inside it s treatment will trigger a call WCF Webservices when, for example, it needs remote formatting information. However, while WCF is ...
At work, we re building a distributed application (possibly across several machines on a LAN, possibly later across several continents on a WAN+VPN). We don t want log files local to each machine (...
I m using py-amqplib to access RabbitMQ in Python. The application receives requests to listen on certain MQ topics from time to time. The first time it receives such a request it creates an AMQP ...
I use the Message Queue to send messages from one application to the other one (this has to work only on one particular machine) I create the queue like this on the receiver side: string ...
Im currently writing a application where i need to let my other server queue base on request and after that, the application from my server send back a small data of whether it s finished, still ...
Is there any way to find out from threadId , if a thread has message queue or not? Basically there are some windows api which only work if a thread has message queue.window
What message oriented middle ware with a Python API are out there? What ones did you use and recommend (or not)?
I am wondering what are the limitation of System-V message queue i.e. Maximum queue size, maximum size per item etc. As I am working on integration of C++ with my PHP scripts and have very large ...
