English 中文(简体)
Use beanstalkd, for periodic tasks, how to always make a job replaced by its latest one?
原标题:

I am trying to use beanstalk for queuing a large number of periodic tasks (for example, tasks need processed every N minutes), for each task, if the last queued job is not completed (not reserved, i mean) when current job to be added, the last queued job should be replaced with current job, in other words, only the latest queued job of a task should be processed.

how can i achieve that using beanstalk?

Ideas i have got right now is: for each task, use memcached store its latest timestamps (set this when add jobs to queue), every time the worker reserved a job successfully, it first checks timestamps for this task in memcached, if timestamps of the job is same as timestamps in memcached, then process this job, otherwise skip this job, and delete it from the queue.

So is there better ways to do such work? please give your suggestions, thanks.

最佳回答

Until named jobs are done and the software released, that may be one of the better solutions.

问题回答

I found a memcache/beanstalk combination also the best solution for an implementation where I didnt want a newer but identical job entering a queue.





相关问题
c# stack queue combination

is there in C# some already defined generic container which can be used as Stack and as Queue at the same time? I just want to be able to append elements either to the end, or to the front of the ...

Packet Queue in Python?

is there any way to queue packets to a socket in Python? I ve been looking for something like the libipq library, but can t find anything equivalent. Here s what I m trying to accomplish: Create ...

Java Queue Merge, Beginner

I m trying to write a method that will take in two Queues (pre-sorted Linked Lists) and return the merged, in ascending order, resulting Queue object. I pasted the Queue class, the merge method ...

Efficient queue in Haskell

How can I efficiently implement a list data structure where I can have 2 views to the head and end of the list, that always point to a head a tail of a list without expensive calls to reverse. i.e: ...

Thread & Queue vs Serial performance

I though it ll be interesting to look at threads and queues, so I ve written 2 scripts, one will break a file up and encrypt each chunk in a thread, the other will do it serially. I m still very new ...

Insert into an STL queue using std::copy

I d like to use std::copy to insert elements into a queue like this: vector<int> v; v.push_back( 1 ); v.push_back( 2 ); queue<int> q; copy( v.begin(), v.end(), insert_iterator< queue&...

Threading in Ruby with a limit

I have a task I need to perform, do_stuff(opts), that will take ~1s each, even while 1 - 10 of them are running in parallel. I need to collect an array of the results for each operation at the end. ...

热门标签