I would take a look at the Condor high throughput computing project. It supports windows (and linux, and OSX) clients and servers, handles complex dependencies between tasks using DAGman and can suspend (and even move) tasks. I ve experience of systems based on Condor that scale to thousands of machines across university campuses.

Platform LSF will do everything you need. It runs on Windows. It is commercial, and can be purchased with support.

Yes. 1. There is a dedicated manager computer node and up to 100 compute nodes. The cluster must be easily expandable.

Yes 2. It is built around job-task concept. A job may have one to 100,000 tasks.

Yes 3. A job, which is initiated by the user on the manager node, results in creation of tasks on the compute node.

Yes 4. Tasks create other tasks on the fly.

Yes 5. Some tasks may run for minutes, while others may take many hours.

Yes 6. The tasks run according to a dependency hierarchy, which may be updated on the fly.

Yes 7. The job may be paused and resumed later.

Yes 8. Each task requires specific resources in terms of CPU (cores), memory and local hard disk space. The manager should be aware of this when scheduling tasks.

Yes 9. The tasks tell their progress and result back to the manager.

Yes 10. The manager is aware if the task is alive or hanged.

Have you looked at Beowulf? Lots of distributions to choose from, and lots of customization options. You ought to be able to find something to meet your needs...

I would recommend Beowulf cause Beowulf behaves more like a single machine rather than many workstations.

give gridgain a try. This should make runtime addition of nodes very easy, and you can monitor/manage the cluster using jmx interfaces

If you don t mind hosting your project in a cloud, you might want to have a look at Windows Azure / Appfabric. AFAIK it allows you to distribute your jobs via workflows and you can dynamically add more worker machines to handle your jobs as the load increases.

You can definitely solve this sort of problem using Data Synapse Grid Server.

1. There is a dedicated manager computer node and up to 100 compute nodes. The cluster must be easily expandable. Yes, a Broker can easily handle 2000 Engines.
2. It is built around job-task concept. A job may have one to 100,000 tasks. Yes, I have queued in excess of 250,000 tasks without issue. Eventually you will run out of memory.
3. A job, which is initiated by the user on the manager node, results in creation of tasks on the compute node. yes
4. Tasks create other tasks on the fly. It can be done, although I would not recommend this sort of model
5. Some tasks may run for minutes, while others may take many hours. yes
6. The tasks run according to a dependency hierarchy, which may be updated on the fly. yes, but I would manage this outside of the grid computing infrastructure
7. The job may be paused and resumed later. yes
8. Each task requires specific resources in terms of CPU (cores), memory and local hard disk space. The manager should be aware of this when scheduling tasks. yes
9. The tasks tell their progress and result back to the manager. yes

` 10. The manager is aware if the task is alive or hanged. yes

Have you examined the SunGrid Engine? It s been a long time since I used it, and I never used it to its full capabilities, but this is my understanding.

1. There is a dedicated manager computer node and up to 100 compute nodes. The cluster must be easily expandable. yes
2. It is built around job-task concept. A job may have one to 100,000 tasks. not sure
3. A job, which is initiated by the user on the manager node, results in creation of tasks on the compute node. yes
4. Tasks create other tasks on the fly. I think so?
5. Some tasks may run for minutes, while others may take many hours. yes
6. The tasks run according to a dependency hierarchy, which may be updated on the fly. not sure
7. The job may be paused and resumed later. not sure
8. Each task requires specific resources in terms of CPU (cores), memory and local hard disk space. The manager should be aware of this when scheduling tasks. pretty sure
9. The tasks tell their progress and result back to the manager. pretty sure

` 10. The manager is aware if the task is alive or hanged. yes