English 中文(简体)
我该如何在.NET中最有效地利用多个核心进行短时计算?
原标题:How can I most effectively take advantage of multiple cores for short computations in .NET?

我正在使用C#为一种叫做Heron的小型编程语言编写解释器,它具有一些可以并行执行的基本列表操作。

我面临的最大挑战之一就是在遇到可并行化操作时,有效地将评估器完成的工作分配到不同的核心上。这可能是一个短操作或长操作,很难事先确定。

我不必担心的一件事是数据同步:并行操作明确禁止修改数据。

所以我最主要的问题是:

  • What is the most effective way to distribute the work across threads, so that I can guarantee that the computer will distribute the work across two cores?

我也对一个相关的问题感兴趣:

  • Roughly how long should an operation take before we can start overcoming the overhead of separating the work onto another thread?
最佳回答

如果您想进行很多并行操作,您会想要从.Net 4.0开始。这里是 .Net并行编程文档。不过,您会想从这里开始。.Net 4.0在多核利用方面增加了很多。以下是一个快速的例子:

当前的3.5串行方法:

for(int i = 0; i < 30000; i++)
{
  doSomething(i);
}

新的.Net 4.0并行方法:

Parallel.For(0, 30000, (i) => doSomething(i));

Parallel.For 方法自动跨越可用的核心数量进行缩放,您可以看到您可以多快地开始利用它。框架中有数十个新库,支持完整的线程/任务管理,如您的示例一样(包括所有的同步、取消等管道)。

有针对并行LINQ(PLINQ)、任务工厂、任务调度器等的库。简而言之,对于您所提出的具体任务,.Net 4.0 对您有巨大的好处,我建议您去下载免费的beta 2(RC即将推出),开始使用。(不,我不是微软的员工......但我很少看到即将发布的产品如此完美地满足需求,所以我强烈推荐您使用.Net 4.0)

问题回答

因为我不想使用VS 2010进行开发,而且我发现ThreadPool在跨核心分配工作时性能不够优化(我认为是因为它启动/停止了太多的线程),所以最终我自己开发了一个。希望其他人也能从中受益。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace HeronEngine
{
    /// <summary>
    /// Represents a work item.
    /// </summary>
    public delegate void Task();

    /// <summary>
    /// This class is intended to efficiently distribute work 
    /// across the number of cores. 
    /// </summary>
    public static class Parallelizer 
    {
        /// <summary>
        /// List of tasks that haven t been yet acquired by a thread 
        /// </summary>
        static List<Task> allTasks = new List<Task>();

        /// <summary>
        /// List of threads. Should be one per core. 
        /// </summary>
        static List<Thread> threads = new List<Thread>();

        /// <summary>
        /// When set signals that there is more work to be done
        /// </summary>
        static ManualResetEvent signal = new ManualResetEvent(false);

        /// <summary>
        /// Used to tell threads to stop working.
        /// </summary>
        static bool shuttingDown = false;

        /// <summary>
        /// Creates a number of high-priority threads for performing 
        /// work. The hope is that the OS will assign each thread to 
        /// a separate core.
        /// </summary>
        /// <param name="cores"></param>
        public static void Initialize(int cores)
        {
            for (int i = 0; i < cores; ++i)
            {
                Thread t = new Thread(ThreadMain);
                // This system is not designed to play well with others
                t.Priority = ThreadPriority.Highest;
                threads.Add(t);
                t.Start();
            }
        }

        /// <summary>
        /// Indicates to all threads that there is work
        /// to be done.
        /// </summary>
        public static void ReleaseThreads()
        {
            signal.Set();
        }

        /// <summary>
        /// Used to indicate that there is no more work 
        /// to be done, by unsetting the signal. Note: 
        /// will not work if shutting down.
        /// </summary>
        public static void BlockThreads()
        {
            if (!shuttingDown)
                signal.Reset();
        }

        /// <summary>
        /// Returns any tasks queued up to perform, 
        /// or NULL if there is no work. It will reset
        /// the global signal effectively blocking all threads
        /// if there is no more work to be done.
        /// </summary>
        /// <returns></returns>
        public static Task GetTask()
        {
            lock (allTasks)
            {
                if (allTasks.Count == 0)
                {
                    BlockThreads();
                    return null;
                }
                Task t = allTasks.Peek();
                allTasks.Pop();
                return t;
            }
        }

        /// <summary>
        /// Primary function for each thread
        /// </summary>
        public static void ThreadMain()
        {
            while (!shuttingDown)
            {
                // Wait until work is available
                signal.WaitOne();

                // Get an available task
                Task task = GetTask();

                // Note a task might still be null becaue
                // another thread might have gotten to it first
                while (task != null)
                {
                    // Do the work
                    task();

                    // Get the next task
                    task = GetTask();
                }
            }
        }

        /// <summary>
        /// Distributes work across a number of threads equivalent to the number 
        /// of cores. All tasks will be run on the available cores. 
        /// </summary>
        /// <param name="localTasks"></param>
        public static void DistributeWork(List<Task> localTasks)
        {
            // Create a list of handles indicating what the main thread should wait for
            WaitHandle[] handles = new WaitHandle[localTasks.Count];

            lock (allTasks)
            {
                // Iterate over the list of localTasks, creating a new task that 
                // will signal when it is done.
                for (int i = 0; i < localTasks.Count; ++i)
                {
                    Task t = localTasks[i];

                    // Create an event used to signal that the task is complete
                    ManualResetEvent e = new ManualResetEvent(false);

                    // Create a new signaling task and add it to the list
                    Task signalingTask = () => { t(); e.Set(); };
                    allTasks.Add(signalingTask);

                    // Set the corresponding wait handler 
                    handles[i] = e;
                }
            }

            // Signal to waiting threads that there is work
            ReleaseThreads();

            // Wait until all of the designated work items are completed.
            Semaphore.WaitAll(handles);
        }

        /// <summary>
        /// Indicate to the system that the threads should terminate
        /// and unblock them.
        /// </summary>
        public static void CleanUp()
        {
            shuttingDown = true;
            ReleaseThreads();
        }
    }    
}

我会选择线程池,尽管它有其问题,但微软正在投资改善它,.NET 4似乎会有一个改进的线程池。此时,我认为最好的是使用包装在你自己对象中的线程池,并等待决定你自己的实现。





相关问题
Anyone feel like passing it forward?

I m the only developer in my company, and am getting along well as an autodidact, but I know I m missing out on the education one gets from working with and having code reviewed by more senior devs. ...

NSArray s, Primitive types and Boxing Oh My!

I m pretty new to the Objective-C world and I have a long history with .net/C# so naturally I m inclined to use my C# wits. Now here s the question: I feel really inclined to create some type of ...

C# Marshal / Pinvoke CBitmap?

I cannot figure out how to marshal a C++ CBitmap to a C# Bitmap or Image class. My import looks like this: [DllImport(@"test.dll", CharSet = CharSet.Unicode)] public static extern IntPtr ...

How to Use Ghostscript DLL to convert PDF to PDF/A

How to user GhostScript DLL to convert PDF to PDF/A. I know I kind of have to call the exported function of gsdll32.dll whose name is gsapi_init_with_args, but how do i pass the right arguments? BTW, ...

Linqy no matchy

Maybe it s something I m doing wrong. I m just learning Linq because I m bored. And so far so good. I made a little program and it basically just outputs all matches (foreach) into a label control. ...

热门标签