1. Are you using a StringBuilder to concat the strings rather than just regular string concatenation?
2. Are you pulling back any more columns from the database then you really need to? If so, try not to. Only pull back the column(s) that you need.
3. Are you pulling back any more rows from the database then you really need to? If so, try not to. Only pull back the row(s) that you need.
4. How much memory does the computer have? Is it maxing out when you run the program or getting close to it? Is the processor at the max much or at all? If you re using too much memory then you may need to do more streaming. This means not pulling the whole result set into memory (i.e. a datatable) but reading each line one at a time. It also might mean that rather than concatting the results into a string (or StringBuilder ) that you might need to be appending them to a file so as to not take up so much memory.

linq 语句后面的一栏有一条条款,将第三栏加在变量中。

 string CSVValues = String.Join(",", dtOutput.AsEnumerable()
                                                .Where(a => a[0].ToString() == value)
                                                .Select(b => b[2].ToString()));

步骤1 - 通过剖析仪运行, 保证您在优化时看到正确的东西 。

举个例子,我们有一个问题 我们确信是 缓慢的数据库互动 和当我们运行的剖析器时, db 几乎没有出现。

说到这里,可能要尝试的事情是:

• if you have the memory available, convert the query to a list, this will force a full db read. Otherwise the linq will probably load in chunks doing multiple db queries.
• push the work to the db - if you can create a query than trims down the data you are looking at, or even calculates the string for you, that might be faster
• if this is something where the query is run often but the data rarely changes, consider copying the data to a local db (eg. sqlite) if you re using a remote db.
• if you re using the local sql-server, try sqlite, it s faster for many things.

var value = dataTable
            .AsEnumerable()
            .Select(row => row.Field<string>("columnName"));

var colValueStr = string.join(",", value.ToArray());

尝试用表达式在表格中添加一个假列。 类似 :

DataColumn dynColumn = new DataColumn();

{
    dynColumn.ColumnName = "FullName";
    dynColumn.DataType = System.Type.GetType("System.String");
    dynColumn.Expression = "LastName+   -ABC";
}
UserDataSet.Tables(0).Columns.Add(dynColumn);

在您的代码中稍后部分, 您可以用这个假列代替。 您不需要旋转任何循环来连接字符串 。

Try using parallel for loop.. Here s the sample code..

Parallel.ForEach(dataTable.AsEnumerable(),
            item => { str += ((item as DataRow)["ColumnName"]).ToString(); });

我将工作分割成小块, 让每块由它自己的线索处理 。 您可以通过更改 nthread 编号来细调线条数 。 用不同的数字来尝试它, 这样您就可以看到性能的差别 。

private string getListOfFileNames(DataTable listWithFileNames)
{
    string whereClause = String.Empty;

    if (listWithFileNames.Columns.Contains("Filename"))
    {
        int nthreads = 8; // You can play with this parameter to fine tune and get your best time.
        int load = listWithFileNames.Rows.Count / nthreads; // This will tell how many items reach thread mush process.

        List<ManualResetEvent> mres = new List<ManualResetEvent>(); // This guys will help the method to know when the work is done.
        List<StringBuilder> sbuilders = new List<StringBuilder>(); // This will be used to concatenate each bis string.

        for (int i = 0; i < nthreads; i++)
        {
            sbuilders.Add(new StringBuilder()); // Create a new string builder
            mres.Add(new ManualResetEvent(false)); // Create a not singaled ManualResetEvent.

            if (i == 0) // We know were to put the very begining of your where clause
            {
                sbuilders[0].Append("where filename in (");
            }

            // Calculate the last item to be processed by the current thread
            int end = i == (nthreads - 1) ? listWithFileNames.Rows.Count : i * load + load;

            // Create a new thread to deal with a part of the big table.
            Thread t = new Thread(new ParameterizedThreadStart((x) =>
            {
                // This is the inside of the thread, we must unbox the parameters
                object[] vars = x as object[];
                int lIndex = (int)vars[0];
                int uIndex = (int)vars[1];
                ManualResetEvent ev = vars[2] as ManualResetEvent;
                StringBuilder sb = vars[3] as StringBuilder;
                bool coma = false;

                // Concatenate the rows in the string builder
                for (int j = lIndex; j < uIndex; j++)
                {
                    if (coma)
                    {
                        sb.Append(", ");
                    }
                    else
                    {
                        coma = true;
                    }

                    sb.Append(" ").Append(listWithFileNames.Rows[j]["Filename"]).Append(" ");
                }

                // Tell the parent Thread that your job is done.
                ev.Set();
            }));

            // Start the thread with the calculated params
            t.Start(new object[] { i * load, end, mres[i], sbuilders[i] });
        }

        // Wait for all child threads to finish their job
        WaitHandle.WaitAll(mres.ToArray());

        // Concatenate the big string.
        for (int i = 1; i < nthreads; i++)
        {
            sbuilders[0].Append(", ").Append(sbuilders[i]);
        }

        sbuilders[0].Append(")"); // Close your where clause

        // Return the finished where clause
        return sbuilders[0].ToString();
    }

    // Returns empty
    return whereClause;
}