English 中文(简体)
chu数字数据
原标题:Enumerate DataTable in chunks

我有一个数据表,填充了360 000英亩的KQ数据。 然而,这个问题已进入OMOM系统。

然而,这是我对于如何在最近1 000次交接之后处理日益老化的问题没有把握的。 或者说,有更好的办法

int catchInt = 0;
string combineWhereClause = string.Empty;

for (int i = 0; i < ThousandLoopTable.Rows.Count; i++)
{
    catchInt++;
    combineWhereClause = combineWhereClause + 
                        " , " + 
                         ThousandLoopTable.Rows[i].ItemArray[0].ToString();

    if (catchInt >= 1000)
    {
        catchInt = 0;
        combineWhereClause = combineWhereClause.TrimStart(   );
        combineWhereClause = combineWhereClause.TrimStart( , );

        Directory.CreateDirectory(ExportDirectory);
        SQLProcessing.SQLProcessor.MasterSqlConnection = 
            SQLProcessing.SQLProcessor.OpenMasterSqlConnection(SQLServer);
        DataTable dtTable = 
            SQLProcessing.SQLProcessor.QueryDataTable(sql_selectionquery);
        for (int m = 0; m < dtTable.Rows.Count; m++)
        {
            string FileName = dtTable.Rows[m].ItemArray[0].ToString() + ".txt";
            string OCR = dtTable.Rows[m].ItemArray[1].ToString();
            File.AppendAllText(ExportDirectory + "\" + FileName, OCR);

        }

        combineWhereClause = string.Empty;
    }
}

例如,如果有3120个牢房,这将达到3 000个,但不会达到最后120个。 然而,我不敢确定如何处理最后120人,因为我实际上不想这样做,就象我这样做吗?

最佳回答

处理数据的方法可能更好(见其他答复),但我认为这是你目前做法所需要的:

Don t reset 每一批中均采用。 相反,把它作为<条码>1,让它作为整个运行的对照。 然后将<代码>if改为:

if (catchInt % 1000 == 0 || catchInt == ThousandLoopTable.Rows.Count)
{
    // Execute your batch
}

http://msdn.microsoft.com/en-us/library/0w4e0fzs%28vs.90%29.aspx”rel=“nofollow” Modulus营运人,在<代码>上识别 斜线/代码可分为1 000。

问题回答

为了避免出现外部例外,可以遵循简单规则:

<><0>

  1. Work with whole dataset
  2. Load whole dataset in memory
  3. Execute long-runing transactional code that blocks server side database

http://www.ohchr.org。

  1. Work with small chunks of data
  2. Load small chunk of data (pagination patter)
  3. Run non-blocking server code
  4. If anything can be done on the server, let it do the work

确保你关于服务器的数据不易变(没有人在改变数据)。 如果无法保证,你可能需要重新思考你的架构,并使用处理数据的问题和补充表格。





相关问题
Anyone feel like passing it forward?

I m the only developer in my company, and am getting along well as an autodidact, but I know I m missing out on the education one gets from working with and having code reviewed by more senior devs. ...

NSArray s, Primitive types and Boxing Oh My!

I m pretty new to the Objective-C world and I have a long history with .net/C# so naturally I m inclined to use my C# wits. Now here s the question: I feel really inclined to create some type of ...

C# Marshal / Pinvoke CBitmap?

I cannot figure out how to marshal a C++ CBitmap to a C# Bitmap or Image class. My import looks like this: [DllImport(@"test.dll", CharSet = CharSet.Unicode)] public static extern IntPtr ...

How to Use Ghostscript DLL to convert PDF to PDF/A

How to user GhostScript DLL to convert PDF to PDF/A. I know I kind of have to call the exported function of gsdll32.dll whose name is gsapi_init_with_args, but how do i pass the right arguments? BTW, ...

Linqy no matchy

Maybe it s something I m doing wrong. I m just learning Linq because I m bored. And so far so good. I made a little program and it basically just outputs all matches (foreach) into a label control. ...

热门标签