Question

I have a file that is about 100Mb that looks like this:

#meta data 1    
skadjflaskdjfasljdfalskdjfl
sdkfjhasdlkgjhsdlkjghlaskdj
asdhfk
#meta data 2
jflaksdjflaksjdflkjasdlfjas
ldaksjflkdsajlkdfj
#meta data 3
alsdkjflasdjkfglalaskdjf

文档中含有一行元数据,数据与数项变量长度数据相对应,仅含有甲型数字特性。将这些数据列入一个简单清单的最佳方式是:

data = [[#meta data 1, skadjflaskdjfasljdfalskdjflsdkfjhasdlkgjhsdlkjghlaskdjasdhfk],
       [#meta data 2, jflaksdjflaksjdflkjasdlfjasldaksjflkdsajlkdfj],
       [#meta data 3, alsdkjflasdjkfglalaskdjf]]

我的初衷是使用<代码>read()方法将整个卷宗读为记忆,然后使用定期表述将数据按预期格式分类。是否有更完善的老路? 所有元数据线从环形开始,所有数据线都是甲型数字。感谢!

Answer 1

itertools.groupby a. 为群体收集线索提供了方便的途径:

import itertools

data=[]
with open( data.txt , r ) as f:
    for key,group in itertools.groupby(f,lambda line: line.startswith( #meta )):
        if key:
            meta=next(group).strip()
        else:
            lines=  .join(group).strip()
            data.append((meta,lines))
print(data)

产量

[( #meta data 1 ,  skadjflaskdjfasljdfalskdjfl
sdkfjhasdlkgjhsdlkjghlaskdj
asdhfk ), ( #meta data 2 ,  jflaksdjflaksjdflkjasdlfjas
ldaksjflkdsajlkdfj ), ( #meta data 3 ,  alsdkjflasdjkfglalaskdjf )]

表达

itertools.groupby(f,lambda line: line.startswith( #meta ))

一位主持人返回。它通过<代码>f的行文,并在每一行打电话<代码>lambda。当它遇到一条从<代码>#meta 开始的线路时,该功能回归代码,否则<代码>False。

<代码>tertools.groupby收集了所有回归同一价值的毗连线。

因此,从<代码>#meta开始的线路放在自己的组别上,随后不从<代码>#meta开始的所有线路都放在下组,等等。

The key is the return value from the lambda function. In this case, it will be either True or False.

Answer 2

我不知道这是否最快,但从我头上看:

data = []
with open( input.file ,  r ) as fp:
    for line in fp:
        line = line.strip()
        if line[0] ==  # :
            data.append((line, []))
        else:
            data[-1][1].append(line)
data = [(X,   .join(Y)) for X, Y in data]

Answer 3

我猜想的是:

result = []
for line in file.readlines():
    if line[0] ==  # :
        result.append([line])
    else:
        if len(result[-1]) == 1:
            result[-1].append(line)
        else:
            result[-1][-1] += line

没有测试。

Answer 4

我很简单,例如:

data = [] # result
lastmeta = None # the last metadata line seen
chunks = [] # lines since the last metadata line
for line in input:
    if line[0] ==  # : # metadata
        if lastmeta: # need to flush data we ve collected
            data.append((lastmeta,   .join(chunks))
        lastmeta = line
    else:
        chunks.append(line)

友情链接