Question

I m trying to write a function that reads files from a "deferred" directory which contains files that contain lists. Here s what the files in the deferred folder contain:

 173378981 ,  45000 ,  343434 ,  3453453 ,  34534545 ,  3452342 ,  234234 ,  42063008 ,  Exempted ,  10000 
 1000014833 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0 
 1000009598 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0 
 279483421 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0 
 1000009600 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0 
 389453080 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0 
 1000009602 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  0 ,  Exempted ,  0

The function used to write the file(s):

def storeDeferredRecords(records):
    """docstring for createFile"""
    now = datetime.datetime.now()
    filename = deferredDir +  /  + now.strftime("%Y%m%d-%H%M%S")
    f = open(filename,  w )
    newlist = map(lambda(x): str(x)[1:-1], records)
    for item in newlist:
        f.write("%s
" % item)
    f.close

I need help with the function used to read the file. I was only able to write this:

def getDeferredRecords():
        """docstring for getDeferredRecords"""
        infiles = [infile for infile in glob.glob(deferredDir +  /* )]
                <code to read the contents of each file here>

Can someone help me out? I need to read the lines and insert them into a list. This list will then be merged with records from separate CSV file.

Answer 1

See the csv module:

BigList = []
for filename in glob.glob(deferredDir +  /* ):
    PartList = csv.reader(open(filename))
    BigList.extend(PartList)

Is that what you had in mind?

Answer 2

First, the last line in the store function needs to be like this f.close()

Your store function saves the values in a newline-separated manner. To read all the files, should be enough:

def getDeferredRecords():
    """docstring for getDeferredRecords"""
    return dict((infile, list(iter(file(infile)))) 
                     for infile in glob.glob(deferredDir +  /* ))

Explanation: a file is iterable, so you can do for line in file: print line for example. With list(iter(file)) you have the lines of a file in a list. dict((a, b) for a, b in foo) returns a dictionary with {a: b} pairs. The return value of the function is a dictionary with the format {filename: list_of_lines_in_file}. Keep in mind that the list elements are strings with a trailing newline.

Answer 3

The Python cvs module is likely a good answer:
http://docs.python.org/library/csv.html

Question:

glob.glob() returns an iterable already, so I do not see the point here...

[infile for infile in glob.glob(deferredDir +  /* )]

Rather:

BigList = []
for filename in glob.glob(deferredDir +  /* ):
    #CVS read code here
    #add to BigList

Food for thought.

Answer 4

Incorporating ideas from Tim Pietzcker, here are the re-written functions:

def storeDeferredRecords(records):
    """docstring for createFile"""
    now = datetime.datetime.now()
    filename = deferredDir +  /  + now.strftime("%Y%m%d-%H%M%S")
    f = csv.writer(open(filename,  w ), delimiter= , )
    f.writerows(records)

def getDeferredRecords():
    """docstring for getDeferredRecords"""
    for filename in glob.glob(deferredDir +  /* ):
        def_records = csv.reader(open(filename, r ))
        records.extend(def_records)

I used csv.writer instead of using the previous code block:

f = open(filename,  w )
newlist = map(lambda(x): str(x)[1:-1], records)
for item in newlist:
        f.write("%s
" % item)
f.close

Thanks to all those who replied!

友情链接