Question

我有一份最初由英特尔·艾因斯汀生公司Python3.7执行的文章。

该书使用“<代码>多处理图书馆。

我从英特尔的猪肉升至英特尔的猪肉。

守则通常可以执行,由于在Python3.7和Python3.9之间变化,我有错误。

I can find a workaround for python3.9 by adding mp.set_start_method("fork") at the top :

import multiprocessing as mp
mp.set_start_method("fork")

but there are significant differences for results of the code between python3.7 and python3.9 ( around 20% of differences for numerical values ) : is it normal ?

这两种版本之间是否取得了相同的结果?

我认为,这个问题来自“<条码>多处理<>/代码”,但我不知道如何用python3.9处理。

Here is an example of the part of code snippet using multiprocessing package for python3.7 :

#Modules import.
import sys
import numpy as np
import scipy.integrate as pyint
import os
from os import path
import glob
from scipy.interpolate import CubicSpline
import multiprocessing as mp
mp.set_start_method("fork")
from multiprocessing import Pool

def integ(I1):
    #The Pobs(k,mu) is duplicated and rolled on the right, lower and right-lower directions to sum each elements 
    function_A = aux_fun_LU(way, ecs, I1[0], I1[1])*delta_x*delta_y
    function_A_10 = np.roll(function_A, -1, axis = 1)
    function_A_01 = np.roll(function_A, -1, axis = 0)
    function_A_11 = np.roll(function_A_01, -1, axis = 1)

    function_A = function_A[0:-1, 0:-1]
    function_A_10 = function_A_10[0:-1, 0:-1]
    function_A_01 = function_A_01[0:-1, 0:-1]
    function_A_11 = function_A_11[0:-1, 0:-1]

    #Integral computation.
    integrale_A = np.sum(function_A + function_A_10 + function_A_01 + function_A_11)/4

    #The integral is saved into the temporar files.
    file=open( tmp_F/LU_table_NL.txt , a )
    file.write(str(I1[0]%N_notRD_params) +     +  str(I1[1]%N_notRD_params) +     + str("%.12e" % integrale_A))
    file.write(str( 
 ))

    return integrale_A

#Function that yields over two index (equivalent to double for loop) for parallel computing.
def g():
    for j in range(N_notRD_params*i, N_notRD_params*i+N_notRD_params):
        for l in range(j, N_notRD_params*i+N_notRD_params):
            yield j, l

#Pool map function for parallel computing.
if __name__ ==  __main__ :
    pool = mp.Pool(12)
    pool.map(integ, g())
    pool.terminate()

因此,我不得不在进口头上添加 for3.9:

import multiprocessing as mp
mp.set_start_method("fork")
from multiprocessing import Pool

But results are really bad with this version for python3.9. I have got a difference of 20% in numerical values in final results.

Is there a way to get the same results than with python3.7 ?

Answer 1

这比回答更重要,但太长了,不能作为评论。

#1) In the function integ, I notice you use way, ecs, delta_x and delta_y without them ever being initialized. Where are these being initialized? Can you verify that they have the same values both times?

#2) Not really your question, but the way you re writing to a file is really bad and is likely to cause race conditions. Having every thread try to write to a common file is a bad idea. A better bet would be to get rid of the writes in integ and just have it instead return integrale_A, I1

You can then write:

with open( ...... ,  w ) as file:
     pool = mp.Pool(12)
     for integral_A, I1 in pool.map(.....):
         file.write(......);
     pool.terminate()

友情链接