Do child processes spawned via multiprocessing share objects created earlier in the program?

这取决于情况。对于全局只读变量，通常可以考虑这样做（除了消耗的内存），否则不应该。

multiprocessing的文档说：

继承比腌制/解腌更好。

On Windows many types from multiprocessing need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead you should arrange the program so that a process which need access to a shared resource created elsewhere can inherit it from an ancestor process.

明确地向子进程传递资源。

On Unix a child process can make use of a shared resource created in a parent process using a global resource. However, it is better to pass the object as an argument to the constructor for the child process.

Apart from making the code (potentially) compatible with Windows this also ensures that as long as the child process is still alive the object will not be garbage collected in the parent process. This might be important if some resource is freed when the object is garbage collected in the parent process.

全局变量

Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that Process.start() was called.

Example

在Windows上（单个CPU）：

#!/usr/bin/env python
import os, sys, time
from multiprocessing import Pool

x = 23000 # replace `23` due to small integers share representation
z = []    # integers are immutable, let s try mutable object

def printx(y):
    global x
    if y == 3:
       x = -x
    z.append(y)
    print os.getpid(), x, id(x), z, id(z) 
    print y
    if len(sys.argv) == 2 and sys.argv[1] == "sleep":
       time.sleep(.1) # should make more apparant the effect

if __name__ ==  __main__ :
    pool = Pool(processes=4)
    pool.map(printx, (1,2,3,4))

带有sleep：

$ python26 test_share.py sleep
2504 23000 11639492 [1] 10774408
1
2564 23000 11639492 [2] 10774408
2
2504 -23000 11639384 [1, 3] 10774408
3
4084 23000 11639492 [4] 10774408
4

没有 睡眠:

$ python26 test_share.py
1148 23000 11639492 [1] 10774408
1
1148 23000 11639492 [1, 2] 10774408
2
1148 -23000 11639324 [1, 2, 3] 10774408
3
1148 -23000 11639324 [1, 2, 3, 4] 10774408
4

"S.Lott先生是正确的。Python的多进程快捷方式会有效地为您提供一个独立的、重复的大块内存。"

在大多数*nix系统中，使用较低级别的调用os.fork()实际上会给您提供写时复制的内存，这可能是您所考虑的。据我所知，在理论上，即使是最简单的程序，您也可以从该数据中读取而不需要重复。

然而，在Python解释器中，情况并不是那么简单。对象数据和元数据存储在同一内存段中，因此，即使对象从未更改，例如对象的引用计数被增加，也会导致内存写入，从而导致副本。几乎任何执行比“打印hello”更多操作的Python程序都会导致引用计数增加，因此您可能永远无法意识到写时复制的好处。

即使有人成功地在Python中破解了共享内存解决方案，尝试在进程间协调垃圾回收也可能会非常痛苦。

如果你在Unix下运行，它们可能会共享同一对象，因为fork是如何工作的（即子进程有单独的内存，但它是写时复制的，因此只要没有人修改它，它就可以共享）。我尝试了以下内容：

import multiprocessing

x = 23

def printx(y):
    print x, id(x)
    print y

if __name__ ==  __main__ :
    pool = multiprocessing.Pool(processes=4)
    pool.map(printx, (1,2,3,4))

并获得以下输出：

$ ./mtest.py
23 22995656
1
23 22995656
2
23 22995656
3
23 22995656
4

当然，这并不能证明没有副本被制作，但您应该能够通过查看ps的输出来验证每个子进程使用多少实际内存，在您的情况下。

不同的进程有不同的地址空间，就像运行不同的解释器实例一样。这就是IPC（进程间通信）的作用。

你可以使用队列或管道来实现这个目的。如果你想在以后分布到网络上，你也可以使用TCP上的RPC。

将此翻译为中文：http://docs.python.org/dev/library/multiprocessing.html#exchanging-objects-between-processes http://docs.python.org/dev/library/multiprocessing.html#exchanging-objects-between-processes

不是直接与多处理相关，但从您的示例中，似乎可以只使用shelve模块或类似的东西。 “big_lookup_object”是否真的必须完全在内存中？

不行，但您可以将数据作为子进程加载，并允许其与其他子进程共享数据，请参见下文。

import time
import multiprocessing

def load_data( queue_load, n_processes )

    ... load data here into some_variable

    """
    Store multiple copies of the data into
    the data queue. There needs to be enough
    copies available for each process to access. 
    """

    for i in range(n_processes):
        queue_load.put(some_variable)


def work_with_data( queue_data, queue_load ):

    # Wait for load_data() to complete
    while queue_load.empty():
        time.sleep(1)

    some_variable = queue_load.get()

    """
    ! Tuples can also be used here
    if you have multiple data files
    you wish to keep seperate.  
    a,b = queue_load.get()
    """

    ... do some stuff, resulting in new_data

    # store it in the queue
    queue_data.put(new_data)


def start_multiprocess():

    n_processes = 5

    processes = []
    stored_data = []

    # Create two Queues
    queue_load = multiprocessing.Queue()
    queue_data = multiprocessing.Queue()

    for i in range(n_processes):

        if i == 0:
            # Your big data file will be loaded here...
            p = multiprocessing.Process(target = load_data,
            args=(queue_load, n_processes))

            processes.append(p)
            p.start()   

        # ... and then it will be used here with each process
        p = multiprocessing.Process(target = work_with_data,
        args=(queue_data, queue_load))

        processes.append(p)
        p.start()

    for i in range(n_processes)
        new_data = queue_data.get()
        stored_data.append(new_data)    

    for p in processes:
        p.join()
    print(processes)