How-to include package data

Question

在使用装置时,我不能让安装机在任何> Pack_data。档案。我所说的一切都说,下面是这样做的正确方式。是否有人?

setup(
   name= myapp ,
   packages=find_packages(),
   package_data={
       myapp : [ data/*.txt ],
   },
   include_package_data=True,
   zip_safe=False,
   install_requires=[ distribute ],
)

<代码>myapp/data/ 是数据文档的位置。

Answer 1

I realize that this is an old question, but for people finding their way here via Google: package_data is a low-down, dirty lie. It is only used when building binary packages (python setup.py bdist ...) but not when building source packages (python setup.py sdist ...). This is, of course, ridiculous -- one would expect that building a source distribution would result in a collection of files that could be sent to someone else to built the binary distribution.

In any case, using MANIFEST.in will work both for binary and for source distributions.

Answer 2

I just had this same issue. The solution, was simply to remove include_package_data=True.

After reading here, I realized that include_package_data aims to include files from version control, as opposed to merely "include package data" as the name implies. From the docs:

The data files [of include_package_data] must be under CVS or Subversion control

...

If you want finer-grained control over what files are included (for example, if you have documentation files in your package directories and want to exclude them from installation), then you can also use the package_data keyword.

Taking that argument out fixed it, which is coincidentally why it also worked when you switched to distutils, since it doesn t take that argument.

Answer 3

Following @Joe s recommendation to remove the include_package_data=True line also worked for me.

To elaborate a bit more, I have no MANIFEST.in file. I use Git and not CVS.

Repository takes this kind of shape:

/myrepo
    - .git/
    - setup.py
    - myproject
        - __init__.py
        - some_mod
            - __init__.py
            - animals.py
            - rocks.py
        - config
            - __init__.py
            - settings.py
            - other_settings.special
            - cool.huh
            - other_settings.xml
        - words
            - __init__.py
            word_set.txt

<代码>up.py:

from setuptools import setup, find_packages
import os.path

setup (
    name= myproject ,
    version = "4.19",
    packages = find_packages(),  
    # package_dir={ mypkg :  src/mypkg },  # didnt use this.
    package_data = {
        # If any package contains *.txt or *.rst files, include them:
          : [ *.txt ,  *.xml ,  *.special ,  *.huh ],
    },

#
    # Oddly enough, include_package_data=True prevented package_data from working.
    # include_package_data=True, # Commented out.
    data_files=[
#               ( bitmaps , [ bm/b1.gif ,  bm/b2.gif ]),
        ( /opt/local/myproject/etc , [ myproject/config/settings.py ,  myproject/config/other_settings.special ]),
        ( /opt/local/myproject/etc , [os.path.join( myproject/config ,  cool.huh )]),
#
        ( /opt/local/myproject/etc , [os.path.join( myproject/config ,  other_settings.xml )]),
        ( /opt/local/myproject/data , [os.path.join( myproject/words ,  word_set.txt )]),
    ],

    install_requires=[  jsonschema ,
         logging , ],

     entry_points = {
         console_scripts : [
            # Blah...
        ], },
)

I run python setup.py sdist for a source distrib (haven t tried binary).

And when inside of a brand new virtual environment, I have a myproject-4.19.tar.gz, file, and I use

(venv) pip install ~/myproject-4.19.tar.gz
...

And other than everything getting installed to my virtual environment s site-packages, those special data files get installed to /opt/local/myproject/data and /opt/local/myproject/etc.

Answer 4

include_package_data=True worked for me.

如您使用电容器,则记住在<代码>内添加“环形”-git。远低于<代码>Manifest ,或包括<代码> 包装_data中的所有途径(如果是,它带有所有类型的静态)

( pasted the comment I made, as k3-rnc mentioned it s actually helpful as is )

Answer 5

Using setup.cfg (setuptools ≥ 30.3.0)

Starting with setuptools 30.3.0 (released 2016-12-08), you can keep your setup.py very small and move the configuration to a setup.cfg file. With this approach, you could put your package data in an [options.package_data] section:

[options.package_data]
* = *.txt, *.rst
hello = *.msg

在这种情况下,请见<代码>setup.py。简言之:

from setuptools import setup
setup()

详情见.configuring setup using setup.cfgfile。

http://stackoverflow.com/a/56523197/974555 发言setup.cfg, 赞成pyproject.toml,如在上提议,但截至2020-02-21年,这仍然是临时性的。

Answer 6

Update: This answer is old and the information is no longer valid. All setup.py configs should use import setuptools. I ve added a more complete answer at https://stackoverflow.com/a/49501350/64313

I solved this by switching to distutils. Looks like distribute is deprecated and/or broken.

from distutils.core import setup

setup(
   name= myapp ,
   packages=[ myapp ],
   package_data={
       myapp : [ data/*.txt ],
   },
)

Answer 7

I had the same problem for a couple of days but even this thread wasn t able to help me as everything was confusing. So I did my research and found the following solution:

基本上,在这种情况下,你应当:

from setuptools import setup

setup(
   name= myapp ,
   packages=[ myapp ],
   package_dir={ myapp : myapp }, # the one line where all the magic happens
   package_data={
       myapp : [ data/*.txt ],
   },
)

此处的所有其他排流答案如下:

Answer 8


我发现,这一职位同时陷入同样的问题。
My experience contradicts the experiences in the other answers.
include_package_data=True does include the data in the
bdist! The explanation in the setuptools
documentation
lacks context and troubleshooting tips, but
include_package_data works as advertised.
我的发言:

Windows / Cygwin
git version 2.21.0
Python 3.8.1 Windows distribution
setuptools v47.3.1
check-manifest v0.42

Here is my how-to guide.
How-to include package data
Here is the file structure for a project I published on PyPI.
(It installs the application in __main__.py).
├── LICENSE.md
├── MANIFEST.in
├── my_package
│   ├── __init__.py
│   ├── __main__.py
│   └── _my_data          <---- folder with data
│       ├── consola.ttf   <---- data file
│       └── icon.png      <---- data file
├── README.md
└── setup.py

Starting point
Here is a generic starting point for the setuptools.setup() in
setup.py.
setuptools.setup(
    ...
    packages=setuptools.find_packages(),
    ...
)

setuptools.find_packages() includes all of my packages in the
distribution. My only package is my_package.
The sub-folder with my data, _my_data, is not considered a
package by Python because it does not contain an __init__.py,
and so find_packages() does not find it.
A solution often-cited, but incorrect, is to put an empty
__init__.py file in the _my_data folder.
This does make it a package, so it does include the folder
_my_data in the distribution. But the data files inside
_my_data are not included.
因此,将<条码>_my_data输入一个包件does not help。
解决办法是:

the sdist already contains the data files
add include_package_data=True to include the data files in the bdist as well

Experiment (how to test the solution)
有三个步骤使这一试验成为重复试验:
$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python setup.py sdist bdist_wheel

I will break these down step-by-step:

Clean out the old build:

$ rm -fr build/ dist/ my_package.egg-info/


Run check-manifest to be sure MANIFEST.in matches the
Git index of files under version control:

$ check-manifest

If MANIFEST.in does not exist yet, create it from the Git
index of files under version control:
$ check-manifest --create

Here is the MANIFEST.in that is created:
include *.md
recursive-include my_package *.png
recursive-include my_package *.ttf

There is no reason to manually edit this file.
As long as everything that should be under version control is
under version control (i.e., is part of the Git index),
check-manifest --create does the right thing.
注:如果是:

ignored in a .gitignore
excluded in a .git/info/exclude
or simply new files that have not been added to the index yet

And if any files are under version control that should not be
under version control, check-manifest issues a warning and
specifies which files it recommends removing from the Git index.

Build:

$ python setup.py sdist bdist_wheel

Now inspect the sdist (source distribution) and bdist_wheel
(build distribution) to see if they include the data files.
Look at the contents of the sdist (only the relevant lines are
shown below):
$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png    <-- yay!
...

So the sdist already includes the data files because they are
listed in MANIFEST.in. There is nothing extra to do to include
the data files in the sdist.
Look at the contents of the bdist (it is a .zip file, parsed
with zipfile.ZipFile):
$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD

Note: you need to create your own check-whl.py script to produce the
above output. It is just three lines:
from zipfile import ZipFile
path = "dist/my_package-0.0.1a6-py3-none-any.whl" # <-- CHANGE
print( 
 .join(ZipFile(path).namelist()))

如预期,<编码>bdist缺失数据文档。

The _my_data rafter is contributed original.

What if I create a _my_data/__init__.py? I repeat the experiment and I find the data files are still not there! The _my_data/ folder is included but it does not contain the data files!

Solution

Contrary to the experience of others, this does work:

setuptools.setup(
    ...
    packages=setuptools.find_packages(),
    include_package_data=True, # <-- adds data files to bdist
    ...
)

现有试验:

$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python.exe setup.py sdist bdist_wheel

确保<代码>sdist 仍然有数据档案:

$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png    <-- yay!
...

查阅<代码>bdist 的内容:

$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package/_my_data/consola.ttf        <--- yay!
my_package/_my_data/icon.png           <--- yay!
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD

How not to test if data files are included

I recommend troubleshooting/testing using the approach outlined above to inspect the sdist and bdist.

pip install in editable mode is not a valid test

Note: pip install -e . does not show if data files are included in the bdist.

The symbolic link causes the installation to behave as if the data files are included (because they already exist locally on the developer s computer).

After pip install my_package, the data files are in the virtual environment s lib/site-packages/my_package/ folder, using the exact same file structure shown above in the list of the whl contents.

Publishing to TestPyPI is a slow way to test

Publishing to TestPyPI and then installing and looking in lib/site-packages/my_packages is a valid test, but it is too time-consuming.

Answer 9

Ancient question and yet... package management of python really leaves a lot to be desired. So I had the use case of installing using pip locally to a specified directory and was surprised both package_data and data_files paths did not work out. I was not keen on adding yet another file to the repo so I ended up leveraging data_files and setup.py option --install-data; something like this

pip install . --install-option="--install-data=$PWD/package" -t package

Answer 10

Like others in this thread, I m more than a little surprised at the combination of longevity and still a lack of clarity, BUT the best answer for me was using check-manifest as recommended in the answer from @mike-gazes

So, using just a setup.cfg and no setup.py and additional text and python files required in the package, what worked for me was keeping this in setup.cfg:

[options]
packages = find:
include_package_data = true

并更新<代码>MANIFEST.in,依据check-manifest输出:

include *.in
include *.txt
include *.yml
include LICENSE
include tox.ini
recursive-include mypkg *.py
recursive-include mypkg *.txt

Answer 11

Moving the folder containing the package data into to module folder solved the problem for me.

See this question: MANIFEST.in ignored on "python setup.py install" - no data files installed?

Answer 12

Just remove the line:

include_package_data=True,

从您的发言稿中删除。 (刚刚用最新的编织物进行试验)

Answer 13

For a directory structure like:

foo/
├── foo
│   ├── __init__.py
│   ├── a.py
│   └── data.txt
└── setup.py

www.un.org/spanish/ga

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from setuptools import setup


NAME =  foo 
DESCRIPTION =  Test library to check how setuptools works 
URL =  https://none.com 
EMAIL =  gzorp@bzorp.com 
AUTHOR =  KT 
REQUIRES_PYTHON =  >=3.6.0 

setup(
    name=NAME,
    version= 0.0.0 ,
    description=DESCRIPTION,
    author=AUTHOR,
    author_email=EMAIL,
    python_requires=REQUIRES_PYTHON,
    url=URL,
    license= MIT ,
    classifiers=[
         Programming Language :: Python ,
         Programming Language :: Python :: 3 ,
         Programming Language :: Python :: 3.6 ,
    ],
    packages=[ foo ],
    package_data={ foo : [ data.txt ]},
    include_package_data=True,
    install_requires=[],
    extras_require={},
    cmdclass={},
)

python setup.py bdist_wheel works.

Answer 14

从Setuptools 623.0<>/strong>开始,您现在可使用recursive Wildcards (**>) ,包括(次)直截留。这样,你就可以把所有文件夹和档案都包含在其中。

For example, when using a pyproject.toml file, this is how you include two folders recursively:

[tool.setuptools.package-data] "ema_workbench.examples.data" = ["**"] "ema_workbench.examples.models" = ["**"]

But you can also only include certain file-types, in a folder and all subfolders. If you want to include all markdown (.md) files for example:

[tool.setuptools.package-data] "ema_workbench.examples.data" = ["**/*.md"]

它还应当使用<代码>setup.py或setup.cfg。

See https://github.com/pypa/setuptools/pull/3309 for the details.

Using setup.cfg (setuptools ≥ 30.3.0)

How-to include package data

Starting point

Experiment (how to test the solution)

Solution

How not to test if data files are included

pip install in editable mode is not a valid test

Publishing to TestPyPI is a slow way to test

友情链接