我有一个我正在使用Travis-CI的requirements.txt文件。在requirements.txt和setup.py中复制需求似乎很愚蠢,所以我希望在setuptools.setup中传递一个文件句柄给install_requires kwarg。

这可能吗?如果是,我该怎么做呢?

这是我的requirements.txt文件:

guessit>=0.5.2
tvdb_api>=1.8.2
hachoir-metadata>=1.3.3
hachoir-core>=1.3.3
hachoir-parser>=1.3.4

当前回答

从表面上看,requirements.txt和setup.py似乎是愚蠢的副本,但重要的是要理解,虽然形式相似,但预期的功能非常不同。

当指定依赖项时,包作者的目标是说“无论你在哪里安装这个包,为了使这个包工作,这些都是你需要的其他包。”

相反,部署作者(在不同的时间可能是同一个人)有不同的工作,因为他们说“这是我们收集和测试的包的列表,我现在需要安装”。

包作者为各种各样的场景编写程序,因为他们将自己的工作以他们可能不知道的方式使用,并且无法知道将在他们的包旁边安装哪些包。为了成为一个好邻居并避免与其他包的依赖版本冲突,它们需要指定尽可能广泛的依赖版本。这就是setup.py中的install_required所做的。

The deployment author writes for a very different, very specific goal: a single instance of an installed application or service, installed on a particular computer. In order to precisely control a deployment, and be sure that the right packages are tested and deployed, the deployment author must specify the exact version and source-location of every package to be installed, including dependencies and dependency's dependencies. With this spec, a deployment can be repeatably applied to several machines, or tested on a test machine, and the deployment author can be confident that the same packages are deployed every time. This is what a requirements.txt does.

So you can see that, while they both look like a big list of packages and versions, these two things have very different jobs. And it's definitely easy to mix this up and get it wrong! But the right way to think about this is that requirements.txt is an "answer" to the "question" posed by the requirements in all the various setup.py package files. Rather than write it by hand, it's often generated by telling pip to look at all the setup.py files in a set of desired packages, find a set of packages that it thinks fits all the requirements, and then, after they're installed, "freeze" that list of packages into a text file (this is where the pip freeze name comes from).

所以结论是:

setup.py should declare the loosest possible dependency versions that are still workable. Its job is to say what a particular package can work with. requirements.txt is a deployment manifest that defines an entire installation job, and shouldn't be thought of as tied to any one package. Its job is to declare an exhaustive list of all the necessary packages to make a deployment work. Because these two things have such different content and reasons for existing, it's not feasible to simply copy one into the other.

引用:

来自Python打包用户指南的install_requires vs Requirements文件。

其他回答

如果你不想强迫你的用户安装pip,你可以模仿它的行为:

import sys

from os import path as p

try:
    from setuptools import setup, find_packages
except ImportError:
    from distutils.core import setup, find_packages


def read(filename, parent=None):
    parent = (parent or __file__)

    try:
        with open(p.join(p.dirname(parent), filename)) as f:
            return f.read()
    except IOError:
        return ''


def parse_requirements(filename, parent=None):
    parent = (parent or __file__)
    filepath = p.join(p.dirname(parent), filename)
    content = read(filename, parent)

    for line_number, line in enumerate(content.splitlines(), 1):
        candidate = line.strip()

        if candidate.startswith('-r'):
            for item in parse_requirements(candidate[2:].strip(), filepath):
                yield item
        else:
            yield candidate

setup(
...
    install_requires=list(parse_requirements('requirements.txt'))
)

另一个parse_requirements黑客也将环境标记解析为extras_require:

from collections import defaultdict
from pip.req import parse_requirements

requirements = []
extras = defaultdict(list)
for r in parse_requirements('requirements.txt', session='hack'):
    if r.markers:
        extras[':' + str(r.markers)].append(str(r.req))
    else:
        requirements.append(str(r.req))

setup(
    ...,
    install_requires=requirements,
    extras_require=extras
)

它应该同时支持sdist和binary dist。

正如其他人所说,parse_requirements有几个缺点,所以这不是您在公共项目中应该做的,但对于内部/个人项目可能足够了。

需求文件使用扩展的pip格式,这只在您需要用更强的约束来补充setup.py时有用,例如指定某些依赖项必须来自的确切url,或者pip freeze的输出将整个包集冻结为已知的工作版本。如果不需要额外的约束,则只使用setup.py。如果你觉得你真的需要发布一个requirements.txt,你可以让它变成一行:

.

它将是有效的,并准确地引用同一目录下的setup.py的内容。

我是这样做的:

import re

def requirements(filename):
    with open(filename) as f:
        ll = f.read().splitlines()
    d = {}
    for l in ll:
        k, v = re.split(r'==|>=', l)
        d[k] = v
    return d

def packageInfo():
    try:
        from pip._internal.operations import freeze
    except ImportError:
        from pip.operations import freeze

    d = {}
    for kv in freeze.freeze():
        k, v = re.split(r'==|>=', kv)
        d[k] = v
    return d

req = getpackver('requirements.txt')
pkginfo = packageInfo()

for k, v in req.items():
    print(f'{k:<16}: {v:<6} -> {pkginfo[k]}')

你可以把它翻转过来,在setup.py中列出依赖项,并只有一个字符——一个点。-在requirements.txt中。


或者,即使不建议,仍然可以使用以下hack(用pip 9.0.1测试)解析requirements.txt文件(如果它没有通过URL引用任何外部需求):

install_reqs = parse_requirements('requirements.txt', session='hack')

但是这并不能过滤环境标记。


在pip的旧版本中,特别是6.0以上的版本中,可以使用一个公共API来实现这一点。需求文件可以包含注释(#),也可以包含一些其他文件(——requirement或-r)。因此,如果你真的想解析一个requirements.txt文件,你可以使用pip解析器:

from pip.req import parse_requirements

# parse_requirements() returns generator of pip.req.InstallRequirement objects
install_reqs = parse_requirements(<requirements_path>)

# reqs is a list of requirement
# e.g. ['django==1.5.1', 'mezzanine==1.4.6']
reqs = [str(ir.req) for ir in install_reqs]

setup(
    ...
    install_requires=reqs
)