小心parse_requirements行为!

请注意，pip.req。Parse_requirements将把下划线改为破折号。这件事让我生气了几天，后来我才发现。示例演示:

from pip.req import parse_requirements  # tested with v.1.4.1

reqs = '''
example_with_underscores
example-with-dashes
'''

with open('requirements.txt', 'w') as f:
    f.write(reqs)

req_deps = parse_requirements('requirements.txt')
result = [str(ir.req) for ir in req_deps if ir.req is not None]
print result

生产

['example-with-underscores', 'example-with-dashes']

你可以把它翻转过来，在setup.py中列出依赖项，并只有一个字符——一个点。-在requirements.txt中。

或者，即使不建议，仍然可以使用以下hack(用pip 9.0.1测试)解析requirements.txt文件(如果它没有通过URL引用任何外部需求):

install_reqs = parse_requirements('requirements.txt', session='hack')

但是这并不能过滤环境标记。

在pip的旧版本中，特别是6.0以上的版本中，可以使用一个公共API来实现这一点。需求文件可以包含注释(#)，也可以包含一些其他文件(——requirement或-r)。因此，如果你真的想解析一个requirements.txt文件，你可以使用pip解析器:

from pip.req import parse_requirements

# parse_requirements() returns generator of pip.req.InstallRequirement objects
install_reqs = parse_requirements(<requirements_path>)

# reqs is a list of requirement
# e.g. ['django==1.5.1', 'mezzanine==1.4.6']
reqs = [str(ir.req) for ir in install_reqs]

setup(
    ...
    install_requires=reqs
)

虽然这不是问题的确切答案，但我推荐Donald Stufft在https://caremad.io/2013/07/setup-vs-requirement/上的博客文章，它很好地回答了这个问题。我用它取得了巨大的成功。

简而言之，requirements.txt不是setup.py的替代品，而是部署的补充。在setup.py中保持包依赖关系的适当抽象。设置requirements.txt或更多的文件，以获取用于开发、测试或生产的软件包依赖关系的特定版本。

例如，在deps/下的回购中包含包:

# fetch specific dependencies
--no-index
--find-links deps/

# install package
# NOTE: -e . for editable mode
.

PIP执行包的setup.py并安装在install_requires中声明的依赖项的特定版本。没有表里不一，这两件文物的用途都被保留了下来。

小心parse_requirements行为!

请注意，pip.req。Parse_requirements将把下划线改为破折号。这件事让我生气了几天，后来我才发现。示例演示:

from pip.req import parse_requirements  # tested with v.1.4.1

reqs = '''
example_with_underscores
example-with-dashes
'''

with open('requirements.txt', 'w') as f:
    f.write(reqs)

req_deps = parse_requirements('requirements.txt')
result = [str(ir.req) for ir in req_deps if ir.req is not None]
print result

生产

['example-with-underscores', 'example-with-dashes']

以下接口在pip 10中已弃用:

from pip.req import parse_requirements
from pip.download import PipSession

所以我把它转换成简单的文本解析:

with open('requirements.txt', 'r') as f:
    install_reqs = [
        s for s in [
            line.split('#', 1)[0].strip(' \t\n') for line in f
        ] if s != ''
    ]

我为此创建了一个可重用函数。它实际上解析需求文件的整个目录，并将它们设置为extras_require。

最新消息请访问:https://gist.github.com/akatrevorjay/293c26fefa24a7b812f5

import glob
import itertools
import os

# This is getting ridiculous
try:
    from pip._internal.req import parse_requirements
    from pip._internal.network.session import PipSession
except ImportError:
    try:
        from pip._internal.req import parse_requirements
        from pip._internal.download import PipSession
    except ImportError:
        from pip.req import parse_requirements
        from pip.download import PipSession


def setup_requirements(
        patterns=[
            'requirements.txt', 'requirements/*.txt', 'requirements/*.pip'
        ],
        combine=True):
    """
    Parse a glob of requirements and return a dictionary of setup() options.
    Create a dictionary that holds your options to setup() and update it using this.
    Pass that as kwargs into setup(), viola

    Any files that are not a standard option name (ie install, tests, setup) are added to extras_require with their
    basename minus ext. An extra key is added to extras_require: 'all', that contains all distinct reqs combined.

    Keep in mind all literally contains `all` packages in your extras.
    This means if you have conflicting packages across your extras, then you're going to have a bad time.
    (don't use all in these cases.)

    If you're running this for a Docker build, set `combine=True`.
    This will set `install_requires` to all distinct reqs combined.

    Example:

    >>> import setuptools
    >>> _conf = dict(
    ...     name='mainline',
    ...     version='0.0.1',
    ...     description='Mainline',
    ...     author='Trevor Joynson <github@trevor.joynson,io>',
    ...     url='https://trevor.joynson.io',
    ...     namespace_packages=['mainline'],
    ...     packages=setuptools.find_packages(),
    ...     zip_safe=False,
    ...     include_package_data=True,
    ... )
    >>> _conf.update(setup_requirements())
    >>> # setuptools.setup(**_conf)

    :param str pattern: Glob pattern to find requirements files
    :param bool combine: Set True to set install_requires to extras_require['all']
    :return dict: Dictionary of parsed setup() options
    """
    session = PipSession()

    # Handle setuptools insanity
    key_map = {
        'requirements': 'install_requires',
        'install': 'install_requires',
        'tests': 'tests_require',
        'setup': 'setup_requires',
    }
    ret = {v: set() for v in key_map.values()}
    extras = ret['extras_require'] = {}
    all_reqs = set()

    files = [glob.glob(pat) for pat in patterns]
    files = itertools.chain(*files)

    for full_fn in files:
        # Parse
        reqs = {
            str(r.req)
            for r in parse_requirements(full_fn, session=session)
            # Must match env marker, eg:
            #   yarl ; python_version >= '3.0'
            if r.match_markers()
        }
        all_reqs.update(reqs)

        # Add in the right section
        fn = os.path.basename(full_fn)
        barefn, _ = os.path.splitext(fn)
        key = key_map.get(barefn)

        if key:
            ret[key].update(reqs)
            extras[key] = reqs

        extras[barefn] = reqs

    if 'all' not in extras:
        extras['all'] = list(all_reqs)

    if combine:
        extras['install'] = ret['install_requires']
        ret['install_requires'] = list(all_reqs)

    def _listify(dikt):
        ret = {}

        for k, v in dikt.items():
            if isinstance(v, set):
                v = list(v)
            elif isinstance(v, dict):
                v = _listify(v)
            ret[k] = v

        return ret

    ret = _listify(ret)

    return ret


__all__ = ['setup_requirements']

if __name__ == '__main__':
    reqs = setup_requirements()
    print(reqs)