小心parse_requirements行为!

请注意，pip.req。Parse_requirements将把下划线改为破折号。这件事让我生气了几天，后来我才发现。示例演示:

from pip.req import parse_requirements  # tested with v.1.4.1

reqs = '''
example_with_underscores
example-with-dashes
'''

with open('requirements.txt', 'w') as f:
    f.write(reqs)

req_deps = parse_requirements('requirements.txt')
result = [str(ir.req) for ir in req_deps if ir.req is not None]
print result

生产

['example-with-underscores', 'example-with-dashes']

另一个parse_requirements黑客也将环境标记解析为extras_require:

from collections import defaultdict
from pip.req import parse_requirements

requirements = []
extras = defaultdict(list)
for r in parse_requirements('requirements.txt', session='hack'):
    if r.markers:
        extras[':' + str(r.markers)].append(str(r.req))
    else:
        requirements.append(str(r.req))

setup(
    ...,
    install_requires=requirements,
    extras_require=extras
)

它应该同时支持sdist和binary dist。

正如其他人所说，parse_requirements有几个缺点，所以这不是您在公共项目中应该做的，但对于内部/个人项目可能足够了。

我为此创建了一个可重用函数。它实际上解析需求文件的整个目录，并将它们设置为extras_require。

最新消息请访问:https://gist.github.com/akatrevorjay/293c26fefa24a7b812f5

import glob
import itertools
import os

# This is getting ridiculous
try:
    from pip._internal.req import parse_requirements
    from pip._internal.network.session import PipSession
except ImportError:
    try:
        from pip._internal.req import parse_requirements
        from pip._internal.download import PipSession
    except ImportError:
        from pip.req import parse_requirements
        from pip.download import PipSession


def setup_requirements(
        patterns=[
            'requirements.txt', 'requirements/*.txt', 'requirements/*.pip'
        ],
        combine=True):
    """
    Parse a glob of requirements and return a dictionary of setup() options.
    Create a dictionary that holds your options to setup() and update it using this.
    Pass that as kwargs into setup(), viola

    Any files that are not a standard option name (ie install, tests, setup) are added to extras_require with their
    basename minus ext. An extra key is added to extras_require: 'all', that contains all distinct reqs combined.

    Keep in mind all literally contains `all` packages in your extras.
    This means if you have conflicting packages across your extras, then you're going to have a bad time.
    (don't use all in these cases.)

    If you're running this for a Docker build, set `combine=True`.
    This will set `install_requires` to all distinct reqs combined.

    Example:

    >>> import setuptools
    >>> _conf = dict(
    ...     name='mainline',
    ...     version='0.0.1',
    ...     description='Mainline',
    ...     author='Trevor Joynson <github@trevor.joynson,io>',
    ...     url='https://trevor.joynson.io',
    ...     namespace_packages=['mainline'],
    ...     packages=setuptools.find_packages(),
    ...     zip_safe=False,
    ...     include_package_data=True,
    ... )
    >>> _conf.update(setup_requirements())
    >>> # setuptools.setup(**_conf)

    :param str pattern: Glob pattern to find requirements files
    :param bool combine: Set True to set install_requires to extras_require['all']
    :return dict: Dictionary of parsed setup() options
    """
    session = PipSession()

    # Handle setuptools insanity
    key_map = {
        'requirements': 'install_requires',
        'install': 'install_requires',
        'tests': 'tests_require',
        'setup': 'setup_requires',
    }
    ret = {v: set() for v in key_map.values()}
    extras = ret['extras_require'] = {}
    all_reqs = set()

    files = [glob.glob(pat) for pat in patterns]
    files = itertools.chain(*files)

    for full_fn in files:
        # Parse
        reqs = {
            str(r.req)
            for r in parse_requirements(full_fn, session=session)
            # Must match env marker, eg:
            #   yarl ; python_version >= '3.0'
            if r.match_markers()
        }
        all_reqs.update(reqs)

        # Add in the right section
        fn = os.path.basename(full_fn)
        barefn, _ = os.path.splitext(fn)
        key = key_map.get(barefn)

        if key:
            ret[key].update(reqs)
            extras[key] = reqs

        extras[barefn] = reqs

    if 'all' not in extras:
        extras['all'] = list(all_reqs)

    if combine:
        extras['install'] = ret['install_requires']
        ret['install_requires'] = list(all_reqs)

    def _listify(dikt):
        ret = {}

        for k, v in dikt.items():
            if isinstance(v, set):
                v = list(v)
            elif isinstance(v, dict):
                v = _listify(v)
            ret[k] = v

        return ret

    ret = _listify(ret)

    return ret


__all__ = ['setup_requirements']

if __name__ == '__main__':
    reqs = setup_requirements()
    print(reqs)

你可以把它翻转过来，在setup.py中列出依赖项，并只有一个字符——一个点。-在requirements.txt中。

或者，即使不建议，仍然可以使用以下hack(用pip 9.0.1测试)解析requirements.txt文件(如果它没有通过URL引用任何外部需求):

install_reqs = parse_requirements('requirements.txt', session='hack')

但是这并不能过滤环境标记。

在pip的旧版本中，特别是6.0以上的版本中，可以使用一个公共API来实现这一点。需求文件可以包含注释(#)，也可以包含一些其他文件(——requirement或-r)。因此，如果你真的想解析一个requirements.txt文件，你可以使用pip解析器:

from pip.req import parse_requirements

# parse_requirements() returns generator of pip.req.InstallRequirement objects
install_reqs = parse_requirements(<requirements_path>)

# reqs is a list of requirement
# e.g. ['django==1.5.1', 'mezzanine==1.4.6']
reqs = [str(ir.req) for ir in install_reqs]

setup(
    ...
    install_requires=reqs
)

下面是一个完整的hack(用pip 9.0.1测试)，基于Romain的回答，解析requirements.txt，并根据当前的环境标记进行过滤:

from pip.req import parse_requirements

requirements = []
for r in parse_requirements('requirements.txt', session='hack'):
    # check markers, such as
    #
    #     rope_py3k    ; python_version >= '3.0'
    #
    if r.match_markers():
        requirements.append(str(r.req))

print(requirements)

使用parse_requirements是有问题的，因为pip API没有公开的文档和支持。在pip 1.6中，该函数实际上是移动的，因此对它的现有使用可能会中断。

消除setup.py和requirements.txt之间重复的一个更可靠的方法是在setup.py中指定你的依赖项，然后输入-e。到requirements.txt文件中。关于为什么这是一种更好的方式，来自pip开发人员的一些信息可以在这里找到:https://caremad.io/blog/setup-vs-requirement/