请求中的URL超过了最大重试次数

我试图获得应用商店>业务的内容:

import requests
from lxml import html

page = requests.get("https://itunes.apple.com/in/genre/ios-business/id6000?mt=8")
tree = html.fromstring(page.text)

flist = []
plist = []
for i in range(0, 100):
    app = tree.xpath("//div[@class='column first']/ul/li/a/@href")
    ap = app[0]
    page1 = requests.get(ap)

当我尝试(0,2)的范围，它工作，但当我把范围在100，它显示这个错误:

Traceback (most recent call last):
  File "/home/preetham/Desktop/eg.py", line 17, in <module>
    page1 = requests.get(ap)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='itunes.apple.com', port=443): Max retries exceeded with url: /in/app/adobe-reader/id469337564?mt=8 (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

当前回答

即使在安装pyopenssl和尝试各种python版本后，我也无法在Windows上工作(而它在mac上工作得很好)，所以我切换到urllib，它可以在python 3.6(从python .org)和3.7 (anaconda)上工作

import urllib 
from urllib.request import urlopen
html = urlopen("http://pythonscraping.com/pages/page1.html")
contents = html.read()
print(contents)

2019-09-27 02:10:10

其他回答

import urllib 
from urllib.request import urlopen
html = urlopen("http://pythonscraping.com/pages/page1.html")
contents = html.read()
print(contents)

2019-09-27 02:10:10

只需要导入时间并补充:

time.sleep(6)

在for循环的某个地方，以避免在短时间内向服务器发送太多请求。数字6意味着:6秒。从1开始测试数字，直到达到有助于避免问题的最小秒数。

2021-02-09 22:57:37

补充我自己的经验，为那些在未来正在经历这种情况的人。我的具体错误是

Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'

事实证明，这实际上是因为我已经达到了系统上打开的文件的最大数量。它与失败的连接无关，甚至与所指出的DNS错误无关。

2019-08-22 22:07:08

首先我运行run.py文件，然后我运行unit_test.py文件，它为我工作

2022-10-12 10:47:51

当我在编写selenium浏览器测试脚本时，在使用JS api调用之前调用driver.quit()时遇到了这个错误。记住，放弃网络驱动是最不应该做的事情!

2019-08-05 17:34:50

请求中的URL超过了最大重试次数

推荐文章

最新文章

标签