正如我们已经在问题和发布的答案中发现的那样，这里有一个名为“蒸馏网络”的反网络抓取和机器人检测服务。根据该公司CEO的采访:

尽管他们可以创造新的机器人，但我们找到了识别的方法 硒是他们使用的一个工具，所以我们阻止硒不 不管他们在机器人上迭代了多少次。我们现在正在做 使用Python和许多不同的技术。一旦我们发现了规律 从一种机器人中脱颖而出，然后我们对其进行逆向工程 他们使用的技术并将其识别为恶意的。

要了解他们究竟是如何检测硒的，还需要时间和更多的挑战，但目前我们可以肯定地说:

it's not related to the actions you take with Selenium. Once you navigate to the site, you get immediately detected and banned. I've tried to add artificial random delays between actions, take a pause after the page is loaded - nothing helped it's not about browser fingerprint either. I tried it in multiple browsers with clean profiles and not, incognito modes, but nothing helped since, according to the hint in the interview, this was "reverse engineering", I suspect this is done with some JavaScript code being executed in the browser revealing that this is a browser automated via Selenium WebDriver

我决定把它作为一个答案，因为很明显:

当你用chromedriver使用selenium时，网站能检测到吗?

Yes.

Also, I haven't experimented with older Selenium and older browser versions. In theory, there could be something implemented/added to Selenium at a certain point that Distil Networks bot detector currently relies on. Then, if this is the case, we might detect (yeah, let's detect the detector) at what point/version a relevant change was made, look into changelog and changesets and, may be, this could give us more information on where to look and what is it they use to detect a webdriver-powered browser. It's just a theory that needs to be tested.

随着硒隐身的可用性，逃避硒驱动的ChromeDriver启动谷歌chrome浏览上下文的检测已经变得更加容易。

selenium-stealth

selenium-stealth是一个Python包，用于防止被探测。这个程序试图使python selenium更加隐秘。然而，目前硒隐身只支持硒铬。

目前硒隐身可以提供的特性:

具有隐身功能的硒隐身通过了所有公共机器人测试。 用硒隐身硒可以做到谷歌账号登录。 硒隐身有助于保持正常的reCAPTCHA v3评分

安装

硒隐身在PyPI上可用，因此您可以按照以下方式安装pip:

pip install selenium-stealth

Selenium4兼容代码

Code Block: from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service from selenium_stealth import stealth options = Options() options.add_argument("start-maximized") # Chrome is controlled by automated test software options.add_experimental_option("excludeSwitches", ["enable-automation"]) options.add_experimental_option('useAutomationExtension', False) s = Service('C:\\BrowserDrivers\\chromedriver.exe') driver = webdriver.Chrome(service=s, options=options) # Selenium Stealth settings stealth(driver, languages=["en-US", "en"], vendor="Google Inc.", platform="Win32", webgl_vendor="Intel Inc.", renderer="Intel Iris OpenGL Engine", fix_hairline=True, ) driver.get("https://bot.sannysoft.com/") Browser Screenshot:

Tl;博士

你可以在下面找到一些相关的详细讨论:

当你用chromedriver使用Selenium时，网站能检测到吗? 如何自动登录到一个网站，这是检测我的尝试登录使用硒隐身 未检测到的Chromedriver没有正确加载

我所要做的就是

my_options = webdriver.ChromeOptions()
my_options.add_argument( '--disable-blink-features=AutomationControlled' )

Some more information to this: This relates to website skyscanner.com. In the past I have been able to scrape it. Yes, it did detect the browser automation and it gave me a captcha to press and hold a button. I used to be able to complete the captcha manually, then search flights and then scrape. But this time around after completing the captcha I get the same captcha again and again, just can't seem to escape from it. I tried some of the most popular suggestions to avoid automation being detected, but they didn't work. Then I found this article which did work, and by process of elimination I found out it only took the option above to get around their browser automation detection. Now I don't even get the captcha and everything else seems to be working normally.

我目前正在运行的版本:

操作系统:Windows 7 64位 win32上的Python 3.8.0 (tags/v3.8.0:fa919fd, 2019-10-14) (MSC v.1916 64位(AMD64)) 浏览器:Chrome版本100.0.4896.60(官方 构建)(64位) 4.1.3硒 ChromeDriver 100.0.4896.60 chromedriver_win32.zip 930ff33ae8babeaa74e0dd1ce1dae7ff

我所看到的机器人检测似乎更复杂，或者至少与我在下面阅读的答案不同。

实验1

我从Python控制台用Selenium打开浏览器和网页。 鼠标已经在一个特定的位置，我知道一旦页面加载，就会出现一个链接。我从不移动鼠标。 我按一次鼠标左键(这对于将焦点从运行Python的控制台转移到浏览器是必要的)。 我再次按下鼠标左键(记住，光标在给定链接的上方)。 链接正常打开，就像它应该的那样。

实验2

像以前一样，我从Python控制台打开一个浏览器和带有Selenium的web页面。 这一次，我不再使用鼠标单击，而是使用Selenium(在Python控制台中)单击带有随机偏移量的同一元素。 链接没有打开，但我被带到一个注册页面。

影响

通过Selenium打开网页浏览器并不妨碍我看起来像人类 像人类一样移动鼠标并不一定要被归类为人类 通过Selenium点击带有偏移量的内容仍然会引起警报

这看起来很神秘，但我猜他们可以确定一个操作是否起源于Selenium，而他们并不关心浏览器本身是否是通过Selenium打开的。或者他们能确定窗口是否有焦点吗?如果有人有什么见解，我会很有趣。

它适用于一些网站，从导航器中删除属性webdriver

from selenium import webdriver
driver = webdriver.Chrome()
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
    "source":
        "const newProto = navigator.__proto__;"
        "delete newProto.webdriver;"
        "navigator.__proto__ = newProto;"
    })

我也有同样的问题，用下面的配置解决了这个问题(用c#)

options.AddArguments("start-maximized");
options.AddArguments("--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36");

options.AddExcludedArgument("enable-automation"); // For hiding chrome being controlled by automation..
options.AddAdditionalCapability("useAutomationExtension", false);

// Import cookies
options.AddArguments("user-data-dir=" + userDataDir);

options.AddArguments("profile-directory=" + profileDir);