在Python中对正则表达式使用compile有什么好处吗?
h = re.compile('hello')
h.match('hello world')
vs
re.match('hello', 'hello world')
在Python中对正则表达式使用compile有什么好处吗?
h = re.compile('hello')
h.match('hello world')
vs
re.match('hello', 'hello world')
当前回答
这个答案可能姗姗来迟,但却是一个有趣的发现。如果你打算多次使用regex,使用compile真的可以节省你的时间(这在文档中也有提到)。下面你可以看到,当直接调用match方法时,使用编译后的正则表达式是最快的。将一个编译好的正则表达式传递给re.match会使它更慢,而将re.match与patter字符串传递在中间的某个地方。
>>> ipr = r'\D+((([0-2][0-5]?[0-5]?)\.){3}([0-2][0-5]?[0-5]?))\D+'
>>> average(*timeit.repeat("re.match(ipr, 'abcd100.10.255.255 ')", globals={'ipr': ipr, 're': re}))
1.5077415757028423
>>> ipr = re.compile(ipr)
>>> average(*timeit.repeat("re.match(ipr, 'abcd100.10.255.255 ')", globals={'ipr': ipr, 're': re}))
1.8324008992184038
>>> average(*timeit.repeat("ipr.match('abcd100.10.255.255 ')", globals={'ipr': ipr, 're': re}))
0.9187896518778871
其他回答
有趣的是,编译对我来说确实更有效(Win XP上的Python 2.5.2):
import re
import time
rgx = re.compile('(\w+)\s+[0-9_]?\s+\w*')
str = "average 2 never"
a = 0
t = time.time()
for i in xrange(1000000):
if re.match('(\w+)\s+[0-9_]?\s+\w*', str):
#~ if rgx.match(str):
a += 1
print time.time() - t
按原样运行上述代码一次,并以相反的方式运行两个if行,编译后的正则表达式的速度将提高一倍
抛开性能差异不考虑,使用re.compile和使用编译后的正则表达式对象进行匹配(任何与正则表达式相关的操作)使得Python运行时的语义更加清晰。
我有过调试一些简单代码的痛苦经历:
compare = lambda s, p: re.match(p, s)
然后我用compare in
[x for x in data if compare(patternPhrases, x[columnIndex])]
其中patternPhrases应该是一个包含正则表达式字符串的变量,x[columnIndex]是一个包含字符串的变量。
我有麻烦,patternPhrases不匹配一些预期的字符串!
但是如果我使用re.compile形式:
compare = lambda s, p: p.match(s)
然后在
[x for x in data if compare(patternPhrases, x[columnIndex])]
Python会抱怨“字符串没有匹配属性”,因为在compare中通过位置参数映射,x[columnIndex]被用作正则表达式!其实我的意思是
compare = lambda p, s: p.match(s)
在我的例子中,使用re.compile更明确地表达了正则表达式的目的,当它的值对肉眼隐藏时,因此我可以从Python运行时检查中获得更多帮助。
因此,我这一课的寓意是,当正则表达式不仅仅是字面字符串时,那么我应该使用re.compile让Python帮助我断言我的假设。
Ubuntu 22.04:
$ python --version
Python 3.10.6
$ for x in 1 10 100 1000 10000 100000 1000000; do python -m timeit -n $x -s 'import re' 're.match("[0-9]{3}-[0-9]{3}-[0-9]{4}", "123-123-1234")'; done
1 loop, best of 5: 972 nsec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (186 usec) was more than four times slower than the best time (972 nsec).
10 loops, best of 5: 819 nsec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (13.9 usec) was more than four times slower than the best time (819 nsec).
100 loops, best of 5: 763 nsec per loop
1000 loops, best of 5: 699 nsec per loop
10000 loops, best of 5: 653 nsec per loop
100000 loops, best of 5: 655 nsec per loop
1000000 loops, best of 5: 656 nsec per loop
$ for x in 1 10 100 1000 10000 100000 1000000; do python -m timeit -n $x -s 'import re' 'r = re.compile("[0-9]{3}-[0-9]{3}-[0-9]{4}")' 'r.match("123-123-1234")'; done
1 loop, best of 5: 985 nsec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (134 usec) was more than four times slower than the best time (985 nsec).
10 loops, best of 5: 775 nsec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (13.9 usec) was more than four times slower than the best time (775 nsec).
100 loops, best of 5: 756 nsec per loop
1000 loops, best of 5: 701 nsec per loop
10000 loops, best of 5: 704 nsec per loop
100000 loops, best of 5: 654 nsec per loop
1000000 loops, best of 5: 651 nsec per loop
大多数情况下,是否使用re.compile没有什么区别。在内部,所有函数都是按照编译步骤实现的:
def match(pattern, string, flags=0):
return _compile(pattern, flags).match(string)
def fullmatch(pattern, string, flags=0):
return _compile(pattern, flags).fullmatch(string)
def search(pattern, string, flags=0):
return _compile(pattern, flags).search(string)
def sub(pattern, repl, string, count=0, flags=0):
return _compile(pattern, flags).sub(repl, string, count)
def subn(pattern, repl, string, count=0, flags=0):
return _compile(pattern, flags).subn(repl, string, count)
def split(pattern, string, maxsplit=0, flags=0):
return _compile(pattern, flags).split(string, maxsplit)
def findall(pattern, string, flags=0):
return _compile(pattern, flags).findall(string)
def finditer(pattern, string, flags=0):
return _compile(pattern, flags).finditer(string)
此外,re.compile()绕过了额外的间接和缓存逻辑:
_cache = {}
_pattern_type = type(sre_compile.compile("", 0))
_MAXCACHE = 512
def _compile(pattern, flags):
# internal: compile pattern
try:
p, loc = _cache[type(pattern), pattern, flags]
if loc is None or loc == _locale.setlocale(_locale.LC_CTYPE):
return p
except KeyError:
pass
if isinstance(pattern, _pattern_type):
if flags:
raise ValueError(
"cannot process flags argument with a compiled pattern")
return pattern
if not sre_compile.isstring(pattern):
raise TypeError("first argument must be string or compiled pattern")
p = sre_compile.compile(pattern, flags)
if not (flags & DEBUG):
if len(_cache) >= _MAXCACHE:
_cache.clear()
if p.flags & LOCALE:
if not _locale:
return p
loc = _locale.setlocale(_locale.LC_CTYPE)
else:
loc = None
_cache[type(pattern), pattern, flags] = p, loc
return p
除了使用re.compile带来的小速度好处外,人们还喜欢命名潜在复杂的模式规范并将其与应用的业务逻辑分离所带来的可读性:
#### Patterns ############################################################
number_pattern = re.compile(r'\d+(\.\d*)?') # Integer or decimal number
assign_pattern = re.compile(r':=') # Assignment operator
identifier_pattern = re.compile(r'[A-Za-z]+') # Identifiers
whitespace_pattern = re.compile(r'[\t ]+') # Spaces and tabs
#### Applications ########################################################
if whitespace_pattern.match(s): business_logic_rule_1()
if assign_pattern.match(s): business_logic_rule_2()
注意,另一位受访者错误地认为pyc文件直接存储已编译的模式;然而,在现实中,每次PYC加载时,它们都会被重新构建:
>>> from dis import dis
>>> with open('tmp.pyc', 'rb') as f:
f.read(8)
dis(marshal.load(f))
1 0 LOAD_CONST 0 (-1)
3 LOAD_CONST 1 (None)
6 IMPORT_NAME 0 (re)
9 STORE_NAME 0 (re)
3 12 LOAD_NAME 0 (re)
15 LOAD_ATTR 1 (compile)
18 LOAD_CONST 2 ('[aeiou]{2,5}')
21 CALL_FUNCTION 1
24 STORE_NAME 2 (lc_vowels)
27 LOAD_CONST 1 (None)
30 RETURN_VALUE
上面的分解来自于一个包含tmp.py的PYC文件:
import re
lc_vowels = re.compile(r'[aeiou]{2,5}')
一般来说,我发现在编译模式时使用标志比内联使用标志更容易(至少更容易记住如何使用),比如re.I。
>>> foo_pat = re.compile('foo',re.I)
>>> foo_pat.findall('some string FoO bar')
['FoO']
vs
>>> re.findall('(?i)foo','some string FoO bar')
['FoO']