我想用一个或条件来过滤我的数据帧,以保持特定列的值超出范围[-0.25,0.25]的行。我尝试了:
df = df[(df['col'] < -0.25) or (df['col'] > 0.25)]
但我得到了错误:
级数的真值不明确。使用a.empty、a.bool()、a.item()、.any()或.all()
我想用一个或条件来过滤我的数据帧,以保持特定列的值超出范围[-0.25,0.25]的行。我尝试了:
df = df[(df['col'] < -0.25) or (df['col'] > 0.25)]
但我得到了错误:
级数的真值不明确。使用a.empty、a.bool()、a.item()、.any()或.all()
当前回答
我在熊猫数据框架中工作时也遇到过同样的问题。
我使用过:numpy.logical_and:
在这里,我试图选择Id与41d7853匹配且degree_type不与Certification匹配的行。
如下所示:
display(df_degrees.loc[np.logical_and(df_degrees['person_id'] == '41d7853' , df_degrees['degree_type'] !='Certification')])
如果我尝试编写如下代码:
display(df_degrees.loc[df_degrees['person_id'] == '41d7853' and df_degrees['degree_type'] !='Certification'])
我们将得到错误:
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
我使用了numpy.logical_,它对我很有用。
其他回答
我在这个命令中遇到了一个错误:
if df != '':
pass
但当我把它改成这样时,它起了作用:
if df is not '':
pass
或者,也可以使用操作员模块。更多详细信息请参见Python文档:
import operator
import numpy as np
import pandas as pd
np.random.seed(0)
df = pd.DataFrame(np.random.randn(5,3), columns=list('ABC'))
df.loc[operator.or_(df.C > 0.25, df.C < -0.25)]
A B C
0 1.764052 0.400157 0.978738
1 2.240893 1.867558 -0.977278
3 0.410599 0.144044 1.454274
4 0.761038 0.121675 0.4438
一件小事,浪费了我的时间。
将条件(如果使用“=”,“!=”进行比较)放在括号中。未能做到这一点也会引发这种例外。
这将起作用:
df[(some condition) conditional operator (some conditions)]
这不会:
df[some condition conditional-operator some condition]
我将尝试给出三种最常见的方法的基准(上面也提到过):
from timeit import repeat
setup = """
import numpy as np;
import random;
x = np.linspace(0,100);
lb, ub = np.sort([random.random() * 100, random.random() * 100]).tolist()
"""
stmts = 'x[(x > lb) * (x <= ub)]', 'x[(x > lb) & (x <= ub)]', 'x[np.logical_and(x > lb, x <= ub)]'
for _ in range(3):
for stmt in stmts:
t = min(repeat(stmt, setup, number=100_000))
print('%.4f' % t, stmt)
print()
结果:
0.4808 x[(x > lb) * (x <= ub)]
0.4726 x[(x > lb) & (x <= ub)]
0.4904 x[np.logical_and(x > lb, x <= ub)]
0.4725 x[(x > lb) * (x <= ub)]
0.4806 x[(x > lb) & (x <= ub)]
0.5002 x[np.logical_and(x > lb, x <= ub)]
0.4781 x[(x > lb) * (x <= ub)]
0.4336 x[(x > lb) & (x <= ub)]
0.4974 x[np.logical_and(x > lb, x <= ub)]
但是,熊猫系列不支持*,NumPy Array比熊猫数据帧快(大约慢1000倍,见数字):
from timeit import repeat
setup = """
import numpy as np;
import random;
import pandas as pd;
x = pd.DataFrame(np.linspace(0,100));
lb, ub = np.sort([random.random() * 100, random.random() * 100]).tolist()
"""
stmts = 'x[(x > lb) & (x <= ub)]', 'x[np.logical_and(x > lb, x <= ub)]'
for _ in range(3):
for stmt in stmts:
t = min(repeat(stmt, setup, number=100))
print('%.4f' % t, stmt)
print()
结果:
0.1964 x[(x > lb) & (x <= ub)]
0.1992 x[np.logical_and(x > lb, x <= ub)]
0.2018 x[(x > lb) & (x <= ub)]
0.1838 x[np.logical_and(x > lb, x <= ub)]
0.1871 x[(x > lb) & (x <= ub)]
0.1883 x[np.logical_and(x > lb, x <= ub)]
注意:添加一行代码x=x.to_numpy()大约需要20µs。
对于喜欢%timeit的人:
import numpy as np
import random
lb, ub = np.sort([random.random() * 100, random.random() * 100]).tolist()
lb, ub
x = pd.DataFrame(np.linspace(0,100))
def asterik(x):
x = x.to_numpy()
return x[(x > lb) * (x <= ub)]
def and_symbol(x):
x = x.to_numpy()
return x[(x > lb) & (x <= ub)]
def numpy_logical(x):
x = x.to_numpy()
return x[np.logical_and(x > lb, x <= ub)]
for i in range(3):
%timeit asterik(x)
%timeit and_symbol(x)
%timeit numpy_logical(x)
print('\n')
结果:
23 µs ± 3.62 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
35.6 µs ± 9.53 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
31.3 µs ± 8.9 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
21.4 µs ± 3.35 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
21.9 µs ± 1.02 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
21.7 µs ± 500 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
25.1 µs ± 3.71 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
36.8 µs ± 18.3 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
28.2 µs ± 5.97 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
我在熊猫数据框架中工作时也遇到过同样的问题。
我使用过:numpy.logical_and:
在这里,我试图选择Id与41d7853匹配且degree_type不与Certification匹配的行。
如下所示:
display(df_degrees.loc[np.logical_and(df_degrees['person_id'] == '41d7853' , df_degrees['degree_type'] !='Certification')])
如果我尝试编写如下代码:
display(df_degrees.loc[df_degrees['person_id'] == '41d7853' and df_degrees['degree_type'] !='Certification'])
我们将得到错误:
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
我使用了numpy.logical_,它对我很有用。