对称差分

如果你只对其中一个数据帧中的行感兴趣，而不是两个数据帧中的行，你在寻找集的差异:

pd.concat([df1,df2]).drop_duplicates(keep=False)

⚠️只有在两个数据帧都不包含任何重复的情况下才有效。

设置差分/关系代数差分

如果你对关系代数差异/集差异感兴趣，即df1-df2或df1\df2:

pd.concat([df1,df2,df2]).drop_duplicates(keep=False) 

⚠️只有在两个数据帧都不包含任何重复的情况下才有效。

通过使用drop_duplicate

pd.concat([df1,df2]).drop_duplicates(keep=False)

更新:

上面的方法只适用于那些本身没有副本的数据帧。例如:

df1=pd.DataFrame({'A':[1,2,3,3],'B':[2,3,4,4]})
df2=pd.DataFrame({'A':[1],'B':[2]})

它将输出如下所示，这是错误的

错误输出:

pd.concat([df1, df2]).drop_duplicates(keep=False)
Out[655]: 
   A  B
1  2  3

正确的输出

Out[656]: 
   A  B
1  2  3
2  3  4
3  3  4

如何实现这一目标?

方法一:将isin与tuple结合使用

df1[~df1.apply(tuple,1).isin(df2.apply(tuple,1))]
Out[657]: 
   A  B
1  2  3
2  3  4
3  3  4

方法二:与指标合并

df1.merge(df2,indicator = True, how='left').loc[lambda x : x['_merge']!='both']
Out[421]: 
   A  B     _merge
1  2  3  left_only
2  3  4  left_only
3  3  4  left_only

我在处理副本时遇到了问题，当一边有副本，另一边至少有一个副本时，所以我使用了Counter。集合做一个更好的差异，确保双方有相同的计数。这不会返回副本，但如果双方有相同的计数，则不会返回任何副本。

from collections import Counter

def diff(df1, df2, on=None):
    """
    :param on: same as pandas.df.merge(on) (a list of columns)
    """
    on = on if on else df1.columns
    df1on = df1[on]
    df2on = df2[on]
    c1 = Counter(df1on.apply(tuple, 'columns'))
    c2 = Counter(df2on.apply(tuple, 'columns'))
    c1c2 = c1-c2
    c2c1 = c2-c1
    df1ondf2on = pd.DataFrame(list(c1c2.elements()), columns=on)
    df2ondf1on = pd.DataFrame(list(c2c1.elements()), columns=on)
    df1df2 = df1.merge(df1ondf2on).drop_duplicates(subset=on)
    df2df1 = df2.merge(df2ondf1on).drop_duplicates(subset=on)
    return pd.concat([df1df2, df2df1])

> df1 = pd.DataFrame({'a': [1, 1, 3, 4, 4]})
> df2 = pd.DataFrame({'a': [1, 2, 3, 4, 4]})
> diff(df1, df2)
   a
0  1
0  2

pandas DataFrame.compare中有一种新的方法，即比较2个不同的dataframe，并返回数据记录中每列中变化的值。

例子

第一个Dataframe

Id Customer Status      Date
1      ABC   Good  Mar 2023
2      BAC   Good  Feb 2024
3      CBA    Bad  Apr 2022

第二个Dataframe

Id Customer Status      Date
1      ABC    Bad  Mar 2023
2      BAC   Good  Feb 2024
5      CBA   Good  Apr 2024

比较Dataframes

print("Dataframe difference -- \n")
print(df1.compare(df2))

print("Dataframe difference keeping equal values -- \n")
print(df1.compare(df2, keep_equal=True))

print("Dataframe difference keeping same shape -- \n")
print(df1.compare(df2, keep_shape=True))

print("Dataframe difference keeping same shape and equal values -- \n")
print(df1.compare(df2, keep_shape=True, keep_equal=True))

结果

Dataframe difference -- 

    Id       Status            Date          
  self other   self other      self     other
0  NaN   NaN   Good   Bad       NaN       NaN
2  3.0   5.0    Bad  Good  Apr 2022  Apr 2024

Dataframe difference keeping equal values -- 

    Id       Status            Date          
  self other   self other      self     other
0    1     1   Good   Bad  Mar 2023  Mar 2023
2    3     5    Bad  Good  Apr 2022  Apr 2024

Dataframe difference keeping same shape -- 

    Id       Customer       Status            Date          
  self other     self other   self other      self     other
0  NaN   NaN      NaN   NaN   Good   Bad       NaN       NaN
1  NaN   NaN      NaN   NaN    NaN   NaN       NaN       NaN
2  3.0   5.0      NaN   NaN    Bad  Good  Apr 2022  Apr 2024

Dataframe difference keeping same shape and equal values -- 

    Id       Customer       Status            Date          
  self other     self other   self other      self     other
0    1     1      ABC   ABC   Good   Bad  Mar 2023  Mar 2023
1    2     2      BAC   BAC   Good  Good  Feb 2024  Feb 2024
2    3     5      CBA   CBA    Bad  Good  Apr 2022  Apr 2024

nice @liangli的解决方案略有变化，不需要改变现有数据框架的索引:

newdf = df1.drop(df1.join(df2.set_index('Name').index))

试试这个: Df_new = df1。merge(df2, how='outer'， indicator=True)。查询('_merge == "left_only"')。下降(_merge, 1)

它将产生一个新的数据框架，其差异是:df1中存在的值，而df2中不存在。