如何用熊猫将一列字典拆分为单独的列?

pd.DataFrame(df['val'].tolist())是分解字典列的规范方法

这是用彩色图表证明的。

基准测试代码供参考。

请注意，我只是计算爆炸的时间，因为这是回答这个问题最有趣的部分——结果构造的其他方面(例如是否使用pop或drop)与讨论无关，可以忽略(但是应该注意的是，使用pop避免了后续的drop调用，因此最终的解决方案性能更好一些，但我们仍然在列出列并将其传递给pd。DataFrame)。

此外，pop破坏性地改变了输入DataFrame，使得在基准测试代码中运行变得更加困难，因为基准测试代码假设输入在测试运行中没有改变。

对其他解决方案的批评

df['val'].apply(pd.Series) is extremely slow for large N as pandas constructs Series objects for each row, then proceeds to construct a DataFrame from them. For larger N the performance dips to the order of minutes or hours. pd.json_normalize(df['val'])) is slower simply because json_normalize is meant to work with a much more complex input data - particularly deeply nested JSON with multiple record paths and metadata. We have a simple flat dict for which pd.DataFrame suffices, so use that if your dicts are flat. Some answers suggest df.pop('val').values.tolist() or df.pop('val').to_numpy().tolist(). I don't think it makes much of a difference whether you listify the series or the numpy array. It's one operation less to listify the series directly and really isn't slower so I'd recommend avoiding generating the numpy array in the intermediate step.

你可以用pop + tolist来使用join。性能与使用drop + tolist的concat相当，但有些人可能会发现这样的语法更干净:

res = df.join(pd.DataFrame(df.pop('b').tolist()))

使用其他方法进行基准测试:

df = pd.DataFrame({'a':[1,2,3], 'b':[{'c':1}, {'d':3}, {'c':5, 'd':6}]})

def joris1(df):
    return pd.concat([df.drop('b', axis=1), df['b'].apply(pd.Series)], axis=1)

def joris2(df):
    return pd.concat([df.drop('b', axis=1), pd.DataFrame(df['b'].tolist())], axis=1)

def jpp(df):
    return df.join(pd.DataFrame(df.pop('b').tolist()))

df = pd.concat([df]*1000, ignore_index=True)

%timeit joris1(df.copy())  # 1.33 s per loop
%timeit joris2(df.copy())  # 7.42 ms per loop
%timeit jpp(df.copy())     # 7.68 ms per loop

我已经在一个方法中连接了这些步骤，你只需要传递数据帧和包含字典的列来展开:

def expand_dataframe(dw: pd.DataFrame, column_to_expand: str) -> pd.DataFrame:
    """
    dw: DataFrame with some column which contain a dict to expand
        in columns
    column_to_expand: String with column name of dw
    """
    import pandas as pd

    def convert_to_dict(sequence: str) -> Dict:
        import json
        s = sequence
        json_acceptable_string = s.replace("'", "\"")
        d = json.loads(json_acceptable_string)
        return d    

    expanded_dataframe = pd.concat([dw.drop([column_to_expand], axis=1),
                                    dw[column_to_expand]
                                    .apply(convert_to_dict)
                                    .apply(pd.Series)],
                                    axis=1)
    return expanded_dataframe

如何用熊猫将一列字典拆分为单独的列?

pd.DataFrame(df['val'].tolist())是分解字典列的规范方法

这是用彩色图表证明的。

基准测试代码供参考。

请注意，我只是计算爆炸的时间，因为这是回答这个问题最有趣的部分——结果构造的其他方面(例如是否使用pop或drop)与讨论无关，可以忽略(但是应该注意的是，使用pop避免了后续的drop调用，因此最终的解决方案性能更好一些，但我们仍然在列出列并将其传递给pd。DataFrame)。

此外，pop破坏性地改变了输入DataFrame，使得在基准测试代码中运行变得更加困难，因为基准测试代码假设输入在测试运行中没有改变。

对其他解决方案的批评

df['val'].apply(pd.Series) is extremely slow for large N as pandas constructs Series objects for each row, then proceeds to construct a DataFrame from them. For larger N the performance dips to the order of minutes or hours. pd.json_normalize(df['val'])) is slower simply because json_normalize is meant to work with a much more complex input data - particularly deeply nested JSON with multiple record paths and metadata. We have a simple flat dict for which pd.DataFrame suffices, so use that if your dicts are flat. Some answers suggest df.pop('val').values.tolist() or df.pop('val').to_numpy().tolist(). I don't think it makes much of a difference whether you listify the series or the numpy array. It's one operation less to listify the series directly and really isn't slower so I'd recommend avoiding generating the numpy array in the intermediate step.

要将字符串转换为实际的字典，可以执行df['污染物级别'].map(eval)。然后，可以使用下面的解决方案将字典转换为不同的列。

举个小例子，你可以使用.apply(pd.Series):

In [2]: df = pd.DataFrame({'a':[1,2,3], 'b':[{'c':1}, {'d':3}, {'c':5, 'd':6}]})

In [3]: df
Out[3]:
   a                   b
0  1           {u'c': 1}
1  2           {u'd': 3}
2  3  {u'c': 5, u'd': 6}

In [4]: df['b'].apply(pd.Series)
Out[4]:
     c    d
0  1.0  NaN
1  NaN  3.0
2  5.0  6.0

为了将它与数据框架的其余部分结合起来，你可以将其他列与上面的结果连接起来:

In [7]: pd.concat([df.drop(['b'], axis=1), df['b'].apply(pd.Series)], axis=1)
Out[7]:
   a    c    d
0  1  1.0  NaN
1  2  NaN  3.0
2  3  5.0  6.0

使用您的代码，如果我省略iloc部分，这也可以工作:

In [15]: pd.concat([df.drop('b', axis=1), pd.DataFrame(df['b'].tolist())], axis=1)
Out[15]:
   a    c    d
0  1  1.0  NaN
1  2  NaN  3.0
2  3  5.0  6.0

my_df = pd.DataFrame.from_dict(my_dict, orient='index'， columns=['my_col'])

. .将正确地解析字典(将每个字典键放入单独的df列，键值放入df行)，因此字典将不会首先被压缩到单个列中。