我试图使用scikit-learn的LabelEncoder来编码字符串标签的pandas DataFrame。由于数据帧有许多(50+)列,我想避免为每一列创建一个LabelEncoder对象;我宁愿只有一个大的LabelEncoder对象,它可以跨所有数据列工作。

将整个DataFrame扔到LabelEncoder中会产生以下错误。请记住,我在这里使用的是虚拟数据;实际上,我正在处理大约50列的字符串标记数据,所以需要一个解决方案,不引用任何列的名称。

import pandas
from sklearn import preprocessing 

df = pandas.DataFrame({
    'pets': ['cat', 'dog', 'cat', 'monkey', 'dog', 'dog'], 
    'owner': ['Champ', 'Ron', 'Brick', 'Champ', 'Veronica', 'Ron'], 
    'location': ['San_Diego', 'New_York', 'New_York', 'San_Diego', 'San_Diego', 
                 'New_York']
})

le = preprocessing.LabelEncoder()

le.fit(df)

回溯(最近一次调用): 文件“”,第1行,在 文件"/Users/bbalin/anaconda/lib/python2.7/site-packages/sklearn/预处理/label.py",第103行 y = column_or_1d(y, warn=True) 文件"/Users/bbalin/anaconda/lib/python2.7/site-packages/sklearn/utils/validation.py",第306行,在column_or_1d中 raise ValueError("错误的输入形状{0}".format(形状)) ValueError:错误的输入形状(6,3)

对于如何解决这个问题有什么想法吗?


当前回答

下面是我一次性转换多列的解决方案,以及精确的inverse_transform

from sklearn import preprocessing
columns = ['buying','maint','lug_boot','safety','cls']  # columns names where transform is required
for X in columns:
  exec(f'le_{X} = preprocessing.LabelEncoder()')  #create label encoder with name "le_X", where X is column name
  exec(f'df.{X} = le_{X}.fit_transform(df.{X})')  #execute fit transform for column X with respective lable encoder "le_X", where X is column name
df.head()  # to display transformed results

for X in columns:
  exec(f'df.{X} = le_{X}.inverse_transform(df.{X})')  #execute inverse_transform for column X with respective lable encoder "le_X", where X is column name
df.head() # to display Inverse transformed results of df

其他回答

下面是我一次性转换多列的解决方案,以及精确的inverse_transform

from sklearn import preprocessing
columns = ['buying','maint','lug_boot','safety','cls']  # columns names where transform is required
for X in columns:
  exec(f'le_{X} = preprocessing.LabelEncoder()')  #create label encoder with name "le_X", where X is column name
  exec(f'df.{X} = le_{X}.fit_transform(df.{X})')  #execute fit transform for column X with respective lable encoder "le_X", where X is column name
df.head()  # to display transformed results

for X in columns:
  exec(f'df.{X} = le_{X}.inverse_transform(df.{X})')  #execute inverse_transform for column X with respective lable encoder "le_X", where X is column name
df.head() # to display Inverse transformed results of df

你可以很容易地做到,

df.apply(LabelEncoder().fit_transform)

EDIT2:

在scikit-learn 0.20中,推荐的方法是

OneHotEncoder().fit_transform(df)

因为OneHotEncoder现在支持字符串输入。 使用ColumnTransformer可以只对某些列应用OneHotEncoder。

编辑:

由于这个最初的答案是一年多前的,并获得了许多赞(包括赏金),我可能应该进一步扩展它。

对于inverse_transform和transform,你需要做一点修改。

from collections import defaultdict
d = defaultdict(LabelEncoder)

这样,您现在将所有列LabelEncoder保留为字典。

# Encoding the variable
fit = df.apply(lambda x: d[x.name].fit_transform(x))

# Inverse the encoded
fit.apply(lambda x: d[x.name].inverse_transform(x))

# Using the dictionary to label future data
df.apply(lambda x: d[x.name].transform(x))

MOAR编辑:

使用Neuraxle的flatforeach步骤,也可以在一次对所有平坦数据使用相同的LabelEncoder:

FlattenForEach(LabelEncoder(), then_unflatten=True).fit_transform(df)

要根据数据列使用单独的LabelEncoders,或者如果只有一些数据列需要进行标签编码,而不需要其他数据列,那么使用ColumnTransformer是一种解决方案,它允许对列选择和LabelEncoder实例进行更多控制。

这是一年半后的事实,但我也需要能够。transform()多个熊猫数据帧列一次(以及能够。inverse_transform()他们)。这扩展了上面@PriceHardman的优秀建议:

class MultiColumnLabelEncoder(LabelEncoder):
    """
    Wraps sklearn LabelEncoder functionality for use on multiple columns of a
    pandas dataframe.

    """
    def __init__(self, columns=None):
        self.columns = columns

    def fit(self, dframe):
        """
        Fit label encoder to pandas columns.

        Access individual column classes via indexig `self.all_classes_`

        Access individual column encoders via indexing
        `self.all_encoders_`
        """
        # if columns are provided, iterate through and get `classes_`
        if self.columns is not None:
            # ndarray to hold LabelEncoder().classes_ for each
            # column; should match the shape of specified `columns`
            self.all_classes_ = np.ndarray(shape=self.columns.shape,
                                           dtype=object)
            self.all_encoders_ = np.ndarray(shape=self.columns.shape,
                                            dtype=object)
            for idx, column in enumerate(self.columns):
                # fit LabelEncoder to get `classes_` for the column
                le = LabelEncoder()
                le.fit(dframe.loc[:, column].values)
                # append the `classes_` to our ndarray container
                self.all_classes_[idx] = (column,
                                          np.array(le.classes_.tolist(),
                                                  dtype=object))
                # append this column's encoder
                self.all_encoders_[idx] = le
        else:
            # no columns specified; assume all are to be encoded
            self.columns = dframe.iloc[:, :].columns
            self.all_classes_ = np.ndarray(shape=self.columns.shape,
                                           dtype=object)
            for idx, column in enumerate(self.columns):
                le = LabelEncoder()
                le.fit(dframe.loc[:, column].values)
                self.all_classes_[idx] = (column,
                                          np.array(le.classes_.tolist(),
                                                  dtype=object))
                self.all_encoders_[idx] = le
        return self

    def fit_transform(self, dframe):
        """
        Fit label encoder and return encoded labels.

        Access individual column classes via indexing
        `self.all_classes_`

        Access individual column encoders via indexing
        `self.all_encoders_`

        Access individual column encoded labels via indexing
        `self.all_labels_`
        """
        # if columns are provided, iterate through and get `classes_`
        if self.columns is not None:
            # ndarray to hold LabelEncoder().classes_ for each
            # column; should match the shape of specified `columns`
            self.all_classes_ = np.ndarray(shape=self.columns.shape,
                                           dtype=object)
            self.all_encoders_ = np.ndarray(shape=self.columns.shape,
                                            dtype=object)
            self.all_labels_ = np.ndarray(shape=self.columns.shape,
                                          dtype=object)
            for idx, column in enumerate(self.columns):
                # instantiate LabelEncoder
                le = LabelEncoder()
                # fit and transform labels in the column
                dframe.loc[:, column] =\
                    le.fit_transform(dframe.loc[:, column].values)
                # append the `classes_` to our ndarray container
                self.all_classes_[idx] = (column,
                                          np.array(le.classes_.tolist(),
                                                  dtype=object))
                self.all_encoders_[idx] = le
                self.all_labels_[idx] = le
        else:
            # no columns specified; assume all are to be encoded
            self.columns = dframe.iloc[:, :].columns
            self.all_classes_ = np.ndarray(shape=self.columns.shape,
                                           dtype=object)
            for idx, column in enumerate(self.columns):
                le = LabelEncoder()
                dframe.loc[:, column] = le.fit_transform(
                        dframe.loc[:, column].values)
                self.all_classes_[idx] = (column,
                                          np.array(le.classes_.tolist(),
                                                  dtype=object))
                self.all_encoders_[idx] = le
        return dframe.loc[:, self.columns].values

    def transform(self, dframe):
        """
        Transform labels to normalized encoding.
        """
        if self.columns is not None:
            for idx, column in enumerate(self.columns):
                dframe.loc[:, column] = self.all_encoders_[
                    idx].transform(dframe.loc[:, column].values)
        else:
            self.columns = dframe.iloc[:, :].columns
            for idx, column in enumerate(self.columns):
                dframe.loc[:, column] = self.all_encoders_[idx]\
                    .transform(dframe.loc[:, column].values)
        return dframe.loc[:, self.columns].values

    def inverse_transform(self, dframe):
        """
        Transform labels back to original encoding.
        """
        if self.columns is not None:
            for idx, column in enumerate(self.columns):
                dframe.loc[:, column] = self.all_encoders_[idx]\
                    .inverse_transform(dframe.loc[:, column].values)
        else:
            self.columns = dframe.iloc[:, :].columns
            for idx, column in enumerate(self.columns):
                dframe.loc[:, column] = self.all_encoders_[idx]\
                    .inverse_transform(dframe.loc[:, column].values)
        return dframe.loc[:, self.columns].values

例子:

如果df和df_copy()是混合类型的pandas数据帧,你可以将MultiColumnLabelEncoder()应用到dtype=object列上,方法如下:

# get `object` columns
df_object_columns = df.iloc[:, :].select_dtypes(include=['object']).columns
df_copy_object_columns = df_copy.iloc[:, :].select_dtypes(include=['object']).columns

# instantiate `MultiColumnLabelEncoder`
mcle = MultiColumnLabelEncoder(columns=object_columns)

# fit to `df` data
mcle.fit(df)

# transform the `df` data
mcle.transform(df)

# returns output like below
array([[1, 0, 0, ..., 1, 1, 0],
       [0, 5, 1, ..., 1, 1, 2],
       [1, 1, 1, ..., 1, 1, 2],
       ..., 
       [3, 5, 1, ..., 1, 1, 2],

# transform `df_copy` data
mcle.transform(df_copy)

# returns output like below (assuming the respective columns 
# of `df_copy` contain the same unique values as that particular 
# column in `df`
array([[1, 0, 0, ..., 1, 1, 0],
       [0, 5, 1, ..., 1, 1, 2],
       [1, 1, 1, ..., 1, 1, 2],
       ..., 
       [3, 5, 1, ..., 1, 1, 2],

# inverse `df` data
mcle.inverse_transform(df)

# outputs data like below
array([['August', 'Friday', '2013', ..., 'N', 'N', 'CA'],
       ['April', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['August', 'Monday', '2014', ..., 'N', 'N', 'NJ'],
       ..., 
       ['February', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['April', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['March', 'Tuesday', '2013', ..., 'N', 'N', 'NJ']], dtype=object)

# inverse `df_copy` data
mcle.inverse_transform(df_copy)

# outputs data like below
array([['August', 'Friday', '2013', ..., 'N', 'N', 'CA'],
       ['April', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['August', 'Monday', '2014', ..., 'N', 'N', 'NJ'],
       ..., 
       ['February', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['April', 'Tuesday', '2014', ..., 'N', 'N', 'NJ'],
       ['March', 'Tuesday', '2013', ..., 'N', 'N', 'NJ']], dtype=object)

你可以通过索引访问单独的列类、列标签和用于适合每个列的列编码器:

mcle.all_classes_ mcle.all_encoders_ mcle.all_labels_

主要使用@Alexander回答,但必须做一些更改-

cols_need_mapped = ['col1', 'col2']

mapper = {col: {cat: n for n, cat in enumerate(df[col].astype('category').cat.categories)} 
     for col in df[cols_need_mapped]}

for c in cols_need_mapped :
    df[c] = df[c].map(mapper[c])

然后,为了将来重用,你可以将输出保存到json文档中,当你需要它时,你可以读入并使用.map()函数,就像我上面所做的那样。

我们可以使用scikit learn中的OrdinalEncoder来代替LabelEncoder,它允许多列编码。

将分类特征编码为整数数组。 这个转换器的输入应该是一个类似数组的整数或字符串,表示分类(离散)特征所取的值。特征被转换为序号整数。这将导致每个特性生成一列整数(0到n_categories - 1)。

>>> from sklearn.preprocessing import OrdinalEncoder
>>> enc = OrdinalEncoder()
>>> X = [['Male', 1], ['Female', 3], ['Female', 2]]
>>> enc.fit(X)
OrdinalEncoder()
>>> enc.categories_
[array(['Female', 'Male'], dtype=object), array([1, 2, 3], dtype=object)]
>>> enc.transform([['Female', 3], ['Male', 1]])
array([[0., 2.],
       [1., 0.]])

描述和示例都是从它的文档页面复制的,你可以在这里找到:

https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html#sklearn.preprocessing.OrdinalEncoder