如果你想把它分成训练集、测试集和验证集，你可以使用这个函数:

from sklearn.model_selection import train_test_split
import pandas as pd

def train_test_val_split(df, test_size=0.15, val_size=0.45):
    temp, test = train_test_split(df, test_size=test_size)
    total_items_count = len(df.index)
    val_length = total_items_count * val_size
    new_val_propotion = val_length / len(temp.index) 
    train, val = train_test_split(temp, test_size=new_val_propotion)
    return train, test, val

我会用K-fold交叉验证。 它已被证明比train_test_split提供更好的结果。下面是一篇关于如何在sklearn中应用它的文章，来自文档本身:https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html

这是我在需要分割数据帧时所写的。我考虑过使用上面安迪的方法，但不喜欢我不能精确地控制数据集的大小(例如，有时是79，有时是81，等等)。

def make_sets(data_df, test_portion):
    import random as rnd

    tot_ix = range(len(data_df))
    test_ix = sort(rnd.sample(tot_ix, int(test_portion * len(data_df))))
    train_ix = list(set(tot_ix) ^ set(test_ix))

    test_df = data_df.ix[test_ix]
    train_df = data_df.ix[train_ix]

    return train_df, test_df


train_df, test_df = make_sets(data_df, 0.2)
test_df.head()

你也可以考虑分层划分为训练集和测试集。设定划分也随机生成训练集和测试集，但保留了原始的类比例。这使得训练集和测试集更好地反映原始数据集的属性。

import numpy as np  

def get_train_test_inds(y,train_proportion=0.7):
    '''Generates indices, making random stratified split into training set and testing sets
    with proportions train_proportion and (1-train_proportion) of initial sample.
    y is any iterable indicating classes of each observation in the sample.
    Initial proportions of classes inside training and 
    testing sets are preserved (stratified sampling).
    '''

    y=np.array(y)
    train_inds = np.zeros(len(y),dtype=bool)
    test_inds = np.zeros(len(y),dtype=bool)
    values = np.unique(y)
    for value in values:
        value_inds = np.nonzero(y==value)[0]
        np.random.shuffle(value_inds)
        n = int(train_proportion*len(value_inds))

        train_inds[value_inds[:n]]=True
        test_inds[value_inds[n:]]=True

    return train_inds,test_inds

df[train_inds]和df[test_inds]为您提供原始DataFrame df的训练和测试集。

在我的例子中，我想用特定的数字分割训练、测试和开发中的数据帧。我在这里分享我的解决方案

首先，为数据帧分配一个唯一的id(如果已经不存在的话)

import uuid
df['id'] = [uuid.uuid4() for i in range(len(df))]

以下是我的分割数字:

train = 120765
test  = 4134
dev   = 2816

分裂函数

def df_split(df, n):
    
    first  = df.sample(n)
    second = df[~df.id.isin(list(first['id']))]
    first.reset_index(drop=True, inplace = True)
    second.reset_index(drop=True, inplace = True)
    return first, second

现在分成培训，测试，开发

train, test = df_split(df, 120765)
test, dev   = df_split(test, 4134)

Scikit Learn的train_test_split就是一个很好的例子。它将拆分numpy数组和数据框架。

from sklearn.model_selection import train_test_split

train, test = train_test_split(df, test_size=0.2)