今天，我非常惊讶地发现，当从数据文件读取数据时(例如)，熊猫能够识别值的类型:
df = pandas.read_csv('test.dat', delimiter=r"\s+", names=['col1','col2','col3'])
例如，可以这样检查:
for i, r in df.iterrows():
print type(r['col1']), type(r['col2']), type(r['col3'])
特别是整数、浮点数和字符串被正确识别。但是，我有一列的日期格式如下:2013-6-4。这些日期被识别为字符串(而不是python date-objects)。
当前回答
Pandas read_csv方法非常适合解析日期。完整的文档请访问http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html
你甚至可以在不同的列中有不同的日期部分，并传递参数:
parse_dates : boolean, list of ints or names, list of lists, or dict
If True -> try parsing the index. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a
separate date column. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date
column. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’
The default sensing of dates works great, but it seems to be biased towards north american Date formats. If you live elsewhere you might occasionally be caught by the results. As far as I can remember 1/6/2000 means 6 January in the USA as opposed to 1 Jun where I live. It is smart enough to swing them around if dates like 23/6/2000 are used. Probably safer to stay with YYYYMMDD variations of date though. Apologies to pandas developers,here but i have not tested it with local dates recently.
可以使用date_parser参数传递一个函数来转换格式。
date_parser : function
Function to use for converting a sequence of string columns to an array of datetime
instances. The default uses dateutil.parser.parser to do the conversion.
其他回答
当将两个列合并为单个datetime列时，接受的答案将生成一个错误(pandas版本0.20.3)，因为列分别发送给date_parser函数。
以下工作:
def dateparse(d,t):
dt = d + " " + t
return pd.datetime.strptime(dt, '%d/%m/%Y %H:%M:%S')
df = pd.read_csv(infile, parse_dates={'datetime': ['date', 'time']}, date_parser=dateparse)
加载csv文件时包含日期列。我们有两种方法来制作熊猫 识别日期列，即
熊猫显式识别格式通过arg date_parser=mydateparser Pandas隐式识别agr infer_datetime_format=True的格式
一些日期列数据
01/01/18
01/02/18
这里我们不知道前两件事，可能是月，也可能是日。在这种情况下，我们要用 方法1: 显式传递格式
mydateparser = lambda x: pd.datetime.strptime(x, "%m/%d/%y")
df = pd.read_csv(file_name, parse_dates=['date_col_name'],
date_parser=mydateparser)
方法2:—隐式或自动识别格式
df = pd.read_csv(file_name, parse_dates=[date_col_name],infer_datetime_format=True)
如果工作表现对你很重要，确保你有时间:
import sys
import timeit
import pandas as pd
print('Python %s on %s' % (sys.version, sys.platform))
print('Pandas version %s' % pd.__version__)
repeat = 3
numbers = 100
def time(statement, _setup=None):
print (min(
timeit.Timer(statement, setup=_setup or setup).repeat(
repeat, numbers)))
print("Format %m/%d/%y")
setup = """import pandas as pd
import io
data = io.StringIO('''\
ProductCode,Date
''' + '''\
x1,07/29/15
x2,07/29/15
x3,07/29/15
x4,07/30/15
x5,07/29/15
x6,07/29/15
x7,07/29/15
y7,08/05/15
x8,08/05/15
z3,08/05/15
''' * 100)"""
time('pd.read_csv(data); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"]); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"],'
'infer_datetime_format=True); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"],'
'date_parser=lambda x: pd.datetime.strptime(x, "%m/%d/%y")); data.seek(0)')
print("Format %Y-%m-%d %H:%M:%S")
setup = """import pandas as pd
import io
data = io.StringIO('''\
ProductCode,Date
''' + '''\
x1,2016-10-15 00:00:43
x2,2016-10-15 00:00:56
x3,2016-10-15 00:00:56
x4,2016-10-15 00:00:12
x5,2016-10-15 00:00:34
x6,2016-10-15 00:00:55
x7,2016-10-15 00:00:06
y7,2016-10-15 00:00:01
x8,2016-10-15 00:00:00
z3,2016-10-15 00:00:02
''' * 1000)"""
time('pd.read_csv(data); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"]); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"],'
'infer_datetime_format=True); data.seek(0)')
time('pd.read_csv(data, parse_dates=["Date"],'
'date_parser=lambda x: pd.datetime.strptime(x, "%Y-%m-%d %H:%M:%S")); data.seek(0)')
打印:
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
Pandas version 0.23.4
Format %m/%d/%y
0.19123052499999993
8.20691274
8.143124389
1.2384357139999977
Format %Y-%m-%d %H:%M:%S
0.5238807110000039
0.9202787830000005
0.9832778819999959
12.002349824999996
因此，对于iso8601格式的日期(%Y-%m-%d %H:% m:%S显然是一个iso8601格式的日期，我猜T可以被删除并被空格取代)，您不应该指定infer_datetime_format(这显然与更常见的日期没有区别)，并且传递您自己的解析器只会削弱性能。另一方面，date_parser与不那么标准的日期格式确实有所不同。像往常一样，在优化之前一定要计时。
你应该在读取时添加parse_dates=True，或者parse_dates=['列名']，这通常足以神奇地解析它。但是总有一些奇怪的格式需要手动定义。在这种情况下，还可以添加日期解析器函数，这是最灵活的方法。
假设你的字符串有一个列'datetime'，那么:
from datetime import datetime
dateparse = lambda x: datetime.strptime(x, '%Y-%m-%d %H:%M:%S')
df = pd.read_csv(infile, parse_dates=['datetime'], date_parser=dateparse)
通过这种方式，你甚至可以将多个列合并到一个datetime列中，这将'date'和'time'列合并到一个'datetime'列中:
dateparse = lambda x: datetime.strptime(x, '%Y-%m-%d %H:%M:%S')
df = pd.read_csv(infile, parse_dates={'datetime': ['date', 'time']}, date_parser=dateparse)
你可以在本页找到strptime和strftime的指令(即用于不同格式的字母)。
Pandas read_csv方法非常适合解析日期。完整的文档请访问http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html
你甚至可以在不同的列中有不同的日期部分，并传递参数:
parse_dates : boolean, list of ints or names, list of lists, or dict
If True -> try parsing the index. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a
separate date column. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date
column. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’
The default sensing of dates works great, but it seems to be biased towards north american Date formats. If you live elsewhere you might occasionally be caught by the results. As far as I can remember 1/6/2000 means 6 January in the USA as opposed to 1 Jun where I live. It is smart enough to swing them around if dates like 23/6/2000 are used. Probably safer to stay with YYYYMMDD variations of date though. Apologies to pandas developers,here but i have not tested it with local dates recently.
可以使用date_parser参数传递一个函数来转换格式。
date_parser : function
Function to use for converting a sequence of string columns to an array of datetime
instances. The default uses dateutil.parser.parser to do the conversion.
