十分钟快速了解Pandas的常用操作!

早起Python

共 20485字,需浏览 41分钟

 ·

2020-09-06 09:15

原文https://pandas.pydata.org/pandas-docs/version/0.18.0/

编译|刘早起(有删改)

Jupyter Notebook整理版可以在后台回复01下载,以下为完整内容

目录

  • 创建数据
  • 数据查看
  • 数据选取
    • 使用[]选取数据
    • 通过标签选取数据
    • 通过位置选取数据
    • 使用布尔索引
    • 修改数据
  • 缺失值处理
    • reindex
    • 删除缺失值
    • 填充缺失值
  • 常用操作
    • 统计
    • Apply函数
    • value_counts()
    • 字符串方法
  • 数据合并
    • Concat
    • Join
    • Append
  • 数据分组
  • 数据重塑
    • 数据堆叠
    • 数据透视表
  • 时间序列
  • 灵活的使用分类数据
  • 数据可视化
  • 导入导出数据
  • 获得帮助

首先导入Python数据处理中常用的三个库

如果没有可以分别执行下方代码框安装

#安装pandas
!pip install pandas
#安装numpy
!pip install numpy
#安装matplotlib
!pip install matoplotlib
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

创建数据

使用pd.Series创建Series对象

s = pd.Series([1,3,5,np.nan,6,8])
s
0    1.0
1 3.0
2 5.0
3 NaN
4 6.0
5 8.0
dtype: float64

通过numpy的array数据来创建DataFrame对象

dates = pd.date_range('20130101', periods=6)
dates
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')
df = pd.DataFrame(np.random.randn(6,4), index=dates, columns=list('ABCD'))
df

ABCD
2013-01-01-0.469364-1.3892910.8440320.042866
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-03-0.1924261.7407650.730479-1.320781
2013-01-040.047348-1.952303-0.691544-1.403883
2013-01-050.2330210.6191120.628579-0.802585
2013-01-060.4939460.8482471.633055-0.740562

通过字典创建DataFrame对象

df2 = pd.DataFrame({ 'A' : 1.,
                     'B' : pd.Timestamp('20130102'),
                     'C' : pd.Series(1,index=list(range(4)),dtype='float32'),
                     'D' : np.array([3] * 4,dtype='int32'),
                     'E' : pd.Categorical(["test","train","test","train"]),
                     'F' : 'foo' })
df2

ABCDEF
01.02013-01-021.03testfoo
11.02013-01-021.03trainfoo
21.02013-01-021.03testfoo
31.02013-01-021.03trainfoo
df2.dtypes
A           float64
B datetime64[ns]
C float32
D int32
E category
F object
dtype: object
dir(df2)
['A',
'B',
'C',
'D',
'E',
'F',
'T',
'_AXIS_ALIASES',
'_AXIS_IALIASES',
'_AXIS_LEN',
'_AXIS_NAMES',
'_AXIS_NUMBERS',
'_AXIS_ORDERS',
'_AXIS_REVERSED',
······
'unstack',
'update',
'values',
'var',
'where',
'xs']

数据查看

基本方法,务必掌握,更多相关查看数据的方法可以参与官方文档[1] 

下面分别是查看数据的顶部和尾部的方法

df.head()

ABCD
2013-01-01-0.469364-1.3892910.8440320.042866
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-03-0.1924261.7407650.730479-1.320781
2013-01-040.047348-1.952303-0.691544-1.403883
2013-01-050.2330210.6191120.628579-0.802585
df.tail(3)

ABCD
2013-01-040.047348-1.952303-0.691544-1.403883
2013-01-050.2330210.6191120.628579-0.802585
2013-01-060.4939460.8482471.633055-0.740562

查看DataFrame对象的索引,列名,数据信息

df.index
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')
df.columns
Index(['A', 'B', 'C', 'D'], dtype='object')
df.values
array([[-0.46936354, -1.38929068,  0.84403157,  0.04286594],
[ 0.98657633, -0.68954348, -0.38326456, -1.10493201],
[-0.19242554, 1.74076522, 0.73047859, -1.32078058],
[ 0.04734752, -1.95230265, -0.6915437 , -1.40388308],
[ 0.23302102, 0.61911183, 0.628579 , -0.80258543],
[ 0.49394583, 0.84824737, 1.633055 , -0.74056229]])

描述性统计

df.describe()

ABCD
count6.0000006.0000006.0000006.000000
mean0.183184-0.1371690.460223-0.888313
std0.5157221.4308930.8558350.528401
min-0.469364-1.952303-0.691544-1.403883
25%-0.132482-1.214354-0.130304-1.266818
50%0.140184-0.0352160.679529-0.953759
75%0.4287150.7909630.815643-0.756068
max0.9865761.7407651.6330550.042866

数据转置

df.T

2013-01-01 00:00:002013-01-02 00:00:002013-01-03 00:00:002013-01-04 00:00:002013-01-05 00:00:002013-01-06 00:00:00
A-0.4693640.986576-0.1924260.0473480.2330210.493946
B-1.389291-0.6895431.740765-1.9523030.6191120.848247
C0.844032-0.3832650.730479-0.6915440.6285791.633055
D0.042866-1.104932-1.320781-1.403883-0.802585-0.740562

根据列名排序

df.sort_index(axis=1, ascending=False)

DCBA
2013-01-010.0428660.844032-1.389291-0.469364
2013-01-02-1.104932-0.383265-0.6895430.986576
2013-01-03-1.3207810.7304791.740765-0.192426
2013-01-04-1.403883-0.691544-1.9523030.047348
2013-01-05-0.8025850.6285790.6191120.233021
2013-01-06-0.7405621.6330550.8482470.493946

根据B列数值排序

df.sort_values(by='B')

ABCD
2013-01-040.047348-1.952303-0.691544-1.403883
2013-01-01-0.469364-1.3892910.8440320.042866
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-050.2330210.6191120.628579-0.802585
2013-01-060.4939460.8482471.633055-0.740562
2013-01-03-0.1924261.7407650.730479-1.320781

数据选取

官方建议使用优化的熊猫数据访问方法.at,.iat,.loc.iloc,部分较早的pandas版本可以使用.ix

这些选取函数的使用需要熟练掌握,我也曾写过相关文章帮助理解

使用[]选取数据

选取单列数据,等效于df.A:

df['A']
2013-01-01   -0.469364
2013-01-02 0.986576
2013-01-03 -0.192426
2013-01-04 0.047348
2013-01-05 0.233021
2013-01-06 0.493946
Freq: D, Name: A, dtype: float64

按行选取数据,使用[]

df[0:3]

ABCD
2013-01-01-0.469364-1.3892910.8440320.042866
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-03-0.1924261.7407650.730479-1.320781
df['20130102':'20130104']

ABCD
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-03-0.1924261.7407650.730479-1.320781
2013-01-040.047348-1.952303-0.691544-1.403883

通过标签选取数据

df.loc[dates[0]]
A   -0.469364
B -1.389291
C 0.844032
D 0.042866
Name: 2013-01-01 00:00:00, dtype: float64
df.loc[:,['A','B']]

AB
2013-01-01-0.469364-1.389291
2013-01-020.986576-0.689543
2013-01-03-0.1924261.740765
2013-01-040.047348-1.952303
2013-01-050.2330210.619112
2013-01-060.4939460.848247
df.loc['20130102':'20130104',['A','B']]

AB
2013-01-020.986576-0.689543
2013-01-03-0.1924261.740765
2013-01-040.047348-1.952303
df.loc['20130102',['A','B']]
A    0.986576
B -0.689543
Name: 2013-01-02 00:00:00, dtype: float64
df.loc[dates[0],'A']
-0.46936353804430075
df.at[dates[0],'A']
-0.46936353804430075

通过位置选取数据

df.iloc[3]
A    0.047348
B -1.952303
C -0.691544
D -1.403883
Name: 2013-01-04 00:00:00, dtype: float64
df.iloc[3:50:2]

AB
2013-01-040.047348-1.952303
2013-01-050.2330210.619112
df.iloc[[1,2,4],[0,2]]

AC
2013-01-020.986576-0.383265
2013-01-03-0.1924260.730479
2013-01-050.2330210.628579
df.iloc[1:3]

ABCD
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-03-0.1924261.7407650.730479-1.320781
df.iloc[:, 1:3]

BC
2013-01-01-1.3892910.844032
2013-01-02-0.689543-0.383265
2013-01-031.7407650.730479
2013-01-04-1.952303-0.691544
2013-01-050.6191120.628579
2013-01-060.8482471.633055
df.iloc[11]
-0.689543482094678
df.iat[11]
-0.689543482094678

使用布尔索引

df[df.A>0]

ABCD
2013-01-020.986576-0.689543-0.383265-1.104932
2013-01-040.047348-1.952303-0.691544-1.403883
2013-01-050.2330210.6191120.628579-0.802585
2013-01-060.4939460.8482471.633055-0.740562
df[df>0]

ABCD
2013-01-01NaNNaN0.8440320.042866
2013-01-020.986576NaNNaNNaN
2013-01-03NaN1.7407650.730479NaN
2013-01-040.047348NaNNaNNaN
2013-01-050.2330210.6191120.628579NaN
2013-01-060.4939460.8482471.633055NaN
df2 = df.copy()
df2['E'] = ['one''one','two','three','four','three']
df2

ABCDE
2013-01-01-0.469364-1.3892910.8440320.042866one
2013-01-020.986576-0.689543-0.383265-1.104932one
2013-01-03-0.1924261.7407650.730479-1.320781two
2013-01-040.047348-1.952303-0.691544-1.403883three
2013-01-050.2330210.6191120.628579-0.802585four
2013-01-060.4939460.8482471.633055-0.740562three
df2[df2['E'].isin(['two','four'])]

ABCDE
2013-01-03-0.1924261.7407650.730479-1.320781two
2013-01-050.2330210.6191120.628579-0.802585four

修改数据

添加新列并自动按索引对齐数据

s1 = pd.Series([1,2,3,4,5,6], index=pd.date_range('20130102', periods=6))
s1
2013-01-02    1
2013-01-03 2
2013-01-04 3
2013-01-05 4
2013-01-06 5
2013-01-07 6
Freq: D, dtype: int64
df['F'] = s1
df.at[dates[0], 'A'] = 0
df.iat[01] = 0
df.loc[:, 'D'] = np.array([5] * len(df)) 
df

ABCDF
2013-01-010.0000000.0000000.8440325NaN
2013-01-020.986576-0.689543-0.38326551.0
2013-01-03-0.1924261.7407650.73047952.0
2013-01-040.047348-1.952303-0.69154453.0
2013-01-050.2330210.6191120.62857954.0
2013-01-060.4939460.8482471.63305555.0
df2 = df.copy()
df2[df2 > 0] = -df2
df2

ABCDF
2013-01-010.0000000.000000-0.844032-5NaN
2013-01-02-0.986576-0.689543-0.383265-5-1.0
2013-01-03-0.192426-1.740765-0.730479-5-2.0
2013-01-04-0.047348-1.952303-0.691544-5-3.0
2013-01-05-0.233021-0.619112-0.628579-5-4.0
2013-01-06-0.493946-0.848247-1.633055-5-5.0

缺失值处理

缺失值处理是Pandas数据处理的一部分,以下仅展示了部分操作

有关缺失值的处理可以查看下面两篇文章:

reindex

Pandas中使用np.nan来表示缺失值,可以使用reindex更改/添加/删除指定轴上的索引

df1 = df.reindex(index=dates[0:4], columns=list(df.columns) + ['E'])
df1.loc[dates[0]:dates[1],'E'] = 1
df1

ABCDFE
2013-01-010.0000000.0000000.8440325NaN1.0
2013-01-020.986576-0.689543-0.38326551.01.0
2013-01-03-0.1924261.7407650.73047952.0NaN
2013-01-040.047348-1.952303-0.69154453.0NaN

删除缺失值

舍弃含有NaN的行

df1.dropna(how='any')

ABCDFE
2013-01-020.986576-0.689543-0.38326551.01.0

填充缺失值

填充缺失数据

df1.fillna(value=5)

ABCDFE
2013-01-010.0000000.0000000.84403255.01.0
2013-01-020.986576-0.689543-0.38326551.01.0
2013-01-03-0.1924261.7407650.73047952.05.0
2013-01-040.047348-1.952303-0.69154453.05.0
pd.isnull(df1)

ABCDFE
2013-01-01FalseFalseFalseFalseTrueFalse
2013-01-02FalseFalseFalseFalseFalseFalse
2013-01-03FalseFalseFalseFalseFalseTrue
2013-01-04FalseFalseFalseFalseFalseTrue

常用操作

在我的Pandas120题系列中有很多关于Pandas常用操作介绍!

欢迎微信搜索公众号【早起Python】关注

后台回复pandas获取相关习题!

统计

在进行统计操作时需要排除缺失值!

「描述性统计?」

纵向求均值

df.mean()
A    0.261411
B 0.094380
C 0.460223
D 5.000000
F 3.000000
dtype: float64

横向求均值

df.mean(1)
2013-01-01    1.461008
2013-01-02 1.182754
2013-01-03 1.855764
2013-01-04 1.080700
2013-01-05 2.096142
2013-01-06 2.595050
Freq: D, dtype: float64
s = pd.Series([1,3,5,np.nan,6,8], index=dates).shift(2)
s
2013-01-01    NaN
2013-01-02 NaN
2013-01-03 1.0
2013-01-04 3.0
2013-01-05 5.0
2013-01-06 NaN
Freq: D, dtype: float64
df.sub(s, axis='index')

ABCDF
2013-01-01NaNNaNNaNNaNNaN
2013-01-02NaNNaNNaNNaNNaN
2013-01-03-1.1924260.740765-0.2695214.01.0
2013-01-04-2.952652-4.952303-3.6915442.00.0
2013-01-05-4.766979-4.380888-4.3714210.0-1.0
2013-01-06NaNNaNNaNNaNNaN

Apply函数

df.apply(np.cumsum)

ABCDF
2013-01-010.0000000.0000000.8440325NaN
2013-01-020.986576-0.6895430.460767101.0
2013-01-030.7941511.0512221.191246153.0
2013-01-040.841498-0.9010810.499702206.0
2013-01-051.074519-0.2819691.1282812510.0
2013-01-061.5684650.5662782.7613363015.0
df.apply(lambda x: x.max() - x.min())
A    1.179002
B 3.693068
C 2.324599
D 0.000000
F 4.000000
dtype: float64

value_counts()

文档中为Histogramming,但示例就是.value_counts()的使用

s = pd.Series(np.random.randint(07, size=10))
s
0    6
1 1
2 4
3 6
4 3
5 2
6 3
7 5
8 2
9 2
dtype: int64
s.value_counts()
2    3
6 2
3 2
5 1
4 1
1 1
dtype: int64

字符串方法

s = pd.Series(['A''B''C''Aaba''Baca', np.nan, 'CABA''dog''cat'])
s.str.lower()
0       a
1 b
2 c
3 aaba
4 baca
5 NaN
6 caba
7 dog
8 cat
dtype: object

数据合并

在我的Pandas120题系列中有很多关于数据合并的操作,

欢迎微信搜索公众号【早起Python】关注

后台回复pandas获取相关习题!

Concat

在连接/合并类型操作的情况下,pandas提供了各种功能,可以轻松地将Series和DataFrame对象与各种用于索引和关系代数功能的集合逻辑组合在一起。

df = pd.DataFrame(np.random.randn(104))
df

0123
00.413620-1.1145270.3226781.207744
1-1.812499-1.3388660.6116220.445057
20.3650980.1779190.8232121.529158
3-0.803774-1.4222551.4113920.400721
40.7327531.413181-0.3386170.088442
5-0.509033-1.2373111.021978-0.596258
60.841053-0.4046841.528639-0.273577
70.966884-2.1425161.0416700.109264
82.2312672.0116250.6010620.533928
9-0.1346410.165157-1.2368271.681187
pieces = [df[:3], df[3:6], df[7:]]
pd.concat(pieces)

0123
00.413620-1.1145270.3226781.207744
1-1.812499-1.3388660.6116220.445057
20.3650980.1779190.8232121.529158
3-0.803774-1.4222551.4113920.400721
40.7327531.413181-0.3386170.088442
5-0.509033-1.2373111.021978-0.596258
70.966884-2.1425161.0416700.109264
82.2312672.0116250.6010620.533928
9-0.1346410.165157-1.2368271.681187

「注意」

将列添加到DataFrame相对较快。

但是,添加一行需要一个副本,并且可能浪费时间

我们建议将预构建的记录列表传递给DataFrame构造函数,而不是通过迭代地将记录追加到其来构建DataFrame

Join

left = pd.DataFrame({'key': ['foo''foo'], 'lval': [12]})
right = pd.DataFrame({'key': ['foo''foo'], 'rval': [45]})
left

keylval
0foo1
1foo2
right

keyrval
0foo4
1foo5
pd.merge(left, right, on='key')

keylvalrval
0foo14
1foo15
2foo24
3foo25

Append

df = pd.DataFrame(np.random.randn(84), columns=['A','B','C','D'])
df

ABCD
0-0.142659-0.941171-0.186519-0.811977
10.5845610.177886-0.1903960.664233
2-1.8078290.2681930.6839900.477042
3-1.474986-1.098600-0.0382802.087236
41.9067030.678425-0.090156-0.444430
50.3297481.1103060.713732-0.714841
61.218329-0.3762640.389029-1.526025
70.4233471.821127-1.795346-0.795738
s = df.iloc[3]
df.append(s, ignore_index=True)

ABCD
0-0.142659-0.941171-0.186519-0.811977
10.5845610.177886-0.1903960.664233
2-1.8078290.2681930.6839900.477042
3-1.474986-1.098600-0.0382802.087236
41.9067030.678425-0.090156-0.444430
50.3297481.1103060.713732-0.714841
61.218329-0.3762640.389029-1.526025
70.4233471.821127-1.795346-0.795738
8-1.474986-1.098600-0.0382802.087236

数据分组

「数据分组」是指涉及以下一个或多个步骤的过程:

  • 根据某些条件将数据分成几组

  • 对每个组进行独立的操作

  • 对结果进行合并

更多操作可以查阅官方文档[2] 

df = pd.DataFrame({'A' : ['foo''bar''foo''bar',
                          'foo''bar''foo''foo'],
                   'B' : ['one''one''two''three',
                           'two''two''one''three'],
                   'C' : np.random.randn(8),
                   'D' : np.random.randn(8)})
df

ABCD
0fooone-1.1452540.974305
1barone1.195757-0.187145
2footwo-0.6994460.248682
3barthree-0.587003-0.200543
4footwo2.046185-1.377637
5bartwo0.444696-0.880975
6fooone0.057713-1.275762
7foothree0.2721960.016167
df.groupby('A').sum()

CD
A

bar1.053451-1.268663
foo0.531394-1.414245
df.groupby(['A''B']).sum()


CD
AB

barone1.195757-0.187145
three-0.587003-0.200543
two0.444696-0.880975
fooone-1.087541-0.301457
three0.2721960.016167
two1.346739-1.128956

数据重塑

详细教程请参阅官方文档[3]「分层索引和重塑」部分。

数据堆叠

可以进行数据压缩

tuples = list(zip(*[['bar''bar''baz''baz',
                     'foo''foo''qux''qux'],
                   ['one''two''one''two',
                    'one''two''one''two']]))
index = pd.MultiIndex.from_tuples(tuples, names=['first''second'])
df = pd.DataFrame(np.random.randn(82), index=index, columns=['A''B'])
df2 = df[:4]
df2


AB
firstsecond

barone-0.6254922.471493
two0.9347081.595349
bazone0.6860790.279957
two0.039190-0.534317
stacked = df2.stack()
stacked
first  second
bar one A -0.625492
B 2.471493
two A 0.934708
B 1.595349
baz one A 0.686079
B 0.279957
two A 0.039190
B -0.534317
dtype: float64

stack()的反向操作是unstack(),默认情况下,它会将最后一层数据进行unstack():

stacked.unstack()


AB
firstsecond

barone-0.6254922.471493
two0.9347081.595349
bazone0.6860790.279957
two0.039190-0.534317
stacked.unstack(1)

secondonetwo
first


barA-0.6254920.934708
B2.4714931.595349
bazA0.6860790.039190
B0.279957-0.534317
stacked.unstack(0)

firstbarbaz
second


oneA-0.6254920.686079
B2.4714930.279957
twoA0.9347080.039190
B1.595349-0.534317

数据透视表

Pandas中实现数据透视表很简单,但是相比之下并没有Excel灵活,可以查看我的文章

df = pd.DataFrame({'A' : ['one''one''two''three'] * 3,
                   'B' : ['A''B''C'] * 4,
                   'C' : ['foo''foo''foo''bar''bar''bar'] * 2,
                   'D' : np.random.randn(12),
                   'E' : np.random.randn(12)})
df

ABCDE
0oneAfoo-0.072719-0.034173
1oneBfoo1.262336-0.907695
2twoCfoo0.093161-1.516473
3threeAbar0.1900560.481209
4oneBbar1.3198550.255924
5oneCbar0.374758-0.019331
6twoAfoo-1.0192820.673759
7threeBfoo-1.526206-0.521203
8oneCfoo1.6001681.632461
9oneAbar-2.410462-0.271305
10twoBbar0.387701-1.039195
11threeCbar-1.367669-1.760517
df.pivot_table(values='D', index=['A''B'], columns='C')

Cbarfoo
AB

oneA-2.410462-0.072719
B1.3198551.262336
C0.3747581.600168
threeA0.190056NaN
BNaN-1.526206
C-1.367669NaN
twoANaN-1.019282
B0.387701NaN
CNaN0.093161

时间序列

对于在频率转换期间执行重采样操作(例如,将秒数据转换为5分钟数据),pandas具有简单、强大和高效的功能。这在金融应用中非常常见,但不仅限于此。参见官方文档[4]「时间序列」部分。

时区表示

rng = pd.date_range('1/1/2012', periods=100, freq='S')
ts = pd.Series(np.random.randint(0500, len(rng)), index=rng)
ts.resample('5Min').sum()
2012-01-01    27339
Freq: 5T, dtype: int64
rng = pd.date_range('3/6/2012 00:00', periods=5, freq='D')
ts = pd.Series(np.random.randn(len(rng)), rng)
ts
2012-03-06   -0.118691
2012-03-07 -1.424038
2012-03-08 0.377441
2012-03-09 -1.116195
2012-03-10 1.180595
Freq: D, dtype: float64
ts_utc = ts.tz_localize('UTC')
ts_utc
2012-03-06 00:00:00+00:00   -0.118691
2012-03-07 00:00:00+00:00 -1.424038
2012-03-08 00:00:00+00:00 0.377441
2012-03-09 00:00:00+00:00 -1.116195
2012-03-10 00:00:00+00:00 1.180595
Freq: D, dtype: float64

时区转换

ts_utc.tz_convert('US/Eastern')
2012-03-05 19:00:00-05:00   -0.118691
2012-03-06 19:00:00-05:00 -1.424038
2012-03-07 19:00:00-05:00 0.377441
2012-03-08 19:00:00-05:00 -1.116195
2012-03-09 19:00:00-05:00 1.180595
Freq: D, dtype: float64

在时间跨度表示之间进行转换

rng = pd.date_range('1/1/2012', periods=5, freq='M')
ts = pd.Series(np.random.randn(len(rng)), index=rng)
ts
2012-01-31    1.138201
2012-02-29 0.677539
2012-03-31 0.272933
2012-04-30 -0.238112
2012-05-31 -1.122162
Freq: M, dtype: float64
ps = ts.to_period()
ps
2012-01    1.138201
2012-02 0.677539
2012-03 0.272933
2012-04 -0.238112
2012-05 -1.122162
Freq: M, dtype: float64
ps.to_timestamp()
2012-01-01    1.138201
2012-02-01 0.677539
2012-03-01 0.272933
2012-04-01 -0.238112
2012-05-01 -1.122162
Freq: MS, dtype: float64

在周期和时间戳之间转换可以使用一些方便的算术函数。

在以下示例中,我们将以11月结束的年度的季度频率转换为季度结束后的月末的上午9点:

prng = pd.period_range('1990Q1''2000Q4', freq='Q-NOV')
ts = pd.Series(np.random.randn(len(prng)), prng)
ts.index = (prng.asfreq('M''e') + 1).asfreq('H''s') + 9
ts.head()
1990-03-01 09:00   -1.555191
1990-06-01 09:00 1.535344
1990-09-01 09:00 -0.092187
1990-12-01 09:00 1.285081
1991-03-01 09:00 1.130063
Freq: H, dtype: float64

事实上,常用有关时间序列的操作远超过上方的官方示例,简单来说与日期有关的操作从创建到转换pandas都能很好的完成!

灵活的使用分类数据

Pandas可以在一个DataFrame中包含分类数据。有关完整文档,请参阅分类介绍和API文档。

df = pd.DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a''b''b''a''a''e']})
df['grade'] = df['raw_grade'].astype("category")
df['grade']
0    a
1 b
2 b
3 a
4 a
5 e
Name: grade, dtype: category
Categories (3, object): [a, b, e]

将类别重命名为更有意义的名称(Series.cat.categories())

df["grade"].cat.categories = ["very good""good""very bad"]

重新排序类别,并同时添加缺少的类别(在有缺失的情况下,string .cat()下的方法返回一个新的系列)。

df["grade"] = df["grade"].cat.set_categories(["very bad""bad""medium""good""very good"])
df["grade"]
0    very good
1 good
2 good
3 very good
4 very good
5 very bad
Name: grade, dtype: category
Categories (5, object): [very bad, bad, medium, good, very good]
df.sort_values(by='grade')

idraw_gradegrade
56every bad
12bgood
23bgood
01avery good
34avery good
45avery good
df.groupby("grade").size()
grade
very bad 1
bad 0
medium 0
good 2
very good 3
dtype: int64

数据可视化

在我的Pandas120题系列中有很多关于数据可视化的操作,

欢迎微信搜索公众号【早起Python】关注

后台回复pandas获取相关习题!

ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
ts.head()
2000-01-01   -1.946554
2000-01-02 -0.354670
2000-01-03 0.361473
2000-01-04 -0.109408
2000-01-05 0.877671
Freq: D, dtype: float64
ts = ts.cumsum() #累加

在Pandas中可以使用.plot()直接绘图,支持多种图形和自定义选项点击可以查阅官方文档[5]

ts.plot()
df = pd.DataFrame(np.random.randn(10004), index=ts.index,
                  columns=['A''B''C''D']) 
df = df.cumsum()

使用plt绘图,具体参数设置可以查阅matplotlib官方文档

plt.figure(); df.plot(); plt.legend(loc='best')

导入导出数据

「将数据写入csv,如果有中文需要注意编码」

df.to_csv('foo.csv')

csv中读取数据

pd.read_csv('foo.csv').head()

Unnamed: 0ABCD
02000-01-01-0.640246-1.846295-0.1817540.981574
12000-01-02-1.580720-2.382281-0.7455800.175213
22000-01-03-2.745502-1.809188-0.371424-0.724011
32000-01-04-2.576642-1.287329-0.615925-1.154665
42000-01-05-2.442921-0.481561-0.2838640.068934

将数据导出为hdf格式

df.to_hdf('foo.h5','df')

hdf文件中读取数据前五行

pd.read_hdf('foo.h5','df').head()

ABCD
2000-01-01-0.640246-1.846295-0.1817540.981574
2000-01-02-1.580720-2.382281-0.7455800.175213
2000-01-03-2.745502-1.809188-0.371424-0.724011
2000-01-04-2.576642-1.287329-0.615925-1.154665
2000-01-05-2.442921-0.481561-0.2838640.068934

将数据保存为xlsx格式

df.to_excel('foo.xlsx', sheet_name='Sheet1')

xlsx格式中按照指定要求读取sheet1中数据

pd.read_excel('foo.xlsx''Sheet1', index_col=None, na_values=['NA']).head()

ABCD
2000-01-01-0.640246-1.846295-0.1817540.981574
2000-01-02-1.580720-2.382281-0.7455800.175213
2000-01-03-2.745502-1.809188-0.371424-0.724011
2000-01-04-2.576642-1.287329-0.615925-1.154665
2000-01-05-2.442921-0.481561-0.2838640.068934

获得帮助

如果你在使用Pandas的过程中遇到了错误,就像下面一样:

>>> if pd.Series([False, T`mrue, False]):
...     print("I was true")
Traceback
    ...
ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all().

可以查阅官方文档来了解该如何解决!

参考资料

[1]

https://pandas.pydata.org/pandas-docs/stable/user_guide/basics.html#basics

[2]

https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#groupby

[3]

https://pandas.pydata.org/pandas-docs/stable/user_guide/advanced.html#advanced-hierarchical

[4]

https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timeseries

[5]

https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html#plotting


-END-


浏览 32
点赞
评论
收藏
分享

手机扫一扫分享

分享
举报
评论
图片
表情
推荐
点赞
评论
收藏
分享

手机扫一扫分享

分享
举报