Pandas

pathpython

Pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

Cookbook

Setup notebook for pandas + bokeh

import os
import pandas as pd
import numpy as np
from bokeh.io import output_notebook, show

# setup some globals we'll use as base paths
BASE_DIR = os.path.realpath(os.path.join(os.getcwd(), '..'))
DATA_DIR = os.path.join(BASE_DIR, 'data')

# tell bokeh to output graphs in the notebook
output_notebook()

# depends on input data, but maybe handy to see untruncated data if you have a lot of columns
pd.options.display.max_columns = 100

# output the computed paths
print('base:', BASE_DIR)
print('data:', DATA_DIR)

Reorder columns in a pandas dataframe

df = df[['this', 'is', 'the', 'new', 'order']]

Construct a DataFrame from a list of dicts

import pandas as pd

ds = []
for user in users:
    ds.append({
        'screen_name': user.screen_name,
        'name': user.name,
    })

df = pd.DataFrame(ds)

Create an index afterward

df = df.set_index('screen_name')

set_index returns a new DataFrame, so assign it back (or pass inplace=True).

A frame to work with

import pandas as pd

df = pd.DataFrame({
    "knight": ["Arthur", "Lancelot", "Galahad", "Robin"],
    "quest": ["grail", "grail", "grail", "flee"],
    "colour": ["blue", "blue", "yellow", "brave"],
    "coconuts": [2, 0, 1, 3],
})
speeds = pd.DataFrame({"knight": ["Arthur", "Robin"], "mph": [11, 9]})

Select rows with a boolean mask

grail = df.loc[df["quest"] == "grail", ["knight", "coconuts"]]

.loc[rows, cols] selects by label; here the row part is a boolean Series and the column part a list.

Group and aggregate

by_colour = df.groupby("colour")["coconuts"].sum().sort_values(ascending=False)

Merge two frames

joined = df.merge(speeds, on="knight", how="left")

how is one of left, right, inner, outer.

Count occurrences

counts = df["quest"].value_counts()

Parse and filter dates

events = pd.DataFrame({
    "when": pd.to_datetime(["0932-01-01", "0932-06-01"], format="%Y-%m-%d"),
    "n": [1, 2],
})
recent = events[events["when"] >= "0932-03-01"]

Pass an explicit format so parsing is consistent and fast.

See also

Related