You have a DataFrame df with a column 'A' of integers. For example: How do you filter out rows which contain the same…
PythonLevel: Intermediate
You have a DataFrame df with a column 'A' of integers. For example:
df = pd.DataFrame({'A': [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})
How do you filter out rows which contain the same integer as the row immediately above?
You should be left with a column containing the following values:
1, 2, 3, 4, 5, 6, 7
Assume import pandas as pd and import numpy as np.
Hints
- First define row granularity, column types and keys.
- Check index alignment, missing values and output size after the transformation.
Answer
df = pd.DataFrame({'A': [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})
df.loc[df['A'].shift() != df['A']]
# Alternatively, we could use drop_duplicates() here. Note
# that this removes *all* duplicates though, so it won't
# work as desired if A is [1, 1, 2, 2, 1, 1] for example.
df.drop_duplicates(subset='A')
Source and licence: 100 pandas puzzles — Alex Riley and contributors · MIT