Skip to content
Quantoros

You have a DataFrame df with a column 'A' of integers. For example: How do you filter out rows which contain the same…

PythonLevel: Intermediate

You have a DataFrame df with a column 'A' of integers. For example:

df = pd.DataFrame({'A': [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})

How do you filter out rows which contain the same integer as the row immediately above?

You should be left with a column containing the following values:

1, 2, 3, 4, 5, 6, 7

Assume import pandas as pd and import numpy as np.

Hints
  • First define row granularity, column types and keys.
  • Check index alignment, missing values and output size after the transformation.

Answer

df = pd.DataFrame({'A': [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})

df.loc[df['A'].shift() != df['A']]

# Alternatively, we could use drop_duplicates() here. Note
# that this removes *all* duplicates though, so it won't
# work as desired if A is [1, 1, 2, 2, 1, 1] for example.

df.drop_duplicates(subset='A')

Source and licence: 100 pandas puzzles — Alex Riley and contributors · MIT

Related questions