Python Pandas - Sorting

Pandas DataFrame - Sorting

Pandas provides efficient methods for sorting data within Series and DataFrames. There are two main types of sorting:

• Sorting by Index Labels: Arranging rows or columns based on their index or column labels.
• Sorting by Values: Arranging rows based on the values in one or more columns.

Let’s create an unsorted DataFrame to demonstrate sorting:

import pandas as pd
import numpy as np

# Create a DataFrame with a jumbled index
unsorted_df = pd.DataFrame(np.random.randn(10, 2),
                           index=[1, 4, 6, 2, 3, 5, 9, 8, 0, 7],
                           columns=['col2', 'col1'])

print("Unsorted DataFrame:")
print(unsorted_df)

Output (random values will differ):

Unsorted DataFrame:
       col2      col1
1 -2.063177  0.537527
4  0.142932 -0.684884
6  0.012667 -0.389340
2 -0.548797  1.848743
3 -1.044160  0.837381
5  0.385605  1.300185
9  1.031425 -1.002967
8 -0.407374 -0.435142
0  2.237453 -1.067139
7 -1.445831 -1.701035

This DataFrame has both unsorted index labels (0-9 out of order) and unsorted values.

Sorting by Index Labels (sort_index)

The sort_index() method sorts a DataFrame or Series based on its index labels. By default, it sorts the row index (axis=0) in ascending order.

import pandas as pd
import numpy as np

unsorted_df = pd.DataFrame(np.random.randn(10, 2),
                           index=[1, 4, 6, 2, 3, 5, 9, 8, 0, 7],
                           columns=['col2', 'col1'])

# Sort by row index (ascending by default)
sorted_by_index_df = unsorted_df.sort_index()

print("DataFrame Sorted by Row Index (Ascending):")
print(sorted_by_index_df)

Output (random values will differ, but index is sorted 0-9):

DataFrame Sorted by Row Index (Ascending):
       col2      col1
0  0.208464  0.627037
1  0.641004  0.331352
2 -0.038067 -0.464730
3 -0.638456 -0.021466
4  0.014646 -0.737438
5 -0.290761 -1.669827
6 -0.797303 -0.018737
7  0.525753  1.628921
8 -0.567031  0.775951
9  0.060724 -0.322425

Controlling Sort Order

Use the ascending parameter (boolean) to control the sort order. ascending=False sorts in descending order.

import pandas as pd
import numpy as np

unsorted_df = pd.DataFrame(np.random.randn(10, 2),
                           index=[1, 4, 6, 2, 3, 5, 9, 8, 0, 7],
                           columns=['col2', 'col1'])

# Sort by row index in descending order
sorted_desc_df = unsorted_df.sort_index(ascending=False)

print("DataFrame Sorted by Row Index (Descending):")
print(sorted_desc_df)

Output (random values will differ, but index is sorted 9-0):

DataFrame Sorted by Row Index (Descending):
       col2      col1
9  0.825697  0.374463
8 -1.699509  0.510373
7 -0.581378  0.622958
6 -0.202951  0.954300
5 -1.289321 -1.551250
4  1.302561  0.851385
3 -0.157915 -0.388659
2 -1.222295  0.166609
1  0.584890 -0.291048
0  0.668444 -0.061294

Sorting Columns

To sort by column labels instead of row index labels, set axis=1.

import pandas as pd
import numpy as np

unsorted_df = pd.DataFrame(np.random.randn(5, 3), # 5 rows, 3 columns
                           index=[1, 4, 0, 3, 2],
                           columns=['col_c', 'col_a', 'col_b'])

print("Original DataFrame (Unsorted Columns):")
print(unsorted_df)

# Sort by column labels (ascending: col_a, col_b, col_c)
sorted_by_cols_df = unsorted_df.sort_index(axis=1)

print("\nDataFrame Sorted by Column Labels (Ascending):")
print(sorted_by_cols_df)

Output (random values will differ, but columns are sorted alphabetically):

Original DataFrame (Unsorted Columns):
      col_c     col_a     col_b
1 -0.291048  0.584890  1.302561
4  0.851385 -0.202951  0.166609
0  0.954300 -1.222295 -1.289321
3 -0.388659 -0.157915  0.825697
2 -1.551250 -1.699509 -0.061294

DataFrame Sorted by Column Labels (Ascending):
      col_a     col_b     col_c
1  0.584890  1.302561 -0.291048
4 -0.202951  0.166609  0.851385
0 -1.222295 -1.289321  0.954300
3 -0.157915  0.825697 -0.388659
2 -1.699509 -0.061294 -1.551250

Sorting by Values (sort_values)

The sort_values() method sorts a DataFrame based on the values in one or more specified columns. The by parameter is used to indicate the column(s) to sort by.

import pandas as pd
import numpy as np

# Create a DataFrame with specific values for sorting demonstration
df = pd.DataFrame({
    'col1': [2, 1, 1, 1, 3],
    'col2': [1, 3, 2, 4, 0]
})
print("Original DataFrame:")
print(df)

# Sort by values in 'col1' (ascending by default)
sorted_by_col1 = df.sort_values(by='col1')

print("\nDataFrame Sorted by 'col1':")
print(sorted_by_col1)

Output:

Original DataFrame:
   col1  col2
0     2     1
1     1     3
2     1     2
3     1     4
4     3     0

DataFrame Sorted by 'col1':
   col1  col2
1     1     3  # Original index 1
2     1     2  # Original index 2
3     1     4  # Original index 3
0     2     1  # Original index 0
4     3     0  # Original index 4

Notice that the rows are now ordered based on the values in col1. The entire row (including col2 and the original index) moves together during the sort.

To sort by multiple columns, pass a list of column names to the by argument. The sorting occurs hierarchically based on the order of columns in the list.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'col1': [2, 1, 1, 1, 3],
    'col2': [1, 3, 2, 4, 0]
})

# Sort first by 'col1' (ascending), then by 'col2' (ascending) for ties in 'col1'
sorted_multi = df.sort_values(by=['col1', 'col2'])

print("DataFrame Sorted by 'col1' then 'col2':")
print(sorted_multi)

Output:

DataFrame Sorted by 'col1' then 'col2':
   col1  col2
2     1     2  # Smallest col2 value where col1 is 1
1     1     3
3     1     4  # Largest col2 value where col1 is 1
0     2     1
4     3     0

You can specify different sort orders (ascending/descending) for each column by passing a list of booleans to the ascending parameter, corresponding to the columns in by.

# Sort by 'col1' ascending, 'col2' descending
sorted_multi_mixed = df.sort_values(by=['col1', 'col2'], ascending=[True, False])
print("\nDataFrame Sorted by 'col1' (Asc), 'col2' (Desc):")
print(sorted_multi_mixed)

Output:

DataFrame Sorted by 'col1' (Asc), 'col2' (Desc):
   col1  col2
3     1     4 # Largest col2 value where col1 is 1
1     1     3
2     1     2 # Smallest col2 value where col1 is 1
0     2     1
4     3     0

Choice of Sorting Algorithm

sort_values() and sort_index() allow specifying the sorting algorithm using the kind parameter. Available options typically include:

• 'quicksort' (Default): Generally fast, but not stable.
• 'mergesort': Stable (preserves the original order of equal elements), potentially slower or uses more memory for some cases.
• 'heapsort': Offers O(n log n) performance like quicksort and mergesort.
• 'stable': Alias for mergesort, emphasizing stability.

Stability means that if two rows have the same value(s) in the sorting column(s), their relative order in the output will be the same as their relative order in the input.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'col1': [2, 1, 1, 1, 3],
    'col2': [1, 3, 2, 4, 0]
})

# Sort using mergesort (guaranteed stable)
sorted_stable = df.sort_values(by='col1', kind='mergesort')

print("DataFrame Sorted by 'col1' using Mergesort (Stable):")
print(sorted_stable)

Output (note the original relative order of rows with col1=1 is preserved: index 1, then 2, then 3):

DataFrame Sorted by 'col1' using Mergesort (Stable):
   col1  col2
1     1     3
2     1     2
3     1     4
0     2     1
4     3     0

Choosing the algorithm might impact performance, but for most use cases, the default (quicksort) is sufficient unless stability is explicitly required.