Tutorials › Python › Pandas DataFrames

Pandas DataFrames

The core Pandas data structure — a 2D table with rows and columns

In my experience, the DataFrame is where analysts either fall in love with Python or give up on it. The ones who stick with it are the ones who understand one thing early: a DataFrame is not a spreadsheet you click around in. It is a programmable data structure — and once you treat it that way, it becomes the most powerful tool you have ever used for data work. Every analysis I have built over twenty-plus years — sales dashboards, customer churn models, operational reports for C-suite — started the same way: load data into a DataFrame, understand its shape, fix its types, filter it, aggregate it, export it. That is the entire workflow. Everything else is detail. The most important habit I can give you: never modify a DataFrame without first understanding what is in it. df.info() and df.describe() before you touch anything. I have seen analysts spend hours debugging wrong numbers because they skipped this step and worked on data with the wrong dtype or unexpected nulls. Two lines of code would have caught it immediately. DataFrames support everything you need: selecting columns, filtering rows, computing new columns, sorting, grouping, merging multiple sources, and exporting to CSV or Excel. Master this one object and you will be ahead of 90% of analysts in any room.

Examples

Creating and exploring DataFrames
import pandas as pd

# Create from dictionary
df = pd.DataFrame({
    "City":    ["Delhi", "Noida", "Gurgaon", "Faridabad"],
    "Sales":   [450000, 320000, 280000, 195000],
    "Costs":   [310000, 220000, 195000, 140000],
    "Quarter": ["Q1", "Q1", "Q1", "Q1"],
})

# Explore
df.shape         # (4, 4) — rows, columns
df.dtypes        # data type of each column
df.columns       # Index(['City', 'Sales', 'Costs', 'Quarter'])
df.head(2)       # first 2 rows
df.tail(2)       # last 2 rows
df.info()        # shape, dtypes, null counts
df.describe()    # statistics for numeric columns

# Select columns
df["Sales"]               # one column → Series
df[["City", "Sales"]]     # multiple columns → DataFrame

# Add computed column
df["Profit"] = df["Sales"] - df["Costs"]
df["Margin"] = df["Profit"] / df["Sales"] * 100

# Rename columns
df = df.rename(columns={"Sales": "Revenue"})

# Drop column
df = df.drop(columns=["Quarter"])
💡 df.info() is one of the most useful methods — it shows column types and how many non-null values each column has.
Filtering rows
# Boolean filter
high_sales = df[df["Sales"] > 300000]

# Multiple conditions
delhi_high = df[(df["City"] == "Delhi") & (df["Sales"] > 300000)]
# Use & (not 'and'), | (not 'or'), ~ (not 'not') with Pandas

# isin — filter by list of values
ncr_cities = df[df["City"].isin(["Delhi", "Noida", "Gurgaon"])]

# String contains
df[df["City"].str.contains("Noi")]   # matches "Noida"

# Sort
df.sort_values("Sales", ascending=False)   # descending
df.sort_values(["Quarter", "Sales"])        # multiple columns

# Reset index after filtering
filtered = df[df["Sales"] > 300000].reset_index(drop=True)

Key Points

  • ✓df.info() shows shape, column types, and null counts — run it first on any new dataset
  • ✓Boolean filters: df[df["Col"] > value] — parentheses around each condition
  • ✓Use & | ~ (not and or not) when combining Pandas filter conditions
  • ✓.reset_index(drop=True) resets row numbers after filtering
  • ✓df.copy() creates an independent copy — without it, modifications affect the original

Practice Question

You want to filter a DataFrame to rows where City is "Delhi" AND Sales > 300000. Which syntax is correct?

Related Topics

Pandas SeriesThe one-dimensional Pandas data structure — a labelled array for a single columnReading CSV and Excel FilesLoad data from CSV, Excel, and multiple sheets into Pandas DataFramesData Cleaning with PandasRename columns, fix data types, remove duplicates, and standardise messy real-world datagroupby and Pivot TablesAggregate data by category with groupby — the Pandas equivalent of Excel pivot tables