Read and write text files, CSV files, and automate file operations for data pipelines
File handling is how Python reads and writes data from disk. In data analysis, you mostly use Pandas for CSV and Excel files (pd.read_csv, pd.read_excel). But understanding Python's built-in file operations is important for: reading log files, writing reports, automating folder operations, and building data pipelines that process batches of files.
Examples
Reading and writing files
# Read a text file
with open("report.txt", "r", encoding="utf-8") as f:
content = f.read() # entire file as string
lines = f.readlines() # list of lines
# Write a text file
with open("output.txt", "w", encoding="utf-8") as f:
f.write("Sales Report\n")
f.write(f"Total: ₹4,50,000\n")
# Append to existing file
with open("log.txt", "a") as f:
f.write("Processed on 2026-08-26\n")
# always use 'with' — it closes the file automatically
# Never: f = open("file.txt") without closing
💡 Always use with open(...) as f — it guarantees the file is closed even if an error occurs.
Working with file paths and folders
import os
import pandas as pd
# Current directory
print(os.getcwd())
# List files in a folder
files = os.listdir("data/")
csv_files = [f for f in files if f.endswith(".csv")]
# Build full path (works on Windows + Mac/Linux)
folder = "monthly_data"
filename = "jan_2026.csv"
full_path = os.path.join(folder, filename)
# "monthly_data/jan_2026.csv" (or \ on Windows)
# Check if file exists
if os.path.exists(full_path):
df = pd.read_csv(full_path)
# Save DataFrame to CSV
df.to_csv("output/cleaned_data.csv", index=False)
# index=False — don't write the row numbers (0,1,2...)
# Save to Excel
df.to_excel("output/report.xlsx", sheet_name="Sales", index=False)
💡 os.path.join() builds file paths correctly for both Windows (backslash) and Mac/Linux (forward slash) — always use it instead of string concatenation.
Key Points
✓Use with open() as f: — it closes files automatically