What You'll Learn Here
- read() — reading entire files
- readline() — reading line by line
- readlines() — reading all lines
- Iteration — efficient file reading
- Large files — handling big data
- CSV files — reading structured data
Reading Files in Python
Reading files is a fundamental skill in Python. Whether you're processing data, reading configuration files, or analyzing logs, you'll often need to read files. Python provides several methods to read files, each suited for different situations.
Think of reading a file like reading a book. You can read the whole book at once, read one page at a time, or just look at specific chapters. Python gives you the flexibility to choose the best approach for your needs.
💡 Key concept: The method you choose for reading files depends on your file size and what you need to do with the data. Small files can be read all at once, while large files should be read line by line to save memory.
The read() Method
1
Reading the Entire File
# The read() method reads the entire file as a single string
# First, let's create a sample file
with open("sample.txt", "w") as f:
f.write("Line 1: Hello World\n")
f.write("Line 2: Python is great\n")
f.write("Line 3: File handling is easy\n")
# 1. Reading the entire file
with open("sample.txt", "r") as file:
content = file.read()
print("Full file content:")
print(content)
print(f"Type: {type(content)}")
# 2. Reading a specific number of characters
with open("sample.txt", "r") as file:
first_10_chars = file.read(10)
print(f"First 10 characters: '{first_10_chars}'")
# 3. Reading the rest after reading some
with open("sample.txt", "r") as file:
part1 = file.read(5)
part2 = file.read()
print(f"Part 1 (5 chars): '{part1}'")
print(f"Part 2 (rest): '{part2}'")
# 4. When to use read()
# ✅ Good for: Small files, configuration files, reading everything at once
# ❌ Not good for: Very large files (uses too much memory)
# 5. Practical example: Reading a configuration file
def read_config(filename):
"""Read a simple configuration file"""
try:
with open(filename, "r") as file:
config_data = {}
for line in file.read().splitlines():
if "=" in line:
key, value = line.split("=", 1)
config_data[key.strip()] = value.strip()
return config_data
except FileNotFoundError:
print(f"Config file '{filename}' not found")
return {}
# Example usage
config = read_config("config.txt")
# print(config)
read() method key points:
- Returns string — reads the entire file as one string
- Optional size — can specify number of characters
- Simple — easiest way to read a file
- Best for small files — under a few MB
- Memory heavy — loads everything at once
Quick Check: What does read() return? (Answer: A string containing the entire file content)
The readline() Method
2
Reading One Line at a Time
# The readline() method reads one line at a time
# 1. Reading a single line
with open("sample.txt", "r") as file:
line1 = file.readline()
line2 = file.readline()
line3 = file.readline()
print(f"Line 1: {line1.strip()}")
print(f"Line 2: {line2.strip()}")
print(f"Line 3: {line3.strip()}")
# 2. Reading with a loop
with open("sample.txt", "r") as file:
print("All lines:")
while True:
line = file.readline()
if not line: # End of file
break
print(f" {line.strip()}")
# 3. Reading a specific line (with counter)
def read_specific_line(filename, line_number):
"""Read a specific line from a file"""
with open(filename, "r") as file:
for i in range(line_number - 1):
file.readline()
return file.readline().strip()
print(f"Line 2: {read_specific_line('sample.txt', 2)}")
# 4. Reading lines with condition
with open("sample.txt", "r") as file:
while True:
line = file.readline()
if not line:
break
if "Python" in line:
print(f"Found: {line.strip()}")
# 5. When to use readline()
# ✅ Good for: Processing files line by line
# ✅ Good for: Large files (memory efficient)
# ✅ Good for: When you need to stop at certain points
readline() method key points:
- Returns string — reads one line at a time
- Includes newline — use strip() to remove
- Returns empty — empty string at EOF
- Memory efficient — doesn't load entire file
- Good for large files — process line by line
Quick Check: What does readline() return when it reaches the end of the file? (Answer: An empty string)
The readlines() Method
3
Reading All Lines as a List
# The readlines() method reads all lines into a list
# 1. Reading all lines as a list
with open("sample.txt", "r") as file:
lines = file.readlines()
print(f"Number of lines: {len(lines)}")
print("All lines:")
for i, line in enumerate(lines, 1):
print(f" Line {i}: {line.strip()}")
# 2. Working with the list
with open("sample.txt", "r") as file:
lines = file.readlines()
print(f"First line: {lines[0].strip()}")
print(f"Last line: {lines[-1].strip()}")
print(f"Lines with 'Python': {[line for line in lines if 'Python' in line]}")
# 3. Removing newline characters
with open("sample.txt", "r") as file:
lines = [line.strip() for line in file.readlines()]
print(f"Clean lines: {lines}")
# 4. When to use readlines()
# ✅ Good for: Files you need to process multiple times
# ✅ Good for: Small to medium files
# ❌ Not good for: Very large files (memory heavy)
# 5. Practical: Reading a CSV file as list
def read_csv_lines(filename):
"""Read CSV file and return list of rows"""
with open(filename, "r") as file:
lines = file.readlines()
return [line.strip().split(",") for line in lines]
# Example with a CSV file
with open("data.csv", "w") as f:
f.write("Name,Age,City\n")
f.write("Alice,25,NYC\n")
f.write("Bob,30,LA\n")
data = read_csv_lines("data.csv")
print(f"CSV data: {data}")
print(f"Header: {data[0]}")
print(f"First row: {data[1]}")
readlines() method key points:
- Returns list — each line is an element
- Includes newlines — use strip() to clean
- Easy to access — index by line number
- Good for small files — under a few MB
- Memory heavy — loads all lines at once
Quick Check: What type does readlines() return? (Answer: A list of strings)
Iterating Over Files
4
Most Efficient Way to Read Files
# Iterating over a file is the most memory-efficient way
# 1. Basic iteration (most efficient)
with open("sample.txt", "r") as file:
print("Iterating over file:")
for line in file:
print(f" {line.strip()}")
# 2. Iterating with index
with open("sample.txt", "r") as file:
print("Lines with index:")
for line_num, line in enumerate(file, 1):
print(f" Line {line_num}: {line.strip()}")
# 3. Iterating with condition
with open("sample.txt", "r") as file:
print("Lines containing 'Python':")
for line in file:
if "Python" in line:
print(f" {line.strip()}")
# 4. Iterating and collecting data
def find_lines_with_pattern(filename, pattern):
"""Find all lines containing a pattern"""
matches = []
with open(filename, "r") as file:
for line in file:
if pattern in line:
matches.append(line.strip())
return matches
matches = find_lines_with_pattern("sample.txt", "Python")
print(f"Lines with 'Python': {matches}")
# 5. Processing large files efficiently
def process_large_file(filename, processor_func):
"""Process a large file line by line"""
with open(filename, "r") as file:
for line in file:
result = processor_func(line.strip())
if result: # Only process non-empty results
yield result
# Example: Count words in a large file
def count_words_in_file(filename):
"""Count total words in a file"""
word_count = 0
with open(filename, "r") as file:
for line in file:
word_count += len(line.split())
return word_count
print(f"Word count: {count_words_in_file('sample.txt')}")
# 6. Comparing methods
# read() - Fastest, but uses most memory
# readlines() - Fast, uses moderate memory
# Iteration - Slowest, but uses least memory
Iteration key points:
- Most memory efficient — reads one line at a time
- Easiest to use — just 'for line in file'
- Works with large files — no memory issues
- Can be combined with enumerate — get line numbers
- Best for most use cases — balanced and efficient
Quick Check: Which reading method is most memory efficient? (Answer: Iteration with 'for line in file')
Reading Large Files
5
Handling Big Data Efficiently
# When dealing with large files, you need special techniques
# 1. Using iteration (already the best approach)
def process_large_file_iter(filename):
"""Process a large file using iteration"""
count = 0
with open(filename, "r") as file:
for line in file:
count += 1
if count % 1000000 == 0:
print(f"Processed {count} lines...")
return count
# 2. Reading chunks (for binary or custom formats)
def read_in_chunks(filename, chunk_size=1024):
"""Read a file in chunks"""
with open(filename, "r") as file:
while True:
chunk = file.read(chunk_size)
if not chunk:
break
yield chunk
# 3. Using a generator for large files
def read_large_file_generator(filename):
"""Generator that yields lines from a large file"""
with open(filename, "r") as file:
for line in file:
yield line
# 4. Processing without loading everything
def count_occurrences(filename, word):
"""Count occurrences of a word in a large file"""
count = 0
with open(filename, "r") as file:
for line in file:
count += line.lower().count(word.lower())
return count
# 5. Streaming processing
def process_stream(filename, batch_size=1000):
"""Process a large file in batches"""
batch = []
with open(filename, "r") as file:
for line in file:
batch.append(line.strip())
if len(batch) >= batch_size:
# Process the batch
# Here we just print the batch size
print(f"Processing batch of {len(batch)} lines")
batch = []
# Process remaining
if batch:
print(f"Processing final batch of {len(batch)} lines")
# 6. Memory-efficient reading with seek
def read_from_position(filename, position=0, size=100):
"""Read from a specific position in a large file"""
with open(filename, "r") as file:
file.seek(position)
return file.read(size)
# 7. Comparing methods for large files
# Using iteration: ~1 second per million lines
# Using readlines(): ~0.5 seconds but uses more memory
# Using read(): ~0.2 seconds but uses the most memory
Large file reading strategies:
- Iteration — best for most large files
- Chunk reading — for binary or custom formats
- Generators — reusable lazy reading
- Batch processing — process in groups
- Don't load everything — avoid read() and readlines()
Quick Check: What should you avoid when reading large files? (Answer: read() and readlines() which load everything into memory)
Reading CSV Files
6
Working with Structured Data
# CSV (Comma Separated Values) files are common for data
# First, create a sample CSV file
with open("employees.csv", "w") as f:
f.write("Name,Age,Department,Salary\n")
f.write("Alice,25,Engineering,75000\n")
f.write("Bob,30,Marketing,65000\n")
f.write("Charlie,35,Sales,70000\n")
f.write("Diana,28,HR,55000\n")
# 1. Reading CSV manually
def read_csv_manual(filename):
"""Read CSV file manually"""
data = []
with open(filename, "r") as file:
# Read header
header = file.readline().strip().split(",")
# Read data
for line in file:
row = line.strip().split(",")
data.append(dict(zip(header, row)))
return data
employees = read_csv_manual("employees.csv")
print("CSV Data:")
for emp in employees:
print(f" {emp}")
# 2. Using the csv module (recommended)
import csv
def read_csv_with_module(filename):
"""Read CSV using the csv module"""
data = []
with open(filename, "r") as file:
reader = csv.DictReader(file)
for row in reader:
data.append(row)
return data
employees = read_csv_with_module("employees.csv")
print("Using csv module:")
for emp in employees:
print(f" {emp['Name']} works in {emp['Department']}")
# 3. Reading CSV with specific columns
def read_csv_columns(filename, columns):
"""Read specific columns from CSV"""
data = []
with open(filename, "r") as file:
reader = csv.DictReader(file)
for row in reader:
data.append({col: row[col] for col in columns if col in row})
return data
names_and_salaries = read_csv_columns("employees.csv", ["Name", "Salary"])
print("Names and salaries:")
for item in names_and_salaries:
print(f" {item}")
# 4. Filtering CSV data
def filter_csv(filename, column, value):
"""Filter CSV rows by column value"""
data = []
with open(filename, "r") as file:
reader = csv.DictReader(file)
for row in reader:
if row[column] == value:
data.append(row)
return data
engineers = filter_csv("employees.csv", "Department", "Engineering")
print(f"Engineers: {len(engineers)}")
for eng in engineers:
print(f" {eng['Name']} - {eng['Salary']}")
# 5. Reading CSV with pandas (if installed)
# import pandas as pd
# df = pd.read_csv("employees.csv")
# print(df.head())
CSV reading key points:
- Manual parsing — simple split by comma
- csv module — recommended for most use cases
- DictReader — reads CSV as dictionaries
- Filtering — can filter rows easily
- Pandas — for advanced data analysis
Quick Check: Which Python module is recommended for reading CSV files? (Answer: csv)
Error Handling When Reading Files
7
Robust File Reading
# Reading files can fail for many reasons
# 1. Comprehensive file reading function
def safe_read_file(filename):
"""Safely read a file with error handling"""
try:
with open(filename, "r") as file:
return file.read()
except FileNotFoundError:
print(f"Error: File '{filename}' not found")
return None
except PermissionError:
print(f"Error: Permission denied for '{filename}'")
return None
except IsADirectoryError:
print(f"Error: '{filename}' is a directory")
return None
except UnicodeDecodeError:
print(f"Error: '{filename}' has encoding issues")
return None
except Exception as e:
print(f"Unexpected error reading '{filename}': {e}")
return None
# 2. Reading with multiple fallbacks
def read_with_fallback(filename, fallback_content=""):
"""Read a file or use fallback content"""
try:
with open(filename, "r") as file:
return file.read()
except FileNotFoundError:
print(f"File '{filename}' not found. Using fallback.")
return fallback_content
# 3. Checking file existence before reading
import os
def read_file_safely(filename):
"""Check if file exists before reading"""
if not os.path.exists(filename):
print(f"File '{filename}' does not exist")
return None
if not os.path.isfile(filename):
print(f"'{filename}' is not a regular file")
return None
if not os.access(filename, os.R_OK):
print(f"'{filename}' is not readable")
return None
try:
with open(filename, "r") as file:
return file.read()
except Exception as e:
print(f"Error reading '{filename}': {e}")
return None
# 4. Reading with encoding handling
def read_file_with_encoding(filename, encodings=['utf-8', 'latin-1']):
"""Try multiple encodings when reading a file"""
for encoding in encodings:
try:
with open(filename, "r", encoding=encoding) as file:
return file.read()
except UnicodeDecodeError:
continue
print(f"Could not decode '{filename}' with any encoding")
return None
Error handling key points:
- FileNotFoundError — file doesn't exist
- PermissionError — don't have permission
- IsADirectoryError — trying to read a directory
- UnicodeDecodeError — encoding issues
- Check before reading — use os.path
Quick Check: What are the most common errors when reading files? (Answer: FileNotFoundError, PermissionError, and UnicodeDecodeError)
Try It Yourself
Experiment with reading files in the editor below. Try different methods and see the results.
Loading Pyodide... 0%
Python Code Editor
========================================
READ FILES PRACTICE
========================================
Sample file created: practice.txt
1. READ() - ENTIRE FILE
Line 1: Hello World
Line 2: Python is amazing
Line 3: File reading is easy
Line 4: Practice makes perfect
2. READLINE() - LINE BY LINE
Line 1: Line 1: Hello World
Line 2: Line 2: Python is amazing
3. READLINES() - ALL LINES
Number of lines: 4
Line 1: Line 1: Hello World
Line 2: Line 2: Python is amazing
Line 3: Line 3: File reading is easy
Line 4: Line 4: Practice makes perfect
4. ITERATION - MOST EFFICIENT
Found: Line 2: Python is amazing
5. READING SPECIFIC LINE
Line 2: Line 2: Python is amazing
Read files practice complete!
READ FILES PRACTICE
========================================
Sample file created: practice.txt
1. READ() - ENTIRE FILE
Line 1: Hello World
Line 2: Python is amazing
Line 3: File reading is easy
Line 4: Practice makes perfect
2. READLINE() - LINE BY LINE
Line 1: Line 1: Hello World
Line 2: Line 2: Python is amazing
3. READLINES() - ALL LINES
Number of lines: 4
Line 1: Line 1: Hello World
Line 2: Line 2: Python is amazing
Line 3: Line 3: File reading is easy
Line 4: Line 4: Practice makes perfect
4. ITERATION - MOST EFFICIENT
Found: Line 2: Python is amazing
5. READING SPECIFIC LINE
Line 2: Line 2: Python is amazing
Read files practice complete!
You've Got It!
You now know multiple ways to read files in Python. You understand when to use read(), readline(), readlines(), and iteration for efficient file reading.
Quick Quiz
Test what you've learned:
1. Which method reads the entire file as a string?
2. Which method is most memory efficient for large files?
3. What does readlines() return?
4. What exception is raised when a file doesn't exist?
5. Which module is recommended for reading CSV files?
Frequently Asked Questions
What is the difference between read() and readlines()?
read() returns the entire file as a single string. readlines() returns a list of strings, where each string is a line from the file. Use read() for small files and readlines() when you need to process lines separately.
How do I read a file line by line efficiently?
The most efficient way to read a file line by line is using iteration:
with open('file.txt', 'r') as f: for line in f:. This reads one line at a time and doesn't load the entire file into memory.
How do I handle different encodings when reading files?
You can specify the encoding with the 'encoding' parameter:
with open('file.txt', 'r', encoding='utf-8') as f:. Common encodings include 'utf-8', 'latin-1', and 'cp1252'. You can also try multiple encodings in a try-except block.
What's a common interview question about reading files?
Common questions include: "How do you read a file in Python?" "What is the difference between read() and readline()?" "How would you handle reading a large file?" and "How do you handle file reading errors?"
Can I read a file from the middle?
Yes! You can use the
seek() method to move to a specific position in the file: file.seek(position). Then use file.read() to read from that position. This is useful for reading large files partially.
How do I read a CSV file without the csv module?
You can read a CSV file manually by splitting each line on commas:
with open('file.csv', 'r') as f: rows = [line.strip().split(',') for line in f]. However, the csv module is recommended because it handles edge cases like quoted fields.
Where to Go From Here
Now that you can read files, check out these related topics:
Write to File
Learn how to write data to files effectively.
Learn More →Rename File
Learn how to rename files in Python.
Learn More →Binary Files
Learn how to read and write binary files.
Learn More →