- What is code optimization — making your code run faster and use less memory
- Why optimize — better performance, user experience
- Measure first — always profile before optimizing
- Efficient data structures — choosing the right one
- Optimizing loops — avoid common performance pitfalls
- List comprehensions — faster than loops
What is Code Optimization?
Code optimization is the process of making your code run faster, use less memory, or both. It's about writing code that does the same thing but more efficiently.
Think of code optimization like packing a suitcase. You want to fit as much as possible in the smallest space. Some things you can fold better (optimize loops), some things you can leave behind (remove unnecessary code), and some things you can use instead of others (better data structures).
But here's the most important rule: Don't optimize too early. First, make it work. Then, make it right. Then, make it fast.
💡 Key concept: Code optimization is about making your code efficient, but always measure first and only optimize when necessary.
Why Optimize?
The Benefits of Optimization
Let's see why optimization matters with a real example.
# Why Optimize?
print("=" * 50)
print("WHY OPTIMIZE?")
print("=" * 50)
# ============================================================
# SLOW CODE - Takes too long
# ============================================================
print("\n1. SLOW CODE")
def slow_function(n):
"""Slow implementation"""
result = []
for i in range(n):
for j in range(n):
for k in range(n):
result.append(i * j * k)
return len(result)
print(" This function does n^3 operations")
print(" For n=100, it does 1,000,000 operations")
print(" For n=1000, it does 1,000,000,000 operations")
# ============================================================
# FAST CODE - Much better
# ============================================================
print("\n2. FAST CODE")
def fast_function(n):
"""Optimized implementation"""
# O(n^2) instead of O(n^3)
result = []
for i in range(n):
for j in range(n):
result.append(i * j)
return len(result) * n # Same result, less work
print(" This function does n^2 operations")
print(" For n=100, it does 10,000 operations")
print(" 100x faster!")
print(" For n=1000, it does 1,000,000 operations")
print(" 1000x faster!")
# ============================================================
# REAL IMPACT
# ============================================================
print("\n3. REAL IMPACT")
print("""
┌─────────────────────────────────────────────────────────────────────┐
│ Why optimization matters: │
├─────────────────────────────────────────────────────────────────────┤
│ • Faster response times for users │
│ • Less CPU usage (lower costs) │
│ • Less memory usage (can handle more users) │
│ • Better scalability │
│ • Reduced infrastructure costs │
│ • Better user experience │
│ • Can handle larger datasets │
└─────────────────────────────────────────────────────────────────────┘
Example:
A web API that handles 1 million requests per day
Saving 10ms per request = 10,000 seconds saved per day
= 2.7 hours of saved time every day!
""")
Benefits of optimization:
- Faster code — better user experience
- Less resources — lower costs
- Better scalability — handle more users
- Handle larger data — process more information
- Environmental impact — less energy consumption
Quick Check: What's the most important rule of optimization? (Answer: Measure first, then optimize. Don't optimize too early.)
Measure Before Optimizing
Profile Your Code First
Before you start optimizing, you need to know what to optimize. Always measure first. Without measurement, you might optimize the wrong thing.
# Measure Before Optimizing
import time
import cProfile
import pstats
print("=" * 50)
print("MEASURE BEFORE OPTIMIZING")
print("=" * 50)
# ============================================================
# SIMPLE TIMING
# ============================================================
print("\n1. SIMPLE TIMING")
def slow_function():
result = []
for i in range(10000):
result.append(i ** 2)
return result
def fast_function():
return [i ** 2 for i in range(10000)]
# Time the slow function
start = time.time()
slow_function()
slow_time = time.time() - start
# Time the fast function
start = time.time()
fast_function()
fast_time = time.time() - start
print(f" Slow function: {slow_time:.4f} seconds")
print(f" Fast function: {fast_time:.4f} seconds")
print(f" Speed improvement: {(slow_time / fast_time):.1f}x")
# ============================================================
# USING cProfile (Profiling Tool)
# ============================================================
print("\n2. USING cProfile")
print("""
# Profile your code to find bottlenecks
import cProfile
def my_function():
total = 0
for i in range(10000):
total += i ** 2
return total
# Profile the function
cProfile.run('my_function()')
# Or save to a file
cProfile.run('my_function()', 'profile_output.prof')
# View the results
import pstats
p = pstats.Stats('profile_output.prof')
p.sort_stats('time').print_stats(10) # Top 10 slowest
""")
# ============================================================
# USING timeit for Micro-benchmarks
# ============================================================
print("\n3. USING timeit")
print("""
import timeit
# Time a single function
time = timeit.timeit('sum(range(100))', number=10000)
print(f"Time: {time:.4f}s")
# Time a function with setup
time = timeit.timeit(
'my_function()',
setup='def my_function(): return sum(range(100))',
number=10000
)
# From the command line
# python -m timeit "sum(range(100))"
""")
# ============================================================
# FINDING BOTTLENECKS
# ============================================================
print("\n4. FINDING BOTTLENECKS")
print("""
Common bottlenecks:
1. Loops (especially nested loops)
2. Function calls (too many small functions)
3. String operations (concatenation in loops)
4. I/O operations (database, files, network)
5. Data structure operations (searching in lists)
Always measure before optimizing!
"Premature optimization is the root of all evil."
- Donald Knuth
""")
Measuring key points:
- Time it — use
time.time()for simple timing - Profile it — use
cProfileto find bottlenecks - Benchmark it — use
timeitfor micro-benchmarks - Measure first — don't guess what's slow
Quick Check: What tool should you use to find bottlenecks? (Answer: cProfile)
Efficient Data Structures
Choosing the Right Data Structure
Choosing the right data structure can make a huge difference in performance.
# Efficient Data Structures
import time
from collections import defaultdict
print("=" * 50)
print("EFFICIENT DATA STRUCTURES")
print("=" * 50)
# ============================================================
# LIST vs SET for Membership Testing
# ============================================================
print("\n1. LIST vs SET for Membership Testing")
# List (O(n) for membership)
my_list = list(range(10000))
# Set (O(1) for membership)
my_set = set(range(10000))
# Test membership
test_value = 5000
# List timing
start = time.time()
for _ in range(10000):
test_value in my_list
list_time = time.time() - start
# Set timing
start = time.time()
for _ in range(10000):
test_value in my_set
set_time = time.time() - start
print(f" List membership: {list_time:.4f}s")
print(f" Set membership: {set_time:.4f}s")
print(f" Set is {(list_time/set_time):.1f}x faster")
print("\n Use SET when you need membership testing (in operator)")
print(" Use LIST when you need ordered data")
# ============================================================
# DICT vs LIST for Lookup
# ============================================================
print("\n2. DICT vs LIST for Lookup")
# Build a list of key-value pairs
data_list = [(i, f"value_{i}") for i in range(10000)]
# Build a dictionary
data_dict = {i: f"value_{i}" for i in range(10000)}
# Lookup by key
key = 5000
# List lookup (O(n) - linear search)
start = time.time()
for _ in range(1000):
for k, v in data_list:
if k == key:
break
list_lookup = time.time() - start
# Dict lookup (O(1))
start = time.time()
for _ in range(1000):
data_dict[key]
dict_lookup = time.time() - start
print(f" List lookup: {list_lookup:.4f}s")
print(f" Dict lookup: {dict_lookup:.4f}s")
print(f" Dict is {(list_lookup/dict_lookup):.1f}x faster")
print("\n Use DICT for key-value lookups")
print(" Use LIST for ordered data")
# ============================================================
# DATA STRUCTURE COMPARISON
# ============================================================
print("\n3. DATA STRUCTURE COMPARISON")
print("""
┌──────────────────┬──────────────────────┬──────────────────────────────┐
│ Data Structure │ Best For │ Operation Complexity │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ List │ Ordered data, │ Access: O(1), Search: O(n) │
│ │ Stack, Queue │ Insert/Delete: O(n) │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Tuple │ Immutable data │ Access: O(1), Search: O(n) │
│ │ Fixed size │ │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Set │ Unique values, │ Membership: O(1) │
│ │ Membership tests │ Insert/Delete: O(1) │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Dict │ Key-value pairs, │ Lookup: O(1) │
│ │ Fast lookups │ Insert/Delete: O(1) │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Deque │ Fast appends/pops │ Append/Pop: O(1) │
│ │ from both ends │ │
└──────────────────┴──────────────────────┴──────────────────────────────┘
""")
Data structures key points:
- List — ordered data, but slow for membership
- Set — fast membership (in), unique values
- Dict — fast key lookups
- Tuple — immutable, memory efficient
Quick Check: Which data structure is fastest for membership testing? (Answer: Set — O(1) vs list O(n))
Optimizing Loops
Make Your Loops Faster
Loops are often where code spends most of its time. Here are ways to make them faster.
# Optimizing Loops
import time
print("=" * 50)
print("OPTIMIZING LOOPS")
print("=" * 50)
# ============================================================
# AVOID LOOKUPS IN LOOPS
# ============================================================
print("\n1. AVOID LOOKUPS IN LOOPS")
# Slow - attribute lookup in loop
class Calculator:
def __init__(self):
self.multiplier = 2
calc = Calculator()
data = list(range(10000))
start = time.time()
result = []
for x in data:
result.append(x * calc.multiplier)
slow_time = time.time() - start
# Fast - local variable before loop
start = time.time()
multiplier = calc.multiplier
result = []
for x in data:
result.append(x * multiplier)
fast_time = time.time() - start
print(f" Slow (lookup in loop): {slow_time:.4f}s")
print(f" Fast (local variable): {fast_time:.4f}s")
print(f" Speedup: {(slow_time/fast_time):.1f}x")
# ============================================================
# USE RANGE LENGTH INSTEAD OF LEN
# ============================================================
print("\n2. USE RANGE LENGTH INSTEAD OF LEN")
data = list(range(10000))
# Slow - calling len() each time
start = time.time()
for i in range(len(data)):
x = data[i]
slow_time = time.time() - start
# Fast - direct iteration
start = time.time()
for x in data:
pass
fast_time = time.time() - start
print(f" Slow (range(len)): {slow_time:.4f}s")
print(f" Fast (direct iteration): {fast_time:.4f}s")
print(f" Speedup: {(slow_time/fast_time):.1f}x")
# ============================================================
# AVOID NESTED LOOPS
# ============================================================
print("\n3. AVOID NESTED LOOPS")
# Slow - nested loops
start = time.time()
result = []
for i in range(100):
for j in range(100):
result.append(i + j)
slow_time = time.time() - start
# Fast - using list comprehension (faster)
start = time.time()
result = [i + j for i in range(100) for j in range(100)]
fast_time = time.time() - start
print(f" Slow (nested loops): {slow_time:.4f}s")
print(f" Fast (list comprehension): {fast_time:.4f}s")
print(f" Speedup: {(slow_time/fast_time):.1f}x")
# ============================================================
# MOVE CONSTANT EXPRESSIONS OUTSIDE LOOPS
# ============================================================
print("\n4. MOVE CONSTANT EXPRESSIONS OUTSIDE LOOPS")
# Slow - calculating inside loop
start = time.time()
result = []
for x in range(10000):
result.append(x * 2 + 3 + 5)
slow_time = time.time() - start
# Fast - calculate constant once
start = time.time()
constant = 2 + 3 + 5
result = []
for x in range(10000):
result.append(x * constant)
fast_time = time.time() - start
print(f" Slow (constant inside loop): {slow_time:.4f}s")
print(f" Fast (constant outside): {fast_time:.4f}s")
print(f" Speedup: {(slow_time/fast_time):.1f}x")
Loop optimization key points:
- Local variables — use local variables instead of attribute lookups
- Direct iteration — use
for x in datainstead offor i in range(len(data)) - Avoid nested loops — use list comprehensions when possible
- Move constants out — don't calculate constants inside loops
Quick Check: Why is direct iteration faster than range(len())? (Answer: Direct iteration avoids index lookups and is more efficient)
List Comprehensions
Faster and Cleaner Than Loops
List comprehensions are often faster than loops and more readable.
# List Comprehensions
import time
print("=" * 50)
print("LIST COMPREHENSIONS")
print("=" * 50)
# ============================================================
# LIST COMPREHENSION vs FOR LOOP
# ============================================================
print("\n1. LIST COMPREHENSION vs FOR LOOP")
data = list(range(10000))
# Loop
start = time.time()
result = []
for x in data:
result.append(x ** 2)
loop_time = time.time() - start
# Comprehension
start = time.time()
result = [x ** 2 for x in data]
comp_time = time.time() - start
print(f" Loop time: {loop_time:.4f}s")
print(f" Comprehension time: {comp_time:.4f}s")
print(f" Comprehension is {(loop_time/comp_time):.1f}x faster")
# ============================================================
# WITH CONDITION (filter)
# ============================================================
print("\n2. WITH CONDITION (filter)")
data = list(range(10000))
# Loop with if
start = time.time()
result = []
for x in data:
if x % 2 == 0:
result.append(x ** 2)
loop_time = time.time() - start
# Comprehension with if
start = time.time()
result = [x ** 2 for x in data if x % 2 == 0]
comp_time = time.time() - start
print(f" Loop with if time: {loop_time:.4f}s")
print(f" Comprehension with if time: {comp_time:.4f}s")
print(f" Comprehension is {(loop_time/comp_time):.1f}x faster")
# ============================================================
# NESTED COMPREHENSIONS
# ============================================================
print("\n3. NESTED COMPREHENSIONS")
# Nested loops
start = time.time()
result = []
for i in range(100):
for j in range(100):
result.append(i * j)
loop_time = time.time() - start
# Nested comprehension
start = time.time()
result = [i * j for i in range(100) for j in range(100)]
comp_time = time.time() - start
print(f" Nested loops time: {loop_time:.4f}s")
print(f" Nested comprehension time: {comp_time:.4f}s")
print(f" Comprehension is {(loop_time/comp_time):.1f}x faster")
# ============================================================
# DICT COMPREHENSIONS
# ============================================================
print("\n4. DICT COMPREHENSIONS")
data = list(range(10000))
# Dict comprehension
result = {x: x ** 2 for x in data}
print(f" Dict comprehension: {len(result)} items")
# ============================================================
# SET COMPREHENSIONS
# ============================================================
print("\n5. SET COMPREHENSIONS")
data = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4]
# Set comprehension (removes duplicates)
result = {x ** 2 for x in data}
print(f" Set comprehension: {result}")
# ============================================================
# GENERATOR EXPRESSIONS (Memory Efficient)
# ============================================================
print("\n6. GENERATOR EXPRESSIONS")
# List comprehension (stores all in memory)
list_comp = [x ** 2 for x in range(1000000)]
print(f" List comprehension memory: ~8MB")
# Generator expression (lazy, memory efficient)
gen_expr = (x ** 2 for x in range(1000000))
print(f" Generator expression memory: ~small")
print(" Use generator expressions for large datasets!")
List comprehensions key points:
- Faster — list comprehensions are faster than loops
- Cleaner — more readable than loops
- Can filter — use
ifcondition - Generator expressions — use
()for memory efficiency
Quick Check: What's the syntax for a list comprehension? (Answer: [expression for item in iterable if condition])
Real-World Example
Optimizing a Data Processing Pipeline
# Real-World Example: Data Processing Pipeline
import time
from collections import Counter
print("=" * 60)
print("DATA PROCESSING PIPELINE OPTIMIZATION")
print("=" * 60)
# ============================================================
# SLOW VERSION - Not Optimized
# ============================================================
def slow_process(data):
"""Slow version - many inefficiencies"""
result = []
# Inefficient loop
for i in range(len(data)):
item = data[i]
# Inefficient string concatenation
text = ""
for char in item:
text = text + char # Bad! Creates new string each time
# Inefficient list membership
if text in result: # O(n) check
continue
# Inefficient append
result.append(text)
return result
# ============================================================
# FAST VERSION - Optimized
# ============================================================
def fast_process(data):
"""Fast version - optimized"""
# Use list comprehension with ''.join()
# and set for membership testing
# Process each item efficiently
processed = []
seen = set()
for item in data:
# Efficient string join
text = ''.join(item)
# Efficient membership test (O(1))
if text in seen:
continue
seen.add(text)
processed.append(text)
return processed
# ============================================================
# EVEN FASTER - Using built-ins
# ============================================================
def fastest_process(data):
"""Fastest version - using built-in functions"""
# Convert each list to string using ''.join
# Use set to remove duplicates
# Convert back to list
processed = {''.join(item) for item in data}
return list(processed)
# ============================================================
# DEMONSTRATION
# ============================================================
print("\n1. GENERATING TEST DATA")
import random
def generate_data(n):
"""Generate test data"""
chars = 'abcdefghijklmnopqrstuvwxyz'
data = []
for _ in range(n):
# Create a list of characters
word = [random.choice(chars) for _ in range(random.randint(3, 8))]
data.append(word)
return data
data = generate_data(10000)
print(f" Generated {len(data)} items")
print("\n2. BENCHMARKING")
# Slow version
print(" Slow version:")
start = time.time()
slow_result = slow_process(data)
slow_time = time.time() - start
print(f" Time: {slow_time:.4f}s")
# Fast version
print(" Fast version:")
start = time.time()
fast_result = fast_process(data)
fast_time = time.time() - start
print(f" Time: {fast_time:.4f}s")
# Fastest version
print(" Fastest version:")
start = time.time()
fastest_result = fastest_process(data)
fastest_time = time.time() - start
print(f" Time: {fastest_time:.4f}s")
print("\n3. COMPARISON")
print(f" Slow vs Fast: {slow_time/fast_time:.1f}x faster")
print(f" Slow vs Fastest: {slow_time/fastest_time:.1f}x faster")
# Verify results are the same
print(f"\n Results match: {set(fast_result) == set(fastest_result)}")
print("\n4. OPTIMIZATION TECHNIQUES USED")
print("""
✅ Used ''.join() instead of string concatenation in loop
✅ Used set for O(1) membership testing
✅ Used list comprehension for processing
✅ Used generator expression for memory efficiency
✅ Avoided range(len()) pattern
✅ Used set comprehension for deduplication
""")
print("\n" + "=" * 60)
print("KEY TAKEAWAYS:")
print("=" * 60)
print("""
- Measure before optimizing
- Use the right data structures
- Avoid nested loops when possible
- Use list comprehensions
- Use set for membership testing
- Use ''.join() for string concatenation
- Use generator expressions for large data
- Profile your code to find bottlenecks
""")
Real-world example key points:
- string join — use
''.join()instead of+in loops - set membership — use set for O(1) lookups
- comprehensions — faster and cleaner
- generator expressions — memory efficient
Quick Check: What's the best way to concatenate many strings? (Answer: Use ''.join(list))
Best Practices
Code Optimization Guidelines
# Best Practices for Code Optimization
print("=" * 60)
print("BEST PRACTICES FOR CODE OPTIMIZATION")
print("=" * 60)
# ============================================================
# 1. MEASURE BEFORE OPTIMIZING
# ============================================================
print("\n1. MEASURE BEFORE OPTIMIZING")
print("""
# Good - measure first
import time
start = time.time()
result = my_function()
print(f"Time: {time.time() - start}")
# Or use cProfile
import cProfile
cProfile.run('my_function()')
# Or use timeit
import timeit
timeit.timeit('my_function()', number=1000)
""")
# ============================================================
# 2. USE THE RIGHT DATA STRUCTURE
# ============================================================
print("\n2. USE THE RIGHT DATA STRUCTURE")
print("""
# Good - use set for membership
seen = set()
if item in seen: # O(1)
pass
# Bad - use list for membership
seen = []
if item in seen: # O(n)
pass
# Good - use dict for lookups
data = {}
value = data.get(key) # O(1)
# Bad - use list for lookups
data = []
for k, v in data:
if k == key: # O(n)
break
""")
# ============================================================
# 3. AVOID PREMATURE OPTIMIZATION
# ============================================================
print("\n3. AVOID PREMATURE OPTIMIZATION")
print("""
# First, make it work
def process_data(data):
result = []
for item in data:
result.append(item * 2)
return result
# Then, make it right (if needed)
def process_data_fast(data):
return [item * 2 for item in data]
# Only optimize when there's a real performance issue
""")
# ============================================================
# 4. USE BUILT-IN FUNCTIONS
# ============================================================
print("\n4. USE BUILT-IN FUNCTIONS")
print("""
# Good - use built-in functions
total = sum(numbers)
max_value = max(numbers)
min_value = min(numbers)
sorted_list = sorted(data)
# Bad - implement yourself
total = 0
for n in numbers:
total += n
# Built-in functions are implemented in C and are faster
""")
# ============================================================
# 5. AVOID GLOBAL LOOKUPS
# ============================================================
print("\n5. AVOID GLOBAL LOOKUPS")
print("""
# Good - local variable
def fast_function(data):
multiplier = 2 # Local
for x in data:
result.append(x * multiplier)
# Bad - global lookup
GLOBAL_MULTIPLIER = 2
def slow_function(data):
for x in data:
result.append(x * GLOBAL_MULTIPLIER) # Global lookup each time
""")
# ============================================================
# 6. USE GENERATORS FOR LARGE DATA
# ============================================================
print("\n6. USE GENERATORS FOR LARGE DATA")
print("""
# Good - generator (memory efficient)
def read_large_file():
for line in open('large_file.txt'):
yield line
# Bad - reading all at once
def read_large_file():
return open('large_file.txt').readlines() # Memory heavy
""")
# ============================================================
# 7. SUMMARY
# ============================================================
print("\n" + "=" * 60)
print("BEST PRACTICES SUMMARY")
print("=" * 60)
print("""
- Measure before optimizing
- Use the right data structure
- Don't optimize prematurely
- Use built-in functions
- Avoid global lookups
- Use generators for large data
- Use list comprehensions
- Profile your code
- Optimize loops (local variables, avoid nested loops)
- Use set for membership tests
- Use dict for lookups
- Use ''.join() for string concatenation
""")
Best practices summary:
- Measure first — always profile before optimizing
- Right data structure — choose based on operations
- Don't prematurely optimize — make it work first
- Use built-ins — implemented in C, faster
- Use generators — for memory efficiency
Quick Check: Should you optimize code before it works? (Answer: No — make it work, then make it right, then make it fast)
Try It Yourself
Experiment with code optimization in the editor below.
CODE OPTIMIZATION - PRACTICE
==================================================
1. SLOW vs FAST
Slow sum: 0.0456s
Fast sum: 0.0123s
Built-in sum is 3.7x faster
2. LIST vs SET
List membership: 0.0234s
Set membership: 0.0012s
Set is 19.5x faster
3. LOOP vs COMPREHENSION
Loop time: 0.0789s
Comprehension time: 0.0345s
Comprehension is 2.3x faster
You've Got It!
You now understand code optimization in Python. You know how to measure performance, choose the right data structures, optimize loops, and use list comprehensions.
Quick Quiz
Test what you've learned:
Frequently Asked Questions
What is code optimization?
When should I optimize my code?
What's faster: list comprehension or for loop?
Why is set faster than list for membership tests?
What's the best tool for profiling Python code?
Should I use generators or lists?
Where to Go From Here
Now that you understand code optimization, check out these related topics:
Debugging
Learn techniques for debugging Python code.
Learn More →Logging
Learn how to add logging to your applications.
Learn More →PEP 8
Learn about Python's style guide for clean code.
Learn More →