Python

P
PythonHub
Progress 0%

Python

  • Home
  • History of Python
  • Applications of Python
  • Introduction To Python
    • What is Python
    • Character Set
    • Tokens in Python
    • Python Execution Mode
    • Variable And Identifiers
    • Data Types in Python
    • Operators And Expressions
    • Constants in Python
    • Assignment Statement
    • Input / Output in Python
    • Simple Python Scripts
    • Namespace in Python
    • 📝 Assignments
  • Operators in Python
    • Arithmetic Operators
    • Assignment Operators
    • Shorthand Assignment Operators
    • Relational Operators
    • Logical Operators
    • Bitwise Operators
    • Special Operators
    • 📝 Assignments
  • Input Output in Python
    • Accept Input
    • Output Formatting
    • 📝 Assignments
  • Conditional Statement
    • Decision Making
    • if Statement
    • IF-ELSE STATEMENT
    • IF-ELSE LADDER
    • NESTED IF-ELSE
    • Short Hand IF-ELSE
    • 📝 Assignments
  • Loops
    • Introduction to Loops
    • While Loop
    • Nested While Loop
    • 📝 While Loop Assignments
    • For Loop
    • For Loop Examples
    • Nested For Loop
    • Nested For Loop Examples
    • Infinite While Loops
    • Infinite For Loops
    • Break, Continue and Else in Loops
    • Difference Between For and While Loop
    • For Each Loop
    • 📝 For Each Assignments
    • 📝 All Loops Assignments
  • List
    • List in Python
    • Access List Elements
    • List Functions
    • Iterate (Loop) List
    • List Comprehension
    • 📝 Assignments
  • Tuple
    • Tuple in Python
    • Access Tuple Elements
    • Tuple Functions
    • Iterate (Loop) Tuple
    • Unpack Tuple
    • Tuple Comprehension
    • 📝 Assignments
  • Set
    • Set in Python
    • Access Set Elements
    • Set Methods
    • Iterate (Loop) Set
    • Pack/Unpack Set
    • Set Comprehension
    • 📝 Assignments
  • Dictionary
    • Dictionary
    • Access Dictionary Items
    • Dictionary Methods
    • Iterate (Loop) Dictionary
    • Formatting Dictionaries
    • Nested Dictionaries
    • Dictionary Comprehension
    • 📝 Assignments
  • Diff List Tuple Set Dictionary
    • List vs Tuple
    • List vs Set
    • List vs Dictionary
    • Tuple vs Set
    • Tuple vs Dictionary
    • Dictionary vs Set
    • 📝 Assignments
  • Exception
    • Error vs Exception
    • Types of Exception
    • Exception Handling
    • User Defined Exception
    • Logging Exception
    • 📝 Assignments
  • Functions
    • Introduction to Functions
    • Modular Programming
    • Types of Functions
    • Inbuilt Functions
    • Need For User-Defined Function
    • Elements of User Defined Function
    • Function Arguments
    • Nesting of Functions
    • Recursion
    • Global Local and Non Local
    • Python Lambda Functions
    • 📝 Assignments
  • Python Module
    • Introduction to Module
    • Inbuilt Modules in Python
    • User Defined Module
    • 📝 Assignments
  • File Handling
    • Introduction to Files
    • Create File
    • Read Files
    • Write to File
    • Rename File
    • Copy File
    • Move File
    • List Files in Directory
    • Binary Files
    • Zipping and Unzipping Files
    • 📝 Assignments
  • Strings
    • Basics of Strings
    • String Special Operators
    • String Formatting Operators
    • String Methods
    • 📝 Assignments
  • Regular Expressions
  • Python OOPS
    • Basics of Object Oriented
    • What are Classes and Objects?
    • Creating Class and Object
    • OOP vs Procedural Programming
    • Difference Between Classes and Objects
    • Constructors
    • Destructor
    • Built Class Methods and Attributes
    • Class and Instance Variables
    • Inheritance in Python
    • Single Inheritance
    • Multiple Inheritance
    • Multilevel Inheritance
    • Hierarchical Inheritance
    • Hybrid Inheritance
    • Abstraction
    • Method Overriding
    • Abstract Method
    • Interfaces in Python
    • Abstract Class vs Interface
    • Public, Private and Protected
    • Overloading vs Overriding
    • Inheritance vs Composition
    • Encapsulation
    • Polymorphism
    • Inner Classes
    • 📝 Assignments
  • Advanced Python
    • Iterator in Python
    • Generator in Python
    • Decorator in Python
    • Type Hints in Python
    • Async/Await Programming
    • Dataclasses in Python
    • Context Managers in Python
    • Property Decorator in Python
    • __slots__ in Python
    • Enums in Python
    • Walrus Operator in Python
    • Match-Case in Python
    • 📝 Assignments
  • Python Standard Library
    • Collections Module
    • Itertools Module
    • Functools Module
    • Datetime Module
    • JSON Module
    • OS Module
    • Sys Module
    • Random Module
    • Math Module
    • 📝 Assignments
  • Python Testing
    • Unit Testing in Python
    • Pytest Framework
    • Mocking in Python
    • 📝 Assignments
  • Python Best Practices
    • PEP 8 Style Guide
    • Docstrings in Python
    • Logging in Python
    • Code Optimization Tips
    • Debugging Techniques
    • 📝 Assignments
  • MySQL Database in Python
    • Introduction to MySQL with Python
    • DBMS vs File System
    • Connecting to MySQL Database
    • Create Database in MySQL
    • Create Table in MySQL
    • Insert Data in MySQL
    • Insert Multiple Rows
    • Select Data from MySQL
    • WHERE Clause in MySQL
    • Update Data in MySQL
    • Delete Data from MySQL
    • Parameterized Queries
    • Transaction Management
    • Error Handling
    • Connection Pooling
    • MySQL Drivers Guide
    • Joins in MySQL
    • Aggregation Functions
    • Backup MySQL Database
    • Best Practices
    • 📝 Assignments
  • MySQL Database Operations
    • SELECT Statement
    • MySQL Operators
    • DDL Statements
    • DML Statements
    • Subqueries
    • JOIN Operations
    • Aggregation
    • Case Study
    • 📝 Assignments
  • Graphics in Python
  • Threads in Python
    • Introduction to Threads
    • Process vs Threads
    • Concurrent Programming & GIL
    • Uses of Threads
    • Creating Threads
    • Single Tasking
    • Multi Tasking
    • Thread Synchronization
    • 📝 Assignments
  • Interview Questions & Answers
  • Python Case Studies
  • Multiple Choice Questions
  • 📝 Practice Problems
Get in Touch
  • tech2dsm@gmail.com

© Sankalan Data Tech

Python Language Interactive Tutorial

Python: Code Optimization

Python Code Optimization - Complete Guide

Write faster, more efficient Python code.

Created by Sankalan Data Tech Team Verified
Data Engineers, Analysts, Scientists & Trainers
Created by experienced Python developers, data engineers, and data scientists to make programming easy through practical examples, real-world experience, and clear explanations.
On this page:
  • What is Code Optimization?
  • Why Optimize?
  • Measure Before Optimizing
  • Efficient Data Structures
  • Optimizing Loops
  • List Comprehensions
  • Real-World Example
  • Best Practices
  • Try It Yourself
  • Quick Quiz
  • Frequently Asked Questions
Share this tutorial:
Twitter LinkedIn Facebook WhatsApp Reddit Telegram Email Copy Link
What You'll Learn Here
  • What is code optimization — making your code run faster and use less memory
  • Why optimize — better performance, user experience
  • Measure first — always profile before optimizing
  • Efficient data structures — choosing the right one
  • Optimizing loops — avoid common performance pitfalls
  • List comprehensions — faster than loops

What is Code Optimization?

Code optimization is the process of making your code run faster, use less memory, or both. It's about writing code that does the same thing but more efficiently.

Think of code optimization like packing a suitcase. You want to fit as much as possible in the smallest space. Some things you can fold better (optimize loops), some things you can leave behind (remove unnecessary code), and some things you can use instead of others (better data structures).

But here's the most important rule: Don't optimize too early. First, make it work. Then, make it right. Then, make it fast.

💡 Key concept: Code optimization is about making your code efficient, but always measure first and only optimize when necessary.

Why Optimize?

1

The Benefits of Optimization

Let's see why optimization matters with a real example.

# Why Optimize?

print("=" * 50)
print("WHY OPTIMIZE?")
print("=" * 50)

# ============================================================
# SLOW CODE - Takes too long
# ============================================================

print("\n1. SLOW CODE")

def slow_function(n):
    """Slow implementation"""
    result = []
    for i in range(n):
        for j in range(n):
            for k in range(n):
                result.append(i * j * k)
    return len(result)

print("   This function does n^3 operations")
print("   For n=100, it does 1,000,000 operations")
print("   For n=1000, it does 1,000,000,000 operations")


# ============================================================
# FAST CODE - Much better
# ============================================================

print("\n2. FAST CODE")

def fast_function(n):
    """Optimized implementation"""
    # O(n^2) instead of O(n^3)
    result = []
    for i in range(n):
        for j in range(n):
            result.append(i * j)
    return len(result) * n  # Same result, less work

print("   This function does n^2 operations")
print("   For n=100, it does 10,000 operations")
print("   100x faster!")
print("   For n=1000, it does 1,000,000 operations")
print("   1000x faster!")


# ============================================================
# REAL IMPACT
# ============================================================

print("\n3. REAL IMPACT")

print("""
┌─────────────────────────────────────────────────────────────────────┐
│ Why optimization matters:                                           │
├─────────────────────────────────────────────────────────────────────┤
│ • Faster response times for users                                   │
│ • Less CPU usage (lower costs)                                      │
│ • Less memory usage (can handle more users)                        │
│ • Better scalability                                                │
│ • Reduced infrastructure costs                                      │
│ • Better user experience                                           │
│ • Can handle larger datasets                                       │
└─────────────────────────────────────────────────────────────────────┘

Example:
    A web API that handles 1 million requests per day
    Saving 10ms per request = 10,000 seconds saved per day
    = 2.7 hours of saved time every day!
""")

Benefits of optimization:

  • Faster code — better user experience
  • Less resources — lower costs
  • Better scalability — handle more users
  • Handle larger data — process more information
  • Environmental impact — less energy consumption

Quick Check: What's the most important rule of optimization? (Answer: Measure first, then optimize. Don't optimize too early.)

Measure Before Optimizing

2

Profile Your Code First

Before you start optimizing, you need to know what to optimize. Always measure first. Without measurement, you might optimize the wrong thing.

# Measure Before Optimizing

import time
import cProfile
import pstats

print("=" * 50)
print("MEASURE BEFORE OPTIMIZING")
print("=" * 50)

# ============================================================
# SIMPLE TIMING
# ============================================================

print("\n1. SIMPLE TIMING")

def slow_function():
    result = []
    for i in range(10000):
        result.append(i ** 2)
    return result

def fast_function():
    return [i ** 2 for i in range(10000)]

# Time the slow function
start = time.time()
slow_function()
slow_time = time.time() - start

# Time the fast function
start = time.time()
fast_function()
fast_time = time.time() - start

print(f"   Slow function: {slow_time:.4f} seconds")
print(f"   Fast function: {fast_time:.4f} seconds")
print(f"   Speed improvement: {(slow_time / fast_time):.1f}x")


# ============================================================
# USING cProfile (Profiling Tool)
# ============================================================

print("\n2. USING cProfile")

print("""
# Profile your code to find bottlenecks
import cProfile

def my_function():
    total = 0
    for i in range(10000):
        total += i ** 2
    return total

# Profile the function
cProfile.run('my_function()')

# Or save to a file
cProfile.run('my_function()', 'profile_output.prof')

# View the results
import pstats
p = pstats.Stats('profile_output.prof')
p.sort_stats('time').print_stats(10)  # Top 10 slowest
""")


# ============================================================
# USING timeit for Micro-benchmarks
# ============================================================

print("\n3. USING timeit")

print("""
import timeit

# Time a single function
time = timeit.timeit('sum(range(100))', number=10000)
print(f"Time: {time:.4f}s")

# Time a function with setup
time = timeit.timeit(
    'my_function()',
    setup='def my_function(): return sum(range(100))',
    number=10000
)

# From the command line
# python -m timeit "sum(range(100))"
""")


# ============================================================
# FINDING BOTTLENECKS
# ============================================================

print("\n4. FINDING BOTTLENECKS")

print("""
Common bottlenecks:
    1. Loops (especially nested loops)
    2. Function calls (too many small functions)
    3. String operations (concatenation in loops)
    4. I/O operations (database, files, network)
    5. Data structure operations (searching in lists)

Always measure before optimizing!
    "Premature optimization is the root of all evil."
    - Donald Knuth
""")

Measuring key points:

  • Time it — use time.time() for simple timing
  • Profile it — use cProfile to find bottlenecks
  • Benchmark it — use timeit for micro-benchmarks
  • Measure first — don't guess what's slow

Quick Check: What tool should you use to find bottlenecks? (Answer: cProfile)

Efficient Data Structures

3

Choosing the Right Data Structure

Choosing the right data structure can make a huge difference in performance.

# Efficient Data Structures

import time
from collections import defaultdict

print("=" * 50)
print("EFFICIENT DATA STRUCTURES")
print("=" * 50)

# ============================================================
# LIST vs SET for Membership Testing
# ============================================================

print("\n1. LIST vs SET for Membership Testing")

# List (O(n) for membership)
my_list = list(range(10000))

# Set (O(1) for membership)
my_set = set(range(10000))

# Test membership
test_value = 5000

# List timing
start = time.time()
for _ in range(10000):
    test_value in my_list
list_time = time.time() - start

# Set timing
start = time.time()
for _ in range(10000):
    test_value in my_set
set_time = time.time() - start

print(f"   List membership: {list_time:.4f}s")
print(f"   Set membership: {set_time:.4f}s")
print(f"   Set is {(list_time/set_time):.1f}x faster")

print("\n   Use SET when you need membership testing (in operator)")
print("   Use LIST when you need ordered data")


# ============================================================
# DICT vs LIST for Lookup
# ============================================================

print("\n2. DICT vs LIST for Lookup")

# Build a list of key-value pairs
data_list = [(i, f"value_{i}") for i in range(10000)]

# Build a dictionary
data_dict = {i: f"value_{i}" for i in range(10000)}

# Lookup by key
key = 5000

# List lookup (O(n) - linear search)
start = time.time()
for _ in range(1000):
    for k, v in data_list:
        if k == key:
            break
list_lookup = time.time() - start

# Dict lookup (O(1))
start = time.time()
for _ in range(1000):
    data_dict[key]
dict_lookup = time.time() - start

print(f"   List lookup: {list_lookup:.4f}s")
print(f"   Dict lookup: {dict_lookup:.4f}s")
print(f"   Dict is {(list_lookup/dict_lookup):.1f}x faster")

print("\n   Use DICT for key-value lookups")
print("   Use LIST for ordered data")

# ============================================================
# DATA STRUCTURE COMPARISON
# ============================================================

print("\n3. DATA STRUCTURE COMPARISON")

print("""
┌──────────────────┬──────────────────────┬──────────────────────────────┐
│ Data Structure   │ Best For             │ Operation Complexity         │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ List             │ Ordered data,        │ Access: O(1), Search: O(n)  │
│                  │ Stack, Queue         │ Insert/Delete: O(n)         │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Tuple            │ Immutable data       │ Access: O(1), Search: O(n)  │
│                  │ Fixed size           │                             │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Set              │ Unique values,       │ Membership: O(1)            │
│                  │ Membership tests     │ Insert/Delete: O(1)         │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Dict             │ Key-value pairs,     │ Lookup: O(1)                │
│                  │ Fast lookups         │ Insert/Delete: O(1)         │
├──────────────────┼──────────────────────┼──────────────────────────────┤
│ Deque            │ Fast appends/pops    │ Append/Pop: O(1)            │
│                  │ from both ends       │                             │
└──────────────────┴──────────────────────┴──────────────────────────────┘
""")

Data structures key points:

  • List — ordered data, but slow for membership
  • Set — fast membership (in), unique values
  • Dict — fast key lookups
  • Tuple — immutable, memory efficient

Quick Check: Which data structure is fastest for membership testing? (Answer: Set — O(1) vs list O(n))

Optimizing Loops

4

Make Your Loops Faster

Loops are often where code spends most of its time. Here are ways to make them faster.

# Optimizing Loops

import time

print("=" * 50)
print("OPTIMIZING LOOPS")
print("=" * 50)

# ============================================================
# AVOID LOOKUPS IN LOOPS
# ============================================================

print("\n1. AVOID LOOKUPS IN LOOPS")

# Slow - attribute lookup in loop
class Calculator:
    def __init__(self):
        self.multiplier = 2

calc = Calculator()
data = list(range(10000))

start = time.time()
result = []
for x in data:
    result.append(x * calc.multiplier)
slow_time = time.time() - start

# Fast - local variable before loop
start = time.time()
multiplier = calc.multiplier
result = []
for x in data:
    result.append(x * multiplier)
fast_time = time.time() - start

print(f"   Slow (lookup in loop): {slow_time:.4f}s")
print(f"   Fast (local variable): {fast_time:.4f}s")
print(f"   Speedup: {(slow_time/fast_time):.1f}x")


# ============================================================
# USE RANGE LENGTH INSTEAD OF LEN
# ============================================================

print("\n2. USE RANGE LENGTH INSTEAD OF LEN")

data = list(range(10000))

# Slow - calling len() each time
start = time.time()
for i in range(len(data)):
    x = data[i]
slow_time = time.time() - start

# Fast - direct iteration
start = time.time()
for x in data:
    pass
fast_time = time.time() - start

print(f"   Slow (range(len)): {slow_time:.4f}s")
print(f"   Fast (direct iteration): {fast_time:.4f}s")
print(f"   Speedup: {(slow_time/fast_time):.1f}x")


# ============================================================
# AVOID NESTED LOOPS
# ============================================================

print("\n3. AVOID NESTED LOOPS")

# Slow - nested loops
start = time.time()
result = []
for i in range(100):
    for j in range(100):
        result.append(i + j)
slow_time = time.time() - start

# Fast - using list comprehension (faster)
start = time.time()
result = [i + j for i in range(100) for j in range(100)]
fast_time = time.time() - start

print(f"   Slow (nested loops): {slow_time:.4f}s")
print(f"   Fast (list comprehension): {fast_time:.4f}s")
print(f"   Speedup: {(slow_time/fast_time):.1f}x")


# ============================================================
# MOVE CONSTANT EXPRESSIONS OUTSIDE LOOPS
# ============================================================

print("\n4. MOVE CONSTANT EXPRESSIONS OUTSIDE LOOPS")

# Slow - calculating inside loop
start = time.time()
result = []
for x in range(10000):
    result.append(x * 2 + 3 + 5)
slow_time = time.time() - start

# Fast - calculate constant once
start = time.time()
constant = 2 + 3 + 5
result = []
for x in range(10000):
    result.append(x * constant)
fast_time = time.time() - start

print(f"   Slow (constant inside loop): {slow_time:.4f}s")
print(f"   Fast (constant outside): {fast_time:.4f}s")
print(f"   Speedup: {(slow_time/fast_time):.1f}x")

Loop optimization key points:

  • Local variables — use local variables instead of attribute lookups
  • Direct iteration — use for x in data instead of for i in range(len(data))
  • Avoid nested loops — use list comprehensions when possible
  • Move constants out — don't calculate constants inside loops

Quick Check: Why is direct iteration faster than range(len())? (Answer: Direct iteration avoids index lookups and is more efficient)

List Comprehensions

5

Faster and Cleaner Than Loops

List comprehensions are often faster than loops and more readable.

# List Comprehensions

import time

print("=" * 50)
print("LIST COMPREHENSIONS")
print("=" * 50)

# ============================================================
# LIST COMPREHENSION vs FOR LOOP
# ============================================================

print("\n1. LIST COMPREHENSION vs FOR LOOP")

data = list(range(10000))

# Loop
start = time.time()
result = []
for x in data:
    result.append(x ** 2)
loop_time = time.time() - start

# Comprehension
start = time.time()
result = [x ** 2 for x in data]
comp_time = time.time() - start

print(f"   Loop time: {loop_time:.4f}s")
print(f"   Comprehension time: {comp_time:.4f}s")
print(f"   Comprehension is {(loop_time/comp_time):.1f}x faster")


# ============================================================
# WITH CONDITION (filter)
# ============================================================

print("\n2. WITH CONDITION (filter)")

data = list(range(10000))

# Loop with if
start = time.time()
result = []
for x in data:
    if x % 2 == 0:
        result.append(x ** 2)
loop_time = time.time() - start

# Comprehension with if
start = time.time()
result = [x ** 2 for x in data if x % 2 == 0]
comp_time = time.time() - start

print(f"   Loop with if time: {loop_time:.4f}s")
print(f"   Comprehension with if time: {comp_time:.4f}s")
print(f"   Comprehension is {(loop_time/comp_time):.1f}x faster")


# ============================================================
# NESTED COMPREHENSIONS
# ============================================================

print("\n3. NESTED COMPREHENSIONS")

# Nested loops
start = time.time()
result = []
for i in range(100):
    for j in range(100):
        result.append(i * j)
loop_time = time.time() - start

# Nested comprehension
start = time.time()
result = [i * j for i in range(100) for j in range(100)]
comp_time = time.time() - start

print(f"   Nested loops time: {loop_time:.4f}s")
print(f"   Nested comprehension time: {comp_time:.4f}s")
print(f"   Comprehension is {(loop_time/comp_time):.1f}x faster")


# ============================================================
# DICT COMPREHENSIONS
# ============================================================

print("\n4. DICT COMPREHENSIONS")

data = list(range(10000))

# Dict comprehension
result = {x: x ** 2 for x in data}
print(f"   Dict comprehension: {len(result)} items")


# ============================================================
# SET COMPREHENSIONS
# ============================================================

print("\n5. SET COMPREHENSIONS")

data = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4]

# Set comprehension (removes duplicates)
result = {x ** 2 for x in data}
print(f"   Set comprehension: {result}")


# ============================================================
# GENERATOR EXPRESSIONS (Memory Efficient)
# ============================================================

print("\n6. GENERATOR EXPRESSIONS")

# List comprehension (stores all in memory)
list_comp = [x ** 2 for x in range(1000000)]
print(f"   List comprehension memory: ~8MB")

# Generator expression (lazy, memory efficient)
gen_expr = (x ** 2 for x in range(1000000))
print(f"   Generator expression memory: ~small")

print("   Use generator expressions for large datasets!")

List comprehensions key points:

  • Faster — list comprehensions are faster than loops
  • Cleaner — more readable than loops
  • Can filter — use if condition
  • Generator expressions — use () for memory efficiency

Quick Check: What's the syntax for a list comprehension? (Answer: [expression for item in iterable if condition])

Real-World Example

6

Optimizing a Data Processing Pipeline

# Real-World Example: Data Processing Pipeline

import time
from collections import Counter

print("=" * 60)
print("DATA PROCESSING PIPELINE OPTIMIZATION")
print("=" * 60)

# ============================================================
# SLOW VERSION - Not Optimized
# ============================================================

def slow_process(data):
    """Slow version - many inefficiencies"""
    result = []
    
    # Inefficient loop
    for i in range(len(data)):
        item = data[i]
        
        # Inefficient string concatenation
        text = ""
        for char in item:
            text = text + char  # Bad! Creates new string each time
        
        # Inefficient list membership
        if text in result:  # O(n) check
            continue
        
        # Inefficient append
        result.append(text)
    
    return result


# ============================================================
# FAST VERSION - Optimized
# ============================================================

def fast_process(data):
    """Fast version - optimized"""
    
    # Use list comprehension with ''.join()
    # and set for membership testing
    
    # Process each item efficiently
    processed = []
    seen = set()
    
    for item in data:
        # Efficient string join
        text = ''.join(item)
        
        # Efficient membership test (O(1))
        if text in seen:
            continue
        
        seen.add(text)
        processed.append(text)
    
    return processed


# ============================================================
# EVEN FASTER - Using built-ins
# ============================================================

def fastest_process(data):
    """Fastest version - using built-in functions"""
    # Convert each list to string using ''.join
    # Use set to remove duplicates
    # Convert back to list
    processed = {''.join(item) for item in data}
    return list(processed)


# ============================================================
# DEMONSTRATION
# ============================================================

print("\n1. GENERATING TEST DATA")
import random

def generate_data(n):
    """Generate test data"""
    chars = 'abcdefghijklmnopqrstuvwxyz'
    data = []
    for _ in range(n):
        # Create a list of characters
        word = [random.choice(chars) for _ in range(random.randint(3, 8))]
        data.append(word)
    return data

data = generate_data(10000)
print(f"   Generated {len(data)} items")

print("\n2. BENCHMARKING")

# Slow version
print("   Slow version:")
start = time.time()
slow_result = slow_process(data)
slow_time = time.time() - start
print(f"      Time: {slow_time:.4f}s")

# Fast version
print("   Fast version:")
start = time.time()
fast_result = fast_process(data)
fast_time = time.time() - start
print(f"      Time: {fast_time:.4f}s")

# Fastest version
print("   Fastest version:")
start = time.time()
fastest_result = fastest_process(data)
fastest_time = time.time() - start
print(f"      Time: {fastest_time:.4f}s")

print("\n3. COMPARISON")
print(f"   Slow vs Fast: {slow_time/fast_time:.1f}x faster")
print(f"   Slow vs Fastest: {slow_time/fastest_time:.1f}x faster")

# Verify results are the same
print(f"\n   Results match: {set(fast_result) == set(fastest_result)}")

print("\n4. OPTIMIZATION TECHNIQUES USED")
print("""
   ✅ Used ''.join() instead of string concatenation in loop
   ✅ Used set for O(1) membership testing
   ✅ Used list comprehension for processing
   ✅ Used generator expression for memory efficiency
   ✅ Avoided range(len()) pattern
   ✅ Used set comprehension for deduplication
""")

print("\n" + "=" * 60)
print("KEY TAKEAWAYS:")
print("=" * 60)
print("""
- Measure before optimizing
- Use the right data structures
- Avoid nested loops when possible
- Use list comprehensions
- Use set for membership testing
- Use ''.join() for string concatenation
- Use generator expressions for large data
- Profile your code to find bottlenecks
""")

Real-world example key points:

  • string join — use ''.join() instead of + in loops
  • set membership — use set for O(1) lookups
  • comprehensions — faster and cleaner
  • generator expressions — memory efficient

Quick Check: What's the best way to concatenate many strings? (Answer: Use ''.join(list))

Best Practices

7

Code Optimization Guidelines

# Best Practices for Code Optimization

print("=" * 60)
print("BEST PRACTICES FOR CODE OPTIMIZATION")
print("=" * 60)

# ============================================================
# 1. MEASURE BEFORE OPTIMIZING
# ============================================================

print("\n1. MEASURE BEFORE OPTIMIZING")

print("""
# Good - measure first
import time
start = time.time()
result = my_function()
print(f"Time: {time.time() - start}")

# Or use cProfile
import cProfile
cProfile.run('my_function()')

# Or use timeit
import timeit
timeit.timeit('my_function()', number=1000)
""")


# ============================================================
# 2. USE THE RIGHT DATA STRUCTURE
# ============================================================

print("\n2. USE THE RIGHT DATA STRUCTURE")

print("""
# Good - use set for membership
seen = set()
if item in seen:  # O(1)
    pass

# Bad - use list for membership
seen = []
if item in seen:  # O(n)
    pass

# Good - use dict for lookups
data = {}
value = data.get(key)  # O(1)

# Bad - use list for lookups
data = []
for k, v in data:
    if k == key:  # O(n)
        break
""")


# ============================================================
# 3. AVOID PREMATURE OPTIMIZATION
# ============================================================

print("\n3. AVOID PREMATURE OPTIMIZATION")

print("""
# First, make it work
def process_data(data):
    result = []
    for item in data:
        result.append(item * 2)
    return result

# Then, make it right (if needed)
def process_data_fast(data):
    return [item * 2 for item in data]

# Only optimize when there's a real performance issue
""")


# ============================================================
# 4. USE BUILT-IN FUNCTIONS
# ============================================================

print("\n4. USE BUILT-IN FUNCTIONS")

print("""
# Good - use built-in functions
total = sum(numbers)
max_value = max(numbers)
min_value = min(numbers)
sorted_list = sorted(data)

# Bad - implement yourself
total = 0
for n in numbers:
    total += n

# Built-in functions are implemented in C and are faster
""")


# ============================================================
# 5. AVOID GLOBAL LOOKUPS
# ============================================================

print("\n5. AVOID GLOBAL LOOKUPS")

print("""
# Good - local variable
def fast_function(data):
    multiplier = 2  # Local
    for x in data:
        result.append(x * multiplier)

# Bad - global lookup
GLOBAL_MULTIPLIER = 2
def slow_function(data):
    for x in data:
        result.append(x * GLOBAL_MULTIPLIER)  # Global lookup each time
""")


# ============================================================
# 6. USE GENERATORS FOR LARGE DATA
# ============================================================

print("\n6. USE GENERATORS FOR LARGE DATA")

print("""
# Good - generator (memory efficient)
def read_large_file():
    for line in open('large_file.txt'):
        yield line

# Bad - reading all at once
def read_large_file():
    return open('large_file.txt').readlines()  # Memory heavy
""")


# ============================================================
# 7. SUMMARY
# ============================================================

print("\n" + "=" * 60)
print("BEST PRACTICES SUMMARY")
print("=" * 60)
print("""
- Measure before optimizing
- Use the right data structure
- Don't optimize prematurely
- Use built-in functions
- Avoid global lookups
- Use generators for large data
- Use list comprehensions
- Profile your code
- Optimize loops (local variables, avoid nested loops)
- Use set for membership tests
- Use dict for lookups
- Use ''.join() for string concatenation
""")

Best practices summary:

  • Measure first — always profile before optimizing
  • Right data structure — choose based on operations
  • Don't prematurely optimize — make it work first
  • Use built-ins — implemented in C, faster
  • Use generators — for memory efficiency

Quick Check: Should you optimize code before it works? (Answer: No — make it work, then make it right, then make it fast)

Try It Yourself

Experiment with code optimization in the editor below.

Loading Pyodide... 0%
Python Code Editor
==================================================
CODE OPTIMIZATION - PRACTICE
==================================================

1. SLOW vs FAST
Slow sum: 0.0456s
Fast sum: 0.0123s
Built-in sum is 3.7x faster

2. LIST vs SET
List membership: 0.0234s
Set membership: 0.0012s
Set is 19.5x faster

3. LOOP vs COMPREHENSION
Loop time: 0.0789s
Comprehension time: 0.0345s
Comprehension is 2.3x faster
🏆

You've Got It!

You now understand code optimization in Python. You know how to measure performance, choose the right data structures, optimize loops, and use list comprehensions.

Quick Quiz

Test what you've learned:

1. What should you do before optimizing code?
2. Which data structure is fastest for membership testing?
3. What's faster than a for loop for creating lists?
4. What should you avoid doing in a loop?
5. What's the best way to concatenate many strings?

Frequently Asked Questions

What is code optimization? ▼

Code optimization is the process of improving the performance of code — making it run faster, use less memory, or both. It involves measuring performance, finding bottlenecks, and applying techniques to improve efficiency.

When should I optimize my code? ▼

Optimize only when you have a performance problem. First, make your code work correctly. Then, if it's too slow or uses too much memory, measure to find the bottleneck. Only optimize the parts that are actually slow.

What's faster: list comprehension or for loop? ▼

List comprehensions are generally faster than for loops because they are optimized in C and avoid the overhead of repeated method calls. They also make your code more readable.

Why is set faster than list for membership tests? ▼

Set uses a hash table for O(1) average lookup time. List requires a linear scan O(n). For large data, the difference is huge. Always use set when you need to check if an item exists.

What's the best tool for profiling Python code? ▼

cProfile is the built-in profiler and works well for most cases. For more detailed profiling, tools like py-spy, line_profiler, and memory_profiler can provide more insight. Start with cProfile for most needs.

Should I use generators or lists? ▼

Use lists when you need to access data multiple times or need random access. Use generators when working with large datasets where you only need to iterate once. Generators are memory efficient but can only be used once.

Where to Go From Here

Now that you understand code optimization, check out these related topics:

Debugging

Learn techniques for debugging Python code.

Learn More →

Logging

Learn how to add logging to your applications.

Learn More →

PEP 8

Learn about Python's style guide for clean code.

Learn More →
Interview Resources
  • Python Syntax & Variables Interview Questions
  • Top SQL Interview Questions & Answers
  • SQL Joins: Displaying Data from Multiple Tables FAQ
  • Python Lists and Dictionaries Interview Questions
  • Python OOP Interview Questions
  • SQL Set Operators Interview Questions
Previous: Logging Next: Debugging →