- What are iterators — objects that allow you to traverse sequences
- Iterables vs iterators — the difference between containers and iterators
- The iterator protocol —
__iter__()and__next__() - Custom iterators — creating your own iterator classes
- Built-in functions —
iter(),next(), and more - Real-world use — practical examples you can use
What are Iterators?
An iterator is an object that allows you to traverse through a sequence of values, one at a time. It's like a pointer that keeps track of where you are in a collection.
Think of an iterator like a bookmark in a book. You can move through the pages one at a time, and you always know where you are. You can go to the next page, but you can't go backward (unless you start over).
Iterators are everywhere in Python. When you write a for loop, Python creates an iterator behind the scenes. When you call list() on something, it's using an iterator.
💡 Key concept: An iterator is an object that returns one value at a time from a sequence. It remembers where it is in the sequence.
Iterables vs Iterators
Understanding the Difference
The terms iterable and iterator are often confused. Here's the difference:
- Iterable — an object that can return an iterator. It has an
__iter__()method. Examples: lists, tuples, dictionaries, strings. - Iterator — an object that produces values one at a time. It has
__iter__()and__next__()methods.
All iterators are iterables, but not all iterables are iterators.
# Iterables vs Iterators
print("=" * 50)
print("ITERABLES vs ITERATORS")
print("=" * 50)
# ----- ITERABLES -----
print("\n1. ITERABLES (can return iterators)")
# Lists, tuples, strings, dictionaries are all iterables
my_list = [1, 2, 3]
my_tuple = (4, 5, 6)
my_string = "Hello"
print(f" List: {my_list}")
print(f" Tuple: {my_tuple}")
print(f" String: {my_string}")
# All iterables have __iter__() method
print(f" List has __iter__: {hasattr(my_list, '__iter__')}")
print(f" List has __next__: {hasattr(my_list, '__next__')}")
# Getting an iterator from an iterable
list_iterator = iter(my_list)
print(f" list_iterator: {list_iterator}")
# ----- ITERATORS -----
print("\n2. ITERATORS (produce values one at a time)")
# An iterator has both __iter__ and __next__
print(f" Iterator has __iter__: {hasattr(list_iterator, '__iter__')}")
print(f" Iterator has __next__: {hasattr(list_iterator, '__next__')}")
# Using next() to get values
print(f" next(list_iterator): {next(list_iterator)}")
print(f" next(list_iterator): {next(list_iterator)}")
print(f" next(list_iterator): {next(list_iterator)}")
# After all values are consumed, it raises StopIteration
try:
next(list_iterator)
except StopIteration as e:
print(f" StopIteration raised: No more values!")
# ----- KEY DIFFERENCE -----
print("\n3. KEY DIFFERENCE")
# Iterable: can be iterated over (for loop, list(), etc.)
# Iterator: produces values, keeps state, can be exhausted
print("""
┌─────────────────┬────────────────────────────────────────────────────────────┐
│ ITERABLE │ ITERATOR │
├─────────────────┼────────────────────────────────────────────────────────────┤
│ Has __iter__() │ Has __iter__() and __next__() │
│ Can be used in │ Produces values one at a time │
│ for loops │ │
│ Can be iterated │ Keeps track of position │
│ multiple times │ │
│ Examples: list, │ Once exhausted, can't be reused │
│ tuple, string │ Examples: list_iterator, range_iterator │
└─────────────────┴────────────────────────────────────────────────────────────┘
""")
Iterables vs iterators key points:
- Iterable — has
__iter__(), can be used in for loops - Iterator — has
__iter__()and__next__(), produces values - All iterators are iterables — but not vice versa
- Iterators are stateful — they remember where they are
- Iterators are consumable — once exhausted, they're done
Quick Check: What's the difference between an iterable and an iterator? (Answer: An iterable can return an iterator; an iterator produces values one at a time and keeps state)
The Iterator Protocol
__iter__() and __next__()
The iterator protocol is a set of rules that objects must follow to be considered iterators. It consists of two methods:
__iter__()— returns the iterator object itself (usuallyself)__next__()— returns the next value from the sequence, or raisesStopIteration
When you use a for loop, Python calls __iter__() to get an iterator, then repeatedly calls __next__() until StopIteration is raised.
# The Iterator Protocol
print("=" * 50)
print("THE ITERATOR PROTOCOL")
print("=" * 50)
# ----- EXAMPLE 1: How for loop works -----
print("\n1. HOW A FOR LOOP WORKS")
numbers = [1, 2, 3, 4, 5]
# This for loop:
print("For loop:")
for num in numbers:
print(f" {num}")
# Is equivalent to this:
print("\nEquivalent while loop with iterator:")
iterator = iter(numbers) # Calls __iter__()
while True:
try:
num = next(iterator) # Calls __next__()
print(f" {num}")
except StopIteration:
print(" (StopIteration raised, loop ends)")
break
# ----- EXAMPLE 2: Custom iterator -----
print("\n2. CUSTOM ITERATOR")
class CountDown:
"""Iterator that counts down from a number to 1"""
def __init__(self, start):
self.current = start
self.start = start
def __iter__(self):
"""Return the iterator object itself"""
return self
def __next__(self):
"""Return the next value or raise StopIteration"""
if self.current < 1:
raise StopIteration
value = self.current
self.current -= 1
return value
# Using the custom iterator
countdown = CountDown(5)
print(" Countdown from 5:")
for num in countdown:
print(f" {num}")
# Can only iterate once! The iterator is exhausted
print("\n Iterating again (won't work):")
for num in countdown:
print(f" {num}") # Nothing printed — already exhausted
# ----- EXAMPLE 3: Iterator that can be reset -----
print("\n3. ITERATOR THAT CAN BE RESET")
class ResetableCountDown:
"""Iterator that can be reset to start over"""
def __init__(self, start):
self.start = start
self.reset()
def reset(self):
self.current = self.start
return self
def __iter__(self):
"""Return the iterator object itself"""
return self
def __next__(self):
"""Return the next value or raise StopIteration"""
if self.current < 1:
raise StopIteration
value = self.current
self.current -= 1
return value
counter = ResetableCountDown(3)
print(" First iteration:")
for num in counter:
print(f" {num}")
print(" Reset and iterate again:")
counter.reset()
for num in counter:
print(f" {num}")
Iterator protocol key points:
- __iter__() — returns the iterator object (usually
self) - __next__() — returns the next value, raises
StopIterationwhen done - for loop — uses
iter()andnext()internally - StopIteration — signals that there are no more items
- Iterators are single-use — they're exhausted after iteration
Quick Check: What two methods make up the iterator protocol? (Answer: __iter__() and __next__())
Creating Custom Iterators
Building Your Own Iterator
You can create your own iterator by defining a class that implements __iter__() and __next__(). This is useful when you need to generate values on the fly, like reading a file line by line or generating a sequence.
# Creating Custom Iterators
print("=" * 50)
print("CREATING CUSTOM ITERATORS")
print("=" * 50)
# ----- EXAMPLE 1: Fibonacci Iterator -----
print("\n1. FIBONACCI ITERATOR")
class Fibonacci:
"""Iterator that generates Fibonacci numbers"""
def __init__(self, max_count):
self.max_count = max_count
self.count = 0
self.a = 0
self.b = 1
def __iter__(self):
return self
def __next__(self):
if self.count >= self.max_count:
raise StopIteration
value = self.a
self.a, self.b = self.b, self.a + self.b
self.count += 1
return value
print(" First 10 Fibonacci numbers:")
for num in Fibonacci(10):
print(f" {num}")
# ----- EXAMPLE 2: Prime Number Iterator -----
print("\n2. PRIME NUMBER ITERATOR")
class Primes:
"""Iterator that generates prime numbers"""
def __init__(self, max_count):
self.max_count = max_count
self.count = 0
self.current = 1
def __iter__(self):
return self
def _is_prime(self, num):
if num < 2:
return False
for i in range(2, int(num ** 0.5) + 1):
if num % i == 0:
return False
return True
def __next__(self):
self.count += 1
self.current += 1
# Find the next prime
while not self._is_prime(self.current):
self.current += 1
return self.current
print(" First 10 prime numbers:")
primes = Primes(10)
for prime in primes:
print(f" {prime}")
# ----- EXAMPLE 3: File Reader Iterator -----
print("\n3. FILE READER ITERATOR")
class FileReader:
"""Iterator that reads a file line by line"""
def __init__(self, filename):
self.filename = filename
self.file = None
def __iter__(self):
self.file = open(self.filename, 'r')
return self
def __next__(self):
line = self.file.readline()
if line == '':
self.file.close()
raise StopIteration
return line.rstrip('\n')
# Simulating a file with a list
print(" Reading data line by line:")
data = ["Line 1", "Line 2", "Line 3", "Line 4"]
class LineReader:
def __init__(self, lines):
self.lines = lines
self.index = 0
def __iter__(self):
return self
def __next__(self):
if self.index >= len(self.lines):
raise StopIteration
value = self.lines[self.index]
self.index += 1
return value
reader = LineReader(data)
for line in reader:
print(f" {line}")
# ----- EXAMPLE 4: Infinite Iterator -----
print("\n4. INFINITE ITERATOR (use with caution!)")
class InfiniteCounter:
"""Iterator that counts forever (use with break)"""
def __init__(self, start=0):
self.current = start
def __iter__(self):
return self
def __next__(self):
value = self.current
self.current += 1
return value
print(" Infinite counter (first 5 values):")
counter = InfiniteCounter()
for i, num in enumerate(counter):
if i >= 5:
break
print(f" {num}")
print("\n Infinite iterators must be broken out of!")
print(" They're useful for generators and streaming data")
Custom iterators key points:
- Implement __iter__ and __next__ — the two required methods
- Maintain state — use instance variables to track position
- Raise StopIteration — when there are no more values
- Useful for lazy evaluation — generate values on the fly
- Can be infinite — but must be controlled with a break
Quick Check: What do you raise when an iterator has no more values? (Answer: StopIteration)
Built-in Iterator Functions
Python's Iterator Tools
Python provides several built-in functions and modules for working with iterators. These make working with sequences more convenient.
# Built-in Iterator Functions
print("=" * 50)
print("BUILT-IN ITERATOR FUNCTIONS")
print("=" * 50)
# ----- 1. iter() and next() -----
print("\n1. iter() and next()")
my_list = [10, 20, 30, 40, 50]
iterator = iter(my_list)
print(f" List: {my_list}")
print(f" Iterator: {iterator}")
print(f" next(): {next(iterator)}")
print(f" next(): {next(iterator)}")
print(f" next(): {next(iterator)}")
# With default value (Python 3.5+)
print(f"\n next with default:")
my_iter = iter([1, 2])
print(f" {next(my_iter, 'Done')}")
print(f" {next(my_iter, 'Done')}")
print(f" {next(my_iter, 'Done')}")
# ----- 2. enumerate() -----
print("\n2. enumerate() - adds a counter")
colors = ['red', 'green', 'blue']
for index, color in enumerate(colors):
print(f" {index}: {color}")
for index, color in enumerate(colors, start=1):
print(f" {index}: {color}")
# ----- 3. zip() -----
print("\n3. zip() - combines multiple iterables")
names = ['Alice', 'Bob', 'Charlie']
ages = [25, 30, 35]
cities = ['NYC', 'LA', 'Chicago']
for name, age, city in zip(names, ages, cities):
print(f" {name}, {age}, {city}")
# zip stops at the shortest iterable
short = [1, 2]
for a, b in zip(short, names):
print(f" {a} - {b}")
# ----- 4. map() -----
print("\n4. map() - applies a function to every item")
numbers = [1, 2, 3, 4, 5]
squared = map(lambda x: x ** 2, numbers)
print(f" Original: {numbers}")
print(f" Squared: {list(squared)}")
# ----- 5. filter() -----
print("\n5. filter() - filters items based on a condition")
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
evens = filter(lambda x: x % 2 == 0, numbers)
print(f" Original: {numbers}")
print(f" Evens: {list(evens)}")
# ----- 6. range() -----
print("\n6. range() - generates a sequence of numbers")
for i in range(5):
print(f" {i}", end=" ")
print()
for i in range(2, 10, 2):
print(f" {i}", end=" ")
print()
# ----- 7. reversed() -----
print("\n7. reversed() - iterates in reverse")
for num in reversed([1, 2, 3, 4]):
print(f" {num}", end=" ")
print()
# ----- 8. sorted() -----
print("\n8. sorted() - returns sorted list")
unsorted = [3, 1, 4, 1, 5, 9, 2]
print(f" Unsorted: {unsorted}")
print(f" Sorted: {sorted(unsorted)}")
print(f" Sorted descending: {sorted(unsorted, reverse=True)}")
# ----- 9. any() and all() -----
print("\n9. any() and all() - boolean checks")
numbers = [1, 2, 3, 4, 5]
print(f" Numbers: {numbers}")
print(f" any(num > 4): {any(num > 4 for num in numbers)}")
print(f" all(num > 0): {all(num > 0 for num in numbers)}")
print(f" all(num > 2): {all(num > 2 for num in numbers)}")
# ----- 10. sum(), min(), max() -----
print("\n10. sum(), min(), max()")
print(f" Sum: {sum(numbers)}")
print(f" Min: {min(numbers)}")
print(f" Max: {max(numbers)}")
Built-in functions key points:
- iter() — gets an iterator from an iterable
- next() — gets the next value from an iterator
- enumerate() — adds a counter to iteration
- zip() — combines multiple iterables
- map() — applies a function to each item
- filter() — filters items by a condition
Quick Check: What function adds a counter to an iteration? (Answer: enumerate())
Real-World Example
Building a Data Stream Processor
# Real-World Example: Data Stream Processor
import time
import random
from datetime import datetime
print("=" * 60)
print("DATA STREAM PROCESSOR")
print("=" * 60)
# ============================================================
# DATA SOURCE - Simulated Sensor Data Stream
# ============================================================
class SensorDataStream:
"""Iterator that generates simulated sensor data"""
def __init__(self, sensor_id, interval=0.5):
self.sensor_id = sensor_id
self.interval = interval
self.count = 0
def __iter__(self):
return self
def __next__(self):
# Simulate data generation
time.sleep(self.interval) # Simulate real-time data
self.count += 1
data = {
"sensor_id": self.sensor_id,
"timestamp": datetime.now().isoformat(),
"reading": round(random.uniform(20, 30), 2),
"status": random.choice(["ok", "ok", "ok", "warning"]),
"count": self.count
}
# Stop after 10 readings (for demo)
if self.count >= 10:
raise StopIteration
return data
# ============================================================
# DATA PROCESSORS - Using iterators
# ============================================================
class DataProcessor:
"""Processes data from a stream using iterators"""
def __init__(self, stream):
self.stream = stream
def process_data(self):
"""Process each piece of data"""
for data in self.stream:
yield self._process_record(data)
def _process_record(self, record):
"""Process a single record"""
# Add processing timestamp
record["processed_at"] = datetime.now().isoformat()
# Convert reading to integer
record["reading_int"] = int(record["reading"])
# Add alert if needed
if record["reading"] > 28:
record["alert"] = "High reading!"
elif record["status"] == "warning":
record["alert"] = "Warning status!"
else:
record["alert"] = "Normal"
return record
# ============================================================
# DATA ANALYZER - Using built-in functions
# ============================================================
class DataAnalyzer:
"""Analyzes data using built-in functions"""
@staticmethod
def analyze_readings(readings):
"""Analyze a list of readings"""
if not readings:
return "No data"
return {
"count": len(readings),
"min": min(readings),
"max": max(readings),
"avg": round(sum(readings) / len(readings), 2)
}
@staticmethod
def filter_high(readings, threshold):
"""Filter readings above threshold"""
return list(filter(lambda x: x > threshold, readings))
@staticmethod
def analyze_by_status(records):
"""Group by status"""
status_groups = {}
for record in records:
status = record["status"]
if status not in status_groups:
status_groups[status] = []
status_groups[status].append(record["reading"])
return status_groups
# ============================================================
# DEMONSTRATION
# ============================================================
print("\n1. STARTING SENSOR DATA STREAM")
sensor = SensorDataStream("SENSOR-001", interval=0.3)
print(" Collecting data...")
processor = DataProcessor(sensor)
print("\n2. PROCESSING DATA")
processed_data = []
for record in processor.process_data():
processed_data.append(record)
print(f" Reading: {record['reading']} - {record['alert']}")
readings = [r["reading"] for r in processed_data]
print(f"\n3. DATA ANALYSIS")
analyzer = DataAnalyzer()
analysis = analyzer.analyze_readings(readings)
print(f" Count: {analysis['count']}")
print(f" Min: {analysis['min']}")
print(f" Max: {analysis['max']}")
print(f" Avg: {analysis['avg']}")
print("\n4. FILTER HIGH READINGS")
high_readings = analyzer.filter_high(readings, 26)
print(f" Readings above 26: {high_readings}")
print("\n5. GROUP BY STATUS")
status_groups = analyzer.analyze_by_status(processed_data)
for status, values in status_groups.items():
print(f" {status}: {values}")
print("\n" + "=" * 60)
print("KEY TAKEAWAYS:")
print("=" * 60)
print(" Iterators are perfect for streaming data")
print(" Custom iterators can generate data on demand")
print(" Built-in functions simplify data analysis")
print(" Iterators enable memory-efficient processing")
Real-world example key points:
- SensorDataStream — custom iterator that simulates real-time data
- DataProcessor — processes data using a generator
- DataAnalyzer — uses built-in functions like
filter(),min(),max() - Memory efficient — data is processed one record at a time
- Real-time — can handle streaming data
Quick Check: Why are iterators good for processing large datasets? (Answer: They process data one item at a time without loading everything into memory)
Best Practices
Using Iterators Effectively
# Best Practices for Iterators
print("=" * 60)
print("BEST PRACTICES FOR ITERATORS")
print("=" * 60)
# ============================================================
# 1. USE GENERATORS FOR SIMPLE ITERATORS
# ============================================================
print("\n1. USE GENERATORS FOR SIMPLE ITERATORS")
# DO: Use generator functions for simple sequences
def countdown(n):
"""Generator that counts down from n to 1"""
while n > 0:
yield n
n -= 1
print(" Generator countdown:")
for num in countdown(5):
print(f" {num}")
# DON'T: Write a class for simple sequences
class CountdownClass:
def __init__(self, n):
self.n = n
self.current = n
def __iter__(self):
return self
def __next__(self):
if self.current < 1:
raise StopIteration
value = self.current
self.current -= 1
return value
# This is more code than needed for simple cases
# ============================================================
# 2. USE GENERATOR EXPRESSIONS FOR SIMPLE TRANSFORMATIONS
# ============================================================
print("\n2. USE GENERATOR EXPRESSIONS")
# DO: Use generator expressions for simple transformations
numbers = [1, 2, 3, 4, 5]
squares = (x ** 2 for x in numbers)
print(f" Squares: {list(squares)}")
# DO: Use generator expressions with built-in functions
evens = (x for x in numbers if x % 2 == 0)
print(f" Evens: {list(evens)}")
# DON'T: Use list comprehensions for large datasets (they load everything)
# big_squares = [x**2 for x in range(10000000)] # Bad for memory
# ============================================================
# 3. HANDLE StopIteration PROPERLY
# ============================================================
print("\n3. HANDLE StopIterATION PROPERLY")
# DO: Use next() with default value
my_iter = iter([1, 2])
print(f" next with default: {next(my_iter, 'empty')}")
print(f" next with default: {next(my_iter, 'empty')}")
print(f" next with default: {next(my_iter, 'empty')}")
# DO: Use for loops (they handle StopIteration automatically)
for value in [1, 2, 3]:
print(f" {value}")
# ============================================================
# 4. MAKE ITERATORS REUSABLE WHEN NEEDED
# ============================================================
print("\n4. MAKE ITERATORS REUSABLE")
# DO: Return a new iterator each time
class Repeatable:
def __init__(self, data):
self.data = data
def __iter__(self):
return iter(self.data) # Returns a NEW iterator
repeatable = Repeatable([1, 2, 3])
print(" First iteration:")
for x in repeatable:
print(f" {x}")
print(" Second iteration:")
for x in repeatable:
print(f" {x}")
# DON'T: Return self (makes single-use iterators)
class SingleUse:
def __init__(self, data):
self.data = data
self.index = 0
def __iter__(self):
return self
def __next__(self):
if self.index >= len(self.data):
raise StopIteration
value = self.data[self.index]
self.index += 1
return value
# ============================================================
# 5. USE itertools FOR ADVANCED ITERATION
# ============================================================
print("\n5. USE itertools FOR ADVANCED ITERATION")
import itertools
# Infinite cycles
colors = ['red', 'green', 'blue']
cycle = itertools.cycle(colors)
print(" Cycle (first 5):", end=" ")
for i in range(5):
print(next(cycle), end=" ")
print()
# Combinations
items = ['A', 'B', 'C']
combs = list(itertools.combinations(items, 2))
print(f" Combinations of {items}: {combs}")
# Chain
list1 = [1, 2, 3]
list2 = [4, 5, 6]
chained = list(itertools.chain(list1, list2))
print(f" Chain {list1} + {list2}: {chained}")
# ============================================================
# 6. SUMMARY
# ============================================================
print("\n" + "=" * 60)
print("BEST PRACTICES SUMMARY")
print("=" * 60)
print("""
┌─────────────────────────────┬─────────────────────────────────────────────┐
│ PRACTICE │ WHY IT MATTERS │
├─────────────────────────────┼─────────────────────────────────────────────┤
│ Use generators for simple │ Less code, more readable │
│ iterators │ │
│ │ │
│ Use generator expressions │ Memory efficient for large datasets │
│ │ │
│ Handle StopIteration │ Avoid errors when iterating │
│ properly │ │
│ │ │
│ Make iterators reusable │ Can iterate multiple times │
│ when needed │ │
│ │ │
│ Use itertools for advanced │ Powerful iteration tools built in │
│ iteration │ │
└─────────────────────────────┴─────────────────────────────────────────────┘
REMEMBER:
• Iterators are memory efficient
• Use generators for simple cases
• Use itertools for complex iteration
• Know when to use for loops vs manual iteration
""")
Best practices summary:
- Use generators — simpler than creating iterator classes
- Use generator expressions — memory efficient for large data
- Handle StopIteration — use default values or for loops
- Make iterators reusable — return new iterators when needed
- Use itertools — powerful iteration tools
Quick Check: What's the simplest way to create an iterator in Python? (Answer: Use a generator function with yield)
Try It Yourself
Experiment with iterators in the editor below.
ITERATORS - PRACTICE
==================================================
1. CUSTOM ITERATOR - NUMBER RANGE
MyRange(0, 5):
0
1
2
3
4
MyRange(10, 0, -2):
10
8
6
4
2
2. GENERATOR FUNCTION
Fibonacci(8):
0
1
1
2
3
5
8
13
3. GENERATOR EXPRESSION
Squares generator:
First 3 squares: 1, 4, 9
Evens: [2, 4, 6, 8, 10]
4. BUILT-IN FUNCTIONS
Data: [15, 8, 22, 4, 17, 9, 31, 12]
Sorted: [4, 8, 9, 12, 15, 17, 22, 31]
Filtered (> 10): [15, 22, 17, 31, 12]
Mapped (x2): [30, 16, 44, 8, 34, 18, 62, 24]
Any > 30: True
All > 0: True
You've Got It!
You now understand iterators in Python. You know how to create custom iterators, use built-in functions, and work with iterables and iterators effectively.
Quick Quiz
Test what you've learned:
Frequently Asked Questions
What is an iterator in Python?
__iter__() and __next__() methods, and raises StopIteration when no more values are available.
What's the difference between iterable and iterator?
__iter__() method that returns an iterator. An iterator is an object that produces values one at a time and keeps track of its position. All iterators are iterables, but not all iterables are iterators.
When should I use a generator vs an iterator class?
yield) for simple sequences and when you need to generate values on the fly. Use an iterator class when you need to maintain complex state or when you need multiple methods beyond just iteration.
Can I iterate over the same iterator twice?
StopIteration if you try to use it again. To iterate multiple times, you need to create a new iterator or use an iterable that returns a fresh iterator each time.
What is the itertools module used for?
itertools module provides a collection of fast, memory-efficient tools for working with iterators. It includes functions for infinite iterators (count, cycle), combinatoric generators (permutations, combinations), and other useful iterators (chain, islice).
Why use iterators instead of lists?
Where to Go From Here
Now that you understand iterators in Python, check out these related topics:
Generators
Learn about generators — a simpler way to create iterators.
Learn More →Decorators
Learn how decorators work with iterators and generators.
Learn More →Collections Module
Learn about collections that work with iterators.
Learn More →