Skip to content

Python itertools: Complete Guide to Iterator Building Blocks

Soren Atelier
Name
Soren Atelier

Updated on

Writing loops is second nature to every Python developer. But look closer at most codebases and you will find the same patterns appearing again and again: flattening nested lists, generating combinations, grouping sorted data, slicing iterators, accumulating running totals. Each one is typically solved with a hand-rolled for loop, temporary variables, and index juggling. The code works, but it is verbose, error-prone, and slow when datasets grow.

These problems compound. A nested loop over two lists produces a Cartesian product, but writing it manually obscures intent. Generating all length-3 combinations from a collection requires careful index management. Grouping consecutive elements by a key needs a state variable that is easy to get wrong. Every one of these patterns has been solved, tested, and optimized in Python's itertools module -- a standard-library toolkit that turns multi-line loop constructs into single, readable function calls that produce memory-efficient iterators.

This guide covers every major function in itertools with practical, runnable examples. By the end, you will have a reference for replacing verbose loop patterns with clean, performant iterator pipelines.

📚

    Importing itertools

    The itertools module is part of Python's standard library. No installation is needed.

    import itertools
 
# Or import specific functions
from itertools import chain, combinations, groupby, islice

    Infinite Iterators

    These functions produce iterators that never terminate on their own. You must use islice(), takewhile(), or another stopping mechanism to avoid infinite loops.

    count() -- Arithmetic Sequences

    count(start, step) generates evenly spaced values beginning at start, incrementing by step (default 1).

    from itertools import count, islice
 
# Count from 10, step 2
counter = count(10, 2)
print(list(islice(counter, 6)))
# [10, 12, 14, 16, 18, 20]
 
# Floating point sequences
floats = count(0.0, 0.25)
print(list(islice(floats, 5)))
# [0.0, 0.25, 0.5, 0.75, 1.0]
 
# Label rows with an index
names = ["Alice", "Bob", "Charlie"]
for idx, name in zip(count(1), names):
    print(f"{idx}. {name}")
# 1. Alice
# 2. Bob
# 3. Charlie

    cycle() -- Repeat a Sequence Forever

    cycle(iterable) saves a copy of the iterable and yields elements from it repeatedly.

    from itertools import cycle, islice
 
colors = cycle(["red", "green", "blue"])
print(list(islice(colors, 7)))
# ['red', 'green', 'blue', 'red', 'green', 'blue', 'red']
 
# Round-robin assignment
tasks = ["task_a", "task_b", "task_c", "task_d", "task_e"]
workers = cycle(["Worker1", "Worker2", "Worker3"])
assignments = {task: worker for task, worker in zip(tasks, workers)}
print(assignments)
# {'task_a': 'Worker1', 'task_b': 'Worker2', 'task_c': 'Worker3',
#  'task_d': 'Worker1', 'task_e': 'Worker2'}

    repeat() -- Yield the Same Value

    repeat(value, times) yields value either infinitely or a fixed number of times.

    from itertools import repeat
 
# Fixed repetitions
print(list(repeat("hello", 3)))
# ['hello', 'hello', 'hello']
 
# Use repeat as a constant argument in map()
import operator
bases = [2, 3, 4, 5]
squared = list(map(operator.pow, bases, repeat(2)))
print(squared)
# [4, 9, 16, 25]

    Finite Iterators: Slicing and Filtering

    These functions consume one or more input iterables and produce a finite output.

    chain() -- Concatenate Multiple Iterables

    chain(*iterables) yields elements from the first iterable until it is exhausted, then from the next, and so on. chain.from_iterable() accepts a single iterable of iterables.

    from itertools import chain
 
# Chain multiple lists
a = [1, 2, 3]
b = [4, 5, 6]
c = [7, 8, 9]
print(list(chain(a, b, c)))
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
 
# Flatten one level of nesting
nested = [[1, 2], [3, 4], [5, 6]]
print(list(chain.from_iterable(nested)))
# [1, 2, 3, 4, 5, 6]
 
# Combine a generator with a list
def evens():
    yield 2; yield 4; yield 6
 
print(list(chain(evens(), [8, 10])))
# [2, 4, 6, 8, 10]

    compress() -- Filter by Selectors

    compress(data, selectors) returns elements from data where the corresponding element in selectors is truthy.

    from itertools import compress
 
data = ["A", "B", "C", "D", "E"]
selectors = [1, 0, 1, 0, 1]
print(list(compress(data, selectors)))
# ['A', 'C', 'E']
 
# Use with boolean conditions
values = [10, 25, 3, 42, 7, 18]
mask = [v > 15 for v in values]
print(list(compress(values, mask)))
# [25, 42, 18]

    islice() -- Slice Any Iterator

    islice(iterable, stop) or islice(iterable, start, stop, step) works like slice notation but on any iterator, without building a list first.

    from itertools import islice, count
 
# First 5 elements of an infinite counter
print(list(islice(count(100), 5)))
# [100, 101, 102, 103, 104]
 
# Elements 3 through 8
print(list(islice(range(20), 3, 9)))
# [3, 4, 5, 6, 7, 8]
 
# Every 3rd element from the first 15
print(list(islice(range(100), 0, 15, 3)))
# [0, 3, 6, 9, 12]
 
# Read first 5 lines from a file iterator
# lines = list(islice(open("data.txt"), 5))

    takewhile() and dropwhile() -- Conditional Slicing

    takewhile(predicate, iterable) yields elements as long as the predicate is true, then stops. dropwhile(predicate, iterable) skips elements while the predicate is true, then yields the rest.

    from itertools import takewhile, dropwhile
 
data = [1, 3, 5, 7, 2, 4, 6, 8]
 
# Take elements while they are less than 6
print(list(takewhile(lambda x: x < 6, data)))
# [1, 3, 5]
 
# Drop elements while they are less than 6
print(list(dropwhile(lambda x: x < 6, data)))
# [7, 2, 4, 6, 8]

    Note that takewhile stops at the first False; it does not resume if later elements satisfy the predicate again.

    starmap() -- Apply a Function to Pre-Grouped Arguments

    starmap(function, iterable) applies a function using argument tuples from the iterable. This is equivalent to function(*args) for each args in the iterable.

    from itertools import starmap
import operator
 
pairs = [(2, 5), (3, 4), (10, 3)]
print(list(starmap(operator.mul, pairs)))
# [10, 12, 30]
 
# Calculate hypotenuse for multiple triangles
import math
triangles = [(3, 4), (5, 12), (8, 15)]
print(list(starmap(math.hypot, triangles)))
# [5.0, 13.0, 17.0]

    filterfalse() -- Inverse Filter

    filterfalse(predicate, iterable) yields elements for which the predicate returns False. It is the complement of the built-in filter().

    from itertools import filterfalse
 
numbers = range(10)
# Keep odd numbers (where "is_even" is false)
odd = list(filterfalse(lambda x: x % 2 == 0, numbers))
print(odd)
# [1, 3, 5, 7, 9]

    Combinatoric Iterators

    These functions produce all possible arrangements of input elements. They are central to brute-force search, testing, and mathematical computation.

    product() -- Cartesian Product

    itertools.product(*iterables, repeat=1) replaces nested for-loops over multiple sequences.

    from itertools import product
 
# Two lists
colors = ["red", "blue"]
sizes = ["S", "M", "L"]
print(list(product(colors, sizes)))
# [('red', 'S'), ('red', 'M'), ('red', 'L'),
#  ('blue', 'S'), ('blue', 'M'), ('blue', 'L')]
 
# Equivalent to:
# [(c, s) for c in colors for s in sizes]
 
# Dice rolls: two six-sided dice
dice = range(1, 7)
all_rolls = list(product(dice, repeat=2))
print(f"Total combinations: {len(all_rolls)}")
# Total combinations: 36
print(all_rolls[:5])
# [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]

    permutations() -- All Orderings

    permutations(iterable, r) generates all possible orderings of length r from the input. Order matters, so (A, B) and (B, A) are both included.

    from itertools import permutations
 
# All 2-letter arrangements from 'ABC'
print(list(permutations("ABC", 2)))
# [('A', 'B'), ('A', 'C'), ('B', 'A'),
#  ('B', 'C'), ('C', 'A'), ('C', 'B')]
 
# Full permutations (all elements)
print(list(permutations([1, 2, 3])))
# [(1, 2, 3), (1, 3, 2), (2, 1, 3),
#  (2, 3, 1), (3, 1, 2), (3, 2, 1)]
 
# Number of permutations: n! / (n-r)!
import math
n, r = 5, 3
print(f"P({n},{r}) = {math.perm(n, r)}")
# P(5,3) = 60

    combinations() -- Unique Subsets

    combinations(iterable, r) generates all unique subsets of length r. Order does not matter, so (A, B) is included but (B, A) is not.

    from itertools import combinations
 
# All 2-element combinations from [1, 2, 3, 4]
print(list(combinations([1, 2, 3, 4], 2)))
# [(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]
 
# Pick 3 toppings from a menu
toppings = ["cheese", "pepperoni", "mushrooms", "onions", "peppers"]
combos = list(combinations(toppings, 3))
print(f"Number of 3-topping combinations: {len(combos)}")
# Number of 3-topping combinations: 10
for c in combos[:3]:
    print(c)
# ('cheese', 'pepperoni', 'mushrooms')
# ('cheese', 'pepperoni', 'onions')
# ('cheese', 'pepperoni', 'peppers')

    combinations_with_replacement() -- Subsets with Repeats

    Elements can be selected more than once.

    from itertools import combinations_with_replacement
 
# Coin denominations: pick 3 coins
coins = [1, 5, 10, 25]
selections = list(combinations_with_replacement(coins, 3))
print(f"Total selections: {len(selections)}")
# Total selections: 35
print(selections[:5])
# [(1, 1, 1), (1, 1, 5), (1, 1, 10), (1, 1, 25), (1, 5, 5)]

    Grouping Data with groupby()

    itertools.groupby(iterable, key=None) groups consecutive elements that share the same key. The input must be sorted by the key first, otherwise identical keys in non-adjacent positions will form separate groups.

    from itertools import groupby
 
# Group consecutive equal elements
data = "AAABBBCCAAB"
for key, group in groupby(data):
    print(f"{key}: {list(group)}")
# A: ['A', 'A', 'A']
# B: ['B', 'B', 'B']
# C: ['C', 'C']
# A: ['A', 'A']
# B: ['B']

    Practical Example: Group Records by Category

    from itertools import groupby
from operator import itemgetter
 
sales = [
    {"product": "Widget", "category": "Hardware", "revenue": 150},
    {"product": "Gadget", "category": "Hardware", "revenue": 300},
    {"product": "App Pro", "category": "Software", "revenue": 500},
    {"product": "Cloud X", "category": "Software", "revenue": 200},
    {"product": "Cable",   "category": "Hardware", "revenue": 50},
]
 
# Sort by category first -- groupby requires sorted input
sales.sort(key=itemgetter("category"))
 
for category, items in groupby(sales, key=itemgetter("category")):
    item_list = list(items)
    total = sum(item["revenue"] for item in item_list)
    print(f"{category}: {len(item_list)} products, ${total} revenue")
# Hardware: 3 products, $500 revenue
# Software: 2 products, $700 revenue

    Grouping Numbers by Property

    from itertools import groupby
 
numbers = sorted(range(1, 16), key=lambda x: x % 3)
for remainder, group in groupby(numbers, key=lambda x: x % 3):
    print(f"Remainder {remainder}: {list(group)}")
# Remainder 0: [3, 6, 9, 12, 15]
# Remainder 1: [1, 4, 7, 10, 13]
# Remainder 2: [2, 5, 8, 11, 14]

    Running Totals with accumulate()

    itertools.accumulate(iterable, func=operator.add, initial=None) produces running (cumulative) results. By default it computes a running sum, but any binary function can be supplied.

    from itertools import accumulate
import operator
 
# Running sum
data = [1, 2, 3, 4, 5]
print(list(accumulate(data)))
# [1, 3, 6, 10, 15]
 
# Running product
print(list(accumulate(data, operator.mul)))
# [1, 2, 6, 24, 120]
 
# Running maximum
temps = [72, 68, 75, 71, 78, 74, 80]
print(list(accumulate(temps, max)))
# [72, 72, 75, 75, 78, 78, 80]
 
# With an initial value
print(list(accumulate(data, operator.add, initial=100)))
# [100, 101, 103, 106, 110, 115]

    Practical: Running Balance

    from itertools import accumulate
 
transactions = [1000, -200, -150, 500, -300, -100, 250]
balances = list(accumulate(transactions))
print("Transactions:", transactions)
print("Balances:    ", balances)
# Transactions: [1000, -200, -150, 500, -300, -100, 250]
# Balances:     [1000, 800, 650, 1150, 850, 750, 1000]

    Complete Function Reference

    FunctionCategoryDescriptionExample Output
    count(start, step)InfiniteArithmetic sequence10, 12, 14, 16, ...
    cycle(iterable)InfiniteRepeats iterable endlesslyA, B, C, A, B, C, ...
    repeat(val, n)InfiniteSame value n times (or forever)5, 5, 5, 5
    chain(*iterables)FiniteConcatenate iterables[1,2] + [3,4] -> 1,2,3,4
    compress(data, sel)FiniteFilter by boolean selectorsABCDE, 10101 -> A,C,E
    islice(iter, start, stop, step)FiniteSlice any iteratorLike list[start:stop:step]
    takewhile(pred, iter)FiniteYield while predicate is trueStops at first false
    dropwhile(pred, iter)FiniteSkip while predicate is trueStarts at first false
    filterfalse(pred, iter)FiniteYield where predicate is falseInverse of filter()
    starmap(func, iter)FiniteApply function to argument tuplesfunc(*args) for each
    accumulate(iter, func)FiniteRunning totals[1,3,6,10,15]
    groupby(iter, key)FiniteGroup consecutive by keyRequires sorted input
    product(*iters)CombinatoricCartesian productReplaces nested loops
    permutations(iter, r)CombinatoricAll r-length orderingsOrder matters
    combinations(iter, r)CombinatoricAll r-length subsetsOrder does not matter
    combinations_with_replacement(iter, r)CombinatoricSubsets with repeatsElements can repeat

    Performance: itertools vs Manual Loops

    The itertools functions are implemented in C, making them significantly faster and more memory-efficient than equivalent Python loops. They produce iterators (lazy evaluation), meaning they yield one element at a time instead of building entire lists in memory.

    import time
from itertools import chain
 
# Benchmark: flatten a list of 1000 sublists, each with 1000 elements
nested = [list(range(1000)) for _ in range(1000)]
 
# Manual approach
start = time.perf_counter()
result_manual = []
for sublist in nested:
    result_manual.extend(sublist)
manual_time = time.perf_counter() - start
 
# itertools approach
start = time.perf_counter()
result_itertools = list(chain.from_iterable(nested))
itertools_time = time.perf_counter() - start
 
print(f"Manual extend: {manual_time:.4f}s")
print(f"chain.from_iterable: {itertools_time:.4f}s")
print(f"Speedup: {manual_time / itertools_time:.2f}x")
# Typical output:
# Manual extend: 0.0180s
# chain.from_iterable: 0.0120s
# Speedup: 1.50x

    Memory Efficiency

    import sys
from itertools import islice, count
 
# A list of 1 million integers
big_list = list(range(1_000_000))
print(f"List memory: {sys.getsizeof(big_list):,} bytes")
# List memory: 8,000,056 bytes
 
# An iterator over the same range (negligible memory)
big_iter = islice(count(), 1_000_000)
print(f"Iterator memory: {sys.getsizeof(big_iter)} bytes")
# Iterator memory: 72 bytes

    Iterators process one element at a time. When chaining multiple transformations, no intermediate lists are created.

    Common Recipes

    The itertools documentation includes several "recipes" -- common patterns built from the module's primitives. Here are the most useful ones.

    Flatten Nested Lists

    from itertools import chain
 
def flatten(nested_list):
    """Flatten one level of nesting."""
    return list(chain.from_iterable(nested_list))
 
data = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
print(flatten(data))
# [1, 2, 3, 4, 5, 6, 7, 8, 9]

    Sliding Window

    from itertools import islice
from collections import deque
 
def sliding_window(iterable, n):
    """Return a sliding window of size n over the iterable."""
    iterator = iter(iterable)
    window = deque(islice(iterator, n), maxlen=n)
    if len(window) == n:
        yield tuple(window)
    for item in iterator:
        window.append(item)
        yield tuple(window)
 
data = [1, 2, 3, 4, 5, 6, 7]
for window in sliding_window(data, 3):
    print(window)
# (1, 2, 3)
# (2, 3, 4)
# (3, 4, 5)
# (4, 5, 6)
# (5, 6, 7)

    Note: Python 3.12+ includes itertools.pairwise() for window size 2, and itertools.batched() for non-overlapping chunks.

    Chunking an Iterable

    from itertools import islice
 
def chunked(iterable, size):
    """Split an iterable into fixed-size chunks."""
    iterator = iter(iterable)
    while True:
        chunk = list(islice(iterator, size))
        if not chunk:
            break
        yield chunk
 
data = list(range(1, 12))
for chunk in chunked(data, 3):
    print(chunk)
# [1, 2, 3]
# [4, 5, 6]
# [7, 8, 9]
# [10, 11]

    In Python 3.12+, use itertools.batched(iterable, n) for the same result with a built-in function.

    Pairwise Iteration

    from itertools import pairwise  # Python 3.10+
 
data = [10, 20, 30, 40, 50]
for a, b in pairwise(data):
    print(f"{a} -> {b}, diff = {b - a}")
# 10 -> 20, diff = 10
# 20 -> 30, diff = 10
# 30 -> 40, diff = 10
# 40 -> 50, diff = 10

    Round-Robin from Multiple Iterables

    from itertools import cycle, islice
 
def roundrobin(*iterables):
    """Yield elements from each iterable in turn."""
    iterators = [iter(it) for it in iterables]
    active = len(iterators)
    nexts = cycle(iter(it).__next__ for it in iterables)
    # Simpler approach:
    pending = len(iterables)
    iters = cycle(iter(it) for it in iterables)
    # Use the recipe from the docs:
    result = []
    iterators = list(map(iter, iterables))
    while iterators:
        next_iterators = []
        for it in iterators:
            try:
                result.append(next(it))
                next_iterators.append(it)
            except StopIteration:
                pass
        iterators = next_iterators
    return result
 
print(roundrobin("ABC", "D", "EF"))
# ['A', 'D', 'E', 'B', 'F', 'C']

    Unique Elements Preserving Order

    from itertools import filterfalse
 
def unique_everseen(iterable):
    """Yield unique elements, preserving first-seen order."""
    seen = set()
    for element in filterfalse(seen.__contains__, iterable):
        seen.add(element)
        yield element
 
data = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]
print(list(unique_everseen(data)))
# [3, 1, 4, 5, 9, 2, 6]

    Chaining itertools Functions Together

    The real power of itertools emerges when you compose multiple functions into a pipeline. Because every function returns an iterator, there is no wasted memory between stages.

    from itertools import chain, compress, accumulate, islice
 
# Pipeline: flatten -> filter -> running sum -> take first 5
nested = [[10, 20, 30], [5, 15, 25], [40, 50]]
mask = [True, False, True, True, False, True, True, False]
 
flat = chain.from_iterable(nested)    # 10, 20, 30, 5, 15, 25, 40, 50
filtered = compress(flat, mask)        # 10, 30, 5, 25, 40
running = accumulate(filtered)         # 10, 40, 45, 70, 110
first_four = list(islice(running, 4))
 
print(first_four)
# [10, 40, 45, 70]

    No intermediate list was created at any step. Each value flows through the pipeline one at a time.

    Experimenting with itertools in Jupyter

    Iterator pipelines can be tricky to debug because iterators are consumed on first pass. An interactive notebook environment makes it easy to inspect intermediate results, test edge cases, and visualize how data flows through each stage. RunCell (opens in a new tab) provides an AI-powered Jupyter environment that is well-suited for this kind of exploration -- you can step through iterator outputs, get AI-assisted explanations when a pipeline behaves unexpectedly, and rapidly prototype recipes before moving them into production code.

    FAQ

    What is itertools in Python?

    itertools is a standard library module that provides a collection of fast, memory-efficient functions for creating and working with iterators. It includes tools for infinite sequences (count, cycle, repeat), finite iteration patterns (chain, islice, groupby), and combinatorics (product, permutations, combinations). All functions return iterators, meaning they generate values lazily without building entire lists in memory.

    What is the difference between itertools.combinations and itertools.permutations?

    combinations(iterable, r) generates all unique subsets of length r where order does not matter -- (A, B) and (B, A) are considered the same and only (A, B) is returned. permutations(iterable, r) generates all orderings of length r where order matters -- both (A, B) and (B, A) are returned. For n elements choosing r, combinations produces n! / (r!(n-r)!) results while permutations produces n! / (n-r)! results.

    How do I flatten a nested list with itertools?

    Use itertools.chain.from_iterable() to flatten one level of nesting. For example, list(chain.from_iterable([[1,2],[3,4],[5,6]])) returns [1, 2, 3, 4, 5, 6]. For deeply nested structures, you need a recursive approach since chain.from_iterable only removes one layer.

    Why does itertools.groupby require sorted input?

    groupby() groups consecutive elements that share the same key. It does not scan the entire iterable to find all matching elements. If your data has [A, A, B, A], groupby produces three groups: A, B, A. To get a single group for each key, sort the data by the key function before passing it to groupby.

    Is itertools faster than regular Python loops?

    Yes. The itertools functions are implemented in C as part of CPython's standard library, making them faster than equivalent hand-written Python loops. They also use lazy evaluation (producing one element at a time), which reduces memory consumption. For large datasets, the combination of C-speed execution and zero intermediate lists can provide significant performance improvements.

    Conclusion

    Python's itertools module replaces dozens of common loop patterns with single function calls that are faster, more readable, and more memory-efficient. The infinite iterators (count, cycle, repeat) handle sequences that have no natural end. The finite iterators (chain, islice, groupby, accumulate, compress, takewhile, dropwhile) cover filtering, slicing, and aggregation. The combinatoric functions (product, permutations, combinations, combinations_with_replacement) eliminate nested loops for exhaustive search.

    The key principle is composition. Because every itertools function returns an iterator, you can pipe the output of one directly into another with no intermediate storage. This makes it possible to process datasets that do not fit in memory, one element at a time, using clean and declarative code.

    📚