PYTHON / MODULES AND PACKAGES
A tour of the standard library
Navigate the batteries-included standard library: know which module owns a problem, inspect it with dir/help/__file__, and tell stdlib from PyPI code.
What you will learn
- Reach for the module that owns the domain: json, csv, datetime, collections, itertools
- Explore an unfamiliar module with dir(), help(), and __file__ instead of guessing
- Check sys.stdlib_module_names to tell shipped modules from pip-installed ones
- Never name your own file after a stdlib module like random.py or json.py
Understanding A tour of the standard library
Every CPython installation carries roughly two hundred importable modules that were built and tested alongside the interpreter itself. That has a practical consequence: `import json` works on any machine with Python 3 installed, with no install step, no version pin, and no chance of a dependency conflict. The library is organised by problem domain, not alphabetically, and the domains are worth memorising even if the function names are not: data formats (json, csv, sqlite3, pickle), text (re, textwrap, string, difflib), time (datetime, zoneinfo, calendar), containers and algorithms (collections, itertools, functools, heapq, bisect, statistics), files and processes (pathlib, shutil, tempfile, subprocess), and program plumbing (sys, argparse, logging, unittest, dataclasses).
Nobody remembers the whole thing, so the real skill is a lookup loop rather than recall. Decide what kind of problem you have, guess the module that owns that domain, import it in a REPL, then run `dir(mod)` to see the names and `help(mod.thing)` to see the signature. Most stdlib modules are ordinary Python files, so `mod.__file__` gives you a path you can open and read when the docs are ambiguous about an edge case; a small core set (sys, math, time, and friends listed in `sys.builtin_module_names`) is compiled into the executable and has no `.py` source at all.
The standard library deliberately stops short of some domains: HTTP clients, numerical arrays, and web frameworks are left to PyPI, even though `urllib.request` and `http.server` will handle a one-off job. It also changes between versions, since modules can be deprecated and eventually removed (distutils disappeared in 3.12, cgi and telnetlib in 3.13), so read the docs for the version you actually run. The flip side of the stdlib being always importable is that its names are easy to collide with: a file called `random.py` in your working directory wins over the real module and quietly breaks anything that imports it.
import json
import statistics
import textwrap
from collections import Counter
from datetime import date
raw = '[{"city": "Oslo", "temp": -3}, {"city": "Lima", "temp": 19}, {"city": "Oslo", "temp": 1}]'
records = json.loads(raw)
temps = [r["temp"] for r in records]
print("mean:", round(statistics.mean(temps), 1))
print("counts:", Counter(r["city"] for r in records).most_common())
print("span:", (date(2024, 3, 1) - date(2024, 1, 1)).days, "days")
print(textwrap.shorten("temperature readings collected by volunteers", width=25, placeholder="..."))The standard library is a large, version-locked toolbox that ships with the interpreter, so the skill is knowing which module owns a problem rather than memorising APIs.
Worked examples
Interrogating a module you just met
Uses dir, __file__, and the two module-name sets in sys to answer 'what is in here, and did it ship with Python?'
import random
import sys
print(random.__name__, "->", random.__file__.endswith("random.py"))
print(sorted(n for n in dir(random) if n.startswith("sh")))
print("json" in sys.stdlib_module_names, "requests" in sys.stdlib_module_names)
print("sys" in sys.builtin_module_names)Example explained
Line 1dir(random) lists every attribute of the module object, so filtering by a prefix finds a half-remembered name fast.
Line 2random.__file__ points at the actual .py source, which is the best reference when the docs leave an edge case open.
Line 3sys.stdlib_module_names is a frozenset of shipped module names; it answers 'do I need pip for this?' without importing anything.
Line 4sys.builtin_module_names is the much smaller set compiled into the interpreter, and those modules have no source file to read.
Three algorithm helpers that replace loops
Shows itertools.groupby, functools.reduce, and itertools.islice over an infinite counter.
from functools import reduce
from itertools import count, groupby, islice
pairs = [("fruit", "fig"), ("fruit", "plum"), ("veg", "kale")]
for key, group in groupby(pairs, key=lambda p: p[0]):
print(key, [name for _, name in group])
print(reduce(lambda a, b: a * b, range(1, 6)))
print(list(islice(count(10, 5), 4)))Example explained
Line 1groupby only merges *adjacent* rows with the same key, so unsorted input silently produces duplicate groups.
Line 2reduce folds a two-argument function across the sequence; here it multiplies 1..5 into 120.
Line 3count(10, 5) never ends, so islice is what makes it safe to turn into a list.
Why the stdlib parser beats str.split
Compares csv.DictReader against a hand-rolled split on a field that contains a quoted comma.
import csv
import io
data = 'name,score\nada,9\ngrace,"7,5"\n'
rows = list(csv.DictReader(io.StringIO(data)))
print(rows[1]["score"])
print(data.split("\n")[2].split(",")[1])Example explained
Line 1csv.DictReader applies the real quoting rules, so the comma inside "7,5" stays inside one field.
Line 2io.StringIO wraps the string in a file-like object, which is the interface csv expects — no temp file needed.
Line 3The naive split cuts at the quoted comma and leaves a stray quote character, the classic bug the csv module exists to prevent.
Important notes
sys.stdlib_module_names was added in Python 3.10; on older interpreters you have to consult the module index in the docs instead.
The stdlib is not frozen — modules are occasionally deprecated and removed, so match the documentation to the version reported by sys.version_info.
Common mistakes
Creating a file named random.py, json.py, or string.py next to your script: your file shadows the stdlib module, and unrelated imports start failing with strange AttributeErrors.
Assuming requests, yaml, or numpy are part of the standard library because they were already installed locally: the script dies with ModuleNotFoundError on any clean machine.
Hand-rolling CSV splitting or day-count arithmetic instead of using csv and datetime: it works on the sample data and then breaks on quoted commas or a leap year.
Try it yourself
Change, predict, then run
Take the string "the quick brown fox jumps over the lazy dog" and use collections.Counter to print the three most common letters ignoring spaces, then print the same sentence passed through textwrap.shorten with width=20.
Open the Python workspaceCheck your understanding
Your script starts with `import json` and `import requests`. It runs on your machine, but a colleague with a fresh Python install gets ModuleNotFoundError for requests only. What explains the difference?
- requests is a third-party package that must be installed separately, while json ships with the interpreter
- requests has to be imported with from-import syntax to be found
- json is a builtin function rather than a module, so it never needs to be installed
- requests can only be imported while a virtual environment is active
Show answer
json is part of the standard library, so it is present in every CPython install and needs no pip step; requests comes from PyPI and exists only where someone installed it. Option 4 is tempting because installs usually happen inside a virtual environment, but activating an environment does not create the package — the missing piece is the installation itself, and requests imports fine outside a venv once it is installed.