PyDepth

What import Actually Does

Importing asyncio costs 21.9 ms and pulls in 100 modules. Importing email costs 0.1 ms and pulls in one. Granularity predicts import cost; size does not.

5 min read

An import statement reads like an assignment. It is closer to a filesystem crawl with a cache in front of it, and the difference shows up as start-up latency in every command-line tool, every serverless cold start and every test run.

The sequence

import x runs four stages, and only the last one has anything to do with your code.

First, sys.modules is consulted. A hit ends the import there — the name is bound and nothing else happens. This is why a module body executes exactly once per process no matter how many files import it, and why a circular import fails with a half-built module rather than a loop.

Second, on a miss, each finder in sys.meta_path is asked in turn for a module spec. The default list is three entries: the builtin finder, the frozen finder, and PathFinder.

Third, PathFinder walks sys.path. For each entry it consults a per-directory FileFinder, which caches that directory’s listing and invalidates the cache on the directory’s mtime. This is the stage that touches the disk, and it is the stage that costs.

Fourth, the loader reads the source or the cached bytecode, compiles if it must, and executes the module body in a fresh namespace.

What it costs

python -X importtime reports cumulative and self time per module. Running it in a fresh interpreter per import, and counting how many modules ended up resident:

$ python3 experiments/importtime/cost.py
python 3.13.2 on darwin
bare interpreter: 36 modules in sys.modules

import          cumulative   modules
json                3.2 ms        24
logging             5.0 ms        31
dataclasses         6.6 ms        38
typing              2.1 ms        14
asyncio            21.9 ms       100
email               0.1 ms         1
unittest           10.1 ms        59

Two entries in that table are worth more than the rest.

asyncio is 21.9 ms and 100 modules. It is a package that imports its own submodules eagerly, each of which imports selectors, socket, ssl, concurrent.futures and the rest. A web service pays this once at start-up and never thinks about it again. A CLI that imports a library that happens to import asyncio pays it on every invocation, and 22 ms is the difference between a command that feels instant and one that does not.

email is 0.1 ms and one module. email is not small — it is a large package with a parser, a generator, a MIME hierarchy and a header registry. Importing the package imports its __init__, and that is all. Its subpackages are separate modules and are imported when they are named. The lesson generalises: the cost of an import is set by how eagerly a package wires its own submodules together, not by how much code the distribution contains.

Reading the tree

The raw output is easier to act on than the summary above, because it is ordered by completion rather than by name:

$ python3 -X importtime -c "import asyncio" 2>&1 | tail -4
import time:       104 |        104 |     asyncio.base_subprocess
import time:      1082 |       1082 |     asyncio.selector_events
import time:       249 |       1434 |   asyncio.unix_events
import time:       116 |      22847 | asyncio

The first column is self time in microseconds, the second cumulative, and the indentation is the import tree. asyncio itself accounts for 116 µs of the 22,847 it is charged with: it is a hub, not a cost, and the fix for a hub is upstream of it. A module with a large self figure is doing real work at import time — building a regex table, reading a data file, registering codecs — and that work is the thing to move, not the import.

The number that is not in the table

Every figure above was measured with the standard library already in the operating system’s page cache. cost.py starts a fresh interpreter per import, which resets sys.modules and nothing else; the kernel still answers the hundred opens and the several hundred stats out of memory.

A cold cache is a different machine. The four stages do not change, but the third one — PathFinder walking sys.path, a FileFinder per directory listing what is in it — stops being memory reads and becomes disk reads, and it is the stage with by far the most of them. A cold container, a fresh CI runner and a cold Lambda are all in that world. The number measured on a warm laptop is the one that gets quoted in the pull request, and it is the one that describes the least of production.

There is no figure for it here on purpose. Measuring a cold cache honestly needs sudo purge on macOS or a drop of /proc/sys/vm/drop_caches on Linux, and the only version of that worth trusting is a reboot — so the harness measures the warm case and says so, rather than quoting a number it cannot reproduce.

The same accounting explains why bytecode caching helps less than people expect. __pycache__ removes the compile step. It does not remove the stat of the source file, the read of the .pyc, the unmarshal or the execution of the module body — and on this workload those, plus the directory scans that precede them, are most of the time.

What to do with this

Measure before moving anything. python -X importtime -c "import yourpackage" prints a tree with self and cumulative microseconds per module; the expensive branch is usually obvious and usually a surprise.

Then, for a CLI, move the heavy imports inside the subcommand that needs them. The argparse layer should be able to print --help without importing the HTTP client. For a library, prefer module-level lazy attributes over eager submodule imports in __init__.py, so that importing your package does not import all of it.

What does not help is the folklore fix of deleting __pycache__ directories or setting PYTHONDONTWRITEBYTECODE. Both make imports slower, not faster.

Where to stop

These are measurements from one interpreter on one filesystem. The relative order — package granularity dominating package size, cold cache dominating everything — will hold anywhere. The absolute milliseconds will not.

And there is a floor. A bare interpreter already carries 36 modules before your first line runs. If start-up matters enough that 20 ms is intolerable, the remaining lever is not the import system; it is not starting a process per invocation.

Frequently asked

Does __pycache__ eliminate the cost?
It eliminates compilation, not I/O. A cached import still stats the source file to check the timestamp, reads the .pyc, unmarshals it and executes the module body. On the measurements here the executing is the small part.
Why is my CLI slow to start when the actual work is fast?
Almost always because a top-level import pulled in a dependency tree the command in question never uses. python -X importtime attributes the time; move the offending import inside the function that needs it.
Is lazy importing worth it?
For a library, rarely — you are moving the cost, not removing it. For a CLI with subcommands, frequently: only the invoked subcommand should pay for its dependencies.
Share

Related posts

Arrow keys to move, Enter to open.