How Imports Cache

23 May 2018

Caching Imports

Whenever Python encounters an import or "from _ import _" statement, it:

1) Looks for that module in the Module Import Path

2) Compiles the file into byte code (maybe)

3) Runs the compiled byte code

Steps 1 and 2 can be pretty taxing, so to ease on computation it will

sys.modules is a dictionary that maps each module import to the explicit filepath of that module’s .py file.

import sys

type(sys.modules)

dict

sys.modules['re']

<module 're' from 'C:\\Users\\nhounshell\\AppData\\Local\\Continuum\\Anaconda3\\lib\\re.py'>

There are a ton of modules that come pre-mapped in Anaconda

len(sys.modules)

Importing a new module appends the value to sys.modules. For example

try:
    sys.modules['localscript']
except Exception as e:
    print(e)

'localscript'

import localscript

sys.modules['localscript']

<module 'localscript' from 'C:\\Users\\nhounshell\\Documents\\github\\BlackBook\\Python\\Modules and Packages\\localscript.py'>

len(sys.modules)

Importing a package actually means first importing the top-level .py file and all of its methods and objects…

For instance, the library pytz contains subdirectories called exceptions, lazy, tzinfo, and tzfile alongside its top-level script.

import pytz
from types import ModuleType

pytzContents = pytz.__dict__

[x for x in pytzContents if type(pytzContents[x]) == ModuleType]

['sys', 'datetime', 'os', 'exceptions', 'lazy', 'tzinfo', 'tzfile']

sys.modules just gets all of them.

list(filter(lambda x: x.startswith('pytz'), sys.modules))

['pytz', 'pytz.exceptions', 'pytz.lazy', 'pytz.tzinfo', 'pytz.tzfile']

So importing something with a ton of submodules, like pandas

import pandas

blows up the number of individual modules that sys.modules keeps track of.

len(sys.modules)