Dynamic importing stuff in Python

Dynamic importing stuff in Python

Before I explain dynamic importing, lemme explain traditional import too for the sake of readers of all levels. To follow along, you must be at least familiar with the basics of Python. If you don't understand anything, read it anyway... you will learn something new. Let's start with the most basic example and increment on it. If you feel bored, skip to the next section.

import module

import os

Very familiar import. Here, we are simply importing os module. os is now available in the scope as a "module" object (Python treats literally everything as an object) identified as os.

import module as something

import random as r

Again, we are simply importing a module. But now, the module will be available as a given identifier (here it is r) in the scope.

import module's module

import urllib.request

Note that you can't import methods and properties of a class without importing a class.

. is the scope resolution operator in Python. It simply allows you to go a level deeper in the namespace and find the next identifier ie. request in the above

example. but urllib whole is imported and the request is not defined! Why is that?

Something seems to be wrong here. But it is how it should be. To keep things interesting, let's continue our journey and pin it for later.

import module's module as something

import urllib.request as request

We have fixed the buggy import we did before. Now the urllib is not available in the scope, request is the scope and refers to urllib.request.

from module import module

from urllib import request

It works exactly like the above fix but is more self-explanatory and simpler. Remember, we still have a question pinned. We need to find the answer.

from module import all

from urllib import *

This one will import modules that have their identifiers listed in **all** list or simply, exist within the namespace of urllib's **init**. Since we used from keyword, urllib does not exist in the scope. But all the modules from within urllib 's namespace now exist with their default identifiers! So many identifiers within the same scope can be dangerous. Consider the following example:

walk = 'Some string'

from os import *

I just committed a crime! walk has been overridden by os.walk ! To avoid such incidents, avoid importing *. As a thumb rule, do imports at the beginning of your code.

from module's module import module

from urllib.request import Request

It works as expected. Nothing new. You can keep going deeper into the modules as long as you want. Anything that comes in between from and import does not exist in the global scope.

import self's module

Let's say that you wanted a module named sys in your package, and you made one. But now, how do you import it when there is already a standard module with a similar name? You use relative importing.

from . import sys

This will import sys from within the current package.

from .sys import mod

This will import mod from within sys module that resides in the current package.

from .. import sys

This imports sys from the parent package. But we still have that question pinned. Let's find the answer to that question. We need to find how import works behind the scene first.

How import works

We can always look in the documentation and find how import works. If not willing to do so, you can always dump global identifiers available with globals(). Other than globals(), __builtins__ list usually contains most of the built-in classes and functions (as strings). If you dir( __builtins__ ), you will find __import__. This is invoked by the import keyword. if you look in help,

help( __import__ )

it explains what __import__ accepts as argument:

Help on built-in function __import__ in module builtins:

__import__ (...)
    __import__ (name, globals=None, locals=None, fromlist=(), level=0) -> module

    Import a module. Because this function is meant for use by the Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.

    The globals argument is only used to determine the context;
    they are not modified. The locals argument is unused. The fromlist
    should be a list of names to emulate ``from name import ...'', or an
    empty list to emulate ``import name''.
    When importing a module from a package, note that __import__ ('A.B', ...)
    returns package A when fromlist is empty, but its submodule B when
    fromlist is not empty. Level is used to determine whether to perform
    absolute or relative imports. 0 is absolute while a positive number
    is the number of parent directories to search relative to the current module.

It also answered our question! When importing modules with scope resolution and fromList is empty, it returns the topmost object.

Note that it also indicates that you can use importlib.import_module() to programmatically import modules. Or should I say dynamically? As **import** provides functionality to import the modules, you can utilize it for dynamic importing. But that would be a bad practice as the documentation depreciates its direct use. So let's see what importlib can do for us.

Hey import library, import module named something...

Let's check out how importlib.import_module() works:

import importlib

As you can see, in the following, import_module accepts the name of the module and package.

Help on function import_module in module importlib:

import_module(name, package=None)
    Import a module.

    The 'package' argument is required when performing a relative import. It
    specifies the package to use as the anchor point from which to resolve the
    relative import to an absolute import.

If you dir() on importlib, you will see that it also contains the default **import** as well. It is just a wrapper over **import** .

But it allows us to import modules by their name:

module = importlib.import_module('yourpackage.'+ modulename)

That's that. You now know how import works and how to do dynamic importing. I hope you learned something new.