Fork me on GitHub

Project Notes

When Modules Start From Here

or why not to put an __init__.py in your top level folder

So this got my attention after a code refactoring left some actual code and its accompanying tests disagreeing on how to reference modules.

We were using nose, and all the tests referenced absolute imports based on the containing folder. But the code didn’t. And because the code didn’t attempt to make any references to sistem or parent modules, it all seemed to work.

Well kinda. It was a hack and a trap.

And after finding this SO question, it was obvious the problem was simply an extraneous __init__.py in the top level folder.

How To Do It

This demonstrates a simple modules structure with imports up and down. The code and tests all agree on how to reference modules. And there is no fiddling of the PYTHONPATH required.

Note that sys.path resolves identically.

$ cd how_to_do_it/
$ python hello.py
Current top of sys.path: /[...]/LittleCodingKata/python/top_level_imports/how_to_do_it
Hello
$ nosetests --nocapture
Current top of sys.path: /[...]/LittleCodingKata/python/top_level_imports/how_to_do_it
..
----------------------------------------------------------------------
Ran 2 tests in 0.004s

OK

Not How To Do It

Exactly the same code, but there’s a __init__.py in the top level folder. Code runs, but nose can’t find the modules.

$ cd not_how_to_do_it
$ python hello.py
Hello
$ nosetests --nocapture
EE
======================================================================
ERROR: Failure: ImportError (No module named constants)
----------------------------------------------------------------------
Traceback (most recent call last):
  [...] not_how_to_do_it/tests/test_hello.py", line 2, in <module>
    import constants
ImportError: No module named constants

======================================================================
ERROR: Failure: ImportError (No module named constants)
----------------------------------------------------------------------
Traceback (most recent call last):
  [...] not_how_to_do_it/tests/test_util.py", line 2, in <module>
    import constants
ImportError: No module named constants

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (errors=2)

Not How To Do It (Hacked)

So if you start off with ./not_how_to_do_it, it is possible to hack the imports in tests to make it look like it works. But it’s a fake, as it is really running with different sys.path in each case. And referencing modules upwards is just impossible to make work in both cases.

So don’t do this;-)

$ cd not_how_to_do_it_hax
$ python hello.py
Current top of sys.path: /[...]/LittleCodingKata/python/top_level_imports/not_how_to_do_it_hax
Hello
$ nosetests
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK
$ nosetests --nocapture
Current top of sys.path: /[...]/LittleCodingKata/python/top_level_imports
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Credits and References

About LCK#78 python
Project Source on GitHub Return to the Project Catalog

This page is a web-friendly rendering of my project notes shared in the LittleCodingKata GitHub repository.

LittleCodingKata is my collection of programming exercises, research and code toys broadly spanning things that relate to programming and software development (languages, frameworks and tools).

These range from the trivial to the complex and serious. Many are inspired by existing work and I'll note credits and references where applicable. The focus is quite scattered, as I variously work on things new and important in the moment, or go back to revisit things from the past.

This is primarily a personal collection for my own edification and learning, but anyone who stumbles by is welcome to borrow, steal or reference the work here. And if you spot errors or issues I'd really appreciate some feedback - create an issue, send me an email or even send a pull-request.