Prev: GUI automation tool (windows)
Next: Creating a custom UI inside Maya with python
From: Frank Millman on 9 Aug 2010 09:19
Hi all

I know the problems related to circular imports, and I know some of the
techniques to get around them. However, I find that I bump my head into them
from time to time, which means, I guess, that I have not fully understood
how to organise my code so that I avoid them in the first place.

It has just happened again. I have organised my code into three modules,
each representing a fairly cohesive functional area of the overall
application. However, there really are times when Module A wants to invoke
something from Module B, ditto for B and C, and ditto for C and A.

I can think of two workarounds. One is to place the import statement inside
the function that actually requires it. It is therefore not executed when
the module itself is imported, thereby avoiding the problem. It works, but
breaks the convention that all imports should be declared at the top of the
program.

A second solution is to avoid invoking the other modules directly, but
rather use global Queue.Queues to pass requests from one module to another.
Again, it works, but adds complication, especially if the 'invoker' needs to
get a return value.

So I think my main question is, is this a symptom of a flaw in my approach,
or is this something that all programmers bump into from time to time?

If the former, I can expand on my current requirement if anyone would like
to suggest a better approach. If the latter, is either of the above
solutions preferred, or are there other techniques to get around the
problem.

Any suggestions will be appreciated.

Thanks

Frank Millman


From: Ethan Furman on 9 Aug 2010 13:33
Frank Millman wrote:
> Hi all
>
> I know the problems related to circular imports...
>
> It has just happened again. I have organised my code into three modules,
> each representing a fairly cohesive functional area of the overall
> application. However, there really are times when Module A wants to
> invoke something from Module B, ditto for B and C, and ditto for C and A.

I believe the issue arises when you have top-level code (module global
code, or unindented code for the visual) that is calling the other
module. If you keep your references to the other module in functions,
you should be able to have your imports at module level.

The below works fine.

a.py
-----
import b
def spam():
b.eggs()
def foo():
print "my circular-import-fu is strong!" # to amuse myself only!)
-----

b.py
-----
import a
def eggs():
print 'sunnyside?'
def ham():
a.foo()
-----

If you put an <a.spam()> in b.py, then you get:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "a.py", line 1, in <module>
import b
File "b.py", line 3, in <module>
a.spam()
AttributeError: 'module' object has no attribute 'spam'


Hope this helps.

~Ethan~
From: Carl Banks on 9 Aug 2010 14:51
On Aug 9, 6:19 am, "Frank Millman" <fr...(a)chagford.com> wrote:
> It has just happened again. I have organised my code into three modules,
> each representing a fairly cohesive functional area of the overall
> application. However, there really are times when Module A wants to invoke
> something from Module B, ditto for B and C, and ditto for C and A.

There's a dozen reasons why circular imports can go wrong. Can you
describe the problem you're having getting them to work? If there's a
traceback when you try to do it, cut-and-paste the traceback and
relevant code here. If it's running, but you are not getting the
behavior you expect, tell us what you expected to happen and what
actually did.


[snip]
> So I think my main question is, is this a symptom of a flaw in my approach,
> or is this something that all programmers bump into from time to time?

I consider the need to resort to circular imports a red flag, not a
manifest error. You say you are organizing the modules
"functionally". I find that when I do that, I will occasionally get
circular references, so I can believe you. I also find that in most
such cases, reorganizing the modules according to "dependencies", so
as to remove the circular import, makes it more confusing. Sometimes
organizing by function makes more sense than organizing by dependency,
and it's better to live with circular imports than to use a less-
sensical organization.


Carl Banks
From: Michael Torrie on 9 Aug 2010 16:06
On Aug 9, 6:19 am, "Frank Millman" <fr...(a)chagford.com> wrote:
> It has just happened again. I have organised my code into three modules,
> each representing a fairly cohesive functional area of the overall
> application. However, there really are times when Module A wants to invoke
> something from Module B, ditto for B and C, and ditto for C and A.

There are a number of ways to avoid circular imports, in order of my own
preference:

1. Make common stuff a new module. So if A needs something from B, and
vice versa, you must factor out the stuff and stick it in its own
module. The fact that you have circular dependencies means that
although things are cohesive, they are way too closely coupled.

2. Instead of having A refer directly to something in B (which would
cause a circular dependency, have the caller pass in as a parameter to
the function in A, whatever is needed from B. This could be just a
variable, complex object, or even a function or method. Take advantage
of the fact that everything in Python is a first-class object.

3. Take advantage of Python's dynamicism. Write an initializer function
in A that allows you to tell it about B and C. In other words, you can
pass B and C to some method in A and have it bind B and C to local
attributes in A. Then you can call B and C's methods just fine from A
since everything is looked up as it is called.
From: Frank Millman on 10 Aug 2010 05:13

"Frank Millman" <frank(a)chagford.com> wrote in message
news:i3ov9e$dug$1(a)dough.gmane.org...
> Hi all
>
> I know the problems related to circular imports, and I know some of the
> techniques to get around them. However, I find that I bump my head into
> them from time to time, which means, I guess, that I have not fully
> understood how to organise my code so that I avoid them in the first
> place.
>
[...]
>
> So I think my main question is, is this a symptom of a flaw in my
> approach, or is this something that all programmers bump into from time to
> time?
>

Thanks for the replies. All good info, but it was Ethan that put me onto the
right track.

I omitted to mention one thing originally, as I did not think it important,
but it turns out to be crucial. My code is organised into three 'packages',
not 'modules'.

To reproduce my situation, I did some tests with the following hierarchy -

top/
a.py
/bb
__init__.py
b.py
/cc
__init__.py
c.py

a.py
----
import bb.b
import cc.c
bb.b.foo()
cc.c.foo()

b.py
----
import cc.c
def foo():
print 'in b.foo, call c.bar'
cc.c.bar()
def bar():
print ' bar in b'

c.py
----
import bb.b
def foo():
print 'in c.foo, call b.bar'
bb.b.bar()
def bar():
print ' bar in c'

If I run 'a.py', I get the correct result -

in b.foo, call c.bar
bar in c
in c.foo, call b.bar
bar in b

I changed 'a.py' -

a.py
----
from bb import b
from cc import c
b.foo()
c.foo()

It still worked.

Next I changed 'b.py' -

b.py
----
from cc import c
def foo():
print 'in b.foo, call c.bar'
c.bar()
def bar():
print ' bar in b'

It still worked.

Then I changed 'c.py' -

c.py
----
from bb import b
def foo():
print 'in b.foo, call c.bar'
b.bar()
def bar():
print ' bar in b'

Now I get the following traceback -
Traceback (most recent call last):
File "F:\dd\a.py", line 1, in <module>
from bb import b
File "F:\dd\bb\b.py", line 1, in <module>
from cc import c
File "F:\dd\cc\c.py", line 1, in <module>
from bb import b
ImportError: cannot import name b

Now that I understand this, I can work around my problem by using
fully-qualified module names.

But it would be interesting to know the underlying reason for this
behaviour.

I am using python 2.6.2.

Thanks for any insights.

Frank


 | 
Pages: 1
Prev: GUI automation tool (windows)
Next: Creating a custom UI inside Maya with python