Prev: Anybody use web2py?
Next: Mechanize/ClientForm - How to select IgnoreControl button and submit form
From: Steven D'Aprano on 23 Dec 2009 08:56
On Wed, 23 Dec 2009 01:41:27 -0800, lordofcode wrote:

> I need to dynamically load a module and unload it and load another
> module.

Why bother unloading it? Unless you're programming for an embedded
device, it's highly unlikely that you're so strapped for memory that a
module will make any real difference.

Just stop using the module and start using another. (See below for
example.)


> For example I have many files(All files in Python are modules right?)
> like mobile_1.py ,mobile_2.py, mobile_3.py etc.. in my project folder
> which contains classes and methods with same name but different
> functionality.(am afraid I cannot change this structure as these files
> are generated randomly by the user)
>
> So initially when my program starts I have to load a default module. I
> do this as follows:
> ##############################
>>>MODULE_name = "mobile_1"
>>>exec "from "+MODULE_name+" import *"
> ##############################
> And use the methods defined in "mobile_1.py" file
>
> Now as the application continues , I may have to use the methods defined
> in "mobile_2.py" or "mobile_3.py" etc instead of the previously loaded
> module,


Sounds like an absolute horrible design. I recommend that you re-think
the design and do something less fragile and difficult.

The obvious way would be something like this:

# Import the first module and use it.
user_module = __import__("mobile_1")
x = user_module.function(123)
y = user_module.SomeClass(123)

# Now change modules.
user_module = __import__("mobile_2")
x = user_module.function(456)
y = user_module.SomeClass(456)


Note that:

(1) The function __import__ loads the module given a name which is only
known at runtime instead of compile time.

(2) Don't use the form "from module import name" form. That injects the
objects into the current namespace, which generally leads to more trouble
than benefit.

(3) After replacing the module, you have to get rid of the old classes
that refer to the old module, and create a new objects. Classes don't
magically change their behaviour just because the name of the class
points to something different:


>>> class C:
.... def method(self):
.... return "Something!"
....
>>> instance = C()
>>> instance.method()
'Something!'
>>> class C: # replace the old class
.... def method(self):
.... return "Something different"
....
>>> instance.method()
'Something!'


I'm betting that once you understand that behaviour, you will understand
why your approach is failing.



> which I incorrectly try to do as below:
> ####################
>>>MODULE_name = "mobile_2"
>>>exec "from "+MODULE_name+" import *"
> #####################
> The above import does not have any impact and the methods called from my
> application still pertain to mobile_1.py as its still in the current
> namespace(?).

The most likely problem is that you have misunderstood how Python's
object model works.

Suppose you do this:

from module_1 import MyClass
x = MyClass(1, 2, 3) # create a class instance
x.method()

The object called "x" is now an instance of MyClass from module_1. So far
so good. Suppose that you do this:

from module_2 import MyClass
x.method()

I'm guessing that you expect that this will call the method from MyClass
in module_2. But that's not what happens: the object x is still the same
object it was before, and it still calls the method from module_1. You
have to throw away that object and create a new one:

from module_2 import MyClass
x = MyClass(1, 2, 3)
x.method()

(This, BTW, is exactly the same situation as the same code above with
class C and instance.method, except an import is added to the mix.)


Now, it's not *impossible* to play tricks like this:

from module_2 import MyClass
# replace the reference to the old MyClass with the new MyClass:
x.__class__ = MyClass
x.method()

which *may* work, for some classes. But better to avoid this unless you
can control the classes yourself, which you can't.


Or find a better design that doesn't rely on dynamically replacing
modules.


> I tried below code with del(), reload() etc but could not figure it out.
> ###Code to unload a dll####

This has nothing to do with DLLs. Python modules are Python modules, not
DLLs, and they don't work the same way.


>>>del sys.modules[MODULE_name] #==> does not delete the reference in
>>>namespace

Deleting the module from the cache does nothing except slow Python down
the next time you import the module.


> 1)How do I unload a module dynamically and completely remove the
> references in the module so a new module with same name references can
> be loaded?

You don't have to. You just have to understand how Python actually works,
and work *with* it instead of *against* it.


> 2)Are there any alternative way to do the above requirement? Currently I
> am working around by restarting the whole initial setup for each new
> module which is unnecessary waste.

What makes you think it's unnecessary?


> Can I avoid this "reset"?

I don't know, because I don't fully understand what "restarting the whole
initial setup" actually means.




--
Steven
From: Jean-Michel Pichavant on 23 Dec 2009 09:31
Steven D'Aprano wrote:
> On Wed, 23 Dec 2009 13:37:06 +0100, Jean-Michel Pichavant wrote:
>
>
>> But believe me, you don't want to mess up with the python import
>> mechanism.
>>
>
> Unless you understand how it works.
>
>
>
Let me quote the OP: 'Not an expert in Python'
I was just answering the OP question, without refering to something he
could do if he was someone else.

JM


From: Steven D'Aprano on 23 Dec 2009 17:04
On Wed, 23 Dec 2009 15:31:53 +0100, Jean-Michel Pichavant wrote:

> Steven D'Aprano wrote:
>> On Wed, 23 Dec 2009 13:37:06 +0100, Jean-Michel Pichavant wrote:
>>
>>
>>> But believe me, you don't want to mess up with the python import
>>> mechanism.
>>>
>>>
>> Unless you understand how it works.
>>
>>
>>
> Let me quote the OP: 'Not an expert in Python' I was just answering the
> OP question, without refering to something he could do if he was someone
> else.

But what he *can* do is to learn how importing works. I'm not sure it's
terribly helpful to tell somebody all the things they can't do instead of
what they can.


--
Steven
From: Carl Banks on 23 Dec 2009 18:18
On Dec 23, 7:40 am, Steven D'Aprano <st...(a)REMOVE-THIS-
cybersource.com.au> wrote:
> On Wed, 23 Dec 2009 13:37:06 +0100, Jean-Michel Pichavant wrote:
> > 3/ if you really need to unload the previous module, it's a little bit
> > tedious.
>
> > import mod1
> > del mod1
> > sys.modules['mod1'] = None
>
> Assigning sys.modules[name] to None is not the same as deleting the
> entry. None has special meaning to imports from packages, and for modules
> it is interpreted as meaning that the module doesn't exist.
>
> >>> import math
> >>> del math
> >>> sys.modules['math'] = None
> >>> import math
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: No module named math
>
> > # will unload mod1 assuming mod1 was the only
> > reference to that module.
>
> Which is highly unlikely. Any classes or functions from the module will
> keep the module alive.

Actually, they won't. Neither classes nor functions directly
reference their module; classes know their module only by name, and
functions only hold references to the module's namespace, not to the
module itself. So if any references to functions defined in the
module remain, the module dict will stick around, but the module
itself may be collected.


> > But believe me, you don't want to mess up with the python import
> > mechanism.
>
> Unless you understand how it works.



Carl Banks
From: lordofcode on 23 Dec 2009 21:32
Hi All,

Thanks you all for your replies ,cleared quiet a few doubts about
importing modules and namespace references .
Currently am going with static importing of all modules. But it may
not be efficient in future as the number of interchangeable modules
that I have to import may run in 30-40's.(Basically this Python code
is part of larger GSM Testing project where each module represents one
model of GSM mobile phone and this number keeps growing). So changing
the design will be a last resort.
But am trying to reduce the number of objects(classes,methods etc) in
each module so importing all 30-40 modules do not impact much.

Anyway am trying out all the mentioned tricks here and get back to you
with any doubts.

Thanks again
Ajay Baragur
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: Anybody use web2py?
Next: Mechanize/ClientForm - How to select IgnoreControl button and submit form