Prev: Problems with cx_Oracle and Oracle 11.1 on Windows
Next: ANN: python-ldap-2.3.10
From: Steven D'Aprano on 1 Nov 2009 17:51
On Sun, 01 Nov 2009 17:34:19 -0300, Gabriel Genellina wrote:

> En Sun, 01 Nov 2009 02:54:15 -0300, Steven D'Aprano
> <steve(a)remove-this-cybersource.com.au> escribió:
>> On Sun, 01 Nov 2009 01:38:16 -0300, Gabriel Genellina wrote:
>
>>>> Incorrect. Simplicity of implementation and API is a virtue, in and
>>>> of itself. The existing module machinery is quite simple to
>>>> understand, use and maintain.
>>>
>>> Uhm... module objects might be quite simple to understand, but module
>>> handling is everything but simple! (simplicity of implem...? quite
>>> simple to WHAT? ROTFLOL!!! )
>>
>> I stand corrected :)
>> Nevertheless, the API is simple: the first time you "import name",
>> Python searches a single namespace (the path) for a module called name.
>> There are other variants of import, but the basics remain:
>>
>> search the path for the module called name, and do something with the
>> first one you find.
>
> Sure, beautiful, a plain and simple search over a list of directories.
> That's how it worked in Python 1.4, I think... Now you have lots of
> "hooks" and even "meta-hooks": sys.meta_path, sys.path_hooks,
> sys.path_importer_cache. And sys.path, of course, which may contain
> other things apart of directory names (zip files, eggs, and even
> instances of custom "loader" objects...).

You'll notice I deliberately didn't refer to directories. I just said
"the path". If the path contains things other than directories, they are
searched too.


> PEP 302 explains this but I'm
> not sure the description is still current. PEP369, if approved, would
> add even more hooks.
> Add packages to the picture, including relative imports and __path__[]
> processing, and it becomes increasingly harder to explain. Bret Cannon
> has rewritten the import system in pure Python (importlib) for 3.1; this
> should help to understand it, I hope. The whole system works, yes, but
> looks to me more like a collection of patches over patches than a
> coherent system. Perhaps this is due to the way it evolved.

You've convinced me that the implementation of the import infrastructure
isn't as clean as I imagined. I'm sure it's a gnarly hack on top of
gnarly hacks, and that maintaining it requires heroic measures worthy of
a medal *grin*.


>>>> Dealing with name clashes doesn't come for free. If you think it
>>>> does, I encourage you to write a patch implementing the behaviour you
>>>> would prefer.
>>>
>>> I'd say it is really a bug, and has existed for a long time.
>>
>> Since import is advertised to return the first module with the given
>> name it finds, I don't see it as a bug even if it doesn't do what the
>> programmer intended it to do. [...] Shadowing a standard library module
>> is no different.
>
> But that's what namespaces are for; if the standard library had its own
> namespace, such collisions would not occur.


Sure. But that's not a bug in the import system. If it's a bug, it's a
bug in the layout of the standard library.


>> You could do it in a backwards compatible way, by adding the std
>> package directory into the path.
>
> Unfortunately you can't, at least not without some special treatment of
> the std package. One of the undocumented rules of the import system is
> that you must not have more than one way to refer to the same module (in
> this case, std.re and re).

*slaps head*

How obvious in hindsight.



--
Steven
From: Gabriel Genellina on 1 Nov 2009 23:50
En Sun, 01 Nov 2009 19:01:42 -0300, MRAB <python(a)mrabarnett.plus.com>
escribi�:
> Gabriel Genellina wrote:

>>>> One way to avoid name clashes would be to put the entire standard
>>>> library under a package; a program that wants the standard re
>>>> module would write "import std.re" instead of "import re", or
>>>> something similar.
>>> You could do it in a backwards compatible way, by adding the std
>>> package directory into the path.
>> Unfortunately you can't, at least not without some special treatment
>> of the std package. One of the undocumented rules of the import
>> system is that you must not have more than one way to refer to the
>> same module (in this case, std.re and re). [...]
> Couldn't the entry in sys.modules be where the module was found, so that
> if 're' was found in 'std' then the entry is 'std.re' even if the import
> said just 're'?

What about a later 'import re'? 're' would not be found in sys.modules
then.
In any case, it requires a change in the current behavior, a PEP, and a
lot of discussion...

--
Gabriel Genellina

From: Gabriel Genellina on 2 Nov 2009 00:11
En Sun, 01 Nov 2009 19:51:04 -0300, Steven D'Aprano
<steve(a)remove-this-cybersource.com.au> escribi�:
> On Sun, 01 Nov 2009 17:34:19 -0300, Gabriel Genellina wrote:
>> En Sun, 01 Nov 2009 02:54:15 -0300, Steven D'Aprano escribi�:

>>> Shadowing a standard library module
>>> is no different.
>>
>> But that's what namespaces are for; if the standard library had its own
>> namespace, such collisions would not occur.
>
> Sure. But that's not a bug in the import system. If it's a bug, it's a
> bug in the layout of the standard library.

Half and half? The standard library cannot have a different structure
because the import system cannot handle it in a backgwards compatible way?

--
Gabriel Genellina

From: Carl Banks on 2 Nov 2009 19:52
On Oct 31, 7:12 am, kj <no.em...(a)please.post> wrote:
> I'm running into an ugly bug, which, IMHO, is really a bug in the
> design of Python's module import scheme.  Consider the following
> directory structure:
>
> ham
> |-- __init__.py
> |-- re.py
> `-- spam.py
>
> ...with the following very simple files:
>
> % head ham/*.py
> ==> ham/__init__.py <==
>
> ==> ham/re.py <==
>
> ==> ham/spam.py <==
> import inspect
>
> I.e. only ham/spam.py is not empty, and it contains the single line
> "import inspect".
>
> If I now run the innocent-looking ham/spam.py, I get the following
> error:
>
> % python26 ham/spam.py
> Traceback (most recent call last):
>   File "ham/spam.py", line 1, in <module>
>     import inspect
>   File "/usr/local/python-2.6.1/lib/python2.6/inspect.py", line 35, in <module>
>     import string
>   File "/usr/local/python-2.6.1/lib/python2.6/string.py", line 122, in <module>
>     class Template:
>   File "/usr/local/python-2.6.1/lib/python2.6/string.py", line 116, in __init__
>     'delim' : _re.escape(cls.delimiter),
> AttributeError: 'module' object has no attribute 'escape'
>
> or, similarly,
>
> % python3 ham/spam.py
> Traceback (most recent call last):
>   File "ham/spam.py", line 1, in <module>
>     import inspect
>   File "/usr/local/python-3.0/lib/python3.0/inspect.py", line 36, in <module>
>     import string
>   File "/usr/local/python-3.0/lib/python3.0/string.py", line 104, in <module>
>     class Template(metaclass=_TemplateMetaclass):
>   File "/usr/local/python-3.0/lib/python3.0/string.py", line 98, in __init__
>     'delim' : _re.escape(cls.delimiter),
> AttributeError: 'module' object has no attribute 'escape'
>
> My sin appears to be having the (empty) file ham/re.py.  So Python
> is confusing it with the re module of the standard library, and
> using it when the inspect module tries to import re.


Python is documented as behaving this way, so this is not a bug.

It is arguably poor design. However, Guido van Rossum already ruled
against using a single package for the standard library, and its not
likely that special case code to detect accidental name-clashes with
the standard library is going to be added, since there are legitimate
reasons to override the standard library.

So for better or worse, you'll just have to deal with it.


Carl Banks
From: Ask Solem on 3 Nov 2009 10:29
On Nov 3, 1:52 am, Carl Banks <pavlovevide...(a)gmail.com> wrote:
> On Oct 31, 7:12 am, kj <no.em...(a)please.post> wrote:
>
>
>
>
>
> > I'm running into an ugly bug, which, IMHO, is really a bug in the
> > design of Python's module import scheme.  Consider the following
> > directory structure:
>
> > ham
> > |-- __init__.py
> > |-- re.py
> > `-- spam.py
>
> > ...with the following very simple files:
>
> > % head ham/*.py
> > ==> ham/__init__.py <==
>
> > ==> ham/re.py <==
>
> > ==> ham/spam.py <==
> > import inspect
>
> > I.e. only ham/spam.py is not empty, and it contains the single line
> > "import inspect".
>
> > If I now run the innocent-looking ham/spam.py, I get the following
> > error:
>
> > % python26 ham/spam.py
> > Traceback (most recent call last):
> >   File "ham/spam.py", line 1, in <module>
> >     import inspect
> >   File "/usr/local/python-2.6.1/lib/python2.6/inspect.py", line 35, in <module>
> >     import string
> >   File "/usr/local/python-2.6.1/lib/python2.6/string.py", line 122, in <module>
> >     class Template:
> >   File "/usr/local/python-2.6.1/lib/python2.6/string.py", line 116, in __init__
> >     'delim' : _re.escape(cls.delimiter),
> > AttributeError: 'module' object has no attribute 'escape'
>
> > or, similarly,
>
> > % python3 ham/spam.py
> > Traceback (most recent call last):
> >   File "ham/spam.py", line 1, in <module>
> >     import inspect
> >   File "/usr/local/python-3.0/lib/python3.0/inspect.py", line 36, in <module>
> >     import string
> >   File "/usr/local/python-3.0/lib/python3.0/string.py", line 104, in <module>
> >     class Template(metaclass=_TemplateMetaclass):
> >   File "/usr/local/python-3.0/lib/python3.0/string.py", line 98, in __init__
> >     'delim' : _re.escape(cls.delimiter),
> > AttributeError: 'module' object has no attribute 'escape'
>
> > My sin appears to be having the (empty) file ham/re.py.  So Python
> > is confusing it with the re module of the standard library, and
> > using it when the inspect module tries to import re.
>
> Python is documented as behaving this way, so this is not a bug.
>
> It is arguably poor design.  However, Guido van Rossum already ruled
> against using a single package for the standard library, and its not
> likely that special case code to detect accidental name-clashes with
> the standard library is going to be added, since there are legitimate
> reasons to override the standard library.
>
> So for better or worse, you'll just have to deal with it.
>
> Carl Banks

Just have to add that you're not just affected by the standard
library.
If you have a module named myapp.django, and someone writes a cool
library called
django that you want to use, you can't use it unless you rename your
local django module.


file myapp/django.py:

from django.utils.functional import curry

ImportError: No module named utils.functional

At least that's what I get, maybe there is some workaround, some way
to say this is an absolute path?
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Problems with cx_Oracle and Oracle 11.1 on Windows
Next: ANN: python-ldap-2.3.10