How to guard against bugs like this one?
in [Python]
|
From: Steve Holden on 3 Feb 2010 14:22 Don't give it another thought. I'd much rather you cared than you didn't ... regards Steve kj wrote: > > Steve, I apologize for the snarkiness of my previous reply to you. > After all, I started the thread by asking the forum for advice on > how to avoid a certain kind of bugs, you were among those who gave > me advice. So nothing other than thanking you for it was in order. > I just let myself get carried away by my annoyance with the Python > import scheme. I'm sorry about it. Even though I don't think I > can put to practice all of your advice, I can still learn a good > deal from it. > > Cheers, > > ~kj > > > Steve Holden <steve(a)holdenweb.com> writes: > >> kj wrote: >>>> First, I don't shadow built in modules. Its really not very hard to avoid. >>> ...*if* you happen to be clairvoyant. I still don't see how the rest of us >>> could have followed this fine principle in the case of numbers.py >>> prior to Python 2.6. >>> >> Clearly the more you know about the standard library the less likely >> this is to be a problem. Had you been migrqating from an earlier version >> the breakage would have alerted you to look for some version-dependent >> difference. > > <snip> > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/
From: Steve Holden on 3 Feb 2010 14:23 kj wrote: > In <hkbv23$c07$1(a)reader2.panix.com> kj <no.email(a)please.post> writes: > > >> Steve, I apologize for the snarkiness of my previous reply to you. >> After all, I started the thread by asking the forum for advice on >> how to avoid a certain kind of bugs, you were among those who gave >> me advice. So nothing other than thanking you for it was in order. >> I just let myself get carried away by my annoyance with the Python >> import scheme. I'm sorry about it. Even though I don't think I >> can put to practice all of your advice, I can still learn a good >> deal from it. > > > Boy, that was dumb of me. The above apology was meant for Stephen > Hansen, not Steve Holden. I guess this is now a meta-apology... > (Sheesh.) > Oh, so you don't like *my* advice? ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/
From: Dan Stromberg on 3 Feb 2010 16:09 kj wrote: > I just spent about 1-1/2 hours tracking down a bug. > > An innocuous little script, let's call it buggy.py, only 10 lines > long, and whose output should have been, at most two lines, was > quickly dumping tens of megabytes of non-printable characters to > my screen (aka gobbledygook), and in the process was messing up my > terminal *royally*. Here's buggy.py: > > > > import sys > import psycopg2 > connection_params = "dbname='%s' user='%s' password='%s'" % tuple(sys.argv[1:]) > conn = psycopg2.connect(connection_params) > cur = conn.cursor() > cur.execute('SELECT * FROM version;') > print '\n'.join(x[-1] for x in cur.fetchall()) > > > (Of course, buggy.py is pretty useless; I reduced the original, > more useful, script to this to help me debug it.) > > Through a *lot* of trial an error I finally discovered that the > root cause of the problem was the fact that, in the same directory > as buggy.py, there is *another* innocuous little script, totally > unrelated, whose name happens to be numbers.py. (This second script > is one I wrote as part of a little Python tutorial I put together > months ago, and is not much more of a script than hello_world.py; > it's baby-steps for the absolute beginner. But apparently, it has > a killer name! I had completely forgotten about it.) > > Both scripts live in a directory filled with *hundreds* little > one-off scripts like the two of them. I'll call this directory > myscripts in what follows. > > It turns out that buggy.py imports psycopg2, as you can see, and > apparently psycopg2 (or something imported by psycopg2) tries to > import some standard Python module called numbers; instead it ends > up importing the innocent myscript/numbers.py, resulting in *absolute > mayhem*. > > (This is no mere Python "wart"; this is a suppurating chancre, and > the fact that it remains unfixed is a neverending source of puzzlement > for me.) > > How can the average Python programmer guard against this sort of > time-devouring bug in the future (while remaining a Python programmer)? > The only solution I can think of is to avoid like the plague the > basenames of all the 200 or so /usr/lib/pythonX.XX/xyz.py{,c} files, > and *pray* that whatever name one chooses for one's script does > not suddenly pop up in the appropriate /usr/lib/pythonX.XX directory > of a future release. > > What else can one do? Let's see, one should put every script in its > own directory, thereby containing the damage. > > Anything else? > > Any suggestion would be appreciated. > > TIA! > > ~k > Here's a pretty simple fix that should work in about any version of python available: Put modules in ~/lib. Put scripts in ~/bin. Your modules end with ..py. Your scripts don't. Your scripts add ~/lib to sys.path as needed. Things that go in ~/lib are named carefully. Things in ~/bin also need to be named carefully, but for an entirely different reason - if you name something "ls", you may get into trouble. Then things in ~/lib plainly could cause issues. Things in ~/bin don't. Ending everything with .py seems to come from the perl tradition of ending everything with .pl. This perl tradition appears to have come from perl advocates wanting everyone to know (by looking at a URL) that they are using a perl CGI. IMO, it's language vanity, and best dispensed with - aside from this issue, it also keeps you from rewriting your program in another language with an identical interface. This does, however, appear to be a scary issue from a security standpoint. I certainly hope that scripts running as root don't search "." for modules.
From: Carl Banks on 3 Feb 2010 17:24 On Feb 3, 8:55 am, Nobody <nob...(a)nowhere.com> wrote: > On Tue, 02 Feb 2010 10:38:53 -0800, Carl Banks wrote: > >> I don't know if that's necessary. Only supporting the "foo.h" case would > >> work fine if Python behaved like gcc, i.e. if the "current directory" > >> referred to the directory contain the file performing the import rather > >> than in the process' CWD. > > >> As it stands, imports are dynamically scoped, when they should be > >> lexically scoped. > > > Mostly incorrect. The CWD is in sys.path only for interactive > > sessions, and when started with -c switch. When running scripts, the > > directory where the script is located is used instead, not the > > process's working directory. > > Okay, so s/CWD/directory containing __main__ script/, but the general > argument still holds. > > > So, no, it isn't anything like dynamic scoping. > > That's what it looks like to me. The way that an import name is resolved > depends upon the run-time context in which the import occurs. Well it has one superficial similarity to dynamic binding, but that's pretty much it. When the directory containing __main__ script is in the path, it doesn't matter if you invoke it from a different directory or os.chdir () during the program, the same modules get imported. I.e., there's nothing dynamic at all about which modules are used. (Unless you fiddle with the path but that's another question.) All I'm saying is the analogy is bad. A better analogy would be if you have lexical binding, but you automatically look in various sister scope before your own scope. Carl Banks
From: Carl Banks on 3 Feb 2010 17:34
On Feb 2, 8:52 pm, Steven D'Aprano <ste...(a)REMOVE.THIS.cybersource.com.au> wrote: > On Tue, 02 Feb 2010 19:55:15 -0800, Carl Banks wrote: > > On Feb 2, 5:49 pm, Steven D'Aprano > > <ste...(a)REMOVE.THIS.cybersource.com.au> wrote: > >> On Tue, 02 Feb 2010 12:26:16 -0800, Carl Banks wrote: > >> > I did not propose obvious module names. I said obvious names like > >> > email.py are bad; more descriptive names like send_email.py are > >> > better. > > >> But surely send_email.py doesn't just send email, it parses email and > >> receives email as well? > > > No, it doesn't. > > Nevertheless, as a general principle, modules will tend to be multi- > purpose and/or generic. Uh, no? If your module is a library with a public API, then you might defensibly have a "generic and/or multi-purpose module", but if that's the case you should have already christened it something unique. Otherwise modules should stick to a single purpose that can be summarized in a short action word or phrase. Carl Banks |