simple python deployment tool
in [Python]
|
From: King on 8 Jul 2010 04:21 Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. 1. Find all the modules/packages and copy to "lib" directory. 2. Find python's *.so dependencies(system libs) and copy them to "syslib" 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" directory. 4. Set up temp environment 5. Run main script using "python <main script>.py" The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. Using above steps I have pulled down a package using wxPython's demo example "pySketch.py". You can download the archive from here: http://rapidshare.com/files/405255400/dist.zip (Note : It's for linux users.) After extracting the contents, run the executable file using "./run". This file sets temporary environment variables and execute "pySketch.py". I apologize for the archive size as all the systems libs are also included. This is eventually not working. I think I am not able to set up temporary environment properly or may be missing some other stuff. I would appreciate if some one can point out mistakes. Here is the practical approach: /lib (all python libs, files and third party modules.) /syslib (all the system libs. libX*.*, cairo, etc.) python (executable) pySketch.py run.sh Contents of "run.sh". ------------------------------ set LD_LIBRARY_PATH="syslib/" set PYTHONPATH="/lib" python pySketch.py Cheers Prashant
From: Alexander Kapps on 8 Jul 2010 05:21 King wrote: > Hi, > > I am writing a python package deployment tool for linux based > platforms. I have tried various existing > tool sets but none of them is up to the mark and they have their own > issues. Initially I'll start with simple approach. I'm sorry, but your approach is not going to work. The Right Way(tm) is to not ship any external libraries, but let the user install the dependencies with the distro's package manager. And to not try to build custom packages, but to let the package maintainers of the different distros package your application for you. > 1. Find all the modules/packages and copy to "lib" directory. > 2. Find python's *.so dependencies(system libs) and copy them to > "syslib" That would include all the way down to libc, your archive is going to be huge, but the actual problem is, what if the libc isn't compatible with the kernel, what if the WX, GTK, X11, etc libraries aren't compatible with the running Xserver? End of the story is, you would need to package a minimal (but almost complete) Linux system into your package, which of course is not want you want. > 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" > directory. > 4. Set up temp environment > 5. Run main script using "python <main script>.py" > > The idea is to produce a cleaner directory structure. Neither I am > creating a boot loader nor I am converting main script > file into executable. It's plain vanilla stuff. It's not vanilla stuff, but a very hard problem. In fact you are fighting against the whole system structure.
From: King on 8 Jul 2010 05:44 On Jul 8, 2:21 pm, Alexander Kapps <alex.ka...(a)web.de> wrote: > King wrote: > > Hi, > > > I am writing a python package deployment tool for linux based > > platforms. I have tried various existing > > tool sets but none of them is up to the mark and they have their own > > issues. Initially I'll start with simple approach. > > I'm sorry, but your approach is not going to work. The Right Way(tm) > is to not ship any external libraries, but let the user install the > dependencies with the distro's package manager. And to not try to > build custom packages, but to let the package maintainers of the > different distros package your application for you. Yes, you an do this by creating a .deb package that will take care of installing the required libraries. It may possible that either required version is not available in the distro's repository or the one available is older one then required. So best deal would be to ship at least all the python's libs along with your package. > > 1. Find all the modules/packages and copy to "lib" directory. > > 2. Find python's *.so dependencies(system libs) and copy them to > > "syslib" > > That would include all the way down to libc, your archive is going > to be huge, but the actual problem is, what if the libc isn't > compatible with the kernel, what if the WX, GTK, X11, etc libraries > aren't compatible with the running Xserver? You are right here. It seems there is no standard definition of system libraries on linux. To over come the problem of binary compatibility, I am using ubuntu (hardy) which is on glibc 2.7. Every other external python library including python is compiled on hardy. This is would give you maximum chances that libs would be compatible in case if you run on any other distro which is using glibc >=2.7 > End of the story is, you would need to package a minimal (but almost > complete) Linux system into your package, which of course is not > want you want. > > > 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" > > directory. > > 4. Set up temp environment > > 5. Run main script using "python <main script>.py" > > > The idea is to produce a cleaner directory structure. Neither I am > > creating a boot loader nor I am converting main script > > file into executable. It's plain vanilla stuff. > > It's not vanilla stuff, but a very hard problem. In fact you are > fighting against the whole system structure. Ok, forget about system libs(libX*.* etc.), I don't know why it sounds too complicated. I am hoping it should work and eventually it's easier then tools that create an executable using bootloader. The problem is in setting the correct python environment. Prashant
From: Alexander Kapps on 8 Jul 2010 08:17 King wrote: > On Jul 8, 2:21 pm, Alexander Kapps <alex.ka...(a)web.de> wrote: >> King wrote: >>> Hi, >>> I am writing a python package deployment tool for linux based >>> platforms. I have tried various existing >>> tool sets but none of them is up to the mark and they have their own >>> issues. Initially I'll start with simple approach. >> I'm sorry, but your approach is not going to work. The Right Way(tm) >> is to not ship any external libraries, but let the user install the >> dependencies with the distro's package manager. And to not try to >> build custom packages, but to let the package maintainers of the >> different distros package your application for you. > > Yes, you an do this by creating a .deb package that will take care of > installing the required libraries. > It may possible that either required version is not available in the > distro's repository or the one available is > older one then required. Yes, that may happen. > So best deal would be to ship at least all > the python's libs along with your package. No, IMHO, the best way to cope with this, is to be a little conservative on what library versions you use. Don't use the most cutting edge versions, but those who are most wildly available. Even if you must use the most recent versions, you should leave the packaging to the distro maintainers. They know their distro a lot better and know how to maintain compatiblity with the rest of the system. >>> 1. Find all the modules/packages and copy to "lib" directory. >>> 2. Find python's *.so dependencies(system libs) and copy them to >>> "syslib" >> That would include all the way down to libc, your archive is going >> to be huge, but the actual problem is, what if the libc isn't >> compatible with the kernel, what if the WX, GTK, X11, etc libraries >> aren't compatible with the running Xserver? > > You are right here. It seems there is no standard definition of system > libraries on linux. To over come the problem of binary > compatibility, I am using ubuntu (hardy) which is on glibc 2.7. Every > other external python library including python is compiled on > hardy. This is would give you maximum chances that libs would be > compatible in case if you run on any other distro which is using glibc >> =2.7 Just because any other disto is based on glibc 2.7 doesn't ensure, that the other parts (like gtk libs vs. X11) are compatible. Actually by doing so, you are limiting your package to Hardy only. Any compatiblity with other Ubuntu versions or other distros would be purely by accident. >> End of the story is, you would need to package a minimal (but almost >> complete) Linux system into your package, which of course is not >> want you want. >> >>> 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" >>> directory. >>> 4. Set up temp environment >>> 5. Run main script using "python <main script>.py" >>> The idea is to produce a cleaner directory structure. Neither I am >>> creating a boot loader nor I am converting main script >>> file into executable. It's plain vanilla stuff. >> It's not vanilla stuff, but a very hard problem. In fact you are >> fighting against the whole system structure. > > Ok, forget about system libs(libX*.* etc.), I don't know why it sounds > too complicated. I am hoping it should work and eventually it's easier > then tools that create an executable using bootloader. The problem is > in setting the correct python environment. It sounds complicated, because it is complicated. :-) Now, imagine, everybody would do this. There would be dozens, maybe hundreds of redundant copies of Python, libXYZ, etc. I don't know what you mean by "create an executable using bootloader", but really, the correct way is to leave the dependency management to the distro and it's package manager.
|
Pages: 1 Prev: Appending to a file using Python API Next: Howto write XML file with comments? |