Python as a scripting language. Alternative to bash script?
in [Python]
|
From: Chris Rebert on 6 Jul 2010 20:27 On Tue, Jul 6, 2010 at 1:35 PM, member thudfoo <thudfoo(a)opensuse.us> wrote: > On Tue, Jul 6, 2010 at 6:40 AM, Michael Torrie <torriem(a)gmail.com> wrote: >> On 07/06/2010 04:12 AM, sturlamolden wrote: >>> On 28 Jun, 19:39, Michael Torrie <torr...(a)gmail.com> wrote: >>>> In python I could simply take the output of "ps ax" and use python's >>>> own, superior, cutting routines (using my module): >>>> >>>> (err, stdout, stderr) = runcmd.run( [ 'ps', 'ax' ] ) >>>> for x in stdout.split('\n'): >>>> Â Â print x.strip().split()[0] >>> >>> Or you just pass the stdout of one command as stdin to another. That >>> is equivalent of piping with bash. >> >> Consider this contrived example: >> >> tail -f /var/log/messages | grep openvpn >> >> While it's possible to set up pipes and spawn programs in parallel to >> operate on the pipes, in practice it's simpler to tell subprocess.Popen >> to use a shell and then just rely on Bash's very nice syntax for setting >> up the pipeline. Â Then just read the final output in python. Â If you set >> the stdout descriptor to non-blocking, you could read output as it came. > > Is this a discussion about the pipes module in the std library? No, though that module is not irrelevant to Mr. Torrie's argument. Cheers, Chris -- http://blog.rebertia.com
From: Chris Rebert on 6 Jul 2010 23:34 On Tue, Jul 6, 2010 at 6:40 AM, Michael Torrie <torriem(a)gmail.com> wrote: > On 07/06/2010 04:12 AM, sturlamolden wrote: >> On 28 Jun, 19:39, Michael Torrie <torr...(a)gmail.com> wrote: >>> In python I could simply take the output of "ps ax" and use python's >>> own, superior, cutting routines (using my module): >>> >>> (err, stdout, stderr) = runcmd.run( [ 'ps', 'ax' ] ) >>> for x in stdout.split('\n'): >>> Â Â print x.strip().split()[0] >> >> Or you just pass the stdout of one command as stdin to another. That >> is equivalent of piping with bash. > > Consider this contrived example: > > tail -f /var/log/messages | grep openvpn > > While it's possible to set up pipes and spawn programs in parallel to > operate on the pipes, in practice it's simpler to tell subprocess.Popen > to use a shell and then just rely on Bash's very nice syntax for setting > up the pipeline. Until there's a Python variable involved that is, unless you want to overlook all the edge cases or do the escaping all by yourself (and then pray you did it right). Cheers, Chris -- http://blog.rebertia.com
From: Stefan Behnel on 7 Jul 2010 03:19 John Nagle, 28.06.2010 19:57: > Programs have "argv" and "argc", plus environment variables, > going in. So, going in, there are essentially subroutine parameters. > But all that comes back is an exit code. They should have had > something similar coming back, with arguments to "exit()" returning > the results. Then the "many small intercommunicating programs" > concept would have worked much better. Except that you just broke the simplicity of the pipe mechanism. Stefan
From: Michael Torrie on 7 Jul 2010 11:31 On 07/06/2010 09:34 PM, Chris Rebert wrote: > On Tue, Jul 6, 2010 at 6:40 AM, Michael Torrie <torriem(a)gmail.com> wrote: >> While it's possible to set up pipes and spawn programs in parallel to >> operate on the pipes, in practice it's simpler to tell subprocess.Popen >> to use a shell and then just rely on Bash's very nice syntax for setting >> up the pipeline. > > Until there's a Python variable involved that is, unless you want to > overlook all the edge cases or do the escaping all by yourself (and > then pray you did it right). Very good point. This is a problem that the pipes module suffers from as well. Although we learned in the other thread on escaping SQL statements that escaping is faster, easier and just as safe as other parameterization mechanisms. Uh huh. Back on target, a library similar to pipes that was safe (pipes is not) and had a pythonic syntax would be cool. pipes module works alright, syntax wise, but it's not a clean syntax.
From: Chris Rebert on 7 Jul 2010 14:27
On Wed, Jul 7, 2010 at 8:31 AM, Michael Torrie <torriem(a)gmail.com> wrote: > On 07/06/2010 09:34 PM, Chris Rebert wrote: >> On Tue, Jul 6, 2010 at 6:40 AM, Michael Torrie <torriem(a)gmail.com> wrote: >>> While it's possible to set up pipes and spawn programs in parallel to >>> operate on the pipes, in practice it's simpler to tell subprocess.Popen >>> to use a shell and then just rely on Bash's very nice syntax for setting >>> up the pipeline. >> >> Until there's a Python variable involved that is, unless you want to >> overlook all the edge cases or do the escaping all by yourself (and >> then pray you did it right). > > Very good point. Â This is a problem that the pipes module suffers from > as well. > > Although we learned in the other thread on escaping SQL statements that > escaping is faster, easier and just as safe as other parameterization > mechanisms. Â Uh huh. > > Back on target, a library similar to pipes that was safe (pipes is not) > and had a pythonic syntax would be cool. Â pipes module works alright, > syntax wise, but it's not a clean syntax. Actually, your original post inspired me to take a crack at writing something like that yesterday: http://rebertia.com/code/subproc_pipelines.py Thoughts anyone? (Written on a whim, so no tests or docs at present.) Cheers, Chris -- http://blog.rebertia.com |