Prev: Access lotus notes using Python 2.5.1
Next: regex to remove lines made of only whitespace
From: RG on 11 Aug 2010 12:59
In article <mailman.1941.1281519759.1673.python-list(a)python.org>,
Cameron Simpson <cs(a)zip.com.au> wrote:

> On 11Aug2010 00:11, RG <rNOSPAMon(a)flownet.com> wrote:
> | When stdin is not a tty, Python seems to buffer all the input through
> | EOF before processing any of it:
> |
> | [ron(a)mickey:~]$ cat | python
> | print 123
> | print 456 <hit ctrl-D here>
> | 123
> | 456
> |
> | Is there a way to get Python to process input line-by-line the way it
> | does when stdin is a TTY even when stdin is not a TTY?
>
> What you're seeing here is not python's behaviour but cat's behaviour.
>
> Almost all programs do line buffering (flush buffer at newline) when the
> file is a terminal (character device) and block buffering (flush when a
> fixed size buffer, typically 8192 bytes or some larger power of 2) when
> the file is not a terminal. This is default behaviour for the stdio
> package.
>
> So "cat" is simply not feeding any data to python until it has a lot of
> it;

I don't think that's right:

[ron(a)mickey:~]$ cat | cat
123
123
321
321

Cat seems to flush its buffer after every newline. Also:

[ron(a)mickey:~]$ cat -u | python
print 123
print 456
123
456


> We would need to know
> more about your specific task to suggest workarounds.

I'm writing a system in a different language but want to use a Python
library. I know of lots of ways to do this (embed a Python interpreter,
fire up a python server) but by far the easiest to implement is to have
the main program spawn a Python interpreter and interact with it through
its stdin/stdout. In my code I explicitly force the output stream that
is being piped to Python's stdin to be flushed so I know it's not a
buffering problem on the input side.

rg
From: RG on 11 Aug 2010 13:02
In article <i3ud8e$p9e$02$1(a)news.t-online.com>,
Peter Otten <__peter__(a)web.de> wrote:

> Grant Edwards wrote:
>
> > On 2010-08-11, Tim Harig <usernet(a)ilthio.net> wrote:
> >> On 2010-08-11, RG <rNOSPAMon(a)flownet.com> wrote:
> >>> When stdin is not a tty, Python seems to buffer all the input through
> >>> EOF before processing any of it:
> >>>
> >>> [ron(a)mickey:~]$ cat | python
> >>> print 123
> >>> print 456 <hit ctrl-D here>
> >>> 123
> >>> 456
> >>>
> >>> Is there a way to get Python to process input line-by-line the way it
> >>> does when stdin is a TTY even when stdin is not a TTY?
> >>
> >> It would be much better to know the overall purpose of what you are
> >> trying
> >> to achieve. There are may be better ways (ie, sockets) depending what
> >> you
> >> are trying to do. Knowing your target platform would also be helpful.
> >>
> >> For the python interpeter itself, you can can get interactive behavior by
> >> invoking it with the -i option.
> >
> > If you're talking about unbuffered stdin/stdout, the option is -u.
> >
> > I don't really see how the -i option is relevent -- it causes the
> > interpreter to go into interactive mode after running the script.
>
> I'd say the following looks like what the OP was asking for:
>
> $ cat | python -i -c'import sys; sys.ps1=""'
> print sys.stdin.isatty()
> False
> print 1
> 1
> print 2
> 2

That is indeed the behavior I'm looking for.

> (Whether it's useful is yet another question)

It's useful to me :-) I'm trying to access a python library from a
program written in another language for which an equivalent library is
not available. The easiest way to do that is to spawn a Python
interpreter and interact with it through stdin/stdout.

Thanks!

rg
From: Tim Harig on 11 Aug 2010 13:56
On 2010-08-11, RG <rNOSPAMon(a)flownet.com> wrote:
> I'm writing a system in a different language but want to use a Python
> library. I know of lots of ways to do this (embed a Python interpreter,
> fire up a python server) but by far the easiest to implement is to have
> the main program spawn a Python interpreter and interact with it through
> its stdin/stdout.

Or, open python using a socket. That way you have total control over how
the information is transfered, as well as bi-directional transfer.
From: RG on 11 Aug 2010 14:37
In article <i3uo7t$6mk$1(a)speranza.aioe.org>,
Tim Harig <usernet(a)ilthio.net> wrote:

> On 2010-08-11, RG <rNOSPAMon(a)flownet.com> wrote:
> > I'm writing a system in a different language but want to use a Python
> > library. I know of lots of ways to do this (embed a Python interpreter,
> > fire up a python server) but by far the easiest to implement is to have
> > the main program spawn a Python interpreter and interact with it through
> > its stdin/stdout.
>
> Or, open python using a socket.

You mean a TCP/IP socket? Or a unix domain socket? The former has
security issues, and the latter seems like a lot of work. Or is there
an easy way to do it that I don't know about?

rg
From: Tim Harig on 11 Aug 2010 15:38
On 2010-08-11, RG <rNOSPAMon(a)flownet.com> wrote:
> In article <i3uo7t$6mk$1(a)speranza.aioe.org>,
> Tim Harig <usernet(a)ilthio.net> wrote:
>
>> On 2010-08-11, RG <rNOSPAMon(a)flownet.com> wrote:
>> > I'm writing a system in a different language but want to use a Python
>> > library. I know of lots of ways to do this (embed a Python interpreter,
>> > fire up a python server) but by far the easiest to implement is to have
>> > the main program spawn a Python interpreter and interact with it through
>> > its stdin/stdout.
>>
>> Or, open python using a socket.
>
> You mean a TCP/IP socket? Or a unix domain socket? The former has
> security issues, and the latter seems like a lot of work. Or is there
> an easy way to do it that I don't know about?

I was referring to unix domain sockets or more specifically stream
pipes. I guess it depends what language you are using and what libraries
you have access to. Under C, working with stream pipes is no more trivial
then using pipe(). You can simply create the socket descriptors using
socketpair(). Keep one of the descriptors for your process and pass the
other to the python child process as both stdin and stdout.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5
Prev: Access lotus notes using Python 2.5.1
Next: regex to remove lines made of only whitespace