Confusing bash behaviour
in [General Linux]
|
Prev: not able to mount rootfs with nfs using static ip
Next: NYC LOCAL: Tuesday 10 August 2010 Lisp NYC Eating and Drinking
From: Vilmos Soti on 9 Aug 2010 16:17 Hello, This is a solved problem for me, but I think people should be aware of this strange and confusing bash behaviour. for x in `command`; do command-list; done command | while read x; do command-list; done are not interchangeable. The problem arises if in the command-list there is a command which reads from stdin. In that case, the for loop behaves as expected, but in the case of a while loop, the stdin reading program will consume some or all of the remaining lines fed into the while loop. Here is an example: for x in *.avi; do mplayer $x done will work as expected. However, ls -1 *.avi | while read x; do mplayer $x done will play only the first avi. The given loop iteration will finish, but since mplayer reads all what left on stdin, the loop will terminate. So for example, if the second file in the *.avi starts with the letter 'b', then mplayer might complain (based on config), that "No bind found for key 'b'.". I spent considerable time yesterday with this problem, and found this solution: .... | ... | ... | while read x; do mplayer $x < /dev/null done Vilmos
From: Chris F.A. Johnson on 9 Aug 2010 16:33 On 2010-08-09, Vilmos Soti wrote: > Hello, > > This is a solved problem for me, but I think people should be aware of > this strange and confusing bash behaviour. > > for x in `command`; do command-list; done > command | while read x; do command-list; done > > are not interchangeable. > > The problem arises if in the command-list there is a command which reads > from stdin. In that case, the for loop behaves as expected, but in the > case of a while loop, the stdin reading program will consume some or > all of the remaining lines fed into the while loop. > > Here is an example: > > for x in *.avi; do > mplayer $x > done > > will work as expected. > > However, > > ls -1 *.avi | while read x; do > mplayer $x > done > > will play only the first avi. The given loop iteration will finish, > but since mplayer reads all what left on stdin, the loop will terminate. > > So for example, if the second file in the *.avi starts with the letter > 'b', then mplayer might complain (based on config), that "No bind found > for key 'b'.". > > I spent considerable time yesterday with this problem, and found this > solution: > > ... | ... | ... | while read x; do > mplayer $x < /dev/null > done That's because, without the redirection, mplayer reads from the standard input and consumes the rest of the output of ls. A simple example is: printf "%s\n" {a..z} | while read x do echo "Received $x" cat done The cat command reads and consumes stdin. -- Chris F.A. Johnson, <http://cfajohnson.com> Author: Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress) Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
From: J G Miller on 9 Aug 2010 17:16 On Monday, August 9th, 2010 at 13:17:12h -0700, Vilmos Soti wrote: > ls -1 *.avi | while read x; do > mplayer $x > done Would not the use of xargs be more appropriate in this type of construct? Also, you should be aware that while loops are sometimes executed in a sub shell which can lead to unexpected results with variables and exit statements.
From: Robert Heller on 9 Aug 2010 17:24 At 09 Aug 2010 13:17:12 -0700 Vilmos Soti <vilmos(a)soti.ca> wrote: > > Hello, > > This is a solved problem for me, but I think people should be aware of > this strange and confusing bash behaviour. > > for x in `command`; do command-list; done > command | while read x; do command-list; done > > are not interchangeable. > > The problem arises if in the command-list there is a command which reads > from stdin. In that case, the for loop behaves as expected, but in the > case of a while loop, the stdin reading program will consume some or > all of the remaining lines fed into the while loop. > > Here is an example: > > for x in *.avi; do > mplayer $x > done > > will work as expected. > > However, > > ls -1 *.avi | while read x; do > mplayer $x > done > > will play only the first avi. The given loop iteration will finish, > but since mplayer reads all what left on stdin, the loop will terminate. > > So for example, if the second file in the *.avi starts with the letter > 'b', then mplayer might complain (based on config), that "No bind found > for key 'b'.". > > I spent considerable time yesterday with this problem, and found this > solution: > > ... | ... | ... | while read x; do > mplayer $x < /dev/null > done from 'man mplayer': -noconsolecontrols Prevent MPlayer from reading key events from standard input. Useful when reading data from standard input. This is automati- cally enabled when - is found on the command line. There are situations where you have to set it manually, e.g. if you open /dev/stdin (or the equivalent on your system), use stdin in a playlist or intend to read from stdin later on via the loadfile or loadlist slave commands. > > Vilmos > -- Robert Heller -- 978-544-6933 Deepwoods Software -- Download the Model Railroad System http://www.deepsoft.com/ -- Binaries for Linux and MS-Windows heller(a)deepsoft.com -- http://www.deepsoft.com/ModelRailroadSystem/
From: Robert Heller on 9 Aug 2010 17:31
At Mon, 9 Aug 2010 21:16:25 +0000 (UTC) J G Miller <miller(a)yoyo.ORG> wrote: > > On Monday, August 9th, 2010 at 13:17:12h -0700, Vilmos Soti wrote: > > > ls -1 *.avi | while read x; do > > mplayer $x > > done > > Would not the use of xargs be more appropriate in this > type of construct? Actually mplayer *.avi Also works. Unless the OP is doing something else with the filenames. One can also do ls -1 *.avi | mplayer -playlist - -noconsolecontrols *mplayer* actually was written by some smart *UNIX* people. There is little need to second guess these people with some funky bash games... > > Also, you should be aware that while loops are sometimes executed > in a sub shell which can lead to unexpected results with variables > and exit statements. > -- Robert Heller -- 978-544-6933 Deepwoods Software -- Download the Model Railroad System http://www.deepsoft.com/ -- Binaries for Linux and MS-Windows heller(a)deepsoft.com -- http://www.deepsoft.com/ModelRailroadSystem/ |