Prev: problem with cat and echo
Next: how to specify all files of a certain string length in Bourne Shell
From: Rahul on 1 Apr 2008 15:09
Is there an easy / elegant way to read from a file, then process the
stream with a set of filters and pipe back to the same file? For
example:

Let's say I have a file

B
C
D

and I want to add an A at the top to get a

A
B
C
D

I'd like to do:
( echo "A" ; cat t ) > t

but that spews an angry "cat: t: input file is output file"

This is a trivial toy example but I find myself running into this sort
of situation every so often. I can always work-around by using a temp
file but I'm just wondering if I'm missing an obvious metaphor...

--
Rahul
From: pk on 1 Apr 2008 15:40
Rahul wrote:

> I'd like to do:
> ( echo "A" ; cat t ) > t
>
> but that spews an angry "cat: t: input file is output file"
>
> This is a trivial toy example but I find myself running into this sort
> of situation every so often. I can always work-around by using a temp
> file but I'm just wondering if I'm missing an obvious metaphor...

There is a recent post where a similar question was asked (subject: "shell
script replacing original file"). See the discussion there.
In short, from what I understand, it's generally unsafe since programs in a
pipeline are executed in parallel, and you don't have any guarantee that
the file will be completely read before the redirection starts writing to
it (especially if it's a large file), and thus what you want can't be done
(without using a temporary file), but read that thread for the details.

--
All the commands are tested with bash and GNU tools, so they may use
nonstandard features. I try to mention when something is nonstandard (if
I'm aware of that), but I may miss something. Corrections are welcome.
From: Icarus Sparry on 2 Apr 2008 14:19
On Tue, 01 Apr 2008 21:40:20 +0200, pk wrote:

> Rahul wrote:
>
>> I'd like to do:
>> ( echo "A" ; cat t ) > t
>>
>> but that spews an angry "cat: t: input file is output file"
>>
>> This is a trivial toy example but I find myself running into this sort
>> of situation every so often. I can always work-around by using a temp
>> file but I'm just wondering if I'm missing an obvious metaphor...
>
> There is a recent post where a similar question was asked (subject:
> "shell script replacing original file"). See the discussion there. In
> short, from what I understand, it's generally unsafe since programs in a
> pipeline are executed in parallel, and you don't have any guarantee that
> the file will be completely read before the redirection starts writing
> to it (especially if it's a large file), and thus what you want can't be
> done (without using a temporary file), but read that thread for the
> details.

It can be done, the usual reason why it isn't done is that if the
operation is interrupted you are not in a good position to recover.

exec < t # give a new name (file descriptor 0 in this case) to t
rm t # remove the old name
{ echo A ; cat ; } > t # generate the new content.
From: Florian Diesch on 1 Apr 2008 23:16
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Rahul <nospam(a)nospam.invalid> wrote:

> Is there an easy / elegant way to read from a file, then process the
> stream with a set of filters and pipe back to the same file? For
> example:

That's what sponge is for. It's part of moreutils
<http://kitenet.net/~joey/code/moreutils.html>



Florian
--
<http://www.florian-diesch.de/>
-----------------------------------------------------------------------
** Hi! I'm a signature virus! Copy me into your signature, please! **
-----------------------------------------------------------------------
From: Janis Papanagnou on 2 Apr 2008 21:13
Icarus Sparry wrote:
> On Tue, 01 Apr 2008 21:40:20 +0200, pk wrote:
>
>
>>Rahul wrote:
>>
>>
>>>I'd like to do:
>>>( echo "A" ; cat t ) > t
>>>
>>>but that spews an angry "cat: t: input file is output file"
>>>
>>>This is a trivial toy example but I find myself running into this sort
>>>of situation every so often. I can always work-around by using a temp
>>>file but I'm just wondering if I'm missing an obvious metaphor...
>>
>>There is a recent post where a similar question was asked (subject:
>>"shell script replacing original file"). See the discussion there. In
>>short, from what I understand, it's generally unsafe since programs in a
>>pipeline are executed in parallel, and you don't have any guarantee that
>>the file will be completely read before the redirection starts writing
>>to it (especially if it's a large file), and thus what you want can't be
>>done (without using a temporary file), but read that thread for the
>>details.
>
>
> It can be done, the usual reason why it isn't done is that if the
> operation is interrupted you are not in a good position to recover.

It's also bad if you attempt that while the file system is full.

>
> exec < t # give a new name (file descriptor 0 in this case) to t
> rm t # remove the old name
> { echo A ; cat ; } > t # generate the new content.

Janis
 |  Next  |  Last
Pages: 1 2 3
Prev: problem with cat and echo
Next: how to specify all files of a certain string length in Bourne Shell