Unexpected splice "always copy" behavior observed
in [Kernel]
|
From: Steven Rostedt on 19 May 2010 11:00 On Wed, 2010-05-19 at 07:39 -0700, Linus Torvalds wrote: > The real limitation is likely always going to be the fact that it has to > be page-aligned and a full page. For a lot of splice inputs, that simply > won't be the case, and you'll end up copying for alignment reasons anyway. That's understandable. For the use cases of splice I use, I work to make it page aligned and full pages. Anyone else using splice for optimizations, should do the same. It only makes sense. The end of buffer may not be a full page, but then it's the end anyway, and I'm not as interested in the speed. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds on 19 May 2010 11:10 On Wed, 19 May 2010, Steven Rostedt wrote: > On Wed, 2010-05-19 at 07:39 -0700, Linus Torvalds wrote: > > > The real limitation is likely always going to be the fact that it has to > > be page-aligned and a full page. For a lot of splice inputs, that simply > > won't be the case, and you'll end up copying for alignment reasons anyway. > > That's understandable. For the use cases of splice I use, I work to make > it page aligned and full pages. Anyone else using splice for > optimizations, should do the same. It only makes sense. > > The end of buffer may not be a full page, but then it's the end anyway, > and I'm not as interested in the speed. Btw, since you apparently have a real case - is the "splice to file" always just an append? IOW, if I'm not right in assuming that the only sane thing people would reasonable care about is "append to a file", then holler now. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Nick Piggin on 19 May 2010 11:20 On Wed, May 19, 2010 at 07:39:11AM -0700, Linus Torvalds wrote: > > > On Wed, 19 May 2010, Nick Piggin wrote: > > > > We can possibly do an attempt to invalidate existing pagecache and > > then try to install the new page. > > Yes, but that's going to be rather hairier. We need to make sure that the > filesystem doesn't have some kind of dirty pointers to the old page etc. > Although I guess that should always show up in the page counters, so I > guess we can always handle the case of page_count() being 1 (only page > cache) and the page being unlocked. Well I mean a full invalidate -- invalidate_mapping_pages -- so there is literally no pagecache there at all. Then we just need to ensure that the filesystem doesn't do anything funny with the page in write_begin (I don't know, such as zero out holes or something strange). I don't think any do except maybe for something obscure like jffs2, but obviously it needs to be looked at. Error handling may need to be looked at too, but shouldn't be much issue I'd think. Even so, it's all going to add branches and complexity to an important fast path, so we'd want to see numbers. > So I'd much rather just handle the "append to the end". > > The real limitation is likely always going to be the fact that it has to > be page-aligned and a full page. For a lot of splice inputs, that simply > won't be the case, and you'll end up copying for alignment reasons anyway. That's true. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Steven Rostedt on 19 May 2010 11:20 On Wed, 2010-05-19 at 07:59 -0700, Linus Torvalds wrote: > > Btw, since you apparently have a real case - is the "splice to file" > always just an append? IOW, if I'm not right in assuming that the only > sane thing people would reasonable care about is "append to a file", then > holler now. My use case is just to move the data from the ring buffer into a file (or network) as fast as possible. It creates a new file and all additions are "append to a file". I believe Mathieu does the same. With me, you are correct. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Miklos Szeredi on 19 May 2010 11:30
On Wed, 19 May 2010, Linus Torvalds wrote: > The real limitation is likely always going to be the fact that it has to > be page-aligned and a full page. For a lot of splice inputs, that simply > won't be the case, and you'll end up copying for alignment reasons anyway. Another limitation I found while splicing from one file to another is that stealing from the source file's page cache does not always succeed. This turned out to be because of a reference from the lru cache for freshly read pages. I'm not sure how this could be fixed. Miklos -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |