Provide a zero-copy method on KVM virtio-net.
in [Kernel]
|
Prev: linux-next: build warning after merge of the devicetree tree
Next: [PATCH]: Compress hibernation image with LZO (in-kernel)
From: Avi Kivity on 30 Jul 2010 01:10 On 07/30/2010 01:31 AM, Shirley Ma wrote: > >> Our goal is to improve the bandwidth and reduce the CPU usage. >> Exact performance data will be provided later. > I did some vhost performance measurement over 10Gb ixgbe, and found that > in order to get consistent BW results, netperf/netserver, qemu, vhost > threads smp affinities are required. > > Looking forward to these results for small message size comparison. For > large message size 10Gb ixgbe BW already reached by doing vhost smp > affinity w/i offloading support, we will see how much CPU utilization it > can be reduced. > > Please provide latency results as well. I did some experimental on > macvtap zero copy sendmsg, what I have found that get_user_pages latency > pretty high. > get_user_pages() is indeed slow. But what about get_user_pages_fast()? Note that when the page is first touched, get_user_pages_fast() falls back to get_user_pages(), so the latency needs to be measured after quite a bit of warm-up. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Arnd Bergmann on 31 Jul 2010 05:40 On Friday 30 July 2010 17:51:52 Shirley Ma wrote: > On Fri, 2010-07-30 at 16:53 +0800, Xin, Xiaohui wrote: > > >Since vhost-net already supports macvtap/tun backends, do you think > > >whether it's better to implement zero copy in macvtap/tun than > > inducing > > >a new media passthrough device here? > > > > > > > I'm not sure if there will be more duplicated code in the kernel. > > I think it should be less duplicated code in the kernel if we use > macvtap to support what media passthrough driver here. Since macvtap has > support virtio_net head and offloading already, the only missing func is > zero copy. Also QEMU supports macvtap, we just need add a zero copy flag > in option. Yes, I fully agree and that was one of the intended directions for macvtap to start with. Thank you so much for following up on that, I've long been planning to work on macvtap zero-copy myself but it's now lower on my priorities, so it's good to hear that you made progress on it, even if there are still performance issues. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 1 Aug 2010 04:20 On 07/30/2010 06:46 PM, Shirley Ma wrote: > Hello Avi, > > On Fri, 2010-07-30 at 08:02 +0300, Avi Kivity wrote: >> get_user_pages() is indeed slow. But what about >> get_user_pages_fast()? >> >> Note that when the page is first touched, get_user_pages_fast() falls >> back to get_user_pages(), so the latency needs to be measured after >> quite a bit of warm-up. > Yes, I used get_user_pages_fast, however if falled back to > get_user_pages() when the apps doesn't allocate buffer on the same page. > If I run a single ping, the RTT is extremely high, but when running > multiple pings, the RTT time reduce significantly, but still it is not > as fast as copy from my initial test. I am thinking that we might need > to pre-pin memory pool. > I don't understand. Under what conditions do you use get_user_pages() instead of get_user_pages_fast()? Why? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 2 Aug 2010 12:20 On 08/02/2010 07:01 PM, Shirley Ma wrote: > Hello Avi, > > On Sun, 2010-08-01 at 11:18 +0300, Avi Kivity wrote: >> I don't understand. Under what conditions do you use >> get_user_pages() >> instead of get_user_pages_fast()? Why? > The code always calls get_user_pages_fast, however, the page will be > unpinned in skb_free if the same page is not used again for a new > buffer. The reason for unpin the page is we don't want to pin all of the > guest kernel memory(memory over commit). That is fine. > So get_user_pages_fast will > call slow path get_user_pages. I don't understand this. gup_fast() only calls gup() if the page is swapped out or read-only. > Your previous comment is suggesting to keep the page pinned for > get_user_pages_fast fast path? > Right now I'm not sure I understand what's happening. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 2 Aug 2010 12:40
On 08/02/2010 07:25 PM, Shirley Ma wrote: > On Mon, 2010-08-02 at 19:11 +0300, Avi Kivity wrote: >> I don't understand this. gup_fast() only calls gup() if the page is >> swapped out or read-only. > Oh, I used the page as read-only on xmit path. Should I use write > instead? No, for xmit getting the page as read only is fine. I was inaccurate, gup_fast() performs as follows: - if .write = 1, gup_fast() will be fast if the page is mapped and writeable - if .write = 0, gup_fast() will be fast if the page is mapped so, using .write = 0 for the xmit path will be faster in more cases than ..write = 1. When are you seeing gup_fast() fall back to gup()? It should be at most once per page (when a guest starts up none of its pages are mapped, it faults them in on demand). -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |