Add a new structure for skb buffer from external.
in [Kernel]
|
From: Dong, Eddie on 23 Jun 2010 04:20 > 3) As I have mentioned above, with this idea, netdev_alloc_skb() will > allocate > as usual, the data pointed by skb->data will be copied into the first > guest buffer. > That means we should reserve sufficient room in guest buffer. For PS > mode > supported driver (for example ixgbe), the room will be more than 128. > After 128bytes, > we will put the first frag data. Look into virtio-net.c the function > page_to_skb() > and receive_mergeable(), that means we should modify guest virtio-net > driver to > compute the offset as the parameter for skb_set_frag(). > > How do you think about this? Attached is a patch to how to modify the > guest driver. > I reserve 512 bytes as an example, and transfer the header len of the > skb in hdr->hdr_len. > Xiaohui & Herbert: Mixing copy of head & 0-copy of bulk data imposes additional challange to find the guest buffer. The backend driver may be unable to find a spare guest buffer from virtqueue at that time which may block the receiving process then. Can't we completely eliminate netdev_alloc_skb here? Assigning guest buffer at this time makes life much easier. Thx, Eddie -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Herbert Xu on 23 Jun 2010 06:00 On Wed, Jun 23, 2010 at 04:09:40PM +0800, Dong, Eddie wrote: > > Xiaohui & Herbert: > Mixing copy of head & 0-copy of bulk data imposes additional challange to find the guest buffer. The backend driver may be unable to find a spare guest buffer from virtqueue at that time which may block the receiving process then. > Can't we completely eliminate netdev_alloc_skb here? Assigning guest buffer at this time makes life much easier. I'm not sure I understand you concern. If you mean that when the guest doesn't give enough pages to the host and the host can't receive on behalf of the guest then isn't that already the case with the original patch-set? Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} <herbert(a)gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Dong, Eddie on 23 Jun 2010 06:10 Herbert Xu wrote: > On Wed, Jun 23, 2010 at 04:09:40PM +0800, Dong, Eddie wrote: >> >> Xiaohui & Herbert: >> Mixing copy of head & 0-copy of bulk data imposes additional >> challange to find the guest buffer. The backend driver may be >> unable to find a spare guest buffer from virtqueue at that time >> which may block the receiving process then. Can't we completely >> eliminate netdev_alloc_skb here? Assigning guest buffer at this time >> makes life much easier. > > I'm not sure I understand you concern. If you mean that when > the guest doesn't give enough pages to the host and the host > can't receive on behalf of the guest then isn't that already > the case with the original patch-set? > I mean once the frontend side driver post the buffers to the backend driver, the backend driver will "immediately" use that buffers to compose skb or gro_frags and post them to the assigned host NIC driver as receive buffers. In that case, if the backend driver recieves a packet from the NIC that requires to do copy, it may be unable to find additional free guest buffer because all of them are already used by the NIC driver. We have to reserve some guest buffers for the possible copy even if the buffer address is not identified by original skb :( Thx, Eddie -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Herbert Xu on 24 Jun 2010 06:10 On Wed, Jun 23, 2010 at 06:05:41PM +0800, Dong, Eddie wrote: > > I mean once the frontend side driver post the buffers to the backend driver, the backend driver will "immediately" use that buffers to compose skb or gro_frags and post them to the assigned host NIC driver as receive buffers. In that case, if the backend driver recieves a packet from the NIC that requires to do copy, it may be unable to find additional free guest buffer because all of them are already used by the NIC driver. We have to reserve some guest buffers for the possible copy even if the buffer address is not identified by original skb :( OK I see what you mean. Can you tell me how does Xiaohui's previous patch-set deal with this problem? Thanks, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} <herbert(a)gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Dong, Eddie on 24 Jun 2010 21:10
Herbert Xu wrote: > On Wed, Jun 23, 2010 at 06:05:41PM +0800, Dong, Eddie wrote: >> >> I mean once the frontend side driver post the buffers to the backend >> driver, the backend driver will "immediately" use that buffers to >> compose skb or gro_frags and post them to the assigned host NIC >> driver as receive buffers. In that case, if the backend driver >> recieves a packet from the NIC that requires to do copy, it may be >> unable to find additional free guest buffer because all of them are >> already used by the NIC driver. We have to reserve some guest >> buffers for the possible copy even if the buffer address is not >> identified by original skb :( > > OK I see what you mean. Can you tell me how does Xiaohui's > previous patch-set deal with this problem? > > Thanks, In current patch, each SKB for the assigned device (SRIOV VF or NIC or a complete queue pairs) uses the buffer from guest, so it eliminates copy completely in software and requires hardware to do so. If we can have an additonal place to store the buffer per skb (may cause copy later on), we can do copy later on or re-post the buffer to assigned NIC driver later on. But that may be not very clean either :( BTW, some hardware may require certain level of packet copy such as for broadcast packets in very old VMDq device, which is not addressed in previous Xiaohui's patch yet. We may address this by implementing an additional virtqueue between guest and host for slow path (broadcast packets only here) with additinal complexity in FE/BE driver. Thx, Eddie-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |