ad7877: keep dma rx buffers in seperate cache lines
in [Kernel]
|
From: Pekka Enberg on 11 May 2010 16:50 Mike Frysinger wrote: > On Tue, May 11, 2010 at 16:38, Pekka Enberg wrote: >> Mike Frysinger wrote: >>> that is a question for David/Grant. i'm not the SPI core maintainer, >>> i'm merely watching over some SPI drivers. however, this answer also >>> doesnt sound like it's thinking big enough because what you're >>> proposing isnt specific to the SPI bus -- any time a DMA safe buffer >>> is needed dynamically, this function could be used. >> Well, we have dma_alloc_coherent(), shouldn't you be using that instead? > > my understanding is that dma_alloc_coherent() gives you a buffer that > is always coherent. the SPI layers take care of flushing and such on > the fly which means allocating coherent memory is overkill and bad for > performance. OK, I'm out of my expert area here but if dma_alloc_coherent() doesn't work for you, you should probably extend the DMA API, not kmalloc(). Pekka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Christoph Lameter on 11 May 2010 17:00 On Tue, 11 May 2010, Mike Frysinger wrote: > > DMA. If the arch can only DMA into cacheline aligned objects then the > > correct method is to force kmalloc alignment to cacheline size. > > these are SPI drivers and are usable on any arch that supports a SPI > bus (which is pretty much every arch). forget about "embedded" > arches. > > the issue here is simple: a SPI driver (AD7877) needs to do a receive > SPI transfer into a DMA safe buffer. what is the exact API to > dynamically allocate memory for the structure with this buffer > embedded in it such that the start of the structure is cached aligned > ? creating a dedicated kmem cache may work, but it isnt a scalable > solution if every SPI driver needs to create its own cache. kmalloc returns a pointer to a DMA safe buffer. There is no requirement on the x86 hardware that the DMA buffers have to be cache aligned. Cachelines will be invalidated as needed. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Mike Frysinger on 11 May 2010 17:00 On Tue, May 11, 2010 at 16:46, Christoph Lameter wrote: > On Tue, 11 May 2010, Mike Frysinger wrote: >> > DMA. If the arch can only DMA into cacheline aligned objects then the >> > correct method is to force kmalloc alignment to cacheline size. >> >> these are SPI drivers and are usable on any arch that supports a SPI >> bus (which is pretty much every arch). forget about "embedded" >> arches. >> >> the issue here is simple: a SPI driver (AD7877) needs to do a receive >> SPI transfer into a DMA safe buffer. what is the exact API to >> dynamically allocate memory for the structure with this buffer >> embedded in it such that the start of the structure is cached aligned >> ? creating a dedicated kmem cache may work, but it isnt a scalable >> solution if every SPI driver needs to create its own cache. > > kmalloc returns a pointer to a DMA safe buffer. There is no requirement on > the x86 hardware that the DMA buffers have to be cache aligned. Cachelines > will be invalidated as needed. so this guarantee is made by the kmalloc() API ? and for arches where the cacheline invalidation is handled in software rather than hardware, they must declare a min alignment value for kmalloc to be at least as big as their cache alignment ? does the phrase "DMA safe buffer" imply cache alignment ? -mike -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Pekka Enberg on 11 May 2010 17:10 Mike Frysinger wrote: > On Tue, May 11, 2010 at 16:46, Christoph Lameter wrote: >> On Tue, 11 May 2010, Mike Frysinger wrote: >>>> DMA. If the arch can only DMA into cacheline aligned objects then the >>>> correct method is to force kmalloc alignment to cacheline size. >>> these are SPI drivers and are usable on any arch that supports a SPI >>> bus (which is pretty much every arch). forget about "embedded" >>> arches. >>> >>> the issue here is simple: a SPI driver (AD7877) needs to do a receive >>> SPI transfer into a DMA safe buffer. what is the exact API to >>> dynamically allocate memory for the structure with this buffer >>> embedded in it such that the start of the structure is cached aligned >>> ? creating a dedicated kmem cache may work, but it isnt a scalable >>> solution if every SPI driver needs to create its own cache. >> kmalloc returns a pointer to a DMA safe buffer. There is no requirement on >> the x86 hardware that the DMA buffers have to be cache aligned. Cachelines >> will be invalidated as needed. > > so this guarantee is made by the kmalloc() API ? and for arches where > the cacheline invalidation is handled in software rather than > hardware, they must declare a min alignment value for kmalloc to be at > least as big as their cache alignment ? > > does the phrase "DMA safe buffer" imply cache alignment ? Yes, you should be able to DMA into kmalloc'd memory. IIRC the block or the SCSI layer depends on that. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Mike Frysinger on 11 May 2010 17:20
On Tue, May 11, 2010 at 17:01, Pekka Enberg wrote: > Mike Frysinger wrote: >> On Tue, May 11, 2010 at 16:46, Christoph Lameter wrote: >>> On Tue, 11 May 2010, Mike Frysinger wrote: >>>>> DMA. If the arch can only DMA into cacheline aligned objects then the >>>>> correct method is to force kmalloc alignment to cacheline size. >>>> >>>> these are SPI drivers and are usable on any arch that supports a SPI >>>> bus (which is pretty much every arch). forget about "embedded" >>>> arches. >>>> >>>> the issue here is simple: a SPI driver (AD7877) needs to do a receive >>>> SPI transfer into a DMA safe buffer. what is the exact API to >>>> dynamically allocate memory for the structure with this buffer >>>> embedded in it such that the start of the structure is cached aligned >>>> ? creating a dedicated kmem cache may work, but it isnt a scalable >>>> solution if every SPI driver needs to create its own cache. >>> >>> kmalloc returns a pointer to a DMA safe buffer. There is no requirement >>> on >>> the x86 hardware that the DMA buffers have to be cache aligned. >>> Cachelines >>> will be invalidated as needed. >> >> so this guarantee is made by the kmalloc() API ? and for arches where >> the cacheline invalidation is handled in software rather than >> hardware, they must declare a min alignment value for kmalloc to be at >> least as big as their cache alignment ? >> >> does the phrase "DMA safe buffer" imply cache alignment ? > > Yes, you should be able to DMA into kmalloc'd memory. IIRC the block or the > SCSI layer depends on that. we can DMA into it just fine, it's just that if the small DMA buffer shares a cacheline with some polled state data (like in current AD7877 driver) instead of getting its own, the core may pull back in stale data before the DMA finishes. so we're looking for something that guarantees both cacheline alignment (when applicable, so this doesnt apply to x86 apparently) and DMA-safe dynamic memory. so if Dmitry's original statement: The data is kmalloced, kmalloc aligns on cacheline boundary AFAIK which means that next kmalloc data chunk will not share "our" cacheline. was revised to say: The data is kmalloced, kmalloc aligns on cacheline boundary (when the arch lacks hardware cacheline invalidation between the core and dma channels) which means that next kmalloc data chunk will not share "our" cacheline. and this revised statement is correct, then Dmitry's revised fix should be sufficient. and i guess we should update the spi-summary document with these tips so we dont have to hash this out again. -mike -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |