USB transfer_buffer allocations on 64bit systems
in [Kernel]
|
Prev: mfd: Initialise WM831x IRQ masks on chip even if interrupts not in use
Next: INFO: task apache2:7601 blocked for more than 120 seconds.
From: Daniel Mack on 7 May 2010 06:30 On Fri, May 07, 2010 at 11:47:37AM +0200, Clemens Ladisch wrote: > Daniel Mack wrote: > > The problem is again (summarized): > > > > On 64bit machines, with 4GB or more, the allocated buffers for USB > > transfers might be beyond the 32bit boundary. In this case, the IOMMU > > should take care and install DMA bounce buffer to copy over the buffer > > before the transfer actually happens. The problem is, however, that this > > copy mechanism takes place when the URB with its associated buffer is > > submitted, not when the EHCI will actually do the transfer. > > > > In the particular case of audio drivers, though, the contents of the > > buffers are likely to change after the submission. What we do here > > is that we map the audio stream buffers which are used by ALSA to > > the output URBs, so they're filled asychronously. Once the buffer is > > actually sent out on the bus, it is believed to contain proper audio > > date. If it doesn't, that's due to too tight audio timing or other > > problems. This breaks once buffers are magically bounced in the > > background. > > At least the audio class and ua101 drivers don't do this and fill the > buffers before they are submitted. Gnaa, you're right. I _thought_ my code does it the way I described, but what I wrote is how I _wanted_ to do it, not how it's currently done. I have a plan to change this in the future. So unfortunately, that doesn't explain it either. Sorry for the noise. Daniel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Oliver Neukum on 7 May 2010 07:50 Am Freitag, 7. Mai 2010 11:47:37 schrieb Clemens Ladisch: > > In the particular case of audio drivers, though, the contents of the > > buffers are likely to change after the submission. What we do here > > is that we map the audio stream buffers which are used by ALSA to > > the output URBs, so they're filled asychronously. Once the buffer is > > actually sent out on the bus, it is believed to contain proper audio > > date. If it doesn't, that's due to too tight audio timing or other > > problems. This breaks once buffers are magically bounced in the > > background. > > At least the audio class and ua101 drivers don't do this and fill the > buffers before they are submitted. > > > So - long story short: these audio buffers need to be DMA coherent. > > Does the USB API actually guarantee that all controllers use DMA, i.e., > that the buffers can be filled after submission? No, you must not touch buffers after submission. This does not even work if we use DMA, because on some architectures this violates guarantees to the dma primitives. It cannot be done. Regards Oliver -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Daniel Mack on 7 May 2010 08:00 On Fri, May 07, 2010 at 01:47:56PM +0200, Oliver Neukum wrote: > Am Freitag, 7. Mai 2010 13:42:34 schrieb Oliver Neukum: > > Am Freitag, 7. Mai 2010 11:47:37 schrieb Clemens Ladisch: > > > > In the particular case of audio drivers, though, the contents of the > > > > buffers are likely to change after the submission. What we do here > > > > is that we map the audio stream buffers which are used by ALSA to > > > > the output URBs, so they're filled asychronously. Once the buffer is > > > > actually sent out on the bus, it is believed to contain proper audio > > > > date. If it doesn't, that's due to too tight audio timing or other > > > > problems. This breaks once buffers are magically bounced in the > > > > background. > > > > > > At least the audio class and ua101 drivers don't do this and fill the > > > buffers before they are submitted. > > > > > > > So - long story short: these audio buffers need to be DMA coherent. > > > > > > Does the USB API actually guarantee that all controllers use DMA, i.e., > > > that the buffers can be filled after submission? > > > > No, you must not touch buffers after submission. This does not even > > work if we use DMA, because on some architectures this violates > > guarantees to the dma primitives. It cannot be done. We do such tricks on other OS where IRQ latency as high as some tens of milliseconds. So we queue EHCI transfers well in advance and update access their contents (for both input and output) from audio context. I considered implementing this idea to the Linux USB audio driver as well. > Or to be precise it could be done with coherent memory, but you'd risk > transfering partially updated buffers, as you cannot know when DMA will > be done and we don't guarantee that DMA will be done right as we transfer. If the memory is coherent, it should be possible. And if buffers are only partially updated, the audio buffer scheduling is too tight. IOW, the configured buffer size of audio clients in userspace is too small. This is OT now I believe, I will start another thread for this topic once I have something to show. Thanks, Daniel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Alan Stern on 7 May 2010 10:50 On Fri, 7 May 2010, Daniel Mack wrote: > > Or to be precise it could be done with coherent memory, but you'd risk > > transfering partially updated buffers, as you cannot know when DMA will > > be done and we don't guarantee that DMA will be done right as we transfer. > > If the memory is coherent, it should be possible. And if buffers are > only partially updated, the audio buffer scheduling is too tight. IOW, > the configured buffer size of audio clients in userspace is too small. Even with coherent memory, this dynamic updating of buffer contents isn't a good idea. There's no guarantee that coherent memory won't be implemented using a bounce buffer during URB submission. Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Alan Stern on 7 May 2010 11:00
On Fri, 7 May 2010, Daniel Mack wrote: > > At least the audio class and ua101 drivers don't do this and fill the > > buffers before they are submitted. > > Gnaa, you're right. I _thought_ my code does it the way I described, but > what I wrote is how I _wanted_ to do it, not how it's currently done. I > have a plan to change this in the future. > > So unfortunately, that doesn't explain it either. Sorry for the noise. At one point we tried an experiment, printing out the buffer and DMA addresses. I don't recall seeing anything obviously wrong, but if an IOMMU was in use then that might not mean anything. Is it possible that the IOMMU mappings sometimes get messed up for addresses above 4 GB? Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |