From: stepanm on
> On Wednesday 28 July 2010, stepanm(a)codeaurora.org wrote:
>> > On Wednesday 28 July 2010 00:41:06 Stepan Moskovchenko wrote:
>> >> Add support for the System MMUs found on the 8x60 and 8x72
>> >> families of Qualcomm chips. These SMMUs allow virtualization
>> >> of the address space used by most of the multimedia cores
>> >> on these chips.
>> >
>> > How is this different from an IOMMU?
>> >
>> > From a very brief look, it seems that you should be using the
>> > existing dma-mapping APIs here instead of making up your own.
>> >
>> > Arnd
>>
>>
>> These are just SMMU APIs, and the DMA-mapping API is one layer above
>> this.
>>
>> We have our own SMMU API for the MSM SoCs because we have muliple
>> IOMMUs,
>> each one having multiple contexts, or even having multiple instances of
>> the same context. Our usage model is also quite a bit different from how
>> the DMA APIs are set up. I believe only two IOMMU drivers actually make
>> use of the DMA API (Intel and AMD) and the other ones (OMAP and other
>> SoCs) have their own APIs for their specific use cases.
>
> The DMA API is extremely flexible, it works just fine with all the
> IOMMUs that I've seen so far. Please take a look at
> include/asm-generic/dma-mapping-common.h and its users to see how
> to use multiple IOMMUs depending on the device.
>
> If the OMAP developers got this wrong, that's not your problem :-)
>
> Arnd

Hi Arnd,

From what I have been able to tell, the IOMMU interface was written by
AMD/Intel to allow the kvm code to work with a common IOMMU interface. To
that end, it isn't really a generic IOMMU interface. We have chosen to use
our own interface because it provides us with a lightweight way of
managing mappings for more esoteric MSM-specific use cases.

These map functions also take into account the way in which we map buffers
that we get from our own physical pool, because the current API was not
intended to deal with prioritized allocation of things like on/off-chip
memory. We are currently evaluating how to use the DMA API with our own
specialized allocator, which has been undergoing some discussion on the
other lists. We would like to use this allocator to maximize TLB
performance, as well as to prioritize the allocation from several
different memory pools.

Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: FUJITA Tomonori on
On Wed, 28 Jul 2010 17:58:50 -0700 (PDT)
stepanm(a)codeaurora.org wrote:

> >> These are just SMMU APIs, and the DMA-mapping API is one layer above
> >> this.
> >>
> >> We have our own SMMU API for the MSM SoCs because we have muliple
> >> IOMMUs,
> >> each one having multiple contexts, or even having multiple instances of
> >> the same context. Our usage model is also quite a bit different from how
> >> the DMA APIs are set up. I believe only two IOMMU drivers actually make
> >> use of the DMA API (Intel and AMD) and the other ones (OMAP and other
> >> SoCs) have their own APIs for their specific use cases.
> >
> > The DMA API is extremely flexible, it works just fine with all the
> > IOMMUs that I've seen so far. Please take a look at
> > include/asm-generic/dma-mapping-common.h and its users to see how
> > to use multiple IOMMUs depending on the device.
> >
> > If the OMAP developers got this wrong, that's not your problem :-)
> >
> > Arnd
>
> Hi Arnd,
>
> From what I have been able to tell, the IOMMU interface was written by
> AMD/Intel to allow the kvm code to work with a common IOMMU interface. To

Don't confuse the IOMMU interface with the DMA API that Arnd
mentioned.

They are not related at all.

The DMA API is defined in Documentation/DMA-API.txt.

Arnd told you that include/asm-generic/dma-mapping-common.h is the
library to support the DMA API with multiple IOMMUs. Lots of
architectures (x86, powerpc, sh, alpha, ia64, microblaze, sparc)
use it.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: FUJITA Tomonori on
On Wed, 28 Jul 2010 22:21:56 +0100
Russell King - ARM Linux <linux(a)arm.linux.org.uk> wrote:

> On Wed, Jul 28, 2010 at 07:50:20PM +0200, Arnd Bergmann wrote:
> > The DMA API is extremely flexible, it works just fine with all the
> > IOMMUs that I've seen so far. Please take a look at
> > include/asm-generic/dma-mapping-common.h and its users to see how
> > to use multiple IOMMUs depending on the device.
>
> We don't yet use those DMA API interface extensions because we haven't
> had the need. If someone who has the need wants to put the effort in
> though...
>
> One of the problems with it though is the abstraction of the sync*
> operations is the wrong way around for stuff like dmabounce - we want
> to be passed the base address of the buffer (so we can look this up),
> plus offset and length. We don't want to know just the region which
> is affected.

We can't pass the base address because the DMA API callers don't pass
the base address for dma_sync_single_for_{device|cpu}.

dma_sync_single_range_for_* requires the base address but they are
obsolete.

So you need to fix dmabounce. Actually, I send you a patch to fix
dmabounce long ago (looks like not applied yet):

http://kerneltrap.org/mailarchive/linux-netdev/2010/4/5/6274046
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Arnd Bergmann on
On Wednesday 28 July 2010 23:21:56 Russell King - ARM Linux wrote:
> On Wed, Jul 28, 2010 at 07:50:20PM +0200, Arnd Bergmann wrote:
> > The DMA API is extremely flexible, it works just fine with all the
> > IOMMUs that I've seen so far. Please take a look at
> > include/asm-generic/dma-mapping-common.h and its users to see how
> > to use multiple IOMMUs depending on the device.
>
> We don't yet use those DMA API interface extensions because we haven't
> had the need. If someone who has the need wants to put the effort in
> though...

Right, it shouldn't be hard now that the groundwork for that is done.
Also, it's only really needed if you have IOMMUs of different types in the
same system. If msm doesn't have any swiotlb or dmabounce devices,
it could always use the same implementation for all devices.

> One of the problems with it though is the abstraction of the sync*
> operations is the wrong way around for stuff like dmabounce - we want
> to be passed the base address of the buffer (so we can look this up),
> plus offset and length. We don't want to know just the region which
> is affected.

Yes, but that is an unrelated (dmabounce specific) problem that seems to
be fixed by an existing patch.

The driver posted by Stepan doesn't even support the dma_sync_single_*
style operations, and I don't think it can run into that specific problem.
Are there even (hardware) IOMMUs that are connected to noncoherent
buses? AFAICT, anything that needs to flush a dcache range in dma_sync_*
has a trivial mapping between bus and phys addresses.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Arnd Bergmann on
On Thursday 29 July 2010 05:35:48 FUJITA Tomonori wrote:
> >
> > From what I have been able to tell, the IOMMU interface was written by
> > AMD/Intel to allow the kvm code to work with a common IOMMU interface. To
>
> Don't confuse the IOMMU interface with the DMA API that Arnd
> mentioned.
>
> They are not related at all.

Exactly, thanks for the clarification. I also didn't realize that there
is now an include/linux/iommu.h file that only describes the PCI SR-IOV
interfaces, unlike the generic IOMMU support that we have in your
include/linux/dma-mapping.h file.

Maybe we should rename linux/iommu.h to something more specific so we
can reduce this confusion in the future.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/