From: Roedel, Joerg on
On Mon, Aug 02, 2010 at 04:30:26AM -0400, FUJITA Tomonori wrote:
> On Mon, 02 Aug 2010 18:03:02 +1000
> Benjamin Herrenschmidt <benh(a)kernel.crashing.org> wrote:
>
> > On Mon, 2010-08-02 at 09:48 +0200, Roedel, Joerg wrote:
> > > > Hrm, indeed I just noticed that. Pretty gross... it should definitly
> > > be
> > > > renamed, is will caused endless confusion with unrelated iommu.h and
> > > > iommu_* interfaces which represent something different.
> > >
> > > The first direction to go should be trying to unify all the different
> > > iommu* interfaces into the iommu-api. The generic api will definitly
> > > need to be extended for that, but since it is an in-kernel interface
> > > thats no problem.
> >
> > Well, I suppose I'm the de-facto candidate to take care of the powerpc
> > side then :-)
>
> We already agreed that what the iommu-api looks like?

Well, we currently trying to figure out how to extend the IOMMU-API
concepts to fit the omap-hardware in. Thats what I currently discuss
with Stephan. It looks to me that we need to add the concept
of device contexts to the IOMMU-API. We should also add IO-TLB
management functions. The TLB management is currently handled completly
in the backend driver. This needs to be changed and makes sense for
AMD IOMMUs and Intel IOMMUs with QPI too.

> ARM's iommu code (arch/plat-omap/include/plat/iommu.h) is a library to
> simplify the IOMMU implementations. It could be useful for all the
> iommu implementations.

To me it looks like a very hardware specific library. But it should fit
well in the domain/device concept the IOMMU-API provides (when we also
add device-contexts to it)

> The current iommu-api (include/linux/iommu.h) provides the common
> interface for specific purposes (for KVM).

The IOMMU-API is not limited to the purposes of KVM. There is
currently development effort to use the IOMMU-API for UIO stuff. So the
IOMMU-API is by no means KVM specific.

> Another question is how the above can work with the DMA-API.

This would work if we handle every device-context the platform provides
as 'struct device'. But does that really need to work with the DMA-API?
What is the driver use-case for that?

Joerg

--
Joerg Roedel - AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Russell King - ARM Linux on
On Mon, Aug 02, 2010 at 06:20:01PM +0900, FUJITA Tomonori wrote:
> On Mon, 2 Aug 2010 10:03:26 +0100
> Russell King - ARM Linux <linux(a)arm.linux.org.uk> wrote:
>
> > On Mon, Aug 02, 2010 at 05:30:26PM +0900, FUJITA Tomonori wrote:
> > > ARM's iommu code (arch/plat-omap/include/plat/iommu.h) is a library to
> > > simplify the IOMMU implementations. It could be useful for all the
> > > iommu implementations.
> >
> > ITYM OMAP's iommu code.
>
> Yeah, I meant that we could extend it to make it useful for other
> iommu implementations. At least, we could make something generic like
> struct iommu_functions, I think. Then we can embed a generic iommu
> structure into an iommu specific struct (like we do with inode).
>
> The current iommu-api (include/linux/iommu.h) is just about domain and
> mapping concept. We can implement it on the top of the above
> infrastructure.
>
> I'm still trying to figure out how the DMA-API can work well with
> them.

I'm not sure it can in totality. The DMA-API solves the host CPU <->
device aspect of DMA support only.

However, there is another use case for IOMMUs, which is to allow two
separate peripheral devices to communicate with each other via system
memory. As the streaming DMA-API involves the idea of buffer ownership
(of a singular agent), it is unsuitable for this use case.

The coherent allocation part of the DMA-API also only deals with the
idea of there being a singular DMA agent accessing the allocated buffer
(in conjunction with the host CPU).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: FUJITA Tomonori on
On Mon, 2 Aug 2010 11:04:19 +0100
Russell King - ARM Linux <linux(a)arm.linux.org.uk> wrote:

> On Mon, Aug 02, 2010 at 06:20:01PM +0900, FUJITA Tomonori wrote:
> > On Mon, 2 Aug 2010 10:03:26 +0100
> > Russell King - ARM Linux <linux(a)arm.linux.org.uk> wrote:
> >
> > > On Mon, Aug 02, 2010 at 05:30:26PM +0900, FUJITA Tomonori wrote:
> > > > ARM's iommu code (arch/plat-omap/include/plat/iommu.h) is a library to
> > > > simplify the IOMMU implementations. It could be useful for all the
> > > > iommu implementations.
> > >
> > > ITYM OMAP's iommu code.
> >
> > Yeah, I meant that we could extend it to make it useful for other
> > iommu implementations. At least, we could make something generic like
> > struct iommu_functions, I think. Then we can embed a generic iommu
> > structure into an iommu specific struct (like we do with inode).
> >
> > The current iommu-api (include/linux/iommu.h) is just about domain and
> > mapping concept. We can implement it on the top of the above
> > infrastructure.
> >
> > I'm still trying to figure out how the DMA-API can work well with
> > them.
>
> I'm not sure it can in totality. The DMA-API solves the host CPU <->
> device aspect of DMA support only.
>
> However, there is another use case for IOMMUs, which is to allow two
> separate peripheral devices to communicate with each other via system
> memory. As the streaming DMA-API involves the idea of buffer ownership
> (of a singular agent), it is unsuitable for this use case.
>
> The coherent allocation part of the DMA-API also only deals with the
> idea of there being a singular DMA agent accessing the allocated buffer
> (in conjunction with the host CPU).

I don't have a clear idea what kinda API works well in the above case
yet.

But we have been talking about more bigger things? Not just about the
interface, how things work together.

- OMAP's iommu code is a library to simplify OMAP implementations.

- include/linux/iommu.h is an interface to IOMMUs for specific
interfaces.

- the DMA API could access to IOMMUs internally.

IOMMU library provides generic iommu interfaces (a structure including
IOMMU implementation specific function pointers). KVM uses some of
them. Architectures could implement the DMA API on the top of some the
interfaces too.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Zach Pfeffer on
On Mon, Aug 02, 2010 at 09:58:02AM +0200, Roedel, Joerg wrote:
> Hi Stephan,
>
> On Fri, Jul 30, 2010 at 01:19:06AM -0400, stepanm(a)codeaurora.org wrote:
> > Unlike a more traditional system with one IOMMU between the bus and
> > memory, MSM has multiple IOMMUs, with each one hard-wired to a dedicated
> > device. Furthermore, each IOMMU can have more than one translation
> > context. One of the use cases is being able to create mappings within
> > multiple instances of one context, and arbitrarily context-switch the
> > IOMMU on the fly.
>
> The IOMMU-API supports multiple IOMMUs (at least multiple AMD/Intel
> IOMMUs). But the face that there are more than one IOMMU is hidden in
> the backend driver implementation. The API itself only works with
> domains and devices. The IOMMU driver needs to know which IOMMU it needs
> to program for a given device. If I understand the concept of your
> hardware correctly you also have this information.
>
> > It sounds like the domain abstraction and attach_device/detach_device can
> > encapsulate this rather nicely and I am in the process of updating my
> > driver to fit this framework.
> >
> > My problem, however, is with iommu_domain_alloc(). This will set up a
> > domain and call the ops function to initialize it, but I want to be able
> > to pass it an ???IOMMU id" that will tell the underlying driver which IOMMU
> > (and which "stream id") is to be associated with that domain instance.
> > This can be a void* parameter that gets passed through to domain_init. I
> > feel like this change will make it easy to deal with multiple
> > IOMMUs/translation contexts, and implementations that have only a singular
> > IOMMU/translation context are free to ignore that parameter.
>
> In the means of the IOMMU-API the domain is the abstraction of an
> address space (in other words a page table). The IOMMU(s) which this domain
> is later assigned to are determined by the iommu_attach_device calls.
> I think the right way to go here is to create the concept of a
> device-context in the IOMMU-API and add functions like
>
> iommu_attach_context(struct iommu_domain *domain,
> struct iommu_context *ctxt);
> iommu_detach_context(struct iommu_context *ctxt);
>
> This would work if you can determine in your iommu-driver which iommu
> you need to program for which device. What do you think?
>

Joerg, I'd like to make sure I understand this. A domain is an address
space separate from the actual page-tables that may be part of an
iommu_context, correct? After I iommu_attach_context the ctxt will
reflect the address space of the domain, correct?

>
> Joerg
>
> --
> Joerg Roedel - AMD Operating System Research Center
>
> Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
> General Managers: Alberto Bozzo, Andrew Bowd
> Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo(a)vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Roedel, Joerg on
On Mon, Aug 02, 2010 at 04:29:38PM -0400, Zach Pfeffer wrote:
> On Mon, Aug 02, 2010 at 09:58:02AM +0200, Roedel, Joerg wrote:

> > In the means of the IOMMU-API the domain is the abstraction of an
> > address space (in other words a page table). The IOMMU(s) which this domain
> > is later assigned to are determined by the iommu_attach_device calls.
> > I think the right way to go here is to create the concept of a
> > device-context in the IOMMU-API and add functions like
> >
> > iommu_attach_context(struct iommu_domain *domain,
> > struct iommu_context *ctxt);
> > iommu_detach_context(struct iommu_context *ctxt);
> >
> > This would work if you can determine in your iommu-driver which iommu
> > you need to program for which device. What do you think?
> >
>
> Joerg, I'd like to make sure I understand this. A domain is an address
> space separate from the actual page-tables that may be part of an
> iommu_context, correct? After I iommu_attach_context the ctxt will
> reflect the address space of the domain, correct?

A domain is defined by a single page-table which can be modified using
the iommu_map/iommu_unmap function calls. I am not completly sure what
you mean by an iommu_context. Can you describe what it means in your
context?

Joerg

--
AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/