From: Benjamin Herrenschmidt on
On Thu, 2010-07-29 at 10:26 +0200, Arnd Bergmann wrote:
> Exactly, thanks for the clarification. I also didn't realize that
> there
> is now an include/linux/iommu.h file that only describes the PCI
> SR-IOV
> interfaces, unlike the generic IOMMU support that we have in your
> include/linux/dma-mapping.h file.
>
> Maybe we should rename linux/iommu.h to something more specific so we
> can reduce this confusion in the future.

Hrm, indeed I just noticed that. Pretty gross... it should definitly be
renamed, is will caused endless confusion with unrelated iommu.h and
iommu_* interfaces which represent something different.

Ben.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Arnd Bergmann on
On Saturday 31 July 2010 00:58:00 stepanm(a)codeaurora.org wrote:
> > If you really need multiple domains across multiple IOMMUs, I'd suggest that
> > we first merge the APIs and then port your code to that, but as a first step
> > you could implement the standard dma-mapping.h API, which allows you to use
> > the IOMMUs in kernel space.
>
> One of our uses cases actually does involve using domains pretty much as
> you had described them, though only on one of the IOMMUs. That is, the
> domain for that IOMMU basically abstracts its page table, and it is a
> legitimate thing to switch out page tables for the IOMMU on the fly. I
> guess the difference is that you described the domain as the set of
> mappings made on ALL the IOMMUs, whereas I had envisioned there being one
> (or more) domains for each IOMMU.

Can you be more specific on what kind of device would use multiple domains
in your case and how you intend to use them? Is this for some kind of DSP
interacting with user processes?

This seems to be a scenario that we haven't dealt with before (or perhaps
avoided intentionally), so if we need to make API changes, we should all
understand what we need them for. It's no problem to extend the API if you
have good reasons for using multiple domains, but I also want to make sure
that there isn't also a way to achieve the same or better result with the
current APIs.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Roedel, Joerg on
On Fri, Jul 30, 2010 at 11:15:24PM -0400, Benjamin Herrenschmidt wrote:
> On Thu, 2010-07-29 at 10:26 +0200, Arnd Bergmann wrote:
> > Exactly, thanks for the clarification. I also didn't realize that
> > there
> > is now an include/linux/iommu.h file that only describes the PCI
> > SR-IOV
> > interfaces, unlike the generic IOMMU support that we have in your
> > include/linux/dma-mapping.h file.
> >
> > Maybe we should rename linux/iommu.h to something more specific so we
> > can reduce this confusion in the future.
>
> Hrm, indeed I just noticed that. Pretty gross... it should definitly be
> renamed, is will caused endless confusion with unrelated iommu.h and
> iommu_* interfaces which represent something different.

The first direction to go should be trying to unify all the different
iommu* interfaces into the iommu-api. The generic api will definitly
need to be extended for that, but since it is an in-kernel interface
thats no problem.

--
Joerg Roedel - AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Roedel, Joerg on
Hi Stephan,

On Fri, Jul 30, 2010 at 01:19:06AM -0400, stepanm(a)codeaurora.org wrote:
> Unlike a more traditional system with one IOMMU between the bus and
> memory, MSM has multiple IOMMUs, with each one hard-wired to a dedicated
> device. Furthermore, each IOMMU can have more than one translation
> context. One of the use cases is being able to create mappings within
> multiple instances of one context, and arbitrarily context-switch the
> IOMMU on the fly.

The IOMMU-API supports multiple IOMMUs (at least multiple AMD/Intel
IOMMUs). But the face that there are more than one IOMMU is hidden in
the backend driver implementation. The API itself only works with
domains and devices. The IOMMU driver needs to know which IOMMU it needs
to program for a given device. If I understand the concept of your
hardware correctly you also have this information.

> It sounds like the domain abstraction and attach_device/detach_device can
> encapsulate this rather nicely and I am in the process of updating my
> driver to fit this framework.
>
> My problem, however, is with iommu_domain_alloc(). This will set up a
> domain and call the ops function to initialize it, but I want to be able
> to pass it an “IOMMU id" that will tell the underlying driver which IOMMU
> (and which "stream id") is to be associated with that domain instance.
> This can be a void* parameter that gets passed through to domain_init. I
> feel like this change will make it easy to deal with multiple
> IOMMUs/translation contexts, and implementations that have only a singular
> IOMMU/translation context are free to ignore that parameter.

In the means of the IOMMU-API the domain is the abstraction of an
address space (in other words a page table). The IOMMU(s) which this domain
is later assigned to are determined by the iommu_attach_device calls.
I think the right way to go here is to create the concept of a
device-context in the IOMMU-API and add functions like

iommu_attach_context(struct iommu_domain *domain,
struct iommu_context *ctxt);
iommu_detach_context(struct iommu_context *ctxt);

This would work if you can determine in your iommu-driver which iommu
you need to program for which device. What do you think?


Joerg

--
Joerg Roedel - AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Benjamin Herrenschmidt on
On Mon, 2010-08-02 at 09:48 +0200, Roedel, Joerg wrote:
> > Hrm, indeed I just noticed that. Pretty gross... it should definitly
> be
> > renamed, is will caused endless confusion with unrelated iommu.h and
> > iommu_* interfaces which represent something different.
>
> The first direction to go should be trying to unify all the different
> iommu* interfaces into the iommu-api. The generic api will definitly
> need to be extended for that, but since it is an in-kernel interface
> thats no problem.

Well, I suppose I'm the de-facto candidate to take care of the powerpc
side then :-)

I don't have the bandwidth right now, but I'll try to have a look when
time permits.

Cheers,
Ben.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/