x86 pci: Add option to not assign BAR's if not already assigned
in [Kernel]
|
Prev: [PATCH 2/8] x86/mrst: add cpu type detection for Medfield
Next: virtio_blk: Use blk-iopoll for host->guest notify
From: Jesse Barnes on 14 May 2010 18:30 On Thu, 13 May 2010 14:02:30 -0600 Bjorn Helgaas <bjorn.helgaas(a)hp.com> wrote: > > > This issue is not specific to x86, so I don't really like having > > > the implementation be x86-specific. > > > > I agree this isn't a x86 specific issue but given the 'norom' > > cmdline option is basically doing the same thing (but for pci > > Expansion ROM BARs) this code was modeled after it. > > IMHO, we should fix both. Yeah, that would be good. Mike, have you looked at this at all? Also, to clarify, this isn't affecting users today, right? Or do you need all this I/O space for multiple IOHs and the drivers that bind to them in current UV systems? Fundamentally, until we have real dynamic PCI resource management (i.e. driver hooks for handling relocation, lazy allocation of resources at driver bind time, etc.) we're going to continue to need hacks like this. However, we could make them slightly more automated by making "nobar" and "norom" the default on systems that typically need them, maybe with a DMI table. Thanks, -- Jesse Barnes, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jesse Barnes on 14 May 2010 18:50 On Fri, 14 May 2010 15:34:01 -0700 Mike Travis <travis(a)sgi.com> wrote: > > > Jesse Barnes wrote: > > On Thu, 13 May 2010 14:02:30 -0600 > > Bjorn Helgaas <bjorn.helgaas(a)hp.com> wrote: > >>>> This issue is not specific to x86, so I don't really like having > >>>> the implementation be x86-specific. > >>> I agree this isn't a x86 specific issue but given the 'norom' > >>> cmdline option is basically doing the same thing (but for pci > >>> Expansion ROM BARs) this code was modeled after it. > > >> IMHO, we should fix both. > > > > Yeah, that would be good. Mike, have you looked at this at all? > > > > Also, to clarify, this isn't affecting users today, right? Or do you > > need all this I/O space for multiple IOHs and the drivers that bind to > > them in current UV systems? > > We have customers that want to install more than 16 PCI-e cards right > now. Our window of opportunity closes very soon (days), so either this > patch makes it in as is (or something close), or we wait for another > release cycle. UV shipments start this month. > > [I wouldn't mind working on an improvement for later.] Wow and they're using cards that want to use I/O space? Funky. It's too late to get this into 2.6.34, but that can't be what you were expecting... I don't see a problem with getting something like this in for 2.6.35. > > Fundamentally, until we have real dynamic PCI resource management (i.e. > > driver hooks for handling relocation, lazy allocation of resources at > > driver bind time, etc.) we're going to continue to need hacks like > > this. However, we could make them slightly more automated by making > > "nobar" and "norom" the default on systems that typically need them, > > maybe with a DMI table. > > It seems that BIOS changes are much more difficult. The real solution > to this problem is for Card Vendors to not request I/O Bars if they > won't be using them. But that's the hardest option of all to accomplish. Right. -- Jesse Barnes, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jesse Barnes on 14 May 2010 19:10 On Fri, 14 May 2010 15:59:11 -0700 Mike Travis <travis(a)sgi.com> wrote: > > > Jesse Barnes wrote: > > On Fri, 14 May 2010 15:34:01 -0700 > > Mike Travis <travis(a)sgi.com> wrote: > > > >> > >> Jesse Barnes wrote: > >>> On Thu, 13 May 2010 14:02:30 -0600 > >>> Bjorn Helgaas <bjorn.helgaas(a)hp.com> wrote: > >>>>>> This issue is not specific to x86, so I don't really like having > >>>>>> the implementation be x86-specific. > >>>>> I agree this isn't a x86 specific issue but given the 'norom' > >>>>> cmdline option is basically doing the same thing (but for pci > >>>>> Expansion ROM BARs) this code was modeled after it. > >>>> IMHO, we should fix both. > >>> Yeah, that would be good. Mike, have you looked at this at all? > >>> > >>> Also, to clarify, this isn't affecting users today, right? Or do you > >>> need all this I/O space for multiple IOHs and the drivers that bind to > >>> them in current UV systems? > >> We have customers that want to install more than 16 PCI-e cards right > >> now. Our window of opportunity closes very soon (days), so either this > >> patch makes it in as is (or something close), or we wait for another > >> release cycle. UV shipments start this month. > >> > >> [I wouldn't mind working on an improvement for later.] > > > > Wow and they're using cards that want to use I/O space? Funky. It's > > too late to get this into 2.6.34, but that can't be what you were > > expecting... I don't see a problem with getting something like this in > > for 2.6.35. > > 2.6.35 would be fine. It's the acceptance that's the key. > > And yes, we're using standard cards like everyone else... ;-) > > [The message is "UV" is just a really, really big PC. ;-)] > > I would appreciate however, some more detail on what's the goal of the > updates to "fix both". Thanks! As Bjorn noted, both the norom and nobar options are listed as x86 specific in the documentation and use x86 specific flags and code to prevent their respective allocations. It would be good if we could move the flags and code to the common PCI layer in drivers/pci so that other arches could take advantage of it, and we could have less resource management fragmentation. -- Jesse Barnes, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jesse Barnes on 14 May 2010 19:30 On Fri, 14 May 2010 16:20:45 -0700 "H. Peter Anvin" <hpa(a)zytor.com> wrote: > On 05/14/2010 03:47 PM, Jesse Barnes wrote: > > > > Wow and they're using cards that want to use I/O space? Funky. It's > > too late to get this into 2.6.34, but that can't be what you were > > expecting... I don't see a problem with getting something like this in > > for 2.6.35. > > > > Most cards on the market provide I/O BARs as a convenience to legacy > BIOSes; they don't need the I/O BAR functionality from inside a > full-featured OS. There are a few, key, exceptions, mainly in the form > of legacy-interface devices like UARTs and VGA (note that VGA has its > own routing bits and is therefore unaffected by this problem.) Yeah, it's the "legacy" part that I'm questioning. I'm just lamenting that it's dying off so slowly... And yes, VGA is an unfortunate standard. -- Jesse Barnes, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jesse Barnes on 14 May 2010 19:40
On Fri, 14 May 2010 16:23:52 -0700 Mike Travis <travis(a)sgi.com> wrote: > Thanks! I was hoping you weren't going to say we need to remove the > need for these options. No, I won't force that on you just yet. :) > I think the dynamic provisioning thing would be a great feature for the > future, but it seems to me that it would entail quite a bit of coordinated > effort between BIOS and the kernel? Not to mention the final death of > all POST related initialization (at least in the case where the device > is not being used in Legacy mode)? [Heck, can we kill POST too?!?] I > haven't looked too closely at non-x86 arch's but unless they run an > x86 emulator, then the POST initialization is pretty much superfluous, > isn't it? The drivers will initialize when the device comes online? > I mean we do have more the 64k of memory now... ;-)] Yeah, we can mostly ignore POST, though some BIOSes need it for their boot time services. Even complex graphics devices tend not to need it these days as long as you have a fully capable driver. As for BIOS coordination for dynamic reallocation, yeah there'd be some of that. I think the basic principles would be: 1) use BIOS allocations wherever possible 2) get an accurate list of available resources from the BIOS for potential remapping later 3) allocate resources for BARs and devices as late as possible (e.g. at driver bind time) to avoid allocating more than we need But that's a good chunk of work, and as we've seen, PCs in particular are really sensitive to having resources moved around too much, so step (2) is probably the hardest part. -- Jesse Barnes, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |