From: H. Peter Anvin on
On 05/13/2010 01:36 PM, Yinghai Lu wrote:
>
> mmio range should be much better, it could use mmio64
>

We're not talking about MMIO.

-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Yinghai Lu on
On 05/13/2010 12:38 PM, Mike Habeck wrote:
>
>
> Nothing really breaks, it's more of a problem that the kernel uses
> up the rest of the I/O Space, and starts spitting out warning
> messages as it tries to assign the rest of the I/O BARs that the
> BIOS didn't assign, something like:
>
> pci 0010:03:00.0: BAR 5: can't allocate I/O resource [0x0-0x7f]
> pci 0012:05:00.0: BAR 5: can't allocate I/O resource [0x0-0x7f]
> ...
>
> And in using up all the I/O space, I think that could prevent a
> hotplug attach of a pci device requiring I/O space (although I
> believe most BIOSes pad the bridge decoders to support that).
> I'm not to familiar with how pci hotplug works on x86 so I may
> be wrong in what I just stated.

assume you have several IOHs on your system, and for hotplug support,
BIOS is supposed to balance the IO ioport range allocation
between IOHs.

mmio range should be much better, it could use mmio64

if your BIOS can not provide right _CRS for peer root buses, some
devices may get wrong allocation from kernel.
or BIOS set small BAR in the bridge, that could cause problem too.

YH


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: H. Peter Anvin on
On 05/14/2010 03:34 PM, Mike Travis wrote:
>
> It seems that BIOS changes are much more difficult. The real solution
> to this problem is for Card Vendors to not request I/O Bars if they
> won't be using them. But that's the hardest option of all to accomplish.
>

That is a non-option. Any device which may have to be a boot or console
device in a legacy system pretty much needs them.

-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Mike Travis on


Jesse Barnes wrote:
> On Thu, 13 May 2010 14:02:30 -0600
> Bjorn Helgaas <bjorn.helgaas(a)hp.com> wrote:
>>>> This issue is not specific to x86, so I don't really like having
>>>> the implementation be x86-specific.
>>> I agree this isn't a x86 specific issue but given the 'norom'
>>> cmdline option is basically doing the same thing (but for pci
>>> Expansion ROM BARs) this code was modeled after it.

>> IMHO, we should fix both.
>
> Yeah, that would be good. Mike, have you looked at this at all?
>
> Also, to clarify, this isn't affecting users today, right? Or do you
> need all this I/O space for multiple IOHs and the drivers that bind to
> them in current UV systems?

We have customers that want to install more than 16 PCI-e cards right
now. Our window of opportunity closes very soon (days), so either this
patch makes it in as is (or something close), or we wait for another
release cycle. UV shipments start this month.

[I wouldn't mind working on an improvement for later.]

>
> Fundamentally, until we have real dynamic PCI resource management (i.e.
> driver hooks for handling relocation, lazy allocation of resources at
> driver bind time, etc.) we're going to continue to need hacks like
> this. However, we could make them slightly more automated by making
> "nobar" and "norom" the default on systems that typically need them,
> maybe with a DMI table.

It seems that BIOS changes are much more difficult. The real solution
to this problem is for Card Vendors to not request I/O Bars if they
won't be using them. But that's the hardest option of all to accomplish.

Thanks,
Mike
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Mike Travis on


H. Peter Anvin wrote:
> On 05/14/2010 03:34 PM, Mike Travis wrote:
>> It seems that BIOS changes are much more difficult. The real solution
>> to this problem is for Card Vendors to not request I/O Bars if they
>> won't be using them. But that's the hardest option of all to accomplish.
>>
>
> That is a non-option. Any device which may have to be a boot or console
> device in a legacy system pretty much needs them.
>
> -hpa

Yes, that's true. We're somewhat fortunate in that our Legacy I/O
devices are confined to those on the first blade, and all of these
other devices are on other blades (PCI segments 1+).

But you're right, unless the card vendor supplied some kind of strapping
option, or something similar, it won't know it's in a UV system or not.

Thanks,
Mike
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/