genapic: optimize & fix APIC mode setup
in [Kernel]
|
From: Andi Kleen on 11 Nov 2006 10:30 On Saturday 11 November 2006 16:14, Ingo Molnar wrote: > From: Ingo Molnar <mingo(a)elte.hu> > > this patch fixes a couple of inconsistencies/problems i found while > reviewing the x86_64 genapic code (when i was chasing mysterious eth0 > timeouts that would only trigger if CPU_HOTPLUG is enabled): > > - AMD systems defaulted to the slower flat-physical mode instead > of the flat-logical mode. The only restriction on AMD systems > is that they should not use clustered APIC mode. This will open a race on CPU hotunplug unfortunately (common for multi core suspend) > > - removed the CPU hotplug hacks, switching the default for small > systems back from phys-flat to logical-flat. The switching to logical > flat mode on small systems fixed sporadic ethernet driver timeouts i > was getting on a dual-core Athlon64 system: That will break CPU hotplug on some Intel systems (Ashok can give details) That is caused ethernet timeouts is weird, probably needs to be root caused. > NETDEV WATCHDOG: eth0: transmit timed out > eth0: Transmit timeout, status 0c 0005 c07f media 80. > eth0: Tx queue start entry 32 dirty entry 28. > eth0: Tx descriptor 0 is 0008a04a. (queue head) > eth0: Tx descriptor 1 is 0008a04a. > eth0: Tx descriptor 2 is 0008a04a. > eth0: Tx descriptor 3 is 0008a04a. > eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1 > > - The use of '<= 8' was a bug by itself (the valid APIC ids > for logical flat mode go from 0 to 7, not 0 to 8). The new logic > is to use logical flat mode on both AMD and Intel systems, and > to only switch to physical mode when logical mode cannot be used. > If CPU hotplug is racy wrt. APIC shutdown then CPU hotplug needs > fixing, not the whole IRQ system be made inconsistent and slowed > down. Yes that needs to be fixed. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 11 Nov 2006 10:40 * Andi Kleen <ak(a)suse.de> wrote: > On Saturday 11 November 2006 16:14, Ingo Molnar wrote: > > From: Ingo Molnar <mingo(a)elte.hu> > > > > this patch fixes a couple of inconsistencies/problems i found while > > reviewing the x86_64 genapic code (when i was chasing mysterious eth0 > > timeouts that would only trigger if CPU_HOTPLUG is enabled): > > > > - AMD systems defaulted to the slower flat-physical mode instead > > of the flat-logical mode. The only restriction on AMD systems > > is that they should not use clustered APIC mode. > > This will open a race on CPU hotunplug unfortunately (common for multi > core suspend) Note that i386 still defaults to logical flat mode, so whatever hotplug CPU races there are, they need to be fixed! Given how rare CPU hotplug systems are i have no problem with having these races in the kernel for a while until it's fixed. Also, distro kernels enable CPU_HOTPLUG frequently. It is just ugly beyond recognition to switch the programming of the IRQ hardware on non-hotplug hardware just because a mostly-software feature (hotplug) is enabled ... if hotplug breaks suspend then fix it, dont hack it around (on one platform) by slowing down the system [and causing other problems] ... Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on 11 Nov 2006 10:50 * Andi Kleen <ak(a)suse.de> wrote: > This will open a race on CPU hotunplug unfortunately > (common for multi core suspend) how can i reproduce this btw, any instructions/pointers for that? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Siddha, Suresh B on 12 Nov 2006 21:20 On Sat, Nov 11, 2006 at 04:20:24PM +0100, Andi Kleen wrote: > On Saturday 11 November 2006 16:14, Ingo Molnar wrote: > > > > - removed the CPU hotplug hacks, switching the default for small > > systems back from phys-flat to logical-flat. The switching to logical > > flat mode on small systems fixed sporadic ethernet driver timeouts i > > was getting on a dual-core Athlon64 system: > > That will break CPU hotplug on some Intel systems (Ashok can give details) There is an issue of using clustered mode along with cpu hotplug. More details are at the below link. http://marc.theaimsgroup.com/?l=linux-kernel&m=113261865814107&w=2 thanks, suresh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Andi Kleen on 12 Nov 2006 21:40
On Monday 13 November 2006 02:50, Siddha, Suresh B wrote: > On Sat, Nov 11, 2006 at 04:20:24PM +0100, Andi Kleen wrote: > > On Saturday 11 November 2006 16:14, Ingo Molnar wrote: > > > > > > - removed the CPU hotplug hacks, switching the default for small > > > systems back from phys-flat to logical-flat. The switching to logical > > > flat mode on small systems fixed sporadic ethernet driver timeouts i > > > was getting on a dual-core Athlon64 system: > > > > That will break CPU hotplug on some Intel systems (Ashok can give details) > > There is an issue of using clustered mode along with cpu hotplug. More details > are at the below link. Thanks Suresh. > > http://marc.theaimsgroup.com/?l=linux-kernel&m=113261865814107&w=2 I should add i have no definite proof this is an issue on AMD systems too, but I changed it there too anyways to better be safe than sorry (and there is not very much performance difference anyways) Now if it causes device driver issues that's different of course. I wasn't aware of this before. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |