kmemleak, cpu usage jump out of nowhere
in [Kernel]
|
From: Damien Wyart on 14 Jul 2010 04:50 Hi, > On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: > > I got a new Intel core-8 i7 processor. > > I am on kernel uname -a > > Linux zenogentoo 2.6.35-rc5 #97 SMP Tue Jul 13 16:13:25 CEST 2010 i686 > > Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux > > Sometimes in the middle of nowhere all of a sudden all of my 8-cores > > are at 100% CPU usage and my machine really lags and hangs and is not > > useable anymore. Some random process just grabs a bunch CPUs according > > to htop. * Pekka Enberg <penberg(a)cs.helsinki.fi> [2010-07-14 11:05]: > Why did you enable CONFIG_DEBUG_KMEMLEAK? Memory leak scanning is > likely the source of these pauses. I am seeing the same problem with a Core i7 920 and 2.6.35-rc5, and I do not have CONFIG_DEBUG_KMEMLEAK enabled, so I think this is not related. I do not see anything special in the logs, just the load becoming mad and almost preventing ssh access. I've been seeing that since the first 2.6.35 rc I tested (-rc2 or -rc3, I don't remember) and I did not have time to report it before but I was surprised nobody else did. No problem with 2.6.34 and 2.6.34.1. -- Damien -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zeno Davatz on 14 Jul 2010 05:00 On Wed, Jul 14, 2010 at 10:38 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: > Zeno Davatz wrote: >> >> On Wed, Jul 14, 2010 at 10:31 AM, Damien Wyart <damien.wyart(a)free.fr> >> wrote: >> >>>> On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: >>>>> >>>>> I got a new Intel core-8 i7 processor. >>>>> I am on kernel uname -a >>>>> Linux zenogentoo 2.6.35-rc5 #97 SMP Tue Jul 13 16:13:25 CEST 2010 i686 >>>>> Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux >>>>> Sometimes in the middle of nowhere all of a sudden all of my 8-cores >>>>> are at 100% CPU usage and my machine really lags and hangs and is not >>>>> useable anymore. Some random process just grabs a bunch CPUs according >>>>> to htop. >>> >>> * Pekka Enberg <penberg(a)cs.helsinki.fi> [2010-07-14 11:05]: >>>> >>>> Why did you enable CONFIG_DEBUG_KMEMLEAK? Memory leak scanning is >>>> likely the source of these pauses. >>> >>> I am seeing the same problem with a Core i7 920 and 2.6.35-rc5, and I do >>> not have CONFIG_DEBUG_KMEMLEAK enabled, so I think this is not related. >>> >>> I do not see anything special in the logs, just the load becoming mad >>> and almost preventing ssh access. I've been seeing that since the first >>> 2.6.35 rc I tested (-rc2 or -rc3, I don't remember) and I did not have >>> time to report it before but I was surprised nobody else did. No problem >>> with 2.6.34 and 2.6.34.1. >> >> same with me. My last build I tested was 2.6.34-rc7. No problems >> there. No CPU jumps out of nowhere. >> >> It is like any application all of a sudden use 400% CPU i.e. htop. > > Interesting. Lets CC some scheduler folks for help. Once it is gdm, once it is firefox-bin once it is htop. Its really od. Nothing crashes one of those just uses lots of CPU. The machine just gets really slow and then calms down again and everything is back to normal. ;) Maybe a bad temper? Best Zeno -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Pekka Enberg on 14 Jul 2010 05:00 On Wed, Jul 14, 2010 at 11:54 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: > On Wed, Jul 14, 2010 at 10:38 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: >> Zeno Davatz wrote: >>> >>> On Wed, Jul 14, 2010 at 10:31 AM, Damien Wyart <damien.wyart(a)free.fr> >>> wrote: >>> >>>>> On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: >>>>>> >>>>>> I got a new Intel core-8 i7 processor. >>>>>> I am on kernel uname -a >>>>>> Linux zenogentoo 2.6.35-rc5 #97 SMP Tue Jul 13 16:13:25 CEST 2010 i686 >>>>>> Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz GenuineIntel GNU/Linux >>>>>> Sometimes in the middle of nowhere all of a sudden all of my 8-cores >>>>>> are at 100% CPU usage and my machine really lags and hangs and is not >>>>>> useable anymore. Some random process just grabs a bunch CPUs according >>>>>> to htop. >>>> >>>> * Pekka Enberg <penberg(a)cs.helsinki.fi> [2010-07-14 11:05]: >>>>> >>>>> Why did you enable CONFIG_DEBUG_KMEMLEAK? Memory leak scanning is >>>>> likely the source of these pauses. >>>> >>>> I am seeing the same problem with a Core i7 920 and 2.6.35-rc5, and I do >>>> not have CONFIG_DEBUG_KMEMLEAK enabled, so I think this is not related. >>>> >>>> I do not see anything special in the logs, just the load becoming mad >>>> and almost preventing ssh access. I've been seeing that since the first >>>> 2.6.35 rc I tested (-rc2 or -rc3, I don't remember) and I did not have >>>> time to report it before but I was surprised nobody else did. No problem >>>> with 2.6.34 and 2.6.34.1. >>> >>> same with me. My last build I tested was 2.6.34-rc7. No problems >>> there. No CPU jumps out of nowhere. >>> >>> It is like any application all of a sudden use 400% CPU i.e. htop. >> >> Interesting. Lets CC some scheduler folks for help. > > Once it is gdm, once it is firefox-bin once it is htop. Its really od. > Nothing crashes one of those just uses lots of CPU. The machine just > gets really slow and then calms down again and everything is back to > normal. That's the part that makes me think it's scheduler and/or cpufreq related. > ;) > > Maybe a bad temper? Yeah, maybe Tux is having a bad day. :-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Catalin Marinas on 14 Jul 2010 06:00 On Wed, 2010-07-14 at 09:27 +0100, Zeno Davatz wrote: > On Wed, Jul 14, 2010 at 10:05 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: > > On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: > >> I am attaching you the file from /sys/kernel/debug/kmemleak > > > > Zeno, can you post your dmesg and .config, please? > > Sure, see attached files. It looks like NO_BOOTMEM is enabled. You can try the attached patch (I need to post it again on the list). kmemleak: Add support for NO_BOOTMEM configurations From: Catalin Marinas <catalin.marinas(a)arm.com> With commits 08677214 and 59be5a8e, alloc_bootmem()/free_bootmem() and friends use the early_res functions for memory management when NO_BOOTMEM is enabled. This patch adds the kmemleak calls in the corresponding code paths for bootmem allocations. Signed-off-by: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Yinghai Lu <yinghai(a)kernel.org> Cc: H. Peter Anvin <hpa(a)zytor.com> --- mm/bootmem.c | 2 ++ mm/page_alloc.c | 1 + 2 files changed, 3 insertions(+), 0 deletions(-) diff --git a/mm/bootmem.c b/mm/bootmem.c index 58c66cc..0747f68 100644 --- a/mm/bootmem.c +++ b/mm/bootmem.c @@ -435,6 +435,7 @@ void __init free_bootmem_node(pg_data_t *pgdat, unsigned long physaddr, { #ifdef CONFIG_NO_BOOTMEM free_early(physaddr, physaddr + size); + kmemleak_free_part(__va(physaddr), size); #else unsigned long start, end; @@ -460,6 +461,7 @@ void __init free_bootmem(unsigned long addr, unsigned long size) { #ifdef CONFIG_NO_BOOTMEM free_early(addr, addr + size); + kmemleak_free_part(__va(addr), size); #else unsigned long start, end; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 431214b..f29f00b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3659,6 +3659,7 @@ void * __init __alloc_memory_core_early(int nid, u64 size, u64 align, ptr = phys_to_virt(addr); memset(ptr, 0, size); reserve_early_without_check(addr, addr + size, "BOOTMEM"); + kmemleak_alloc(ptr, size, 1, 0); return ptr; } -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Pekka Enberg on 14 Jul 2010 06:00
Catalin Marinas wrote: > On Wed, 2010-07-14 at 09:27 +0100, Zeno Davatz wrote: >> On Wed, Jul 14, 2010 at 10:05 AM, Pekka Enberg <penberg(a)cs.helsinki.fi> wrote: >>> On Wed, Jul 14, 2010 at 9:12 AM, Zeno Davatz <zdavatz(a)gmail.com> wrote: > >>>> I am attaching you the file from /sys/kernel/debug/kmemleak >>> Zeno, can you post your dmesg and .config, please? >> Sure, see attached files. > > It looks like NO_BOOTMEM is enabled. You can try the attached patch (I > need to post it again on the list). > > > kmemleak: Add support for NO_BOOTMEM configurations > > From: Catalin Marinas <catalin.marinas(a)arm.com> > > With commits 08677214 and 59be5a8e, alloc_bootmem()/free_bootmem() and > friends use the early_res functions for memory management when > NO_BOOTMEM is enabled. This patch adds the kmemleak calls in the > corresponding code paths for bootmem allocations. > > Signed-off-by: Catalin Marinas <catalin.marinas(a)arm.com> > Cc: Yinghai Lu <yinghai(a)kernel.org> > Cc: H. Peter Anvin <hpa(a)zytor.com> Makes sense. Acked-by: Pekka Enberg <penberg(a)cs.helsinki.fi> Zeno, this should fix the kmemleak false positives but not the big pauses you're seeing. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |