From: Borislav Petkov on
From: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sat, Apr 10, 2010 at 01:40:39PM -0700

> Yes. So the patches I actually think are important are:
>
> - the RCU fix is real, although admittedly the race window is probably
> too small to ever really hit.
>
> - the simplification rule to find_mergeable_anon_vma's is required,
> because otherwise our anon_vma_merge() will do the wrong thing (maybe
> Johannes' patch would be an alternative, but quite frankly, I think we
> want the simpler code, and I don't think we even _want_ to share
> anon_vma's that are complex due to forking)
>
> I like my "cleanup" version (the bigger one with lots of comments) more
> than the two-liner version, but they should be equivalent.
>
> - the vma_adjust() fix is the one that I think may actually end up fixing
> your problems for good. Knock wood.
>
> So I think they are all required, but I suspect that the vma_adjust() one
> is finally the most direct explanation of the problem you've seen.

Damn, nope, still no joy :(. It looked like it was fixed but one of the
test was to hibernate right after the 3 kvm guests were shut down and I
guess the mem freeing pattern kinda hits it where it most hurts.

Anyways, I'm going to bed soon, will test whatever you come up with guys
tomorrow morning when I can think again.

By the way, do we want to create a new thread - the mailchain is off the
screen limits of my netbook :)

Thanks.

p.s. Oopsie:


[ 647.288638] PM: Syncing filesystems ... done.
[ 647.307459] Freezing user space processes ... (elapsed 0.01 seconds) done.
[ 647.320981] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
[ 647.334152] PM: Preallocating image memory...
[ 647.492781] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 647.493001] IP: [<ffffffff810c60a0>] page_referenced+0xee/0x1dc
[ 647.493001] PGD 22a1d1067 PUD 1cb6a9067 PMD 0
[ 647.493001] Oops: 0000 [#1] PREEMPT SMP
[ 647.493001] last sysfs file: /sys/power/state
[ 647.493001] CPU 0
[ 647.493001] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod 8250_pnp ohci_hcd 8250 serial_core pcspkr k10temp edac_core
[ 647.493001]
[ 647.493001] Pid: 3231, comm: hib.sh Not tainted 2.6.34-rc3-00503-g8b3334b #6 M3A78 PRO/System Product Name
[ 647.493001] RIP: 0010:[<ffffffff810c60a0>] [<ffffffff810c60a0>] page_referenced+0xee/0x1dc
[ 647.493001] RSP: 0018:ffff880223b6f8b8 EFLAGS: 00010283
[ 647.493001] RAX: ffff88022aa316c8 RBX: ffffea0006882fc0 RCX: 0000000000000000
[ 647.493001] RDX: ffff880223b6fcf8 RSI: ffff88022aa316a0 RDI: ffff88022de6de60
[ 647.493001] RBP: ffff880223b6f938 R08: 0000000000000002 R09: 0000000000000000
[ 647.493001] R10: ffff880228cb03a8 R11: ffffffff00000012 R12: 0000000000000000
[ 647.493001] R13: ffffffffffffffe0 R14: ffff88022aa31688 R15: ffff880223b6fa00
[ 647.493001] FS: 00007f0eea2086f0(0000) GS:ffff88000a000000(0000) knlGS:0000000000000000
[ 647.493001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 647.493001] CR2: 0000000000000000 CR3: 0000000223df5000 CR4: 00000000000006f0
[ 647.493001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 647.493001] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 647.493001] Process hib.sh (pid: 3231, threadinfo ffff880223b6e000, task ffff88022de6de60)
[ 647.493001] Stack:
[ 647.493001] ffff88022aa316c8 00000000810c5dbf ffff880223b6f918 ffffffff810c5f28
[ 647.493001] <0> ffff880223b6f8f8 ffffffff00000001 ffffea0006867570 ffffea0006889070
[ 647.493001] <0> ffffea0006889070 0000000223b6fcf8 ffffea0006889070 ffffea0006882fe8
[ 647.493001] Call Trace:
[ 647.493001] [<ffffffff810c5f28>] ? try_to_unmap_anon+0xa2/0xb4
[ 647.493001] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 647.493001] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 647.493001] [<ffffffff810b1155>] ? shrink_zone+0x11a/0x3d6
[ 647.493001] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 647.493001] [<ffffffff8140f000>] ? _raw_spin_lock_irq+0x19/0x79
[ 647.493001] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 647.493001] [<ffffffff810b155b>] ? shrink_slab+0x14a/0x15c
[ 647.493001] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 647.493001] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 647.493001] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 647.493001] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 647.493001] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 647.493001] [<ffffffff8140bdd4>] ? printk+0x41/0x45
[ 647.493001] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 647.493001] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 647.493001] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 647.493001] [<ffffffff8118f5d7>] kobj_attr_store+0x17/0x19
[ 647.493001] [<ffffffff8112e490>] sysfs_write_file+0x108/0x144
[ 647.493001] [<ffffffff810db69f>] vfs_write+0xb2/0x153
[ 647.493001] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 647.493001] [<ffffffff810db803>] sys_write+0x4a/0x71
[ 647.493001] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 647.493001] Code: 3b 56 10 73 1e 48 83 fa f2 74 18 48 8d 4d cc 4d 89 f8 48 89 df e8 11 f2 ff ff 41 01 c4 83 7d cc 00 74 19 4d 8b 6d 20 49 83 ed 20 <49> 8b 45 20 0f 18 08 49 8d 45 20 48 39 45 80 75 aa 4c 89 f7 e8
[ 647.493001] RIP [<ffffffff810c60a0>] page_referenced+0xee/0x1dc
[ 647.493001] RSP <ffff880223b6f8b8>
[ 647.493001] CR2: 0000000000000000
[ 647.508991] ---[ end trace 91f57fb5ef398fd2 ]---
[ 647.509150] note: hib.sh[3231] exited with preempt_count 2
[ 647.509311] BUG: scheduling while atomic: hib.sh/3231/0x10000003
[ 647.509462] INFO: lockdep is turned off.
[ 647.509610] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod 8250_pnp ohci_hcd 8250 serial_core pcspkr k10temp edac_core
[ 647.511093] Pid: 3231, comm: hib.sh Tainted: G D 2.6.34-rc3-00503-g8b3334b #6
[ 647.511353] Call Trace:
[ 647.511504] [<ffffffff810658df>] ? __debug_show_held_locks+0x1b/0x24
[ 647.511658] [<ffffffff8102dfac>] __schedule_bug+0x72/0x77
[ 647.511811] [<ffffffff8140c1e8>] schedule+0xe3/0x7ff
[ 647.511962] [<ffffffff810bd0e4>] ? unmap_vmas+0x90c/0x911
[ 647.512191] [<ffffffff81030ecb>] __cond_resched+0x18/0x24
[ 647.512337] [<ffffffff8140c9d1>] _cond_resched+0x2c/0x37
[ 647.512550] [<ffffffff810bcef1>] unmap_vmas+0x719/0x911
[ 647.512697] [<ffffffff810c1781>] exit_mmap+0x102/0x1e4
[ 647.512911] [<ffffffff810c16e8>] ? exit_mmap+0x69/0x1e4
[ 647.513082] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 647.513233] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 647.513387] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 647.513538] [<ffffffff81039e2f>] ? kmsg_dump+0x13b/0x155
[ 647.513690] [<ffffffff8100616b>] ? oops_end+0x47/0x93
[ 647.513859] [<ffffffff810061b2>] oops_end+0x8e/0x93
[ 647.514009] [<ffffffff8101f3e5>] no_context+0x1fc/0x20b
[ 647.514172] [<ffffffff8118b72b>] ? cfq_insert_request+0x7a/0x3b1
[ 647.514321] [<ffffffff8101f580>] __bad_area_nosemaphore+0x18c/0x1af
[ 647.514473] [<ffffffff8101f7bb>] ? do_page_fault+0xa8/0x32d
[ 647.514625] [<ffffffff8101f5b6>] bad_area_nosemaphore+0x13/0x15
[ 647.514777] [<ffffffff8101f886>] do_page_fault+0x173/0x32d
[ 647.514929] [<ffffffff814103a3>] ? error_sti+0x5/0x6
[ 647.515084] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 647.515242] [<ffffffff8140ecfb>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 647.515397] [<ffffffff814101bf>] page_fault+0x1f/0x30
[ 647.515549] [<ffffffff810c60a0>] ? page_referenced+0xee/0x1dc
[ 647.515701] [<ffffffff810c6032>] ? page_referenced+0x80/0x1dc
[ 647.515853] [<ffffffff810c5f28>] ? try_to_unmap_anon+0xa2/0xb4
[ 647.516010] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 647.516167] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 647.516323] [<ffffffff810b1155>] ? shrink_zone+0x11a/0x3d6
[ 647.516474] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 647.516627] [<ffffffff8140f000>] ? _raw_spin_lock_irq+0x19/0x79
[ 647.516780] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 647.516931] [<ffffffff810b155b>] ? shrink_slab+0x14a/0x15c
[ 647.517086] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 647.517243] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 647.517398] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 647.517551] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 647.517703] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 647.517856] [<ffffffff8140bdd4>] ? printk+0x41/0x45
[ 647.518011] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 647.518168] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 647.518322] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 647.518473] [<ffffffff8118f5d7>] kobj_attr_store+0x17/0x19
[ 647.518625] [<ffffffff8112e490>] sysfs_write_file+0x108/0x144
[ 647.518777] [<ffffffff810db69f>] vfs_write+0xb2/0x153
[ 647.518928] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 647.519084] [<ffffffff810db803>] sys_write+0x4a/0x71
[ 647.519240] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 699.648857] SysRq : HELP : loglevel(0-9) reBoot Crash show-all-locks(D) terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)
[ 700.234923] SysRq : Emergency Sync
[ 700.235341] Emergency Sync complete
[ 700.982072] SysRq : Emergency Remount R/O
[ 701.600802] SysRq : Resetting

--
Regards/Gruss,
Boris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Borislav Petkov on
From: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sat, Apr 10, 2010 at 02:30:49PM -0700

> On Sat, 10 Apr 2010, Borislav Petkov wrote:
> >
> > Damn, nope, still no joy :(. It looked like it was fixed but one of the
> > test was to hibernate right after the 3 kvm guests were shut down and I
> > guess the mem freeing pattern kinda hits it where it most hurts.
>
> Damn, I really hoped that was it. Three independent bugs found and fixed,
> and still no joy? Oh well.

Yep, I'll redo the testing tomorrow, so that we are sure that even with
the _three_ bugs fixed we still hit the funky list element issue.

> > By the way, do we want to create a new thread - the mailchain is off the
> > screen limits of my netbook :)
>
> I prefer to keep it in one thread so that they all show up together if I
> need to, but feel free to start a new one. Not a biggie.

I'll keep the thread then - I didn't know it mattered. Mine was just a
suggestion, nevermind.

> > [ 647.492781] BUG: unable to handle kernel NULL pointer dereference at (null)
> > [ 647.493001] IP: [<ffffffff810c60a0>] page_referenced+0xee/0x1dc
>
> Well, it sure is consistent. I'll start to think about what else could go
> wrong..

Which could mean that even with those issues fixed, the real issue is
yet something else. Because obviously the fixes you throw at it don't
seem to change it - even the traces remain consistent across tests.
And if it is use-after-free case, the funny patterns could be some
shifted SLUB poison values which we happen to "see" through the dangling
pointer... I dunno.

Hmm.

--
Regards/Gruss,
Boris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Borislav Petkov on
From: Borislav Petkov <bp(a)alien8.de>
Date: Sat, Apr 10, 2010 at 11:51:15PM +0200

> > Damn, I really hoped that was it. Three independent bugs found and fixed,
> > and still no joy? Oh well.
>
> Yep, I'll redo the testing tomorrow, so that we are sure that even with
> the _three_ bugs fixed we still hit the funky list element issue.

Ok, I could verify that the three patches we were talking about still
can't fix the issue. However, just to make sure I'm sending the versions
of the patches I used for you guys to check.

[ 529.667108] PM: Preallocating image memory...
[ 529.930881] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 529.931275] IP: [<ffffffff810c603c>] page_referenced+0xee/0x1dc
[ 529.931377] PGD 22e33d067 PUD 22ddc1067 PMD 0
[ 529.931377] Oops: 0000 [#1] PREEMPT SMP
[ 529.931377] last sysfs file: /sys/power/state
[ 529.931377] CPU 3
[ 529.931377] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod 8250_pnp 8250 ohci_hcd edac_core serial_core pcspkr k10temp
[ 529.931377]
[ 529.931377] Pid: 3354, comm: hib.sh Tainted: G W 2.6.34-rc3-00503-g0fcc334 #1 M3A78 PRO/System Product Name
[ 529.931377] RIP: 0010:[<ffffffff810c603c>] [<ffffffff810c603c>] page_referenced+0xee/0x1dc
[ 529.931377] RSP: 0018:ffff880105a118b8 EFLAGS: 00010283
[ 529.931377] RAX: ffff88022dc896c8 RBX: ffffea0007a15e10 RCX: 0000000000000000
[ 529.931377] RDX: ffff880105a11cf8 RSI: ffff88022dc896a0 RDI: ffff88022b760000
[ 529.931377] RBP: ffff880105a11938 R08: 0000000000000002 R09: 0000000000000000
[ 529.931377] R10: 0000000000000000 R11: ffffffff00000012 R12: 0000000000000000
[ 529.931377] R13: ffffffffffffffe0 R14: ffff88022dc89688 R15: ffff880105a11a00
[ 529.931377] FS: 00007f21045876f0(0000) GS:ffff88000a600000(0000) knlGS:0000000000000000
[ 529.931377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 529.931377] CR2: 0000000000000000 CR3: 000000022b33f000 CR4: 00000000000006e0
[ 529.931377] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 529.931377] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 529.931377] Process hib.sh (pid: 3354, threadinfo ffff880105a10000, task ffff88022b760000)
[ 529.931377] Stack:
[ 529.931377] ffff88022dc896c8 00000000810b0082 0000000000000000 0000000000000000
[ 529.931377] <0> 0000000000000000 0000000000000000 0000000000000000 0000000000000020
[ 529.931377] <0> 0000000000000000 0000000200000000 7fffffffffffffff ffffea0007a15e38
[ 529.931377] Call Trace:
[ 529.931377] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 529.931377] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 529.931377] [<ffffffff810af59c>] ? isolate_pages_global+0xd0/0x1fc
[ 529.931377] [<ffffffff8140f9f6>] ? _raw_spin_unlock_irq+0x30/0x58
[ 529.931377] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 529.931377] [<ffffffff810b0556>] ? shrink_active_list+0x232/0x244
[ 529.931377] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 529.931377] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 529.931377] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 529.931377] [<ffffffff81078e1e>] ? memory_bm_test_bit+0x1/0x30
[ 529.931377] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 529.931377] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 529.931377] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 529.931377] [<ffffffff8140bd74>] ? printk+0x41/0x45
[ 529.931377] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 529.931377] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 529.931377] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 529.931377] [<ffffffff8118f573>] kobj_attr_store+0x17/0x19
[ 529.931377] [<ffffffff8112e42c>] sysfs_write_file+0x108/0x144
[ 529.931377] [<ffffffff810db63b>] vfs_write+0xb2/0x153
[ 529.931377] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 529.931377] [<ffffffff810db79f>] sys_write+0x4a/0x71
[ 529.931377] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 529.931377] Code: 3b 56 10 73 1e 48 83 fa f2 74 18 48 8d 4d cc 4d 89 f8 48 89 df e8 11 f2 ff ff 41 01 c4 83 7d cc 00 74 19 4d 8b 6d 20 49 83 ed 20 <49> 8b 45 20 0f 18 08 49 8d 45 20 48 39 45 80 75 aa 4c 89 f7 e8
[ 529.931377] RIP [<ffffffff810c603c>] page_referenced+0xee/0x1dc
[ 529.931377] RSP <ffff880105a118b8>
[ 529.931377] CR2: 0000000000000000
[ 529.945250] ---[ end trace caa5471c993e6461 ]---
[ 529.945558] note: hib.sh[3354] exited with preempt_count 2
[ 529.945710] BUG: scheduling while atomic: hib.sh/3354/0x10000003
[ 529.945858] INFO: lockdep is turned off.
[ 529.946005] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod 8250_pnp 8250 ohci_hcd edac_core serial_core pcspkr k10temp
[ 529.947595] Pid: 3354, comm: hib.sh Tainted: G D W 2.6.34-rc3-00503-g0fcc334 #1
[ 529.947848] Call Trace:
[ 529.947993] [<ffffffff810658df>] ? __debug_show_held_locks+0x1b/0x24
[ 529.948147] [<ffffffff8102dfac>] __schedule_bug+0x72/0x77
[ 529.948296] [<ffffffff8140c188>] schedule+0xe3/0x7ff
[ 529.948449] [<ffffffff810bd0e4>] ? unmap_vmas+0x90c/0x911
[ 529.948599] [<ffffffff81030ecb>] __cond_resched+0x18/0x24
[ 529.948748] [<ffffffff8140c971>] _cond_resched+0x2c/0x37
[ 529.948896] [<ffffffff810bcef1>] unmap_vmas+0x719/0x911
[ 529.949049] [<ffffffff8140f01e>] ? _raw_spin_lock_irqsave+0x1e/0x85
[ 529.949199] [<ffffffff8105a878>] ? up+0x14/0x3e
[ 529.949347] [<ffffffff810c171f>] exit_mmap+0x102/0x1e4
[ 529.949639] [<ffffffff810c1686>] ? exit_mmap+0x69/0x1e4
[ 529.949787] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 529.949935] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 529.950087] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 529.950236] [<ffffffff81039e2f>] ? kmsg_dump+0x13b/0x155
[ 529.950525] [<ffffffff8100616b>] ? oops_end+0x47/0x93
[ 529.950671] [<ffffffff810061b2>] oops_end+0x8e/0x93
[ 529.950819] [<ffffffff8101f3e5>] no_context+0x1fc/0x20b
[ 529.950967] [<ffffffff8101f580>] __bad_area_nosemaphore+0x18c/0x1af
[ 529.951120] [<ffffffff8101f7bb>] ? do_page_fault+0xa8/0x32d
[ 529.951276] [<ffffffff8101f5b6>] bad_area_nosemaphore+0x13/0x15
[ 529.951572] [<ffffffff8101f886>] do_page_fault+0x173/0x32d
[ 529.951719] [<ffffffff81410363>] ? error_sti+0x5/0x6
[ 529.951867] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 529.952018] [<ffffffff8140ec9b>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 529.952170] [<ffffffff8141017f>] page_fault+0x1f/0x30
[ 529.952319] [<ffffffff810c603c>] ? page_referenced+0xee/0x1dc
[ 529.952615] [<ffffffff810c5fce>] ? page_referenced+0x80/0x1dc
[ 529.952762] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 529.952911] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 529.953065] [<ffffffff810af59c>] ? isolate_pages_global+0xd0/0x1fc
[ 529.953214] [<ffffffff8140f9f6>] ? _raw_spin_unlock_irq+0x30/0x58
[ 529.953363] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 529.953627] [<ffffffff810b0556>] ? shrink_active_list+0x232/0x244
[ 529.953775] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 529.953924] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 529.954077] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 529.954226] [<ffffffff81078e1e>] ? memory_bm_test_bit+0x1/0x30
[ 529.954486] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 529.954632] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 529.954782] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 529.954931] [<ffffffff8140bd74>] ? printk+0x41/0x45
[ 529.955083] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 529.955233] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 529.955457] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 529.955604] [<ffffffff8118f573>] kobj_attr_store+0x17/0x19
[ 529.955752] [<ffffffff8112e42c>] sysfs_write_file+0x108/0x144
[ 529.955900] [<ffffffff810db63b>] vfs_write+0xb2/0x153
[ 529.956053] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 529.956202] [<ffffffff810db79f>] sys_write+0x4a/0x71
[ 529.956351] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 537.634362] SysRq : HELP : loglevel(0-9) reBoot Crash show-all-locks(D) terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)
[ 538.129750] SysRq : Emergency Sync
[ 538.130161] Emergency Sync complete
[ 538.902386] SysRq : Emergency Remount R/O
[ 539.328830] SysRq : Resetting

--
Regards/Gruss,
Boris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Borislav Petkov on
From: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sun, Apr 11, 2010 at 10:16:10AM -0700

> Conversely, if you still see the oops (rather than the watchdog), that
> means that we actually have pages that are still marked mapped, and that
> despite that mapped state have a stale page->mapping pointer. I actually
> find that the more likely case, because otherwise the window is _so_ small
> that I don't see how you can hit the oops so reliably.

Ok, did test with the all 5 patches applied. It oopsed with the same
trace, see below. Except one kernel/sched.c:3555 warning checking
spinlock count overflowing, nothing else. :(

I tried to see whether the page->mapping pointer is stale, I dunno,
maybe there could be something in the register dump which could tell us
what's happening. This is how I see it, I could very well be wrong and
missing something though:


So, yes, we oops at the same place, however, a bit early we do

anon_vma = page_lock_anon_vma(page);
if (!anon_vma)
return referenced;

which compiles here to

.loc 1 496 0
movq %rbx, %rdi # page,
call page_lock_anon_vma #
..LVL288:
.loc 1 497 0
testq %rax, %rax # anon_vma
..LVL289:
.loc 1 496 0
movq %rax, %r14 #, anon_vma

and I checked that on the path before the instruction where we oops we
don't touch %r14 so the value in the register dump below should be that
anon_vma. Which looks like valid kernel pointer. We dereference it later
to get anon_vma->head.next with

.loc 1 501 0
movq 64(%r14), %r13 # <variable>.head.next, <variable>.head.next
..LBE1287:
leaq 64(%r14), %rax #,
movq %rax, -128(%rbp) #, %sfp
..LBB1288:
subq $32, %r13 #, avc

which ends up in %r13 as ffffffffffffffe0.

So, it really looks like at least that list_head in anon_vma is
bollocks, or even the whole anon_vma. So if this is correct, it is
highly likely that the anon_vma is already freed material or not
initialized at all.

Hm...


[ 616.317201] Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
[ 616.329964] PM: Preallocating image memory...
[ 616.586463] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 616.586851] IP: [<ffffffff810c614f>] page_referenced+0xee/0x1dc
[ 616.587045] PGD 225dcf067 PUD 22627f067 PMD 0
[ 616.587126] Oops: 0000 [#1] PREEMPT SMP
[ 616.587126] last sysfs file: /sys/power/state
[ 616.587126] CPU 1
[ 616.587126] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod ohci_hcd edac_core 8250_pnp 8250 serial_core pcspkr k10temp
[ 616.587126]
[ 616.587126] Pid: 3453, comm: hib.sh Tainted: G W 2.6.34-rc3-00505-g1d9bb34 #1 M3A78 PRO/System Product Name
[ 616.587126] RIP: 0010:[<ffffffff810c614f>] [<ffffffff810c614f>] page_referenced+0xee/0x1dc
[ 616.587126] RSP: 0018:ffff88022b3258b8 EFLAGS: 00010283
[ 616.587126] RAX: ffff880200ba4b88 RBX: ffffea00076b2b30 RCX: ffff88022eacaa58
[ 616.587126] RDX: ffffffff810c5e7a RSI: ffff880200ba4b60 RDI: ffff88022fa492e0
[ 616.587126] RBP: ffff88022b325938 R08: 0000000000000002 R09: 0000000000000000
[ 616.587126] R10: ffff88022eacaa30 R11: 0000000000000001 R12: 0000000000000000
[ 616.587126] R13: ffffffffffffffe0 R14: ffff880200ba4b48 R15: ffff88022b325a00
[ 616.587126] FS: 00007f0b140306f0(0000) GS:ffff88000a200000(0000) knlGS:0000000000000000
[ 616.587126] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 616.587126] CR2: 0000000000000000 CR3: 000000022c44f000 CR4: 00000000000006e0
[ 616.587126] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 616.587126] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 616.587126] Process hib.sh (pid: 3453, threadinfo ffff88022b324000, task ffff88022fa492e0)
[ 616.587126] Stack:
[ 616.587126] ffff880200ba4b88 00000000810c5e5f ffff88022b325918 ffffffff810c5fd7
[ 616.587126] <0> ffff880200000000 ffffffff00000001 ffff88022b325fd8 ffffea00076c1a80
[ 616.587126] <0> ffffea00076c1a80 000000022b325cf8 ffffea00076c1a80 ffffea00076b2b58
[ 616.587126] Call Trace:
[ 616.587126] [<ffffffff810c5fd7>] ? try_to_unmap_anon+0xa2/0xb4
[ 616.587126] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 616.587126] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 616.587126] [<ffffffff810af59c>] ? isolate_pages_global+0xd0/0x1fc
[ 616.587126] [<ffffffff8140fb06>] ? _raw_spin_unlock_irq+0x30/0x58
[ 616.587126] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 616.587126] [<ffffffff810b0556>] ? shrink_active_list+0x232/0x244
[ 616.587126] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 616.587126] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 616.587126] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 616.587126] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 616.587126] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 616.587126] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 616.587126] [<ffffffff8140be84>] ? printk+0x41/0x45
[ 616.587126] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 616.587126] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 616.587126] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 616.587126] [<ffffffff8118f687>] kobj_attr_store+0x17/0x19
[ 616.587126] [<ffffffff8112e540>] sysfs_write_file+0x108/0x144
[ 616.587126] [<ffffffff810db74f>] vfs_write+0xb2/0x153
[ 616.587126] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 616.587126] [<ffffffff810db8b3>] sys_write+0x4a/0x71
[ 616.587126] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 616.587126] Code: 3b 56 10 73 1e 48 83 fa f2 74 18 48 8d 4d cc 4d 89 f8 48 89 df e8 02 f2 ff ff 41 01 c4 83 7d cc 00 74 19 4d 8b 6d 20 49 83 ed 20 <49> 8b 45 20 0f 18 08 49 8d 45 20 48 39 45 80 75 aa 4c 89 f7 e8
[ 616.587126] RIP [<ffffffff810c614f>] page_referenced+0xee/0x1dc
[ 616.587126] RSP <ffff88022b3258b8>
[ 616.587126] CR2: 0000000000000000
[ 616.600838] ---[ end trace 0ea0c6b4ead21c8f ]---
[ 616.600984] note: hib.sh[3453] exited with preempt_count 2
[ 616.601282] BUG: scheduling while atomic: hib.sh/3453/0x10000003
[ 616.601431] INFO: lockdep is turned off.
[ 616.601584] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod ohci_hcd edac_core 8250_pnp 8250 serial_core pcspkr k10temp
[ 616.603115] Pid: 3453, comm: hib.sh Tainted: G D W 2.6.34-rc3-00505-g1d9bb34 #1
[ 616.603460] Call Trace:
[ 616.603605] [<ffffffff810658df>] ? __debug_show_held_locks+0x1b/0x24
[ 616.603755] [<ffffffff8102dfac>] __schedule_bug+0x72/0x77
[ 616.603903] [<ffffffff8140c298>] schedule+0xe3/0x7ff
[ 616.604051] [<ffffffff810bd0e4>] ? unmap_vmas+0x90c/0x911
[ 616.604230] [<ffffffff81030ecb>] __cond_resched+0x18/0x24
[ 616.604381] [<ffffffff8140ca81>] _cond_resched+0x2c/0x37
[ 616.604529] [<ffffffff810bcef1>] unmap_vmas+0x719/0x911
[ 616.604678] [<ffffffff810c16c0>] exit_mmap+0x102/0x1e4
[ 616.604826] [<ffffffff810c1627>] ? exit_mmap+0x69/0x1e4
[ 616.604975] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 616.605124] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 616.605280] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 616.605430] [<ffffffff81039e2f>] ? kmsg_dump+0x13b/0x155
[ 616.605579] [<ffffffff8100616b>] ? oops_end+0x47/0x93
[ 616.605727] [<ffffffff810061b2>] oops_end+0x8e/0x93
[ 616.605875] [<ffffffff8101f3e5>] no_context+0x1fc/0x20b
[ 616.606023] [<ffffffff8101f580>] __bad_area_nosemaphore+0x18c/0x1af
[ 616.606176] [<ffffffff8101f7bb>] ? do_page_fault+0xa8/0x32d
[ 616.606330] [<ffffffff8101f5b6>] bad_area_nosemaphore+0x13/0x15
[ 616.606479] [<ffffffff8101f886>] do_page_fault+0x173/0x32d
[ 616.606628] [<ffffffff81410463>] ? error_sti+0x5/0x6
[ 616.606776] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 616.606926] [<ffffffff8140edab>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 616.607076] [<ffffffff8141027f>] page_fault+0x1f/0x30
[ 616.607227] [<ffffffff810c5e7a>] ? page_lock_anon_vma+0x0/0xbb
[ 616.607381] [<ffffffff810c614f>] ? page_referenced+0xee/0x1dc
[ 616.607530] [<ffffffff810c60e1>] ? page_referenced+0x80/0x1dc
[ 616.607678] [<ffffffff810c5fd7>] ? try_to_unmap_anon+0xa2/0xb4
[ 616.607827] [<ffffffff810b06bc>] shrink_page_list+0x154/0x4c7
[ 616.607976] [<ffffffff81067149>] ? print_lock_contention_bug+0x1b/0xe1
[ 616.608131] [<ffffffff810af59c>] ? isolate_pages_global+0xd0/0x1fc
[ 616.608284] [<ffffffff8140fb06>] ? _raw_spin_unlock_irq+0x30/0x58
[ 616.608435] [<ffffffff810b0d8a>] shrink_inactive_list+0x35b/0x60c
[ 616.608585] [<ffffffff810b0556>] ? shrink_active_list+0x232/0x244
[ 616.608734] [<ffffffff810b1347>] shrink_zone+0x30c/0x3d6
[ 616.608883] [<ffffffff810b1f3d>] do_try_to_free_pages+0x191/0x29a
[ 616.609031] [<ffffffff810b20db>] shrink_all_memory+0x95/0xc4
[ 616.609183] [<ffffffff810af4cc>] ? isolate_pages_global+0x0/0x1fc
[ 616.609337] [<ffffffff81079c9c>] ? count_data_pages+0x65/0x79
[ 616.609486] [<ffffffff81079f03>] hibernate_preallocate_memory+0x1aa/0x2cb
[ 616.609636] [<ffffffff8140be84>] ? printk+0x41/0x45
[ 616.609784] [<ffffffff8107878f>] hibernation_snapshot+0x36/0x1e1
[ 616.609933] [<ffffffff81078a08>] hibernate+0xce/0x172
[ 616.610080] [<ffffffff81077775>] state_store+0x5c/0xd3
[ 616.610233] [<ffffffff8118f687>] kobj_attr_store+0x17/0x19
[ 616.610383] [<ffffffff8112e540>] sysfs_write_file+0x108/0x144
[ 616.610532] [<ffffffff810db74f>] vfs_write+0xb2/0x153
[ 616.610680] [<ffffffff810663c9>] ? trace_hardirqs_on_caller+0x1f/0x14b
[ 616.610830] [<ffffffff810db8b3>] sys_write+0x4a/0x71
[ 616.610978] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 682.501863] SysRq : HELP : loglevel(0-9) reBoot Crash show-all-locks(D) terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)
[ 683.552767] SysRq : Emergency Sync
[ 683.553147] Emergency Sync complete
[ 684.180708] SysRq : Emergency Remount R/O
[ 684.927560] SysRq : Resetting

--
Regards/Gruss,
Boris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Borislav Petkov on
From: Linus Torvalds <torvalds(a)linux-foundation.org>
Date: Sun, Apr 11, 2010 at 06:04:39PM -0700

> It checks each anonymous page at unmap time against the vma it gets
> unmapped from. It depends on the previous vma_verify debugging patch, and
> it would be interesting to hear whether this patch causes any new warnngs
> for you..
>
> If the warnings do happen, they are not going to be printing out any
> hugely informative data apart from the fact that the bad case happened at
> all. But If they do trigger, I can try to improve on them - it's just not
> worth trying to make them any more interesting if they never trigger.

Haa, I think you're gonna want to improve them :)

WARN_ONCE(1, "page->mapping does not exist in vma chain");

triggered on the first resume showing a rather messy 4 WARN_ONCEs. Had I
more cores, there maybe would've been more of them :) Maybe need locking
if clean output is of interest (see below).

So, anyway, if I can read this correctly, there is a page->mapping
anon_vma which is _not_ in the anon_vmas chain of the vma
(avc->same_vma).

And the spot we oops on is in page_referenced_anon():

list_for_each_entry(avc, &anon_vma->head, same_anon_vma) {

which is actually where we iterate over all vmas associated with this
anon_vma.

So if that previous anon_vma pointed to by the page_mapping has been
falsely unlinked at some point, no wonder we boom on that later.

By the way, I completely understand when you say that your head hurts
from looking at this :).


[ 486.580872] Restarting tasks ... done.
[ 494.167242] [drm] Resetting GPU
[ 495.422354] ------------[ cut here ]------------
[ 495.422407] WARNING: at mm/memory.c:909 unmap_vmas+0x548/0xa29()
[ 495.422442] Hardware name: System Product Name
[ 495.422474] page->mapping does not exist in vma chain
[ 495.422504] Modules linked in:
[ 495.422545] ------------[ cut here ]------------
[ 495.422555] ------------[ cut here ]------------
[ 495.422565] powernow_k8
[ 495.422583] WARNING: at mm/memory.c:909 unmap_vmas+0x548/0xa29()
[ 495.422591] cpufreq_ondemand
[ 495.422597] Hardware name: System Product Name
[ 495.422602] page->mapping does not exist in vma chain cpufreq_powersave
[ 495.422612] Modules linked in: cpufreq_userspace powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table freq_table cpufreq_conservative cpufreq_conservative binfmt_misc binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt kvm_amd dm_mod 8250_pnp kvm 8250 serial_core edac_core pcspkr k10temp ohci_hcd
[ 495.422676] ipv6Pid: 2919, comm: udevd Tainted: G W 2.6.34-rc3-00506-g6c62fe4 #1
[ 495.422689] Call Trace:
[ 495.422694] vfat
[ 495.422700] ------------[ cut here ]------------
[ 495.422721] WARNING: at mm/memory.c:909 unmap_vmas+0x548/0xa29()
[ 495.422729] fat [<ffffffff81038fe0>] warn_slowpath_common+0x7c/0x94
[ 495.422746] dm_crypt
[ 495.422751] Hardware name: System Product Name
[ 495.422758] dm_modpage->mapping does not exist in vma chain
[ 495.422767] Modules linked in: 8250_pnp [<ffffffff8103904f>] warn_slowpath_fmt+0x41/0x43
[ 495.422784] powernow_k8 cpufreq_ondemand 8250 cpufreq_powersave [<ffffffff810bcd20>] unmap_vmas+0x548/0xa29
[ 495.422807] serial_core cpufreq_userspace [<ffffffff810bd021>] ? unmap_vmas+0x849/0xa29
[ 495.422828] edac_core freq_table pcspkr cpufreq_conservative [<ffffffff810c17d8>] exit_mmap+0x102/0x1e4
[ 495.422851] binfmt_misc [<ffffffff810c173f>] ? exit_mmap+0x69/0x1e4
[ 495.422863] k10temp [<ffffffff810368bc>] mmput+0x48/0xb9
[ 495.422876] kvm_amd [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 495.422889] ohci_hcd kvm
[ 495.422903] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 495.422909] ipv6Pid: 2916, comm: udevd Tainted: G W 2.6.34-rc3-00506-g6c62fe4 #1
[ 495.422927] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 495.422934] Call Trace:
[ 495.422940] vfat [<ffffffff8141016d>] ? retint_swapgs+0xe/0x13
[ 495.422956] fat [<ffffffff81038fe0>] warn_slowpath_common+0x7c/0x94
[ 495.422972] dm_crypt dm_mod 8250_pnp [<ffffffff8103cf8a>] do_group_exit+0x84/0xb0
[ 495.422989] 8250 serial_core [<ffffffff8103cfcd>] sys_exit_group+0x17/0x1b
[ 495.423013] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 495.423019] edac_core
[ 495.423025] ---[ end trace d9664ac54d1edb0e ]---
[ 495.423031] pcspkr k10temp ohci_hcd
[ 495.423043] Pid: 2914, comm: udevd Tainted: G W 2.6.34-rc3-00506-g6c62fe4 #1
[ 495.423055] [<ffffffff8103904f>] warn_slowpath_fmt+0x41/0x43
[ 495.423063] Call Trace:
[ 495.423073] [<ffffffff810bcd20>] unmap_vmas+0x548/0xa29
[ 495.423087] [<ffffffff81038fe0>] warn_slowpath_common+0x7c/0x94
[ 495.423100] [<ffffffff810bd021>] ? unmap_vmas+0x849/0xa29
[ 495.423111] [<ffffffff8103904f>] warn_slowpath_fmt+0x41/0x43
[ 495.423123] [<ffffffff810bcd20>] unmap_vmas+0x548/0xa29
[ 495.423134] [<ffffffff810bd021>] ? unmap_vmas+0x849/0xa29
[ 495.423147] [<ffffffff810c17d8>] exit_mmap+0x102/0x1e4
[ 495.423159] [<ffffffff810c17d8>] exit_mmap+0x102/0x1e4
[ 495.423172] [<ffffffff810c173f>] ? exit_mmap+0x69/0x1e4
[ 495.423184] [<ffffffff810c173f>] ? exit_mmap+0x69/0x1e4
[ 495.423194] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 495.423204] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 495.423214] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 495.423225] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 495.423236] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 495.423246] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 495.423266] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 495.423277] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 495.423292] [<ffffffff8141016d>] ? retint_swapgs+0xe/0x13
[ 495.423303] [<ffffffff8141016d>] ? retint_swapgs+0xe/0x13
[ 495.423315] [<ffffffff8103cf8a>] do_group_exit+0x84/0xb0
[ 495.423325] [<ffffffff8103cfcd>] sys_exit_group+0x17/0x1b
[ 495.423334] [<ffffffff8103cf8a>] do_group_exit+0x84/0xb0
[ 495.423346] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 495.423357] [<ffffffff8103cfcd>] sys_exit_group+0x17/0x1b
[ 495.423365] ---[ end trace d9664ac54d1edb0f ]---
[ 495.423386] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 495.423402] ---[ end trace d9664ac54d1edb10 ]---
[ 495.424191] WARNING: at mm/memory.c:909 unmap_vmas+0x548/0xa29()
[ 495.424215] Hardware name: System Product Name
[ 495.424238] page->mapping does not exist in vma chain
[ 495.424259] Modules linked in: powernow_k8 cpufreq_ondemand cpufreq_powersave cpufreq_userspace freq_table cpufreq_conservative binfmt_misc kvm_amd kvm ipv6 vfat fat dm_crypt dm_mod 8250_pnp 8250 serial_core edac_core pcspkr k10temp ohci_hcd
[ 495.424693] Pid: 1923, comm: udevd Tainted: G W 2.6.34-rc3-00506-g6c62fe4 #1
[ 495.424723] Call Trace:
[ 495.424758] [<ffffffff81038fe0>] warn_slowpath_common+0x7c/0x94
[ 495.424788] [<ffffffff8103904f>] warn_slowpath_fmt+0x41/0x43
[ 495.424816] [<ffffffff810bcd20>] unmap_vmas+0x548/0xa29
[ 495.424843] [<ffffffff810bd021>] ? unmap_vmas+0x849/0xa29
[ 495.424875] [<ffffffff810c17d8>] exit_mmap+0x102/0x1e4
[ 495.424901] [<ffffffff810c173f>] ? exit_mmap+0x69/0x1e4
[ 495.424926] [<ffffffff810368bc>] mmput+0x48/0xb9
[ 495.424954] [<ffffffff8103ad90>] exit_mm+0x110/0x11d
[ 495.424981] [<ffffffff8103c9e6>] do_exit+0x1c5/0x6e5
[ 495.425008] [<ffffffff81065387>] ? trace_hardirqs_off_caller+0x1f/0xa9
[ 495.425038] [<ffffffff8141016d>] ? retint_swapgs+0xe/0x13
[ 495.425065] [<ffffffff8103cf8a>] do_group_exit+0x84/0xb0
[ 495.425091] [<ffffffff8103cfcd>] sys_exit_group+0x17/0x1b
[ 495.425119] [<ffffffff8100221b>] system_call_fastpath+0x16/0x1b
[ 495.425156] ---[ end trace d9664ac54d1edb11 ]---


--
Regards/Gruss,
Boris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/