From: Ingo Molnar on

Hi,

* Herbert Xu <herbert(a)gondor.apana.org.au> wrote:

> crypto: pcrypt - Add pcrypt crypto parallelization wrapper

-tip testing started triggering the following frequent boot crash yesterday,
on 64-bit x86:

| calling pcrypt_init+0x0/0xee @ 1
| BUG: unable to handle kernel NULL pointer dereference at (null)
| IP: [<ffffffff8126550b>] memcpy+0xb/0xb0
| Call Trace:
| [<ffffffff810c2f38>] ? padata_alloc+0x98/0x150
| [<ffffffff82158c51>] ? pcrypt_init+0x0/0xee
| [<ffffffff82158ccc>] pcrypt_init+0x7b/0xee

( Full oops attached below. Config attached as well. )

I have bisected it back to the following upstream commit:

|
| 5068c7a883d1694f0fb3631b664827644153cd08 is the first bad commit
| commit 5068c7a883d1694f0fb3631b664827644153cd08
| Author: Steffen Klassert <steffen.klassert(a)secunet.com>
| Date: Thu Jan 7 15:57:19 2010 +1100
|
| crypto: pcrypt - Add pcrypt crypto parallelization wrapper
|

And have confirmed it via more testing that reverting the following two
commits:

5a1436b: crypto: pcrypt - call the complete function on error
5068c7a: crypto: pcrypt - Add pcrypt crypto parallelization wrapper

Solves the crash. I have pushed out the reverts into tip:out-of-tree for now,
and can test any real fix as well.

Thanks,

Ingo

-------------->
Calling crypto_ctr_module_init+0x0/0x3e @ 1
initcall crypto_ctr_module_init+0x0/0x3e returned 0 after 0 usecs
calling pcrypt_init+0x0/0xee @ 1
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<ffffffff8126550b>] memcpy+0xb/0xb0
PGD 0
Oops: 0002 [#1] SMP
last sysfs file:
CPU 1
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.33-01768-g43a834d #19338 A8N-E/System Product Name
RIP: 0010:[<ffffffff8126550b>] [<ffffffff8126550b>] memcpy+0xb/0xb0
RSP: 0018:ffff88003f875e88 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff81f5f740 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffffffff81f5f740 RDI: 0000000000000000
RBP: ffff88003f875ec0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003e5317b0
R13: ffff88003f97a690 R14: ffff88003e5317b0 R15: ffff88003f9287d0
FS: 0000000000000000(0000) GS:ffff880004200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 0000000001be8000 CR4: 00000000000006a0
DR0: ffffffff82c160bc DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Process swapper (pid: 1, threadinfo ffff88003f874000, task ffff88003f878000)
Stack:
ffffffff810c2f38 ffff880000000018 ffffffff81f5f740 ffffffff82158c51
<0> 00000001552321fe 0000000000000002 0000000000000000 ffff88003f875ee0
<0> ffffffff82158ccc 00000001552321fe 0000000000000000 ffff88003f875f10
Call Trace:
[<ffffffff810c2f38>] ? padata_alloc+0x98/0x150
[<ffffffff82158c51>] ? pcrypt_init+0x0/0xee
[<ffffffff82158ccc>] pcrypt_init+0x7b/0xee
[<ffffffff810001dc>] do_one_initcall+0x3c/0x200
[<ffffffff8213e73c>] kernel_init+0x161/0x1b7
[<ffffffff81004184>] kernel_thread_helper+0x4/0x10
[<ffffffff817d2150>] ? restore_args+0x0/0x30
[<ffffffff8213e5db>] ? kernel_init+0x0/0x1b7
[<ffffffff81004180>] ? kernel_thread_helper+0x0/0x10
Code: 81 ea d8 1f 00 00 48 3b 42 20 73 07 48 8b 50 f9 31 c0 c3 31 d2 48 c7 c0 f2 ff ff ff c3 90 90 90 48 89 f8 89 d1 c1 e9 03 83 e2 07 <f3> 48 a5 89 d1 f3 a4 c3 8b 1e 4c 8b 46 08 4c 89 1f 4c 89 47 08
RIP [<ffffffff8126550b>] memcpy+0xb/0xb0
RSP <ffff88003f875e88>
CR2: 0000000000000000
---[ end trace 5a5d197966b56a2e ]---
swapper used greatest stack depth: 5016 bytes left
Kernel panic - not syncing: Attempted to kill init!

From: Steffen Klassert on
Hi.

On Mon, Mar 01, 2010 at 08:50:24AM +0100, Ingo Molnar wrote:
>
> Hi,
>
> * Herbert Xu <herbert(a)gondor.apana.org.au> wrote:
>
> > crypto: pcrypt - Add pcrypt crypto parallelization wrapper
>
> -tip testing started triggering the following frequent boot crash yesterday,
> on 64-bit x86:
>
> | calling pcrypt_init+0x0/0xee @ 1
> | BUG: unable to handle kernel NULL pointer dereference at (null)
> | IP: [<ffffffff8126550b>] memcpy+0xb/0xb0
> | Call Trace:
> | [<ffffffff810c2f38>] ? padata_alloc+0x98/0x150
> | [<ffffffff82158c51>] ? pcrypt_init+0x0/0xee
> | [<ffffffff82158ccc>] pcrypt_init+0x7b/0xee
>

Ugh, it seems that I forgot to allocate one of the cpumasks. Looking at the
configs of my test systems I noticed, that CONFIG_CPUMASK_OFFSTACK was
not set on all the configs, so I did not notice it.

The patch below fixes the boot crash if CONFIG_CPUMASK_OFFSTACK is
enabled on my test systems. Does the patch fix it for you too?

Thanks,

Steffen

Subject: [PATCH] padata: allocate the cpumask for the padata instance

The cpumask of the padata instance was used without allocated.
This caused boot crashes if CONFIG_CPUMASK_OFFSTACK is enabled.
This patch fixes this by doing proper allocation for this cpumask.

Signed-off-by: Steffen Klassert <steffen.klassert(a)secunet.com>
---
kernel/padata.c | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/kernel/padata.c b/kernel/padata.c
index 6f9bcb8..93caf65 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -642,6 +642,9 @@ struct padata_instance *padata_alloc(const struct cpumask *cpumask,
if (!pd)
goto err_free_inst;

+ if (!alloc_cpumask_var(&pinst->cpumask, GFP_KERNEL))
+ goto err_free_pd;
+
rcu_assign_pointer(pinst->pd, pd);

pinst->wq = wq;
@@ -654,12 +657,14 @@ struct padata_instance *padata_alloc(const struct cpumask *cpumask,
pinst->cpu_notifier.priority = 0;
err = register_hotcpu_notifier(&pinst->cpu_notifier);
if (err)
- goto err_free_pd;
+ goto err_free_cpumask;

mutex_init(&pinst->lock);

return pinst;

+err_free_cpumask:
+ free_cpumask_var(pinst->cpumask);
err_free_pd:
padata_free_pd(pd);
err_free_inst:
@@ -685,6 +690,7 @@ void padata_free(struct padata_instance *pinst)

unregister_hotcpu_notifier(&pinst->cpu_notifier);
padata_free_pd(pinst->pd);
+ free_cpumask_var(pinst->cpumask);
kfree(pinst);
}
EXPORT_SYMBOL(padata_free);
--
1.5.6.5

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Herbert Xu on
On Mon, Mar 01, 2010 at 03:55:19PM +0100, Steffen Klassert wrote:
>
> On Mon, Mar 01, 2010 at 08:50:24AM +0100, Ingo Molnar wrote:
> >
> > * Herbert Xu <herbert(a)gondor.apana.org.au> wrote:
> >
> > > crypto: pcrypt - Add pcrypt crypto parallelization wrapper
> >
> > -tip testing started triggering the following frequent boot crash yesterday,
> > on 64-bit x86:
> >
> > | calling pcrypt_init+0x0/0xee @ 1
> > | BUG: unable to handle kernel NULL pointer dereference at (null)
> > | IP: [<ffffffff8126550b>] memcpy+0xb/0xb0
> > | Call Trace:
> > | [<ffffffff810c2f38>] ? padata_alloc+0x98/0x150
> > | [<ffffffff82158c51>] ? pcrypt_init+0x0/0xee
> > | [<ffffffff82158ccc>] pcrypt_init+0x7b/0xee
> >
>
> Ugh, it seems that I forgot to allocate one of the cpumasks. Looking at the
> configs of my test systems I noticed, that CONFIG_CPUMASK_OFFSTACK was
> not set on all the configs, so I did not notice it.
>
> The patch below fixes the boot crash if CONFIG_CPUMASK_OFFSTACK is
> enabled on my test systems. Does the patch fix it for you too?

Ingo, any luck with this patch from Steffen?

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert(a)gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* Herbert Xu <herbert(a)gondor.apana.org.au> wrote:

> On Mon, Mar 01, 2010 at 03:55:19PM +0100, Steffen Klassert wrote:
> >
> > On Mon, Mar 01, 2010 at 08:50:24AM +0100, Ingo Molnar wrote:
> > >
> > > * Herbert Xu <herbert(a)gondor.apana.org.au> wrote:
> > >
> > > > crypto: pcrypt - Add pcrypt crypto parallelization wrapper
> > >
> > > -tip testing started triggering the following frequent boot crash yesterday,
> > > on 64-bit x86:
> > >
> > > | calling pcrypt_init+0x0/0xee @ 1
> > > | BUG: unable to handle kernel NULL pointer dereference at (null)
> > > | IP: [<ffffffff8126550b>] memcpy+0xb/0xb0
> > > | Call Trace:
> > > | [<ffffffff810c2f38>] ? padata_alloc+0x98/0x150
> > > | [<ffffffff82158c51>] ? pcrypt_init+0x0/0xee
> > > | [<ffffffff82158ccc>] pcrypt_init+0x7b/0xee
> > >
> >
> > Ugh, it seems that I forgot to allocate one of the cpumasks. Looking at the
> > configs of my test systems I noticed, that CONFIG_CPUMASK_OFFSTACK was
> > not set on all the configs, so I did not notice it.
> >
> > The patch below fixes the boot crash if CONFIG_CPUMASK_OFFSTACK is
> > enabled on my test systems. Does the patch fix it for you too?
>
> Ingo, any luck with this patch from Steffen?

I'm still busy with the merge window - please dont wait for me and consider it
fixed, i'll re-report if there's still any problems.

Thanks,

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Herbert Xu on
On Thu, Mar 04, 2010 at 04:00:52AM +0100, Ingo Molnar wrote:
>
> I'm still busy with the merge window - please dont wait for me and consider it
> fixed, i'll re-report if there's still any problems.

OK, I'll apply the patch.

Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert(a)gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/