From: Martin Pirker on
Hi list...

My shiny new kernel wants to tell me something.
I do not understand him, but maybe you do.... see below

HTH,
Martin


[ 27.026441]
[ 27.026443] =================================
[ 27.029003] [ INFO: inconsistent lock state ]
[ 27.030490] 2.6.35-rc5 #1
[ 27.031984] ---------------------------------
[ 27.033520] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 27.035118] swapper/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 27.035121] (&(&bdi->wb_lock)->rlock){+.?...}, at:
[<ffffffff8116cffe>] bdi_queue_work+0x2e/0xb0
[ 27.035133] {SOFTIRQ-ON-W} state was registered at:
[ 27.035136] [<ffffffff81093fde>] __lock_acquire+0x5de/0x1470
[ 27.035144] [<ffffffff81094f16>] lock_acquire+0xa6/0x160
[ 27.035149] [<ffffffff81578ec1>] _raw_spin_lock+0x31/0x40
[ 27.035157] [<ffffffff811132ce>] bdi_task_init+0x3e/0xa0
[ 27.035164] [<ffffffff811136c1>] bdi_forker_task+0x31/0x390
[ 27.035169] [<ffffffff8107dd26>] kthread+0xa6/0xb0
[ 27.035176] [<ffffffff8100bee4>] kernel_thread_helper+0x4/0x10
[ 27.035183] irq event stamp: 114300
[ 27.035185] hardirqs last enabled at (114300):
[<ffffffff8113aa1d>] kmem_cache_alloc_notrace+0xcd/0x110
[ 27.035193] hardirqs last disabled at (114299):
[<ffffffff8113a9a6>] kmem_cache_alloc_notrace+0x56/0x110
[ 27.035200] softirqs last enabled at (114274):
[<ffffffff81065a62>] __do_softirq+0x142/0x260
[ 27.035208] softirqs last disabled at (114287):
[<ffffffff8100bfdc>] call_softirq+0x1c/0x30
[ 27.035213]
[ 27.035214] other info that might help us debug this:
[ 27.035217] 1 lock held by swapper/0:
[ 27.035219] #0:
(&q->backing_dev_info.laptop_mode_wb_timer){+.-...}, at:
[<ffffffff8106e9f9>] run_timer_softirq+0x139/0x3f0
[ 27.035229]
[ 27.035230] stack backtrace:
[ 27.035234] Pid: 0, comm: swapper Not tainted 2.6.35-rc5 #1
[ 27.035237] Call Trace:
[ 27.035239] <IRQ> [<ffffffff810924aa>] print_usage_bug+0x18a/0x190
[ 27.035250] [<ffffffff81018ccf>] ? save_stack_trace+0x2f/0x50
[ 27.035255] [<ffffffff81092620>] ? check_usage_forwards+0x0/0xf0
[ 27.035260] [<ffffffff810932a2>] mark_lock+0x322/0x3f0
[ 27.035265] [<ffffffff81093f79>] __lock_acquire+0x579/0x1470
[ 27.035270] [<ffffffff81018ccf>] ? save_stack_trace+0x2f/0x50
[ 27.035275] [<ffffffff81090037>] ? graph_unlock+0x47/0xa0
[ 27.035280] [<ffffffff81094f16>] lock_acquire+0xa6/0x160
[ 27.035284] [<ffffffff8116cffe>] ? bdi_queue_work+0x2e/0xb0
[ 27.035289] [<ffffffff810933db>] ? mark_held_locks+0x6b/0xa0
[ 27.035294] [<ffffffff81578ec1>] _raw_spin_lock+0x31/0x40
[ 27.035298] [<ffffffff8116cffe>] ? bdi_queue_work+0x2e/0xb0
[ 27.035303] [<ffffffff8109373d>] ? trace_hardirqs_on+0xd/0x10
[ 27.035307] [<ffffffff8116cffe>] bdi_queue_work+0x2e/0xb0
[ 27.035313] [<ffffffff8116d508>] __bdi_start_writeback+0x88/0x110
[ 27.035317] [<ffffffff8116d5d5>] bdi_start_writeback+0x15/0x20
[ 27.035324] [<ffffffff8110345b>] laptop_mode_timer_fn+0x5b/0x70
[ 27.035329] [<ffffffff8106ea7f>] run_timer_softirq+0x1bf/0x3f0
[ 27.035334] [<ffffffff8106e9f9>] ? run_timer_softirq+0x139/0x3f0
[ 27.035339] [<ffffffff81082516>] ? __run_hrtimer+0xa6/0x1b0
[ 27.035343] [<ffffffff81103400>] ? laptop_mode_timer_fn+0x0/0x70
[ 27.035349] [<ffffffff81065999>] ? __do_softirq+0x79/0x260
[ 27.035354] [<ffffffff810659f2>] __do_softirq+0xd2/0x260
[ 27.035359] [<ffffffff81082920>] ? hrtimer_interrupt+0x140/0x250
[ 27.035363] [<ffffffff8100bfdc>] call_softirq+0x1c/0x30
[ 27.035368] [<ffffffff8100daad>] do_softirq+0x9d/0xd0
[ 27.035373] [<ffffffff810655b5>] irq_exit+0x95/0xa0
[ 27.035378] [<ffffffff81580f30>] smp_apic_timer_interrupt+0x70/0x9b
[ 27.035386] [<ffffffff8100ba93>] apic_timer_interrupt+0x13/0x20
[ 27.035389] <EOI> [<ffffffff812fdb8a>] ? intel_idle+0xfa/0x180
[ 27.035399] [<ffffffff812fdb83>] ? intel_idle+0xf3/0x180
[ 27.035406] [<ffffffff8157d6a0>] ? __atomic_notifier_call_chain+0x0/0xa0
[ 27.035412] [<ffffffff81459a87>] cpuidle_idle_call+0xa7/0x140
[ 27.035418] [<ffffffff81009dd8>] cpu_idle+0xb8/0x110
[ 27.035424] [<ffffffff8157276b>] start_secondary+0x1fd/0x204
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Rafael J. Wysocki on
On Wednesday, July 14, 2010, Martin Pirker wrote:
> Hi list...
>
> My shiny new kernel wants to tell me something.
> I do not understand him, but maybe you do.... see below

Hmm, kind of looks like a writeback issue (adding CCs).

Rafael


> [ 27.026441]
> [ 27.026443] =================================
> [ 27.029003] [ INFO: inconsistent lock state ]
> [ 27.030490] 2.6.35-rc5 #1
> [ 27.031984] ---------------------------------
> [ 27.033520] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
> [ 27.035118] swapper/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
> [ 27.035121] (&(&bdi->wb_lock)->rlock){+.?...}, at:
> [<ffffffff8116cffe>] bdi_queue_work+0x2e/0xb0
> [ 27.035133] {SOFTIRQ-ON-W} state was registered at:
> [ 27.035136] [<ffffffff81093fde>] __lock_acquire+0x5de/0x1470
> [ 27.035144] [<ffffffff81094f16>] lock_acquire+0xa6/0x160
> [ 27.035149] [<ffffffff81578ec1>] _raw_spin_lock+0x31/0x40
> [ 27.035157] [<ffffffff811132ce>] bdi_task_init+0x3e/0xa0
> [ 27.035164] [<ffffffff811136c1>] bdi_forker_task+0x31/0x390
> [ 27.035169] [<ffffffff8107dd26>] kthread+0xa6/0xb0
> [ 27.035176] [<ffffffff8100bee4>] kernel_thread_helper+0x4/0x10
> [ 27.035183] irq event stamp: 114300
> [ 27.035185] hardirqs last enabled at (114300):
> [<ffffffff8113aa1d>] kmem_cache_alloc_notrace+0xcd/0x110
> [ 27.035193] hardirqs last disabled at (114299):
> [<ffffffff8113a9a6>] kmem_cache_alloc_notrace+0x56/0x110
> [ 27.035200] softirqs last enabled at (114274):
> [<ffffffff81065a62>] __do_softirq+0x142/0x260
> [ 27.035208] softirqs last disabled at (114287):
> [<ffffffff8100bfdc>] call_softirq+0x1c/0x30
> [ 27.035213]
> [ 27.035214] other info that might help us debug this:
> [ 27.035217] 1 lock held by swapper/0:
> [ 27.035219] #0:
> (&q->backing_dev_info.laptop_mode_wb_timer){+.-...}, at:
> [<ffffffff8106e9f9>] run_timer_softirq+0x139/0x3f0
> [ 27.035229]
> [ 27.035230] stack backtrace:
> [ 27.035234] Pid: 0, comm: swapper Not tainted 2.6.35-rc5 #1
> [ 27.035237] Call Trace:
> [ 27.035239] <IRQ> [<ffffffff810924aa>] print_usage_bug+0x18a/0x190
> [ 27.035250] [<ffffffff81018ccf>] ? save_stack_trace+0x2f/0x50
> [ 27.035255] [<ffffffff81092620>] ? check_usage_forwards+0x0/0xf0
> [ 27.035260] [<ffffffff810932a2>] mark_lock+0x322/0x3f0
> [ 27.035265] [<ffffffff81093f79>] __lock_acquire+0x579/0x1470
> [ 27.035270] [<ffffffff81018ccf>] ? save_stack_trace+0x2f/0x50
> [ 27.035275] [<ffffffff81090037>] ? graph_unlock+0x47/0xa0
> [ 27.035280] [<ffffffff81094f16>] lock_acquire+0xa6/0x160
> [ 27.035284] [<ffffffff8116cffe>] ? bdi_queue_work+0x2e/0xb0
> [ 27.035289] [<ffffffff810933db>] ? mark_held_locks+0x6b/0xa0
> [ 27.035294] [<ffffffff81578ec1>] _raw_spin_lock+0x31/0x40
> [ 27.035298] [<ffffffff8116cffe>] ? bdi_queue_work+0x2e/0xb0
> [ 27.035303] [<ffffffff8109373d>] ? trace_hardirqs_on+0xd/0x10
> [ 27.035307] [<ffffffff8116cffe>] bdi_queue_work+0x2e/0xb0
> [ 27.035313] [<ffffffff8116d508>] __bdi_start_writeback+0x88/0x110
> [ 27.035317] [<ffffffff8116d5d5>] bdi_start_writeback+0x15/0x20
> [ 27.035324] [<ffffffff8110345b>] laptop_mode_timer_fn+0x5b/0x70
> [ 27.035329] [<ffffffff8106ea7f>] run_timer_softirq+0x1bf/0x3f0
> [ 27.035334] [<ffffffff8106e9f9>] ? run_timer_softirq+0x139/0x3f0
> [ 27.035339] [<ffffffff81082516>] ? __run_hrtimer+0xa6/0x1b0
> [ 27.035343] [<ffffffff81103400>] ? laptop_mode_timer_fn+0x0/0x70
> [ 27.035349] [<ffffffff81065999>] ? __do_softirq+0x79/0x260
> [ 27.035354] [<ffffffff810659f2>] __do_softirq+0xd2/0x260
> [ 27.035359] [<ffffffff81082920>] ? hrtimer_interrupt+0x140/0x250
> [ 27.035363] [<ffffffff8100bfdc>] call_softirq+0x1c/0x30
> [ 27.035368] [<ffffffff8100daad>] do_softirq+0x9d/0xd0
> [ 27.035373] [<ffffffff810655b5>] irq_exit+0x95/0xa0
> [ 27.035378] [<ffffffff81580f30>] smp_apic_timer_interrupt+0x70/0x9b
> [ 27.035386] [<ffffffff8100ba93>] apic_timer_interrupt+0x13/0x20
> [ 27.035389] <EOI> [<ffffffff812fdb8a>] ? intel_idle+0xfa/0x180
> [ 27.035399] [<ffffffff812fdb83>] ? intel_idle+0xf3/0x180
> [ 27.035406] [<ffffffff8157d6a0>] ? __atomic_notifier_call_chain+0x0/0xa0
> [ 27.035412] [<ffffffff81459a87>] cpuidle_idle_call+0xa7/0x140
> [ 27.035418] [<ffffffff81009dd8>] cpu_idle+0xb8/0x110
> [ 27.035424] [<ffffffff8157276b>] start_secondary+0x1fd/0x204
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo(a)vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Maciej Rutecki on
On środa, 14 lipca 2010 o 22:33:10 Martin Pirker wrote:
> Hi list...
>
> My shiny new kernel wants to tell me something.
> I do not understand him, but maybe you do.... see below
>

I created a Bugzilla entry at
https://bugzilla.kernel.org/show_bug.cgi?id=16400
for your bug report, please add your address to the CC list in there, thanks!
--
Maciej Rutecki
http://www.maciek.unixy.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Dan Carpenter on
This is from:

commit 31373d09da5b7fe21fe6f781e92bd534a3495f00
Author: Matthew Garrett <mjg(a)redhat.com>
Date: Tue Apr 6 14:25:14 2010 +0200

laptop-mode: Make flushes per-device

One of the features of laptop-mode is that it forces a writeout of dirty
pages if something else triggers a physical read or write from a device.
The current implementation flushes pages on all devices, rather than only
the one that triggered the flush. This patch alters the behaviour so that
only the recently accessed block device is flushed, preventing other
disks being spun up for no terribly good reason.

One way to fix it might be to change all the places that call
spin_lock(&bdi->wb_lock); to spin_lock_bh(&bdi->wb_lock); but I'm not
sure that's the right way.

I don't think Matthew Garrett has a bugzilla account?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Jens Axboe on
On 07/17/2010 01:04 PM, Dan Carpenter wrote:
> This is from:
>
> commit 31373d09da5b7fe21fe6f781e92bd534a3495f00
> Author: Matthew Garrett <mjg(a)redhat.com>
> Date: Tue Apr 6 14:25:14 2010 +0200
>
> laptop-mode: Make flushes per-device
>
> One of the features of laptop-mode is that it forces a writeout of dirty
> pages if something else triggers a physical read or write from a device.
> The current implementation flushes pages on all devices, rather than only
> the one that triggered the flush. This patch alters the behaviour so that
> only the recently accessed block device is flushed, preventing other
> disks being spun up for no terribly good reason.
>
> One way to fix it might be to change all the places that call
> spin_lock(&bdi->wb_lock); to spin_lock_bh(&bdi->wb_lock); but I'm not
> sure that's the right way.
>
> I don't think Matthew Garrett has a bugzilla account?

I posted a patch for this the other day, but I just now notice
that it was a private list of CC addresses.

Can anyone try this completed untested patch?

diff --git a/block/blk-core.c b/block/blk-core.c
index 5ab3ac2..a108b8e 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -451,7 +451,7 @@ void blk_cleanup_queue(struct request_queue *q)
*/
blk_sync_queue(q);

- del_timer_sync(&q->backing_dev_info.laptop_mode_wb_timer);
+ cancel_delayed_work_sync(&q->backing_dev_info.laptop_mode_work);
mutex_lock(&q->sysfs_lock);
queue_flag_set_unlocked(QUEUE_FLAG_DEAD, q);
mutex_unlock(&q->sysfs_lock);
@@ -515,8 +515,9 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id)
return NULL;
}

- setup_timer(&q->backing_dev_info.laptop_mode_wb_timer,
- laptop_mode_timer_fn, (unsigned long) q);
+
+ INIT_DELAYED_WORK(&q->backing_dev_info.laptop_mode_work,
+ laptop_mode_work_fn);
init_timer(&q->unplug_timer);
setup_timer(&q->timeout, blk_rq_timed_out_timer, (unsigned long) q);
INIT_LIST_HEAD(&q->timeout_list);
diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index e536f3a..d51acff 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
@@ -84,7 +84,7 @@ struct backing_dev_info {

struct device *dev;

- struct timer_list laptop_mode_wb_timer;
+ struct delayed_work laptop_mode_work;

#ifdef CONFIG_DEBUG_FS
struct dentry *debug_dir;
diff --git a/include/linux/writeback.h b/include/linux/writeback.h
index c24eca7..936ef5a 100644
--- a/include/linux/writeback.h
+++ b/include/linux/writeback.h
@@ -87,8 +87,7 @@ static inline void inode_sync_wait(struct inode *inode)
#ifdef CONFIG_BLOCK
void laptop_io_completion(struct backing_dev_info *info);
void laptop_sync_completion(void);
-void laptop_mode_sync(struct work_struct *work);
-void laptop_mode_timer_fn(unsigned long data);
+void laptop_mode_work_fn(struct work_struct *);
#else
static inline void laptop_sync_completion(void) { }
#endif
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 3d2111a..9978e8e 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -698,18 +698,20 @@ int dirty_writeback_centisecs_handler(ctl_table *table, int write,
}

#ifdef CONFIG_BLOCK
-void laptop_mode_timer_fn(unsigned long data)
+void laptop_mode_work_fn(struct work_struct *work)
{
- struct request_queue *q = (struct request_queue *)data;
+ struct backing_dev_info *bdi;
int nr_pages = global_page_state(NR_FILE_DIRTY) +
global_page_state(NR_UNSTABLE_NFS);

+ bdi = container_of(work, struct backing_dev_info, laptop_mode_work.work);
+
/*
* We want to write everything out, not just down to the dirty
* threshold
*/
- if (bdi_has_dirty_io(&q->backing_dev_info))
- bdi_start_writeback(&q->backing_dev_info, nr_pages);
+ if (bdi_has_dirty_io(bdi))
+ bdi_start_writeback(bdi, nr_pages);
}

/*
@@ -717,9 +719,12 @@ void laptop_mode_timer_fn(unsigned long data)
* of all dirty data a few seconds from now. If the flush is already scheduled
* then push it back - the user is still using the disk.
*/
-void laptop_io_completion(struct backing_dev_info *info)
+void laptop_io_completion(struct backing_dev_info *bdi)
{
- mod_timer(&info->laptop_mode_wb_timer, jiffies + laptop_mode);
+ if (work_pending(&bdi->laptop_mode_work.work))
+ mod_timer(&bdi->laptop_mode_work.timer, jiffies + laptop_mode);
+ else
+ schedule_delayed_work(&bdi->laptop_mode_work, laptop_mode);
}

/*
@@ -734,7 +739,7 @@ void laptop_sync_completion(void)
rcu_read_lock();

list_for_each_entry_rcu(bdi, &bdi_list, bdi_list)
- del_timer(&bdi->laptop_mode_wb_timer);
+ __cancel_delayed_work(&bdi->laptop_mode_work);

rcu_read_unlock();
}

--
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/