Prev: [PATCH 0/3] Get maintainer script cleanups
Next: Enhanced reimplemention of the kfifo API request for merge
From: Benjamin Li on 1 Mar 2010 20:30
Hi Bruno,

On Tue, 2010-02-23 at 04:15 -0800, Bruno Pr�mont wrote:
> Hi Benjamin,
>
> On Fri, 19 February 2010 "Benjamin Li" <benli(a)broadcom.com> wrote:
> > >From your logs it looks like the device came up using MSI, but in the
> > MSI-X poll routine was being called:
> >
> > [ 9.836673] bnx2: eth0: using MSI
> > ...
> >
> > [ 134.643459] [<ffffffffa004019e>] bnx2_poll_msix+0x3e/0xd0 [bnx2]
> > [ 134.643465] [<ffffffff8135bcd1>] netpoll_poll+0xe1/0x3c0
> >
> > which is incorrect. If we are in MSI mode, the bnx2_poll() routine
> > should be used.
> >
> > I think what is going on here is that during the bnx2x driver
> > initialization the current bnx2 driver adds all possible NAPI
> > structures that map to all the hardware vectors (BNX2_MAX_MSIX_VEC=9)
> > to the NAPI list in the net_device structure regardless if they are
> > used or not (Seen in drivers/net/bnx2.c:bnx2_init_napi()). This can
> > cause uninitialized NAPI structures to be placed on the napi_list.
> > Because this device is in MSI mode, only 1 vector is initialized.
> > Now, the problem is triggered when net/core/netpoll.c:poll_napi() is
> > called. This is because this routine will run through the entire
> > napi_list calling all the poll routines. In your particular case, it
> > is calling the poll routine on an uninitialized vector causing the
> > kernel panic.
> >
> > Please try the patch below to see if it solves your problem. Note,
> > this only have been compile tested and tested against basic traffic
> > runs. Unfortunately, I could not reproduce the kernel panic with the
> > instructions below to verify the patch.
> >
> > Thanks again for all your help in helping us track this down.
>
> I applied the patch today and tried to reproduce with my showcases.
>
> Seems that it's harder to trigger now but I still end up being able to
> crash the box. Don't know if it's the same cause or not (could also
> be the tcp-retransmit ghost)...
>
> This time I had to run a few paralell scp's (8Mb/s each) to the box and
> 'echo t > /proc/sysrq-trigger' multiple times via ssh session for it to
> happen. It didn't trigger with by netbomb though I will try some more
> and see)
>
> I don't know if it's the same reason or not (hopefully something
> reached disk as serial console is dead and pings are not
> answered anymore.
> It's probably some printk/bug/warn that triggers in network stack and
> deadlocks with netconsole.

Thanks for trying the patch. I still haven't been able to reproduce
what you are seeing here. I am able to run scp and 'echo t
> /proc/sysrq-trigger' multiple times. I was wondering if you had any
success reproducing the problem with a stack trace?

Thanks again.

-Ben

>
> Regards,
> Bruno
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Bruno Prémont on 2 Mar 2010 02:20
Hi Benjamin,

On Mon, 1 Mar 2010 17:26:10 "Benjamin Li" <benli(a)broadcom.com> wrote:
> Thanks for trying the patch. I still haven't been able to reproduce
> what you are seeing here. I am able to run scp and 'echo t
> > /proc/sysrq-trigger' multiple times. I was wondering if you had any
> success reproducing the problem with a stack trace?

Last week I couldn't trigger it again after booting with console=ttyS1,
but today I finally got the box to crash (same procedure but matter of
"luck" as to when it happens)

System continues printing the bnx2 debug messages but does not accept any
input (not even on serial console). Every few minutes it complains about
collectd being blocked for more than 120 seconds (collectd wants to send
UDP packets with system state)

The running kernel is 2.6.33-rc8 (though I did not forward-port your
patch in this thread, that is bnx2_dump_ftq() is missing - half of it
having been applied to 2.6.33). Looking at the trace I got I will add
that part and try again.

Regards,
Bruno

[575872.163771] ------------[ cut here ]------------
[575872.172467] WARNING: at /usr/src/linux-2.6.33-rc8-git7/kernel/softirq.c:143 local_bh_enable_ip+0x72/0xa0()
[575872.172890] Hardware name: ProLiant DL360 G5
[575872.172890] Modules linked in: qla2xxx netbomb ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci uhci_hcd ipmi_si ipmi_msghandler hpwdt bnx2 ehci_hcd libata [last unloaded: qla2xxx]
[575872.172890] Pid: 32564, comm: cat Not tainted 2.6.33-rc8-git7-x86_64 #1
[575872.172890] Call Trace:
[575872.172890] [<ffffffff8103f002>] ? local_bh_enable_ip+0x72/0xa0
[575872.172890] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
[575872.172890] [<ffffffff810393cf>] warn_slowpath_null+0xf/0x20
[575872.172890] [<ffffffff8103f002>] local_bh_enable_ip+0x72/0xa0
[575872.172890] [<ffffffff814002af>] _raw_spin_unlock_bh+0xf/0x20
[575872.172890] [<ffffffffa0026ed4>] bnx2_reg_rd_ind+0x44/0x60 [bnx2]
[575872.172890] [<ffffffffa0026eff>] bnx2_shmem_rd+0xf/0x20 [bnx2]
[575872.172890] [<ffffffffa0030ff4>] bnx2_poll+0x194/0x228 [bnx2]
[575872.172890] [<ffffffff8135c081>] netpoll_poll+0xe1/0x3c0
[575872.172890] [<ffffffff8135c518>] netpoll_send_skb+0x118/0x210
[575872.172890] [<ffffffff8135c80b>] netpoll_send_udp+0x1fb/0x210
[575872.172890] [<ffffffffa00501c5>] write_msg+0x95/0xd0 [netbomb]
[575872.172890] [<ffffffffa0050255>] netbomb_write+0x55/0xa4 [netbomb]
[575872.172890] [<ffffffff810f6571>] proc_reg_write+0x71/0xb0
[575872.172890] [<ffffffff810ab6db>] vfs_write+0xcb/0x180
[575872.172890] [<ffffffff810ab880>] sys_write+0x50/0x90
[575872.172890] [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
[575872.172890] ---[ end trace d4f601a1b34bd327 ]---
[575878.950008] ------------[ cut here ]------------
[575878.955297] WARNING: at /usr/src/linux-2.6.33-rc8-git7/net/sched/sch_generic.c:255 dev_watchdog+0x25e/0x270()
[575878.966541] Hardware name: ProLiant DL360 G5
[575878.972352] NETDEV WATCHDOG: eth0 (bnx2): transmit queue 0 timed out
[575878.976435] Modules linked in: qla2xxx netbomb ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci uhci_hcd ipmi_si ipmi_msghandler hpwdt bnx2 ehci_hcd libata [last unloaded: qla2xxx]
[575878.985325] Pid: 0, comm: swapper Tainted: G W 2.6.33-rc8-git7-x86_64 #1
[575878.988929] Call Trace:
[575878.990124] <IRQ> [<ffffffff8135f84e>] ? dev_watchdog+0x25e/0x270
[575878.994317] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
[575878.998118] [<ffffffff81039444>] warn_slowpath_fmt+0x64/0x70
[575879.010352] [<ffffffff81362206>] ? nlmsg_notify+0x46/0xc0
[575879.012956] [<ffffffff8119f339>] ? strlcpy+0x49/0x60
[575879.015487] [<ffffffff81349b33>] ? netdev_drivername+0x43/0x50
[575879.027713] [<ffffffff8135f84e>] dev_watchdog+0x25e/0x270
[575879.030339] [<ffffffff810354fc>] ? scheduler_tick+0x12c/0x290
[575879.034202] [<ffffffff8135f5f0>] ? dev_watchdog+0x0/0x270
[575879.047296] [<ffffffff810440fc>] run_timer_softirq+0x13c/0x210
[575879.051101] [<ffffffff8105b4b7>] ? clockevents_program_event+0x57/0xa0
[575879.054422] [<ffffffff8103edb6>] __do_softirq+0xa6/0x130
[575879.065396] [<ffffffff81003bcc>] call_softirq+0x1c/0x30
[575879.070825] [<ffffffff81005be5>] do_softirq+0x55/0x90
[575879.073266] [<ffffffff8103eb35>] irq_exit+0x75/0x90
[575879.075618] [<ffffffff8101aeed>] smp_apic_timer_interrupt+0x6d/0xa0
[575879.094357] [<ffffffff81003693>] apic_timer_interrupt+0x13/0x20
[575879.099195] <EOI> [<ffffffff8100b186>] ? mwait_idle+0x66/0x80
[575879.102225] [<ffffffff81001f90>] ? enter_idle+0x20/0x30
[575879.119572] [<ffffffff81002003>] cpu_idle+0x63/0xb0
[575879.121925] [<ffffffff818ab5de>] start_secondary+0x158/0x1aa
[575879.125708] ---[ end trace d4f601a1b34bd328 ]---
[575879.130792] bnx2: eth0 DEBUG: intr_sem[0]
[575879.132880] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575879.149587] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575879.152959] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575883.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575883.961580] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575883.974574] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575883.985269] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575888.951879] bnx2: eth0 DEBUG: intr_sem[0]
[575888.955092] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575888.967583] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575888.973761] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575893.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575893.953954] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575893.964160] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575893.968596] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575898.951879] bnx2: eth0 DEBUG: intr_sem[0]
[575898.957913] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575898.962225] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575898.972121] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575903.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575903.956728] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575903.961380] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575903.972171] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575908.951880] bnx2: eth0 DEBUG: intr_sem[0]
[575908.956790] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575908.961398] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575908.974628] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575913.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575913.958417] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575913.962159] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575913.965457] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575918.951879] bnx2: eth0 DEBUG: intr_sem[0]
[575918.958634] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575918.965095] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575918.974149] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575923.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575923.964361] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575923.970549] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575923.975969] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575928.951879] bnx2: eth0 DEBUG: intr_sem[0]
[575928.961375] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575928.965103] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575928.969552] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575933.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575933.963067] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575933.970543] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575933.993032] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575938.951879] bnx2: eth0 DEBUG: intr_sem[0]
[575938.958134] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575938.961695] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575938.968855] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575943.951883] bnx2: eth0 DEBUG: intr_sem[0]
[575943.959389] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[575943.964102] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[575943.967612] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[575948.951879] bnx2: eth0 DEBUG: intr_sem[0]
.....

Blocked collectd task:
[576003.053144] INFO: task collectd:3019 blocked for more than 120 seconds.
[576003.066206] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[576003.072659] collectd D 0000000000000000 0 3019 1 0x00020000
[576003.072663] ffff8802aa3afa78 0000000000200086 0000000000000000 0000000000000008
[576003.072666] 0000000000012780 000000000000dda8 ffff8802aa3affd8 ffff8802aa256fa0
[576003.072670] ffff8802af86e9c0 ffff8802aa257208 00000005ab1a3000 00000001036e43c3
[576003.072673] Call Trace:
[576003.072682] [<ffffffff813ff33e>] __mutex_lock_slowpath+0x12e/0x180
[576003.072687] [<ffffffff813fecfe>] mutex_lock+0x1e/0x40
[576003.072690] [<ffffffff81357fd8>] rtnetlink_rcv+0x18/0x40
[576003.072694] [<ffffffff813621b5>] netlink_unicast+0x285/0x290
[576003.072698] [<ffffffff81344f83>] ? memcpy_fromiovec+0x63/0x80
[576003.072701] [<ffffffff81362c63>] netlink_sendmsg+0x1d3/0x2c0
[576003.072705] [<ffffffff810bdd9e>] ? __d_instantiate+0x5e/0xd0
[576003.072708] [<ffffffff8133981b>] sock_sendmsg+0xbb/0xf0
[576003.072712] [<ffffffff810c72aa>] ? seq_open+0x4a/0xa0
[576003.072714] [<ffffffff81338ed9>] ? copy_from_user+0x9/0x10
[576003.072717] [<ffffffff81338f0c>] ? move_addr_to_kernel+0x2c/0x40
[576003.072720] [<ffffffff8135e09c>] ? verify_compat_iovec+0x9c/0x110
[576003.072723] [<ffffffff8133b734>] sys_sendmsg+0x184/0x320
[576003.072728] [<ffffffff810f5d60>] ? proc_delete_inode+0x0/0x50
[576003.072731] [<ffffffff810700e0>] ? call_rcu_sched+0x10/0x20
[576003.072733] [<ffffffff810700f9>] ? call_rcu+0x9/0x10
[576003.072736] [<ffffffff810bca9a>] ? d_free+0x3a/0x60
[576003.072741] [<ffffffff810acae8>] ? __fput+0x178/0x1f0
[576003.072744] [<ffffffff81058014>] ? getnstimeofday+0x64/0xf0
[576003.072747] [<ffffffff8135d94c>] compat_sys_socketcall+0xcc/0x210
[576003.072751] [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Bruno Prémont on 2 Mar 2010 03:30
Hi Benjamin,

> The running kernel is 2.6.33-rc8 (though I did not forward-port your
> patch in this thread, that is bnx2_dump_ftq() is missing - half of it
> having been applied to 2.6.33). Looking at the trace I got I will add
> that part and try again.

Here it is with bnx2_dump_ftq added:

[ 3405.422963] ------------[ cut here ]------------
[ 3405.428958] WARNING: at /usr/src/linux-2.6.33-rc8-git7/kernel/softirq.c:143 local_bh_enable_ip+0x72/0xa0()
[ 3405.431858] Hardware name: ProLiant DL360 G5
[ 3405.431858] Modules linked in: netbomb bnx2 ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci ipmi_si ipmi_msghandler uhci_hcd qla2xxx libata hpwdt ehci_hcd [last unloaded: bnx2]
[ 3405.431858] Pid: 25763, comm: cat Not tainted 2.6.33-rc8-git7-x86_64 #1
[ 3405.431858] Call Trace:
[ 3405.431858] [<ffffffff8103f002>] ? local_bh_enable_ip+0x72/0xa0
[ 3405.431858] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
[ 3405.431858] [<ffffffff810393cf>] warn_slowpath_null+0xf/0x20
[ 3405.431858] [<ffffffff8103f002>] local_bh_enable_ip+0x72/0xa0
[ 3405.431858] [<ffffffff814002af>] _raw_spin_unlock_bh+0xf/0x20
[ 3405.431858] [<ffffffffa0108ed4>] bnx2_reg_rd_ind+0x44/0x60 [bnx2]
[ 3405.431858] [<ffffffffa0108eff>] bnx2_shmem_rd+0xf/0x20 [bnx2]
[ 3405.431858] [<ffffffffa0113464>] bnx2_poll+0x194/0x228 [bnx2]
[ 3405.431858] [<ffffffff8135c081>] netpoll_poll+0xe1/0x3c0
[ 3405.431858] [<ffffffff8135c518>] netpoll_send_skb+0x118/0x210
[ 3405.431858] [<ffffffff8135c80b>] netpoll_send_udp+0x1fb/0x210
[ 3405.431858] [<ffffffffa00131c5>] write_msg+0x95/0xd0 [netbomb]
[ 3405.431858] [<ffffffffa0013255>] netbomb_write+0x55/0xa4 [netbomb]
[ 3405.431858] [<ffffffff810f6571>] proc_reg_write+0x71/0xb0
[ 3405.431858] [<ffffffff810ab6db>] vfs_write+0xcb/0x180
[ 3405.431858] [<ffffffff810ab880>] sys_write+0x50/0x90
[ 3405.431858] [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
[ 3405.431858] ---[ end trace b4ac1510884bf2bc ]---
[ 3411.050005] ------------[ cut here ]------------
[ 3411.054851] WARNING: at /usr/src/linux-2.6.33-rc8-git7/net/sched/sch_generic.c:255 dev_watchdog+0x25e/0x270()
[ 3411.059546] Hardware name: ProLiant DL360 G5
[ 3411.061569] NETDEV WATCHDOG: eth0 (bnx2): transmit queue 0 timed out
[ 3411.064582] Modules linked in: netbomb bnx2 ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci ipmi_si ipmi_msghandler uhci_hcd qla2xxx libata hpwdt ehci_hcd [last unloaded: bnx2]
[ 3411.064597] Pid: 0, comm: swapper Tainted: G W 2.6.33-rc8-git7-x86_64 #1
[ 3411.064599] Call Trace:
[ 3411.064601] <IRQ> [<ffffffff8135f84e>] ? dev_watchdog+0x25e/0x270
[ 3411.064609] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
[ 3411.064612] [<ffffffff81039444>] warn_slowpath_fmt+0x64/0x70
[ 3411.064616] [<ffffffff8103486d>] ? default_wake_function+0xd/0x10
[ 3411.064620] [<ffffffff8119f339>] ? strlcpy+0x49/0x60
[ 3411.064623] [<ffffffff81349b33>] ? netdev_drivername+0x43/0x50
[ 3411.064626] [<ffffffff8135f84e>] dev_watchdog+0x25e/0x270
[ 3411.064630] [<ffffffff8104c000>] ? delayed_work_timer_fn+0x0/0x40
[ 3411.064633] [<ffffffff8104bf87>] ? __queue_work+0x77/0x90
[ 3411.064636] [<ffffffff8103558b>] ? scheduler_tick+0x1bb/0x290
[ 3411.064639] [<ffffffff8135f5f0>] ? dev_watchdog+0x0/0x270
[ 3411.064642] [<ffffffff810440fc>] run_timer_softirq+0x13c/0x210
[ 3411.064645] [<ffffffff8105b4b7>] ? clockevents_program_event+0x57/0xa0
[ 3411.064649] [<ffffffff8103edb6>] __do_softirq+0xa6/0x130
[ 3411.064652] [<ffffffff81003bcc>] call_softirq+0x1c/0x30
[ 3411.064655] [<ffffffff81005be5>] do_softirq+0x55/0x90
[ 3411.064658] [<ffffffff8103eb35>] irq_exit+0x75/0x90
[ 3411.064661] [<ffffffff8101aeed>] smp_apic_timer_interrupt+0x6d/0xa0
[ 3411.064664] [<ffffffff81003693>] apic_timer_interrupt+0x13/0x20
[ 3411.064666] <EOI> [<ffffffff8100b186>] ? mwait_idle+0x66/0x80
[ 3411.064670] [<ffffffff81001f90>] ? enter_idle+0x20/0x30
[ 3411.064673] [<ffffffff81002003>] cpu_idle+0x63/0xb0
[ 3411.064676] [<ffffffff813f2f14>] rest_init+0x74/0x80
[ 3411.064680] [<ffffffff81880c15>] start_kernel+0x2f8/0x336
[ 3411.064683] [<ffffffff8188026d>] x86_64_start_reservations+0x7d/0x84
[ 3411.064686] [<ffffffff81880354>] x86_64_start_kernel+0xe0/0xf2
[ 3411.064688] ---[ end trace b4ac1510884bf2bd ]---
[ 3411.064689] bnx2: <--- start FTQ dump on eth0 --->
[ 3411.064691] bnx2: eth0: BNX2_RV2P_PFTQ_CTL 10000
[ 3411.064693] bnx2: eth0: BNX2_RV2P_TFTQ_CTL 20000
[ 3411.064695] bnx2: eth0: BNX2_RV2P_MFTQ_CTL 20000
[ 3411.064697] bnx2: eth0: BNX2_TBDR_FTQ_CTL 4002
[ 3411.064699] bnx2: eth0: BNX2_TDMA_FTQ_CTL 10002
[ 3411.064702] bnx2: eth0: BNX2_TXP_FTQ_CTL 10002
[ 3411.064704] bnx2: eth0: BNX2_TPAT_FTQ_CTL 10002
[ 3411.064706] bnx2: eth0: BNX2_RXP_CFTQ_CTL 8000
[ 3411.064708] bnx2: eth0: BNX2_RXP_FTQ_CTL 100000
[ 3411.064710] bnx2: eth0: BNX2_COM_COMXQ_FTQ_CTL 10000
[ 3411.064712] bnx2: eth0: BNX2_COM_COMTQ_FTQ_CTL 20000
[ 3411.064715] bnx2: eth0: BNX2_COM_COMQ_FTQ_CTL 10000
[ 3411.064717] bnx2: eth0: BNX2_CP_CPQ_FTQ_CTL 8000
[ 3411.064724] bnx2: eth0: TXP mode b84c state 80001000 evt_mask 500 pc 8000bcc pc 8000bec instr 30620001
[ 3411.064732] bnx2: eth0: TPAT mode b84c state 80005000 evt_mask 500 pc 8000694 pc 80006a4 instr 8e310458
[ 3411.064740] bnx2: eth0: RXP mode b84c state 80001000 evt_mask 500 pc 80044a8 pc 80044a8 instr ac2400f8
[ 3411.064748] bnx2: eth0: COM mode b84c state 80001000 evt_mask 500 pc 8000a60 pc 8000a74 instr 3c030800
[ 3411.064756] bnx2: eth0: CP mode b84c state 80008000 evt_mask 500 pc 8000434 pc 8000700 instr 39ee0001
[ 3411.064758] bnx2: <--- end FTQ dump on eth0 --->
[ 3411.064759] bnx2: eth0 DEBUG: intr_sem[0]
[ 3411.064762] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 3411.064766] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 3411.064768] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 3416.050005] bnx2: <--- start FTQ dump on eth0 --->
[ 3416.054456] bnx2: eth0: BNX2_RV2P_PFTQ_CTL 10000
[ 3416.057683] bnx2: eth0: BNX2_RV2P_TFTQ_CTL 20000
[ 3416.059944] bnx2: eth0: BNX2_RV2P_MFTQ_CTL 20000
[ 3416.069481] bnx2: eth0: BNX2_TBDR_FTQ_CTL 4002
[ 3416.078819] bnx2: eth0: BNX2_TDMA_FTQ_CTL 10002
[ 3416.083896] bnx2: eth0: BNX2_TXP_FTQ_CTL 10002
[ 3416.085968] bnx2: eth0: BNX2_TPAT_FTQ_CTL 10002
[ 3416.088084] bnx2: eth0: BNX2_RXP_CFTQ_CTL 8000
[ 3416.090129] bnx2: eth0: BNX2_RXP_FTQ_CTL 100000
[ 3416.092285] bnx2: eth0: BNX2_COM_COMXQ_FTQ_CTL 10000
[ 3416.096710] bnx2: eth0: BNX2_COM_COMTQ_FTQ_CTL 20000
[ 3416.099065] bnx2: eth0: BNX2_COM_COMQ_FTQ_CTL 10000
[ 3416.101506] bnx2: eth0: BNX2_CP_CPQ_FTQ_CTL 8000
[ 3416.105567] bnx2: eth0: TXP mode b84c state 80001000 evt_mask 500 pc 8000bf0 pc 8000bc0 instr b8302b
[ 3416.109963] bnx2: eth0: TPAT mode b84c state 80001000 evt_mask 500 pc 8000674 pc 8000694 instr af8d0034
[ 3416.120862] bnx2: eth0: RXP mode b84c state 80001000 evt_mask 500 pc 80044c4 pc 800447c instr 3c050800
[ 3416.125237] bnx2: eth0: COM mode b84c state 80001000 evt_mask 500 pc 8000a1c pc 8000a28 instr 8c4200b8
[ 3416.130656] bnx2: eth0: CP mode b84c state 80000000 evt_mask 500 pc 8000728 pc 800040c instr 18d2821
[ 3416.139721] bnx2: <--- end FTQ dump on eth0 --->
[ 3416.142905] bnx2: eth0 DEBUG: intr_sem[0]
[ 3416.145814] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 3416.151452] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 3416.154701] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
....


Regards,
Bruno
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Michael Chan on 2 Mar 2010 17:30

On Tue, 2010-03-02 at 00:20 -0800, Bruno Prémont wrote:
> [ 3405.422963] ------------[ cut here ]------------
> [ 3405.428958] WARNING: at /usr/src/linux-2.6.33-rc8-git7/kernel/softirq.c:143 local_bh_enable_ip+0x72/0xa0()

In normal NAPI mode, we are in softirq context and we correctly use
spin_lock_bh() and spin_unlock_bh() here. In netpoll mode, IRQs are
disabled and so we get a warning from spin_unlock_bh().

> [ 3405.431858] Hardware name: ProLiant DL360 G5
> [ 3405.431858] Modules linked in: netbomb bnx2 ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci ipmi_si ipmi_msghandler uhci_hcd qla2xxx libata hpwdt ehci_hcd [last unloaded: bnx2]
> [ 3405.431858] Pid: 25763, comm: cat Not tainted 2.6.33-rc8-git7-x86_64 #1
> [ 3405.431858] Call Trace:
> [ 3405.431858] [<ffffffff8103f002>] ? local_bh_enable_ip+0x72/0xa0
> [ 3405.431858] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
> [ 3405.431858] [<ffffffff810393cf>] warn_slowpath_null+0xf/0x20
> [ 3405.431858] [<ffffffff8103f002>] local_bh_enable_ip+0x72/0xa0
> [ 3405.431858] [<ffffffff814002af>] _raw_spin_unlock_bh+0xf/0x20
> [ 3405.431858] [<ffffffffa0108ed4>] bnx2_reg_rd_ind+0x44/0x60 [bnx2]
> [ 3405.431858] [<ffffffffa0108eff>] bnx2_shmem_rd+0xf/0x20 [bnx2]
> [ 3405.431858] [<ffffffffa0113464>] bnx2_poll+0x194/0x228 [bnx2]
> [ 3405.431858] [<ffffffff8135c081>] netpoll_poll+0xe1/0x3c0
> [ 3405.431858] [<ffffffff8135c518>] netpoll_send_skb+0x118/0x210
> [ 3405.431858] [<ffffffff8135c80b>] netpoll_send_udp+0x1fb/0x210
> [ 3405.431858] [<ffffffffa00131c5>] write_msg+0x95/0xd0 [netbomb]
> [ 3405.431858] [<ffffffffa0013255>] netbomb_write+0x55/0xa4 [netbomb]
> [ 3405.431858] [<ffffffff810f6571>] proc_reg_write+0x71/0xb0
> [ 3405.431858] [<ffffffff810ab6db>] vfs_write+0xcb/0x180
> [ 3405.431858] [<ffffffff810ab880>] sys_write+0x50/0x90
> [ 3405.431858] [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
> [ 3405.431858] ---[ end trace b4ac1510884bf2bc ]---
> [ 3411.050005] ------------[ cut here ]------------
> [ 3411.054851] WARNING: at /usr/src/linux-2.6.33-rc8-git7/net/sched/sch_generic.c:255 dev_watchdog+0x25e/0x270()
> [ 3411.059546] Hardware name: ProLiant DL360 G5
> [ 3411.061569] NETDEV WATCHDOG: eth0 (bnx2): transmit queue 0 timed out
> [ 3411.064582] Modules linked in: netbomb bnx2 ipmi_devintf loop dm_mod sg sr_mod cdrom ata_piix ahci ipmi_si ipmi_msghandler uhci_hcd qla2xxx libata hpwdt ehci_hcd [last unloaded: bnx2]
> [ 3411.064597] Pid: 0, comm: swapper Tainted: G W 2.6.33-rc8-git7-x86_64 #1
> [ 3411.064599] Call Trace:

Do we have timers running in this environment? The timer in the bnx2
driver, bnx2_timer(), needs to run to provide a heart beat to the
firmware. In netpoll mode without timer interrupts, if we are regularly
calling the NAPI poll function, it should also be able to provide the
heartbeat. Without the heartbeat, the firmware will reset the chip and
result in the NETDEV WATCHDOG.

> [ 3411.064601] <IRQ> [<ffffffff8135f84e>] ? dev_watchdog+0x25e/0x270
> [ 3411.064609] [<ffffffff81039368>] warn_slowpath_common+0x78/0xd0
> [ 3411.064612] [<ffffffff81039444>] warn_slowpath_fmt+0x64/0x70
> [ 3411.064616] [<ffffffff8103486d>] ? default_wake_function+0xd/0x10
> [ 3411.064620] [<ffffffff8119f339>] ? strlcpy+0x49/0x60
> [ 3411.064623] [<ffffffff81349b33>] ? netdev_drivername+0x43/0x50
> [ 3411.064626] [<ffffffff8135f84e>] dev_watchdog+0x25e/0x270
> [ 3411.064630] [<ffffffff8104c000>] ? delayed_work_timer_fn+0x0/0x40
> [ 3411.064633] [<ffffffff8104bf87>] ? __queue_work+0x77/0x90
> [ 3411.064636] [<ffffffff8103558b>] ? scheduler_tick+0x1bb/0x290
> [ 3411.064639] [<ffffffff8135f5f0>] ? dev_watchdog+0x0/0x270
> [ 3411.064642] [<ffffffff810440fc>] run_timer_softirq+0x13c/0x210
> [ 3411.064645] [<ffffffff8105b4b7>] ? clockevents_program_event+0x57/0xa0
> [ 3411.064649] [<ffffffff8103edb6>] __do_softirq+0xa6/0x130
> [ 3411.064652] [<ffffffff81003bcc>] call_softirq+0x1c/0x30
> [ 3411.064655] [<ffffffff81005be5>] do_softirq+0x55/0x90
> [ 3411.064658] [<ffffffff8103eb35>] irq_exit+0x75/0x90
> [ 3411.064661] [<ffffffff8101aeed>] smp_apic_timer_interrupt+0x6d/0xa0
> [ 3411.064664] [<ffffffff81003693>] apic_timer_interrupt+0x13/0x20
> [ 3411.064666] <EOI> [<ffffffff8100b186>] ? mwait_idle+0x66/0x80
> [ 3411.064670] [<ffffffff81001f90>] ? enter_idle+0x20/0x30
> [ 3411.064673] [<ffffffff81002003>] cpu_idle+0x63/0xb0
> [ 3411.064676] [<ffffffff813f2f14>] rest_init+0x74/0x80
> [ 3411.064680] [<ffffffff81880c15>] start_kernel+0x2f8/0x336
> [ 3411.064683] [<ffffffff8188026d>] x86_64_start_reservations+0x7d/0x84
> [ 3411.064686] [<ffffffff81880354>] x86_64_start_kernel+0xe0/0xf2
> [ 3411.064688] ---[ end trace b4ac1510884bf2bd ]---


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Brian Haley on 4 Mar 2010 15:40
Hi Michael,

Michael Chan wrote:
> Do we have timers running in this environment? The timer in the bnx2
> driver, bnx2_timer(), needs to run to provide a heart beat to the
> firmware. In netpoll mode without timer interrupts, if we are regularly
> calling the NAPI poll function, it should also be able to provide the
> heartbeat. Without the heartbeat, the firmware will reset the chip and
> result in the NETDEV WATCHDOG.

We have also been seeing watchdog timeouts with bnx2, below is a
stack trace with Benjamin's debug patch applied. Normally we were
only seeing them under heavy load, but this one was at boot. We haven't
tried the latest firmware/driver from 2.6.33 yet. You can contact me
offline if you need more detailed info.

Thanks,

-Brian


[ 2.428093] bnx2 0000:04:00.0: firmware: requesting bnx2/bnx2-rv2p-06-5.0.0.j3.fw
[ 2.432526] eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 41, node addr 00:1c:c4:e1:cc:ea
[ 2.439520] bnx2 0000:42:00.0: PCI INT A -> GSI 34 (level, low) -> IRQ 34

<snip>

[ 223.805014] ------------[ cut here ]------------
[ 223.805023] WARNING: at net/sched/sch_generic.c:261 dev_watchdog+0x12d/0x1d5()
[ 223.805026] Hardware name: ProLiant DL385 G2
[ 223.805028] NETDEV WATCHDOG: eth0 (bnx2): transmit queue 0 timed out
[ 223.805031] Modules linked in: itapi iptable_filter ip_tables x_tables mptctl ipmi_devintf deflate zlib_deflate ctr twofish twofish_common camellia serpent blowfish cast5 des_generic cbc cryptd aes_x86_64 aes_generic xcbc rmd160 sha256_generic sha1_generic crypto_null af_key dm_snapshot dm_mirror dm_region_hash dm_log dm_mod sg bonding sctp crc32c libcrc32c loop psmouse serio_raw amd64_edac_mod edac_core k8temp container i2c_piix4 i2c_core ipmi_si ipmi_msghandler shpchp pci_hotplug hpilo processor evdev ext3 jbd mbcache ses enclosure sd_mod crc_t10dif ide_cd_mod cdrom ata_generic libata ide_pci_generic usbhid hid mptsas bnx2 mptscsih mptbase scsi_transport_sas serverworks ehci_hcd scsi_mod ide_core ohci_hcd uhci_hcd button thermal fan thermal_sys edd [last unloaded: scsi_wait_scan]
[ 223.805102] Pid: 0, comm: swapper Not tainted 2.6.32-clim-4-amd64 #1
[ 223.805105] Call Trace:
[ 223.805108] <IRQ> [<ffffffff812697a0>] ? dev_watchdog+0x12d/0x1d5
[ 223.805118] [<ffffffff81049914>] warn_slowpath_common+0x77/0xa4
[ 223.805123] [<ffffffff810499b6>] warn_slowpath_fmt+0x64/0x66
[ 223.805128] [<ffffffff81045df7>] ? default_wake_function+0xd/0xf
[ 223.805133] [<ffffffff81035fa7>] ? __wake_up_common+0x46/0x76
[ 223.805138] [<ffffffff8103b414>] ? __wake_up+0x43/0x50
[ 223.805143] [<ffffffff81253829>] ? netdev_drivername+0x43/0x4b
[ 223.805147] [<ffffffff812697a0>] dev_watchdog+0x12d/0x1d5
[ 223.805152] [<ffffffff8105e84a>] ? delayed_work_timer_fn+0x0/0x3d
[ 223.805156] [<ffffffff8105e7ee>] ? __queue_work+0x35/0x3d
[ 223.805159] [<ffffffff81269673>] ? dev_watchdog+0x0/0x1d5
[ 223.805164] [<ffffffff8105655a>] run_timer_softirq+0x1ff/0x2a1
[ 223.805169] [<ffffffff810205a1>] ? lapic_next_event+0x18/0x1c
[ 223.805174] [<ffffffff8104f9e3>] __do_softirq+0xde/0x19f
[ 223.805179] [<ffffffff8100ccec>] call_softirq+0x1c/0x28
[ 223.805183] [<ffffffff8100e8b1>] do_softirq+0x41/0x81
[ 223.805187] [<ffffffff8104f7bd>] irq_exit+0x36/0x75
[ 223.805191] [<ffffffff81020f33>] smp_apic_timer_interrupt+0x88/0x96
[ 223.805195] [<ffffffff8100c6b3>] apic_timer_interrupt+0x13/0x20
[ 223.805198] <EOI> [<ffffffff81027740>] ? native_safe_halt+0x6/0x8
[ 223.805207] [<ffffffff81012da3>] ? default_idle+0x55/0x74
[ 223.805210] [<ffffffff810131ce>] ? c1e_idle+0xf4/0xfb
[ 223.805215] [<ffffffff81065529>] ? atomic_notifier_call_chain+0x13/0x15
[ 223.805219] [<ffffffff8100aeec>] ? cpu_idle+0x5b/0x93
[ 223.805225] [<ffffffff81304144>] ? start_secondary+0x1a8/0x1ac
[ 223.805228] ---[ end trace b04d103e6c8c23de ]---
[ 223.805231] bnx2: eth0 DEBUG: intr_sem[0]
[ 223.805236] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 223.805242] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 223.805245] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 228.805016] bnx2: eth0 DEBUG: intr_sem[0]
[ 228.805023] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 228.805029] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 228.805033] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 233.805014] bnx2: eth0 DEBUG: intr_sem[0]
[ 233.805019] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 233.805024] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 233.805028] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 238.805013] bnx2: eth0 DEBUG: intr_sem[0]
[ 238.805019] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 238.805025] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 238.805029] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 243.805015] bnx2: eth0 DEBUG: intr_sem[0]
[ 243.805021] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 243.805027] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 243.805031] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 248.805014] bnx2: eth0 DEBUG: intr_sem[0]
[ 248.805019] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 248.805025] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 248.805028] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 253.805015] bnx2: eth0 DEBUG: intr_sem[0]
[ 253.805021] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 253.805027] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 253.805031] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 258.805016] bnx2: eth0 DEBUG: intr_sem[0]
[ 258.805022] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 258.805027] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 258.805031] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 263.805013] bnx2: eth0 DEBUG: intr_sem[0]
[ 263.805018] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 263.805023] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 263.805027] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 268.805014] bnx2: eth0 DEBUG: intr_sem[0]
[ 268.805019] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 268.805025] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 268.805028] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 273.805015] bnx2: eth0 DEBUG: intr_sem[0]
[ 273.805022] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 273.805028] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 273.805032] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 278.805012] bnx2: eth0 DEBUG: intr_sem[0]
[ 278.805017] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 278.805023] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 278.805026] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 283.805012] bnx2: eth0 DEBUG: intr_sem[0]
[ 283.805016] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 283.805022] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 283.805025] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 288.805015] bnx2: eth0 DEBUG: intr_sem[0]
[ 288.805022] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 288.805028] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 288.805031] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 293.805011] bnx2: eth0 DEBUG: intr_sem[0]
[ 293.805016] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 293.805022] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 293.805026] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 298.805015] bnx2: eth0 DEBUG: intr_sem[0]
[ 298.805021] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 298.805027] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 298.805030] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 303.805013] bnx2: eth0 DEBUG: intr_sem[0]
[ 303.805020] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 303.805026] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 303.805030] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 308.805013] bnx2: eth0 DEBUG: intr_sem[0]
[ 308.805018] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 308.805023] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 308.805027] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
[ 313.805011] bnx2: eth0 DEBUG: intr_sem[0]
[ 313.805016] bnx2: eth0 DEBUG: EMAC_TX_STATUS[00000008] RPM_MGMT_PKT_CTRL[00000000]
[ 313.805022] bnx2: eth0 DEBUG: MCP_STATE_P0[00000000] MCP_STATE_P1[00000000]
[ 313.805025] bnx2: eth0 DEBUG: HC_STATS_INTERRUPT_STATUS[00000000]
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: [PATCH 0/3] Get maintainer script cleanups
Next: Enhanced reimplemention of the kfifo API request for merge