From: Philip Langdale on
On Fri, 05 Feb 2010 12:45:21 -0500 (EST)
Len Brown <lenb(a)kernel.org> wrote:

> Jeff,
> What do you see if you apply just the patch below?
>
> Also, in addition to "powertop -d" to show what the kernel requests,
> please run turbostat to show what the hardware actually did:
>
> http://userweb.kernel.org/~lenb/acpi/utils/pmtools-latest/turbostat/turbostat.c
>
> eg.
> # turbostat -d -v sleep 5
>
> thanks,
> -Len Brown, Intel Open Source Technology Center
> ---

To resurrect this thread...

I have a giga-byte GA-P55M-UD4 motherboard and I have this same problem
as well. Len's patch "works" in that I see C6 being used, but it also
cripples the system - if I do a make -j16 kernel build, I see most jobs
serialized onto one or two cores. Without the patch, I see the
full utilization of all 8 hyper-threads as expected.

Now, gigabyte have already b0rked these boards up by using the UHCI
controllers on the PCH instead of the rate matching hubs. Maybe that's
directly the cause of BM activity - maybe they screwed something else
up - is it possible for BIOS/ACPI mistakes to lead to this behaviour?

Jeff - is your board gigabyte too?

--phil

> diff --git a/drivers/acpi/processor_idle.c
> b/drivers/acpi/processor_idle.c index 7c0441f..f528625 100644
> --- a/drivers/acpi/processor_idle.c
> +++ b/drivers/acpi/processor_idle.c
> @@ -763,7 +763,7 @@ static const struct file_operations
> acpi_processor_power_fops = { static int acpi_idle_bm_check(void)
> {
> u32 bm_status = 0;
> -
> +return bm_status;
> acpi_read_bit_register(ACPI_BITREG_BUS_MASTER_STATUS,
> &bm_status); if (bm_status)
> acpi_write_bit_register(ACPI_BITREG_BUS_MASTER_STATUS,
> 1);
>
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to majordomo(a)vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
>




--phil
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Len Brown on

On Mon, 26 Apr 2010, Philip Langdale wrote:

> On Fri, 05 Feb 2010 12:45:21 -0500 (EST)
> Len Brown <lenb(a)kernel.org> wrote:
>
> > Jeff,
> > What do you see if you apply just the patch below?
> >
> > Also, in addition to "powertop -d" to show what the kernel requests,
> > please run turbostat to show what the hardware actually did:
> >
> > http://userweb.kernel.org/~lenb/acpi/utils/pmtools-latest/turbostat/turbostat.c
> >
> > eg.
> > # turbostat -d -v sleep 5
> >
> > thanks,
> > -Len Brown, Intel Open Source Technology Center
> > ---
>
> To resurrect this thread...
>
> I have a giga-byte GA-P55M-UD4 motherboard and I have this same problem
> as well. Len's patch "works" in that I see C6 being used, but it also
> cripples the system - if I do a make -j16 kernel build, I see most jobs
> serialized onto one or two cores. Without the patch, I see the
> full utilization of all 8 hyper-threads as expected.

Curious failure.
I could imagine that there is something in the design of this board
where we want to not enter a deep C-state, and thus the board and
Linux are doing the right thing by avoiding the C-state.
However, ignoring the bm-status check and blindly going to that state
I would expect to impact throughput and latency, but don't see
how that might 'serialize' the workload or otherwise cause it
to use some cores and not others.

It is possible that we jump into those deep states just to be
immediately forced to jump right back out. You'd see this in
high usage counts under /sys/devices/system/cpu/cpu*/cpuidle

turbostat, of course, would tell you the actual residency in those states.
Of course there is a twist... The hardware has a feature to recognize
thrashing and may demote an OS request for a deep state into
an actual hardware request for a shallower state. this is one reason
that the output of powertop (request) and turbostat (result)
may be different.

cheers,
-Len


> Now, gigabyte have already b0rked these boards up by using the UHCI
> controllers on the PCH instead of the rate matching hubs. Maybe that's
> directly the cause of BM activity - maybe they screwed something else
> up - is it possible for BIOS/ACPI mistakes to lead to this behaviour?
>
> Jeff - is your board gigabyte too?
>
> --phil
>
> > diff --git a/drivers/acpi/processor_idle.c
> > b/drivers/acpi/processor_idle.c index 7c0441f..f528625 100644
> > --- a/drivers/acpi/processor_idle.c
> > +++ b/drivers/acpi/processor_idle.c
> > @@ -763,7 +763,7 @@ static const struct file_operations
> > acpi_processor_power_fops = { static int acpi_idle_bm_check(void)
> > {
> > u32 bm_status = 0;
> > -
> > +return bm_status;
> > acpi_read_bit_register(ACPI_BITREG_BUS_MASTER_STATUS,
> > &bm_status); if (bm_status)
> > acpi_write_bit_register(ACPI_BITREG_BUS_MASTER_STATUS,
> > 1);
> >
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Jeff Garrett on
On Mon, Apr 26, 2010 at 07:40:02PM -0700, Philip Langdale wrote:
> On Fri, 05 Feb 2010 12:45:21 -0500 (EST)
> Len Brown <lenb(a)kernel.org> wrote:
> > Jeff,
> > What do you see if you apply just the patch below?
> >
> > Also, in addition to "powertop -d" to show what the kernel requests,
> > please run turbostat to show what the hardware actually did:
> >
> > http://userweb.kernel.org/~lenb/acpi/utils/pmtools-latest/turbostat/turbostat.c
> >
> > eg.
> > # turbostat -d -v sleep 5
> >
> > thanks,
> > -Len Brown, Intel Open Source Technology Center
> > ---
>
> To resurrect this thread...
>
> I have a giga-byte GA-P55M-UD4 motherboard and I have this same problem
> as well. Len's patch "works" in that I see C6 being used, but it also
> cripples the system - if I do a make -j16 kernel build, I see most jobs
> serialized onto one or two cores. Without the patch, I see the
> full utilization of all 8 hyper-threads as expected.
>
> Now, gigabyte have already b0rked these boards up by using the UHCI
> controllers on the PCH instead of the rate matching hubs. Maybe that's
> directly the cause of BM activity - maybe they screwed something else
> up - is it possible for BIOS/ACPI mistakes to lead to this behaviour?
>
> Jeff - is your board gigabyte too?
>
> --phil

My board identifies it as a Dell. No idea if they rebranded a gigabyte.

The patch seems to work for me as well, powertop shows 97.5% c3,
turbostat shows 93.6% c6 now. I do get weird latency spikes (on I/O)
from time to time.

When I was investigating, I completely configured USB off, and it still
wouldn't go into deep sleep. Not sure how well that meshes with your
UHCI theory.

-Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Philip Langdale on
On Tue, 27 Apr 2010 03:26:34 -0400 (EDT)
Len Brown <lenb(a)kernel.org> wrote:

>
>
> Curious failure.
> I could imagine that there is something in the design of this board
> where we want to not enter a deep C-state, and thus the board and
> Linux are doing the right thing by avoiding the C-state.
> However, ignoring the bm-status check and blindly going to that state
> I would expect to impact throughput and latency, but don't see
> how that might 'serialize' the workload or otherwise cause it
> to use some cores and not others.

Hmm - and now I can't reproduce it. I got proper parallelization across
the kernel compile. I guess some sort of runtime state was messed up,
and I obviously lost that then I rebooted. :-/

> It is possible that we jump into those deep states just to be
> immediately forced to jump right back out. You'd see this in
> high usage counts under /sys/devices/system/cpu/cpu*/cpuidle
>
> turbostat, of course, would tell you the actual residency in those
> states. Of course there is a twist... The hardware has a feature to
> recognize thrashing and may demote an OS request for a deep state into
> an actual hardware request for a shallower state. this is one reason
> that the output of powertop (request) and turbostat (result)
> may be different.

Without the patch, Turbostat showed C3 residency of 99% for most
hyper-threads with one or two getting ~15% C6 residency. PC3 was 75%.
Cores were at their lowest P state.

With the patch, C6 residency is 99%, PC6 is 75% and 7 hyper-threads at
lowest P state with one stubborning running at a higher level.

I have a very similarly configured machine with an asus motherboard and
it doesn't have this problem - which is another reason I'm wondering if
it's an OEM screwup.

--phil
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Philip Langdale on
On Tue, 27 Apr 2010 07:47:03 -0500
jeff(a)jgarrett.org (Jeff Garrett) wrote:


> My board identifies it as a Dell. No idea if they rebranded a
> gigabyte.
>
> The patch seems to work for me as well, powertop shows 97.5% c3,
> turbostat shows 93.6% c6 now. I do get weird latency spikes (on I/O)
> from time to time.
>
> When I was investigating, I completely configured USB off, and it
> still wouldn't go into deep sleep. Not sure how well that meshes
> with your UHCI theory.

Does your board expose UHCI controllers or just EHCI with the rate
matching hubs? When you say 'configured USB off'?, do you mean off
in the BIOS or just no drivers?

--phil
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/