From: Denys Fedoryshchenko on
> * Ingo Molnar <mingo(a)elte.hu> wrote:
> with e1000e i get:
>
> 64 bytes from europe (10.0.1.15): icmp_seq=1 ttl=64 time=0.212 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=2 ttl=64 time=0.372 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=3 ttl=64 time=0.815 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=4 ttl=64 time=0.961 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=5 ttl=64 time=0.201 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=6 ttl=64 time=0.788 ms
>
> TCP latencies are fine too - ssh feels snappy again.
>
> it still does not have nearly as good latencies as say forcedeth though:
>
> 64 bytes from mercury (10.0.1.13): icmp_seq=1 ttl=64 time=0.076 ms
> 64 bytes from mercury (10.0.1.13): icmp_seq=2 ttl=64 time=0.085 ms
> 64 bytes from mercury (10.0.1.13): icmp_seq=3 ttl=64 time=0.045 ms
> 64 bytes from mercury (10.0.1.13): icmp_seq=4 ttl=64 time=0.053 ms
>
> that's 10 times better packet latencies.
>
> and even an ancient Realtek RTL-8139 over 10 megabit Ethernet (!) has
> better latencies than the e1000e over 1000 megabit:
>
> 64 bytes from pluto (10.0.1.10): icmp_seq=2 ttl=64 time=0.309 ms
> 64 bytes from pluto (10.0.1.10): icmp_seq=3 ttl=64 time=0.333 ms
> 64 bytes from pluto (10.0.1.10): icmp_seq=4 ttl=64 time=0.329 ms
> 64 bytes from pluto (10.0.1.10): icmp_seq=5 ttl=64 time=0.311 ms
> 64 bytes from pluto (10.0.1.10): icmp_seq=6 ttl=64 time=0.302 ms
>
> is it done intentionally perhaps? I dont think it makes much sense to
> delay rx/tx processing on a completely idle box for such a long time.
Idle box, ICH8 chipset, e1000e, latest git.

MegaRouterCore-KARAM ~ # ping 192.168.20.26
PING 192.168.20.26 (192.168.20.26) 56(84) bytes of data.
64 bytes from 192.168.20.26: icmp_seq=1 ttl=64 time=0.109 ms
64 bytes from 192.168.20.26: icmp_seq=2 ttl=64 time=0.134 ms
64 bytes from 192.168.20.26: icmp_seq=3 ttl=64 time=0.120 ms
64 bytes from 192.168.20.26: icmp_seq=4 ttl=64 time=0.117 ms
64 bytes from 192.168.20.26: icmp_seq=5 ttl=64 time=0.117 ms
64 bytes from 192.168.20.26: icmp_seq=6 ttl=64 time=0.113 ms

Disabling interrupt moderation
MegaRouterCore-KARAM ~ # ethtool -C eth0 rx-usecs 0
MegaRouterCore-KARAM ~ # ping 192.168.20.26
PING 192.168.20.26 (192.168.20.26) 56(84) bytes of data.
64 bytes from 192.168.20.26: icmp_seq=1 ttl=64 time=0.072 ms
64 bytes from 192.168.20.26: icmp_seq=2 ttl=64 time=0.091 ms
64 bytes from 192.168.20.26: icmp_seq=3 ttl=64 time=0.066 ms
64 bytes from 192.168.20.26: icmp_seq=4 ttl=64 time=0.065 ms
64 bytes from 192.168.20.26: icmp_seq=5 ttl=64 time=0.077 ms
64 bytes from 192.168.20.26: icmp_seq=6 ttl=64 time=0.073 ms

Maybe try the same?
ethtool -C eth0 rx-usecs 0

--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* Denys Fedoryshchenko <denys(a)visp.net.lb> wrote:

> > * Ingo Molnar <mingo(a)elte.hu> wrote:
> > with e1000e i get:
> >
> > 64 bytes from europe (10.0.1.15): icmp_seq=1 ttl=64 time=0.212 ms
> > 64 bytes from europe (10.0.1.15): icmp_seq=2 ttl=64 time=0.372 ms
> > 64 bytes from europe (10.0.1.15): icmp_seq=3 ttl=64 time=0.815 ms
> > 64 bytes from europe (10.0.1.15): icmp_seq=4 ttl=64 time=0.961 ms
> > 64 bytes from europe (10.0.1.15): icmp_seq=5 ttl=64 time=0.201 ms
> > 64 bytes from europe (10.0.1.15): icmp_seq=6 ttl=64 time=0.788 ms
> >
> > TCP latencies are fine too - ssh feels snappy again.
> >
> > it still does not have nearly as good latencies as say forcedeth though:
> >
> > 64 bytes from mercury (10.0.1.13): icmp_seq=1 ttl=64 time=0.076 ms
> > 64 bytes from mercury (10.0.1.13): icmp_seq=2 ttl=64 time=0.085 ms
> > 64 bytes from mercury (10.0.1.13): icmp_seq=3 ttl=64 time=0.045 ms
> > 64 bytes from mercury (10.0.1.13): icmp_seq=4 ttl=64 time=0.053 ms
> >
> > that's 10 times better packet latencies.
> >
> > and even an ancient Realtek RTL-8139 over 10 megabit Ethernet (!) has
> > better latencies than the e1000e over 1000 megabit:
> >
> > 64 bytes from pluto (10.0.1.10): icmp_seq=2 ttl=64 time=0.309 ms
> > 64 bytes from pluto (10.0.1.10): icmp_seq=3 ttl=64 time=0.333 ms
> > 64 bytes from pluto (10.0.1.10): icmp_seq=4 ttl=64 time=0.329 ms
> > 64 bytes from pluto (10.0.1.10): icmp_seq=5 ttl=64 time=0.311 ms
> > 64 bytes from pluto (10.0.1.10): icmp_seq=6 ttl=64 time=0.302 ms
> >
> > is it done intentionally perhaps? I dont think it makes much sense to
> > delay rx/tx processing on a completely idle box for such a long time.
> Idle box, ICH8 chipset, e1000e, latest git.
>
> MegaRouterCore-KARAM ~ # ping 192.168.20.26
> PING 192.168.20.26 (192.168.20.26) 56(84) bytes of data.
> 64 bytes from 192.168.20.26: icmp_seq=1 ttl=64 time=0.109 ms
> 64 bytes from 192.168.20.26: icmp_seq=2 ttl=64 time=0.134 ms
> 64 bytes from 192.168.20.26: icmp_seq=3 ttl=64 time=0.120 ms
> 64 bytes from 192.168.20.26: icmp_seq=4 ttl=64 time=0.117 ms
> 64 bytes from 192.168.20.26: icmp_seq=5 ttl=64 time=0.117 ms
> 64 bytes from 192.168.20.26: icmp_seq=6 ttl=64 time=0.113 ms

ok, that looks much better! i have another box with e1000, ich7:

64 bytes from titan (10.0.1.14): icmp_seq=5 ttl=64 time=0.345 ms
64 bytes from titan (10.0.1.14): icmp_seq=6 ttl=64 time=1.03 ms
64 bytes from titan (10.0.1.14): icmp_seq=7 ttl=64 time=0.383 ms
64 bytes from titan (10.0.1.14): icmp_seq=8 ttl=64 time=0.320 ms
64 bytes from titan (10.0.1.14): icmp_seq=9 ttl=64 time=0.996 ms
64 bytes from titan (10.0.1.14): icmp_seq=10 ttl=64 time=0.248 ms

> Disabling interrupt moderation
> MegaRouterCore-KARAM ~ # ethtool -C eth0 rx-usecs 0
> MegaRouterCore-KARAM ~ # ping 192.168.20.26
> PING 192.168.20.26 (192.168.20.26) 56(84) bytes of data.
> 64 bytes from 192.168.20.26: icmp_seq=1 ttl=64 time=0.072 ms
> 64 bytes from 192.168.20.26: icmp_seq=2 ttl=64 time=0.091 ms
> 64 bytes from 192.168.20.26: icmp_seq=3 ttl=64 time=0.066 ms
> 64 bytes from 192.168.20.26: icmp_seq=4 ttl=64 time=0.065 ms
> 64 bytes from 192.168.20.26: icmp_seq=5 ttl=64 time=0.077 ms
> 64 bytes from 192.168.20.26: icmp_seq=6 ttl=64 time=0.073 ms
>
> Maybe try the same?
> ethtool -C eth0 rx-usecs 0

well i tend not to tweak my drivers with such options because i want to
experience and test what 99.9% of our users will experience in the
field. The reality is that if it's not the default behavior, it's almost
as if it didnt exist at all.

but even with that tune on e1000e (on the t60, ich7) i still get rather
large numbers:

earth4:~/s> ping eu
PING europe (10.0.1.15) 56(84) bytes of data.
64 bytes from europe (10.0.1.15): icmp_seq=1 ttl=64 time=0.250 ms
64 bytes from europe (10.0.1.15): icmp_seq=2 ttl=64 time=0.250 ms
64 bytes from europe (10.0.1.15): icmp_seq=3 ttl=64 time=0.225 ms
64 bytes from europe (10.0.1.15): icmp_seq=4 ttl=64 time=0.932 ms
64 bytes from europe (10.0.1.15): icmp_seq=5 ttl=64 time=0.251 ms
64 bytes from europe (10.0.1.15): icmp_seq=6 ttl=64 time=0.915 ms
64 bytes from europe (10.0.1.15): icmp_seq=7 ttl=64 time=0.250 ms
64 bytes from europe (10.0.1.15): icmp_seq=8 ttl=64 time=0.238 ms
64 bytes from europe (10.0.1.15): icmp_seq=9 ttl=64 time=0.390 ms
64 bytes from europe (10.0.1.15): icmp_seq=10 ttl=64 time=0.260 ms

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: David Miller on
From: "Kok, Auke" <auke-jan.h.kok(a)intel.com>
Date: Wed, 18 Jun 2008 14:25:28 -0700

> You only complain and do not provide a single solution to your
> problem. Your continued screaming and whining is totally not
> productive nor constructive at all, and frankly is insulting since
> you completely ignore the fact that we worked with the the community
> more than two-year to come to some maintainable situation.

I completely and %100 agree with you.

> Until I see such a thing I can't do much else than ignore
> your childish whining.

Join the club.

I'm also ignoring everything he writes until he changes his modus
operandi to one that is more constructive than the pure hurtful
whining he is emitting as of late.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Denys Fedoryshchenko on
On Thursday 19 June 2008 01:05, Ingo Molnar wrote:
>
> ok, that looks much better! i have another box with e1000, ich7:
>
> 64 bytes from titan (10.0.1.14): icmp_seq=5 ttl=64 time=0.345 ms
> 64 bytes from titan (10.0.1.14): icmp_seq=6 ttl=64 time=1.03 ms
> 64 bytes from titan (10.0.1.14): icmp_seq=7 ttl=64 time=0.383 ms
> 64 bytes from titan (10.0.1.14): icmp_seq=8 ttl=64 time=0.320 ms
> 64 bytes from titan (10.0.1.14): icmp_seq=9 ttl=64 time=0.996 ms
> 64 bytes from titan (10.0.1.14): icmp_seq=10 ttl=64 time=0.248 ms
Maybe there is some flow-control involved?
ethtool -S eth0 ?

This is Interrupt throttling i guess in e1000. In e1000 also parameters, but available only on insmod stage
parm: TxIntDelay:Transmit Interrupt Delay (array of int)
parm: TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)
parm: RxIntDelay:Receive Interrupt Delay (array of int)
parm: RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)
parm: InterruptThrottleRate:Interrupt Throttling Rate (array of int)


> well i tend not to tweak my drivers with such options because i want to
> experience and test what 99.9% of our users will experience in the
> field. The reality is that if it's not the default behavior, it's almost
> as if it didnt exist at all.

Each coin have two sides. On one side - low latencies(difference 1ms, it is matter anywhere?)
, on another - performance.


>
> but even with that tune on e1000e (on the t60, ich7) i still get rather
> large numbers:
>
> earth4:~/s> ping eu
> PING europe (10.0.1.15) 56(84) bytes of data.
> 64 bytes from europe (10.0.1.15): icmp_seq=1 ttl=64 time=0.250 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=2 ttl=64 time=0.250 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=3 ttl=64 time=0.225 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=4 ttl=64 time=0.932 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=5 ttl=64 time=0.251 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=6 ttl=64 time=0.915 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=7 ttl=64 time=0.250 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=8 ttl=64 time=0.238 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=9 ttl=64 time=0.390 ms
> 64 bytes from europe (10.0.1.15): icmp_seq=10 ttl=64 time=0.260 ms


Is all this hosts on same switch? Is the switch manageable or not?
For example i am having problems with packetloss on long fiber link between two cheap Linksys switches.
Without flow-control i cannot survive, and as result i have 1-2ms additional delay on load, and +-0.500ms jitter "inside" this switches (probably from switches).

There is many things matter. Maybe even processor sleep latencies involved? bus latency, PCI latency, whatever.

Also on laptops is dynamic frequency running (Speedstep)
with 600 Mhz PentiumM (Speedstep - ondemand)
64 bytes from 127.0.0.1: icmp_seq=17 ttl=64 time=0.017 ms
full speed 1.7 Ghz
64 bytes from 127.0.0.1: icmp_seq=33 ttl=64 time=0.007 ms

on network also i see difference -0.030ms when i am running burnP6 (from CPUburn package).





--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Ingo Molnar on

* Kok, Auke <auke-jan.h.kok(a)intel.com> wrote:

> You only complain and do not provide a single solution to your
> problem. [...]

i have reported the problem and even provided a fix.

I have triggered an e1000/e1000e related problem that got introduced in
the v2.6.25 merge window - one of my testboxes came up with no
networking and it took me an hour to figure out why. (i wasnt
particularly focusing on e1000, i just happened to hit that bug in 9
million lines of Linux kernel code)

I have reported it here, two and a half months ago:

http://lkml.org/lkml/2008/4/8/256

I even showed you which commit introduced the problem and gave you a
oneliner fix that i tested (it solved the problem):

http://bugzilla.kernel.org/attachment.cgi?id=15704&amp;action=view

You were Cc:-ed to that. (attached below again for reference) The bug
was added to the regression list of v2.6.25. I never expected to spend
more than 10 minutes on this problem once i found out what's happening -
we fix dozens of bugs like this per stable kernel release.

I just checked latest -git, my fix is still not upstream (or any
equivalent solution - i really dont mind how it's solved and i'm not
maintaining this code).

no alternative patch was sent to me - i offered to test any solution
back then.

FYI, since i first reported it i've been hit by that problem roughly a
dozen times. (it happened sporadically so i forgot about it - until i
again had a system come up with no networking.) It caused me lost time
and lost work that could have been spent on better things.

Ingo

------------------------>
Subject: e1000=y && e1000e=m regression fix
From: Ingo Molnar <mingo(a)elte.hu>
Date: Wed Apr 09 21:09:35 CEST 2008

fix a regression from v2.6.24: do not transfer the e1000e PCI IDs from
e1000 to e1000e if e1000 is built-in and e1000e is a module.

Built-in drivers take precedence over modules in many ways - and in this
case it's clear that the user intended the e1000 driver to be the
primary one. "Silently change behavior and break existing configs" is
never a good migration strategy. Most users will use distro kernels that
are not affected by this problem at all - nor are they affected by this
patch - but this problem can hit users and developers who build their
kernels themselves and migrate from v2.6.24 to v2.6.25.

this fixes: http://bugzilla.kernel.org/show_bug.cgi?id=10427

Signed-off-by: Ingo Molnar <mingo(a)elte.hu>
---
drivers/net/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-x86.q/drivers/net/Kconfig
===================================================================
--- linux-x86.q.orig/drivers/net/Kconfig
+++ linux-x86.q/drivers/net/Kconfig
@@ -2022,7 +2022,7 @@ config E1000E
will be called e1000e.

config E1000E_ENABLED
- def_bool E1000E != n
+ def_bool E1000E = y || ((E1000E != n) && (E1000 = E1000E))

config IP1000
tristate "IP1000 Gigabit Ethernet support"
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/