From: john stultz on
On Wed, 2010-05-26 at 14:44 -0400, Brian Bloniarz wrote:
> On 05/26/2010 12:25 PM, john stultz wrote:
> > Brian: is this something the NTPd folks actually want? Has anyone
> > checked with them before we hand down the solution from high upon on
> > lkml mountain?
>
> I haven't checked, it's been a while since I dealt with
> this problem. The NTP maintainers definitely complain about the
> quick TSC calibration code like it's a bug:
> (e.g. http://www.mail-archive.com/questions(a)lists.ntp.org/msg02079.html).
> Anyway I'll reach out before I spend any time investing in
> a solution that they don't want (and you don't like :).
>
> > Personally I think NTPd should be a little more savvy about how far it
> > trusts the drift file when it starts up. Since I believe its
> > fast-startup mode can quickly estimate the drift well within 100ppm,
> > which is about the maximum variance I've seen from the calibration code.
>
> The workaround we went with was to remove the drift file on
> every reboot. But in our experience, even with iburst, converging takes
> a long time. I don't have hard numbers since it's been a long time since
> I investigated the problem, but we defined failure as >1ms offset syncing
> to a server in our LAN, and a cold NTP boot takes 10-20 hours to get
> there.

Ok. If its been awhile, you may find recent kernels (2.6.31+) are much
faster to converge due to adjustments made to the SHIFT_PLL constant.
This was done explicitly to address issues similar to what you describe
above.

thanks
-john

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: john stultz on
On Wed, 2010-05-26 at 11:51 -0700, H. Peter Anvin wrote:
> On 05/26/2010 11:44 AM, Brian Bloniarz wrote:
> > On 05/26/2010 12:25 PM, john stultz wrote:
> >> Brian: is this something the NTPd folks actually want? Has anyone
> >> checked with them before we hand down the solution from high upon on
> >> lkml mountain?
> >
> > I haven't checked, it's been a while since I dealt with
> > this problem. The NTP maintainers definitely complain about the
> > quick TSC calibration code like it's a bug:
> > (e.g. http://www.mail-archive.com/questions(a)lists.ntp.org/msg02079.html).
> > Anyway I'll reach out before I spend any time investing in
> > a solution that they don't want (and you don't like :).
> >
>
> Yes, Prof. Mills in particular (for those who don't know, he's "Mr.
> NTP") really gets upset about the way Linux does timekeeping.
> Unfortunately it's not clear to me that he's willing to work with us as
> opposed to wanting things to work exactly like BSD, and swive the
> non-NTP users.

Although I suspect his dislike for Linux is historical, as Roman
reworked the ntp code to follow the NTPv4 reference implementation back
in the 2.6.19 timeframe.

However I'd be more then happy to try to address any specific
deficiencies with Linux's NTP implementation if someone can better
express Prof Mills' critiques are.

thanks
-john




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Brian Bloniarz on
On 05/26/2010 03:49 PM, john stultz wrote:
> On Wed, 2010-05-26 at 14:44 -0400, Brian Bloniarz wrote:
>> On 05/26/2010 12:25 PM, john stultz wrote:
>>> Brian: is this something the NTPd folks actually want? Has anyone
>>> checked with them before we hand down the solution from high upon on
>>> lkml mountain?
>>
>> I haven't checked, it's been a while since I dealt with
>> this problem. The NTP maintainers definitely complain about the
>> quick TSC calibration code like it's a bug:
>> (e.g. http://www.mail-archive.com/questions(a)lists.ntp.org/msg02079.html).
>> Anyway I'll reach out before I spend any time investing in
>> a solution that they don't want (and you don't like :).
>>
>>> Personally I think NTPd should be a little more savvy about how far it
>>> trusts the drift file when it starts up. Since I believe its
>>> fast-startup mode can quickly estimate the drift well within 100ppm,
>>> which is about the maximum variance I've seen from the calibration code.
>>
>> The workaround we went with was to remove the drift file on
>> every reboot. But in our experience, even with iburst, converging takes
>> a long time. I don't have hard numbers since it's been a long time since
>> I investigated the problem, but we defined failure as >1ms offset syncing
>> to a server in our LAN, and a cold NTP boot takes 10-20 hours to get
>> there.
>
> Ok. If its been awhile, you may find recent kernels (2.6.31+) are much
> faster to converge due to adjustments made to the SHIFT_PLL constant.
> This was done explicitly to address issues similar to what you describe
> above.

My tests were pre-2.6.31, this is really good to know. I'll take
another look on recent kernels.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: H. Peter Anvin on
On 05/26/2010 01:19 PM, john stultz wrote:
>>
>> Yes, Prof. Mills in particular (for those who don't know, he's "Mr.
>> NTP") really gets upset about the way Linux does timekeeping.
>> Unfortunately it's not clear to me that he's willing to work with us as
>> opposed to wanting things to work exactly like BSD, and swive the
>> non-NTP users.
>
> Although I suspect his dislike for Linux is historical, as Roman
> reworked the ntp code to follow the NTPv4 reference implementation back
> in the 2.6.19 timeframe.
>
> However I'd be more then happy to try to address any specific
> deficiencies with Linux's NTP implementation if someone can better
> express Prof Mills' critiques are.
>

That's the $10M question... I think it starts with asking the NTP
community for advice.

-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Pavel Machek on
Hi!

> > Yes, understood. But the kernel doesn't expose a "gettimeofday
> > performance sucks" flag either. If it did (or in the case of
> > the patch, if tsc_reliable is zero) the application could at least
> > choose to turn off the 10000-100000 timestamps/second and log
> > a message saying "you are running on old hardware so you get
> > fewer features".
>
> I don't think anyone would object to exporting such a flag if
> it's cleanly designed.
>
> Getting the semantics right for that might be somewhat tricky
> though. How is "slow" defined?

Well... if you want to know how fast gettimeofday is, perhaps doing

gettimeofday(); gettimeofday();

is good enough?

If not, perhaps you can export 'how many clocks is gettimeofday
expected to take' variable somewhere, but...

Pavel


> > A CPU-hotplugable system is a good example of a case where
> > the kernel should expose that tsc_reliable is 0. (I've heard
>
> That would mean that a large class of systems which
> are always hotplug capable (even if it's not used)
> would never get fast TSC time.
>
> Wasn't the goal here to be faster?
>
> > anecdotally that CPU hotplug into a QPI or Hypertransport system
> > will have some other interesting challenges, so may require some
> > special kernel parameters anyway.) Even if tsc_reliable were
> > only enabled if a "no-cpu_hotplug" kernel parameter is set,
> > that is still useful. And with cores-per-socket (and even
> > nodes-per-socket) going up seemingly every day, multi-socket
> > systems will likely be an ever smaller percentage of new
> > systems.
>
> Still the people running them will expect as good performance
> as possible.
>
> -Andi
>

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/