Implementation of system time

[bzt]

I disagree. Timestamps are important and must have far better precision than one second.

Not that I know of, you don't. Can you name one protocol where it really matters? Internet time is second precision (see rfc822, rfc2822, and even rfc3339 states in section 5.3 that "A format which includes rarely used options is likely to cause interoperability problems. [...] The format defined below includes only one rarely used option: fractions of a second.").

You use them to log communication or events, for instance, and then you want decent precision (at least milliseconds, but I have microseconds).

That's why struct time in POSIX has two fields: a second precision (which comes from the wallclock and must be accurate), and a nanosec field which can be implemented with whatever precision the hardware supports (and allowed to be less accurate, it doesn't matter if subsec is a bit off).

You also want a wall clock, but it might be implemented by adding an offset to the system clock

And right there, that's where you lost accuracy for good. System time CAN'T be implemented by adding offsets, that just doesn't work. Not on real hardware, but gets a lot worse when you try to do this on a VM. (And indeed, many hobby OS loose keeping the time in a VM, I bet rdos does too. For the records, they will lost it on real hardware too, just after such a long time that their developer never notices, but eventually they will without a doubt.)

You can also compensate for a slightly unstable system time by slowly modifying the offset, or synching it with RTC, NTP, or some other time reference.

You're confusing your offsets with NTP protocol's drift. What RTP does, is knowing the exact time from an external universal clock source (typically a nuclear-clock), but when system time and that external clock source differs, it does not set that time at once, rather iterates to it in smaller steps. This has nothing to do with keeping the exact time using an unstable system time with whatever offsets (that's just not possible without an external clock source).
FYI, the RTC chip is such an external clock source, operates independently to the PC and the CPU, and works even when your computer is turned off. That's why your system time IRQ handler should read the time from CMOS, and never adding offsets.

Cheers,
bzt

[rdos]

I disagree. Timestamps are important and must have far better precision than one second.

Not that I know of, you don't. Can you name one protocol where it really matters? Internet time is second precision (see rfc822, rfc2822, and even rfc3339 states in section 5.3 that "A format which includes rarely used options is likely to cause interoperability problems. [...] The format defined below includes only one rarely used option: fractions of a second.").

I typically use NTP, not Internet time.

You use them to log communication or events, for instance, and then you want decent precision (at least milliseconds, but I have microseconds).

That's why struct time in POSIX has two fields: a second precision (which comes from the wallclock and must be accurate), and a nanosec field which can be implemented with whatever precision the hardware supports (and allowed to be less accurate, it doesn't matter if subsec is a bit off).

If you want to log the timing of events, you cannot have the sub-second part drifting. It will result in incorrect calculations of time between events. Also, the sub-second part naturally must go through zero as the second part increase, otherwise you have inconsistent time. What this means is that both must have the same precision.

You also want a wall clock, but it might be implemented by adding an offset to the system clock

And right there, that's where you lost accuracy for good. System time CAN'T be implemented by adding offsets, that just doesn't work.

I'm unsure if you understood the method. If I set system time from the RTC at boot up, but later discover that it runs five seconds behind, then I simply set the "offset" to five seconds. When I read out the wall clock I take system time and add the "offset" (five seconds) and now I have the correct wall clock, and still a consistent system time that increase properly.

This in no way affects the acuracy of system time or the wall clock. It only allows you to fix it if the system time for some reason is drifting or if the RTC was incorrect to begin with.

Not on real hardware, but gets a lot worse when you try to do this on a VM. (And indeed, many hobby OS loose keeping the time in a VM, I bet rdos does too. For the records, they will lost it on real hardware too, just after such a long time that their developer never notices, but eventually they will without a doubt.)

RDOS doesn't run on emulators.

You can also compensate for a slightly unstable system time by slowly modifying the offset, or synching it with RTC, NTP, or some other time reference.

You're confusing your offsets with NTP protocol's drift. What RTP does, is knowing the exact time from an external universal clock source (typically a nuclear-clock), but when system time and that external clock source differs, it does not set that time at once, rather iterates to it in smaller steps. This has nothing to do with keeping the exact time using an unstable system time with whatever offsets (that's just not possible without an external clock source).
FYI, the RTC chip is such an external clock source, operates independently to the PC and the CPU, and works even when your computer is turned off. That's why your system time IRQ handler should read the time from CMOS, and never adding offsets.

You cannot use the RTC to effectively sync system time. It simply has too low precision. You could read it in a loop to decide when it increments the count, but the next time you must do it again if you want the exact point. If you have a high precision system time you could calculate drift by checking when it increments, but it is wasting a lot of CPU time.

In my home controller I ask an NTP server for the time once an hour or so and calibrate the difference to system time and this way I do have the exact time with high precision AND high accuracy.

[nullplan]

I suspect that Linux puts the burden of adapting to the reference on the application developer instead, that needs to figure out which frequency the TSC runs at.

Not particularly, no. You call clock_gettime(CLOCK_MONOTONIC), you get the clock time in seconds and nanoseconds, and how fast the TSC runs is not important to the application. And before you ask for system call delay, clock_gettime() is implemented in the VDSO on most architectures, including x86. That essentially means the kernel maps in a page of code and a page of data in front of that. Userspace can call a function in the VDSO, and it reads the time from the data page and corrects it with hardware counters. Nanosecond precision without a syscall.

Not that I know of, you don't. Can you name one protocol where it really matters?

ext2/3/4 can record file times with nanosecond precision. Actually, most FSes can. Except FAT, which famously can only store times to a precision of two seconds. And that is important for make, which tries to figure out which of two files is newer, which is a problem when you only have second precision but an otherwise fast computer, so a lot of files are created at the same time.

Syslog is another protocol that comes to mind. The number of times I was foiled by second precision log messages in my day job is not even funny anymore.

[rdos]

I suspect that Linux puts the burden of adapting to the reference on the application developer instead, that needs to figure out which frequency the TSC runs at.

Not particularly, no. You call clock_gettime(CLOCK_MONOTONIC), you get the clock time in seconds and nanoseconds, and how fast the TSC runs is not important to the application. And before you ask for system call delay, clock_gettime() is implemented in the VDSO on most architectures, including x86. That essentially means the kernel maps in a page of code and a page of data in front of that. Userspace can call a function in the VDSO, and it reads the time from the data page and corrects it with hardware counters. Nanosecond precision without a syscall.

Interesting. This is a bit how I might eventually implement it too, along with Posix timers.

ext2/3/4 can record file times with nanosecond precision. Actually, most FSes can. Except FAT, which famously can only store times to a precision of two seconds. And that is important for make, which tries to figure out which of two files is newer, which is a problem when you only have second precision but an otherwise fast computer, so a lot of files are created at the same time.

Exactly

Syslog is another protocol that comes to mind. The number of times I was foiled by second precision log messages in my day job is not even funny anymore.

Only having seconds in logs sucks big time. It's not the exact time that matters most, rather that you can see the timing of events and relate them to real time.

[rdos]

You're confusing your offsets with NTP protocol's drift. What RTP does, is knowing the exact time from an external universal clock source (typically a nuclear-clock), but when system time and that external clock source differs, it does not set that time at once, rather iterates to it in smaller steps. This has nothing to do with keeping the exact time using an unstable system time with whatever offsets (that's just not possible without an external clock source).

It's certainly possible to let the system time drift slowly so it gets closer to the wall clock, but only if the adjustment is small. If you set the wall clock from the RTC, it can be up to a second off (or more if it is incorrectly set) because of the poor precision, and this large differences cannot be drifted. By using the RTC IRQ you could get the time at a more exact point, but interupt latency will introduce random errors. Difference from NTP can be drifted too rather than applied directly, but only if you have a high precision system time.

Drifting the system time can be done by having a parts counter which is added / subtracted by a part of the elapsed time from each readout. The factors are modified when the drift is measured and this creates an algorithm that can give a more stable system time. I have this when I sync the system time with the RTC IRQ.

[zberry7]

ext2/3/4 can record file times with nanosecond precision. Actually, most FSes can. Except FAT, which famously can only store times to a precision of two seconds. And that is important for make, which tries to figure out which of two files is newer, which is a problem when you only have second precision but an otherwise fast computer, so a lot of files are created at the same time.

Syslog is another protocol that comes to mind. The number of times I was foiled by second precision log messages in my day job is not even funny anymore.

Honestly I’m more confused than when I first asked the question haha. I looked into what Linux does (briefly), and it checks if the TSC counters are synchronized between CPUs on the system, and also if it’s invariant using CPUID, according to the source I was reading. Obviously falling back on other clock sources when they’re not invariant or consistent between CPUs.

I’ve erased the code I’ve written at least 3 times the past few days, really I was just hoping to have a single clock source that’s both as accurate as possible while being nanosecond precise. I’ve seen people saying to just have two clocks, one clock, use IRQs, don’t use IRQs, use TSC, don’t use TSC.

I don’t see the point of keeping a separate clock for wall time when I can get potentially highly precise results with invariant TSC, or HPET, and just correct for drift periodically. As for the implementation, whenever someone wants to read the time, spin lock, read counter, compare to last value and check if it overflowed, convert to nanoseconds, add to some global uint64_t, release the lock and return. At least that’s what I gleaned from earlier responses.

Then I if I need the wall clock, I can figure out the number of nanoseconds from a reference date to when I started the clock. And have drift correction every so often to ensure the wall clock stays fairly accurate.

[nullplan]

Honestly I’m more confused than when I first asked the question haha

Well, you asked the internet for an opinion. It should not be surprising there are many different ones out there.

I’ve erased the code I’ve written at least 3 times the past few days, really I was just hoping to have a single clock source that’s both as accurate as possible while being nanosecond precise. I’ve seen people saying to just have two clocks, one clock, use IRQs, don’t use IRQs, use TSC, don’t use TSC.

There is no single option that is always present and precise. You have to look through the different possibilities to select a source on the computer you are currently executing on. Familiarize yourself with the possibilities, then make a choice. I would suggest using TSC if invariant, else HPET, ACPI PM timer, LAPIC timer, or PIT, in that order. Other people may suggest different things. Make your own choice. It is your OS.

As for the implementation, whenever someone wants to read the time, spin lock, read counter, compare to last value and check if it overflowed, convert to nanoseconds, add to some global uint64_t, release the lock and return.

In a good counter, you can discount overflow as a possibility. A 3 GHz 64 bit counter will overflow once every century or so. So I don't even handle that case. If someone wants to sue me in a hundred years, go right ahead.

[bzt]

I typically use NTP, not Internet time.

That two has absolutely nothing to do with each other. It's like saying you're not using a browser, because you use SVG.

If you want to log the timing of events, you cannot have the sub-second part drifting. It will result in incorrect calculations of time between events. Also, the sub-second part naturally must go through zero as the second part increase, otherwise you have inconsistent time. What this means is that both must have the same precision.

You are wrong. First, when the second part increases, you simply zero the fractional part. Second, no matter how hard you want it, no machine can give you nanosecond precision. For example, on a machine with RTC only, the fractional part might be in nanosec units, but that will be updated in best case every 1/ 32768th of a second. Keeping the fractional part actually increasing by 1 every nanosec requires so huge overhead that no mainline kernels do that (assuming the hardware can handle 1 billion IRQs per second in the first place, which is very unlikely).

I'm unsure if you understood the method. If I set system time from the RTC at boot up, but later discover that it runs five seconds behind, then I simply set the "offset" to five seconds. When I read out the wall clock I take system time and add the "offset" (five seconds) and now I have the correct wall clock, and still a consistent system time that increase properly.

No, I haven't misunderstood, your wallclock will be off 100% guaranteed. What you are not keeping in mind, you say "discover that it runs five seconds behind", without realizing what that actually means and how that can be implemented. If you do, then you'll get to the offset-less solution I was mentioning.

This in no way affects the acuracy of system time or the wall clock.

Yes, it does, you said it yourself that you might "discover that it runs behind". That could not happen if it were accurate, but getting behind happens because it's not.

RDOS doesn't run on emulators.

That explains everything. That's why you're unaware of these things, because you haven't tested RDOS properly under different circumstances. A properly written OS should have no issues running in an emulator.

ext2/3/4 can record file times with nanosecond precision. Actually, most FSes can. Except FAT, which famously can only store times to a precision of two seconds. And that is important for make, which tries to figure out which of two files is newer, which is a problem when you only have second precision but an otherwise fast computer, so a lot of files are created at the same time.

Which is totally and completely irrelevant, because the VFS can handle timestamp with nanosec precision regardless to the actual FS used. If a process needs nanosec precision on file timestamps, then it will only communicating with the VFS-node only, with in-memory file records only, there's simple no time to write the files to the storage and read them back in less than a nanosec. That's just not possible.

Syslog is another protocol that comes to mind.

RFC5425 states it pretty clearly, that time format must comply with RFC3339, which in turn, as I've already quoted "A format which includes rarely used options is likely to cause interoperability problems. [...] The format defined below includes only one rarely used option: fractions of a second."

In other words, using fractions of a second with syslog is an Undefined Behaviour, threat it as such. (Not to mention if you're correlating two loglines there's absolutely no guarantee that their originator machines had the same time precision, most likely they haven't.) I was the IT development manager at the company which produced syslog-ng, I know a thing or two about syslogs Sadly a couple of years ago syslog-ng was sold to the sharks, and went down on the toilet.

Cheers,
bzt

[rdos]

You are wrong. First, when the second part increases, you simply zero the fractional part.

So, you might not have the full range of nanoseconds, or more than 1G of them? Not really acceptable, but at least better than inconsistent.

Second, no matter how hard you want it, no machine can give you nanosecond precision. For example, on a machine with RTC only, the fractional part might be in nanosec units, but that will be updated in best case every 1/ 32768th of a second. Keeping the fractional part actually increasing by 1 every nanosec requires so huge overhead that no mainline kernels do that (assuming the hardware can handle 1 billion IRQs per second in the first place, which is very unlikely).

Wrong. When you read system time you also read the hardware (TSC, HPET or PIT) and add this to get a current value. Just like in the Linux solution nullplan wrote about. It's only really primitive kernels that read system time from a fixed location that is updated by IRQs.

No, I haven't misunderstood, your wallclock will be off 100% guaranteed. What you are not keeping in mind, you say "discover that it runs five seconds behind", without realizing what that actually means and how that can be implemented. If you do, then you'll get to the offset-less solution I was mentioning.

A better solution is to drift the offset slowly based on how much time is off when you compare it with NTP or RTC. If you also keep statistics about the difference and calculate the average drift of your hardware then you can over time get very close to a wall clock that is extremely accurate if you use an NTP server. However, again, it requires high precision time keeping, and second precision and IRQs simply are not adequate.

RDOS doesn't run on emulators.

That explains everything. That's why you're unaware of these things, because you haven't tested RDOS properly under different circumstances. A properly written OS should have no issues running in an emulator.

It's the emulators that do a bad job with segmentation that causes these issues. They simply have too many short-cuts that real CPUs don't have.

[bzt]

So, you might not have the full range of nanoseconds, or more than 1G of them? Not really acceptable, but at least better than inconsistent.

And how do you do that with offsets? You can't. The one and only way to get the full range of nanoseconds is an IRQ per each nanosecond, which is not viable. Any other solution (even your offsets) won't get the full range.

Wrong. When you read system time you also read the hardware (TSC, HPET or PIT) and add this to get a current value.

Not wrong. When you add that to the current value, a) you won't get the full range of nanoseconds, b) you do not count for the time spent in the handler after the addition and your timestamp will be off little-by-little. On a real computer it takes long time to accumulate, but on slow emulators it will happen pretty fast. You should not "add", but "overwrite" with an external, monotonic timestamp, such as the RTC's date time, that way the handler's overhead won't matter at all, and your system time will always be accurate and never off.

It's only really primitive kernels that read system time from a fixed location that is updated by IRQs.

Now what has preemptiveness to do with time precision and accuracy? This makes no sense at all, sorry.

A better solution is to drift the offset slowly based on how much time is off when you compare it with NTP or RTC.

Don't be stupid. Both NTP and RTC are external clock sources, which provide absolute time so there's no need for "adding" anything as I've already pointed out. I've also pointed out that drifting to a specific time is irrelevant from the point of how you get that specific time. Drifting is only implemented in NTP because there are supposedly badly written software which might misbehave if the system time would change much at once. I don't know what happens to those supposedly existing software on summertime change or when the user sets the time manually... Drifting time is just an overcomplicated solution to a non-existent problem.

Cheers,
bzt

[klange]

bzt, it is clear from several parts of your latest post that you are intentionally misinterpreting or misconstruing everything rdos is saying so that you can continue to argue. As I have lost confidence in my own ability to remain civil in responding to the blithering idiocy you have demonstrated here, I am locking this thread before it gets out of hand.

@zberry7: To offer a final conclusion to this thread and not leave you high and dry... everyone outside of bzt (and probably even bzt, too, if he shuts up for a minute and actually tries to do something other than argue for the sake of arguing) is essentially in agreement: You want to take an external, continuous, high-precious source (of which there are many, and which one is best can vary based on the particular hardware and your exact usecase, but any of an invariant TSC, an HPET, or an APIC timer is a fine choice) and use it as an offset from an accurate (if imprecise) base with real wall-clock time. The simplest strategy, though perfectly viable for a hobby OS, is to just take the RTC time - you start by doing it once at boot, and at your leisure you can do it again later to resynchronize.

Everything else is strategies for getting better accuracy, better monotonicity, and faster user access out of this same approach: Push the time basis (as data) and precise hardware source read (as the direct hardware access instruction) into userspace to avoid syscall overhead; if a change in the time basis is so drastic it would cause a visible jump, make the change happen to gradually so user applications don't freak out; use network time protocols to get more accuracy than your RTC offers.