Kernel System Time

[johnsa]

Hi,

Would I be correct in thinking that using the CMOS/RTC ports initially would be the correct way to manage the system date/time?
IE: once in the Kernel I could keep something like kernel_date/time values up to date via
mov al,09h
out 70h,al
jmp short $+2
in al,71h

etc?

Thanks!

[Brendan]

Hi,

Would I be correct in thinking that using the CMOS/RTC ports initially would be the correct way to manage the system date/time?

In general the kernel should get the time and date from the CMOS/RTC once during boot (and convert it to a single integer, like "number of milli-seconds since the beginning of 2000"), and then use some sort of timer (e.g. PIT IRQ, local APIC IRQ, HPET IRQ, CMOS/RTC periodic IRQ or RDTSC) to keep track of time after that.

This is partly because reading the time and date from the CMOS/RTC is insanely slow (lots of I/O port reads and writes), partly because it lacks precision (e.g. seconds rather than milli-seconds, micro-seconds or nano-seconds), and partly because you need a timer IRQ anyway (for preempting tasks that have used all of their time slice, and waking up tasks that were sleeping - e.g. "sleep()").

If you want to do it well then there's a lot of things to complicate it too though. Hassles include:

• Determining which timer IRQ is the best one to use during boot (given that some aren't supported by the hardware) and selecting/calculating the best frequency to use. Note: there's a compromise between timer precision and overhead, where slower frequencies with less overhead are better for older computers and faster frequencies with higher precision are better for newer computers.
• The CMOS/RTC should be set to UTC (but you can't assume that - some dodgy OSs set the RTC to local time so other OSs get messed up by daylight savings changes), and your OS should keep track of UTC and then convert it into local time if/when necessary (including handling daylight savings, etc).
• Each process or thread should have it's own time zone that it can change without effecting anything else, and each user should have their own time zone too; so that (for e.g.) if the OS is running on a computer in Australia and someone uses something like telnet or SSH to log into your OS from England then they can get the correct time for England and not Australian time.
• Need to keep track of drift and adjust for it (e.g. if the OS loses 5 seconds per day then you can add a tiny adjustment to the counter to correct the drift and keep the time accurate). Ideally, you should also be able to use something like NTP to improve accuracy and automatically configure drift compensation.
• When the time is changed you may need to support "soft" changes to avoid confusing other software. For example, if the time is wrong and you need to add an hour to it, then software might not expect the time to suddenly jump an entire hour, and things like device drivers and animation (which need specific delays) might stop working properly if the time suddenly jumps. To fix this you can change the time slowly - for example, for each second add one and a half seconds to the timer count until the time slowly becomes correct (in this case if you need to add an hour to the timer, then it'd take an hour and a half). This can mean that the OS needs to keep track of the "target time" (with jumps) and the "current time" (with slow changes) until they are equal.
• When the OS is shutdown you can set a new/correct time in the RTC to avoid drift that occurs while the OS is running from effecting the RTC. You can also store the "time when the RTC was updated" somewhere (e.g. on the hard drive) during shutdown, and use this during boot to calculate how long the RTC thinks the computer was turned off, and then calculate the amount of drift that would have occurred while the computer was turned off. That way if the RTC loses 5 seconds per day you can adjust for that, even when the computer is turned off.

A simple OS running on "average" hardware that does none of this might gain or lose a few minutes per day and might only have a precision of 1 second. An OS that does all of this might end up being accurate to within 10 ms and have 10 us precision on the same "average" hardware.

I should point out that high precision timing is very nice - for example, imagine if your application wants to update the screen 100 times per second:

Code: Select all

    last_time = get_current_time();
    while(running) {
        draw_the_screen();
        delay_time = last_time + 10ms - get_current_time();
        if(delay_time > 0) sleep(delay_time);
        last_time += 10ms;
    }

In this case, if it takes 6.123 ms to update the screen then the code might be asking to sleep for 3.877 ms. If the OS's timing only has 1 second precision then the timing wouldn't be precise enough for this to work correctly - the code would update the screen 10 times with no delay between updates, then it'd ask for a 1 second delay that would actually only be 38.77 ms (and you'd get clunky animation instead of smooth animation). With 1 ms precision the results might be acceptable (between 9 ms and 11 ms between screen updates), but if the OS's timing had 100 us precision it'll be much better (between 9.9 ms and 10.1 ms between screen updates).


Cheers,

Brendan

[johnsa]

Many thanks for the detailed response!
I see the RTC/CMOS only return the year as two-digit (EG: "09"). So I presume it's safe to just prefix the 20 ;)No Y2k bug for 1000 year's.. long after I'll care about it.
I'm not familliar with the HPET, are they any docs/details on it?

Thanks!
John

[Love4Boobies]

• When the OS is shutdown you can set a new/correct time in the RTC to avoid drift that occurs while the OS is running from effecting the RTC. You can also store the "time when the RTC was updated" somewhere (e.g. on the hard drive) during shutdown, and use this during boot to calculate how long the RTC thinks the computer was turned off, and then calculate the amount of drift that would have occurred while the computer was turned off. That way if the RTC loses 5 seconds per day you can adjust for that, even when the computer is turned off.

Checking if anything change when booting might also save you some headaches. But I disagree with not updating the system time all at once. Applications shouldn't count on the system time when doing real time processing, they should use other OS services that would set up timers to handle whatever is needed. Otherwise, you won't really know what the time is for another hour. Not to mention that applications that used system time would still be kind of screwed.

I'm not familliar with the HPET, are they any docs/details on it?

Yes, you may find Intel's specification here.

[Brendan]

Hi,

I see the RTC/CMOS only return the year as two-digit (EG: "09"). So I presume it's safe to just prefix the 20 ;)No Y2k bug for 1000 year's.. long after I'll care about it.

Most modern RTC chips do keep track of the century, but it's in a non-standard location in the CMOS and you need to parse ACPI tables to find if the RTC supports it and which location in the CMOS it's at.

But I disagree with not updating the system time all at once. Applications shouldn't count on the system time when doing real time processing, they should use other OS services that would set up timers to handle whatever is needed. Otherwise, you won't really know what the time is for another hour. Not to mention that applications that used system time would still be kind of screwed.

I'm thinking about things like "make" failing to work because the new file's have timestamps that are older than the old binary's timestamp, and incremental backup utilities that don't backup new files because the new file's timestamps are older than the last backup, and scheduled tasks that are meant to be done at 12:00 each day but are skipped completely because the time went from 11:30 to 12:30 (or are done twice because the time went from 12:30 back to 11:30).

Some of these sorts of problems (but not all of them) can be avoided with careful programming, but even when the problem can be avoided most application programmers won't realize the problem exists and they'll stuff it up anyway...

Note: For OS programming, it's bad to assume the hardware is sane, bad to assume the firmware is sane, bad to assume the applications are sane, and bad to assume your own OS/kernel code is sane...


Cheers,

Brendan

[Love4Boobies]

I'm thinking about things like "make" failing to work because the new file's have timestamps that are older than the old binary's timestamp, and incremental backup utilities that don't backup new files because the new file's timestamps are older than the last backup, and scheduled tasks that are meant to be done at 12:00 each day but are skipped completely because the time went from 11:30 to 12:30 (or are done twice because the time went from 12:30 back to 11:30).

And things could still fail in the same way even in your scenario, only the probability that this would happen would be lower, esp. on slower machines. However, think about using make to build an OS. You probably also have a lot of files that might get updated in less than half a second...

Some of these sorts of problems (but not all of them) can be avoided with careful programming, but even when the problem can be avoided most application programmers won't realize the problem exists and they'll stuff it up anyway...

Actually, now that you've mentioned it... I see no obvious workaround for the make problem. The same applies when the user himself chooses to change the system time. Should he be disallowed to do so? Maybe sending some sort of signal to applications to notify them that the time has changed so they can handle it appropriately would be best.

[Combuster]

Using two time counters would be the best fix for a changing time (including leap seconds and DST changes)

That means you have one counter that keeps track of actual time, and one counter tracking time since OS startup (and make the second one easier to find for programmers). Then if you want to check for an interval, you should use the uptime counter, and when you check for a certain time, you should check the real time counter. And of course, you can only update the first timer during OS operation.

[Brendan]

Hi,

I'm thinking about things like "make" failing to work because the new file's have timestamps that are older than the old binary's timestamp, and incremental backup utilities that don't backup new files because the new file's timestamps are older than the last backup, and scheduled tasks that are meant to be done at 12:00 each day but are skipped completely because the time went from 11:30 to 12:30 (or are done twice because the time went from 12:30 back to 11:30).

And things could still fail in the same way even in your scenario, only the probability that this would happen would be lower, esp. on slower machines. However, think about using make to build an OS. You probably also have a lot of files that might get updated in less than half a second...

Lots of files being updated in less than half a second is easy enough to fix - just use high precision timestamps (e.g. nano-seconds since the epoch) and/or make sure each timestamp is unique. For example, (in the extreme case) instead of using a timestamp you could just use a counter that's increased by one each time a "timestamp" is requested, so the first file that's modified gets "timestamp = 1", the second file gets "timestamp = 2", etc.

Of course you could combine both techniques - for example, you could have 64-bit timestamps where the highest 32-bits contains "seconds since the epoch" and the lower 32-bits is a counter that's increased by one each time a timestamp is requested (where the counter is reset to zero if the second changes). In this case timestamps might go 0x1234567800000000, 0x1234567800000001, 0x1234567800000002, 0x1234567800000003, ..., 0x1234567900000000, 0x1234567900000001, 0x1234567900000002.

Some of these sorts of problems (but not all of them) can be avoided with careful programming, but even when the problem can be avoided most application programmers won't realize the problem exists and they'll stuff it up anyway...

Actually, now that you've mentioned it... I see no obvious workaround for the make problem. The same applies when the user himself chooses to change the system time. Should he be disallowed to do so?

A user (e.g. root or admin, but not necessarily any user) should be allowed to change the "target" system time, and the OS should gradually make the "current" system time match the "target" system time. Of course there's cases where the OS can cheat. For example, you could keep track of the last used timestamp in the file system/s, and then during boot you could do something like:

Code: Select all

    if(target < current) {
        if(target > previous_FS_timestamp) {
            current = target;
        } else {
            current = previous_FS_timestamp + 1;
        }
    } else {
        current = target;
    }

You could also remember when the system time was last read, and apply changes retro-actively. For example, if the current time is 1:00 and the user sets the target time to 2:00, then if the time hasn't been read for 30 minutes you could pretend that you started adding 1.5 seconds to the system time each second half an hour ago (e.g. immediately set the current time to 1:15 and add 1.5 seconds per second after that).

Maybe sending some sort of signal to applications to notify them that the time has changed so they can handle it appropriately would be best.

That can get very complicated very quickly - imagine if the root user can't make up their mind and changes the "target" time 20 times in half an hour. Applications would need to track an arbitrary number of changes, and when an application reads a timestamp for a file it'd need to figure out which changes have been applied since the file was created to figure out what the timestamp should have been. Of course this isn't possible if the time has been turned back - you could create a file with 12:00 as the timestamp, then an hour later you could turn the clock back by an hour and create a new file that also has 12:00 as it's timestamp, and no application would be able to figure out which file was created when (even if it does track time changes).

For most of these cases it's enough for the OS to guarantee that later timestamps are always numerically higher than earlier timestamps (with no need to make the timestamps correspond to any time scale). For other cases it's not so simple, and the OS would benefit from having "sleep_until(specific_time)" which is effected by system time changes and "sleep_for(specific_duration") which is never effected by system time changes.


Cheers,

Brendan