FPU state on multicore processors

[halofreak1990]

Since loading/saving FPU state is likely part of the scheduling logic, wouldn't it be simpler to simply make (part of) the actual scheduling uninterruptible?
Something like

Code: Select all

critical_enter();

// save thread and/or FPU state

critical_leave();

... interruptible code here ...

critical_enter();

// load new thread and/or FPU state

critical_leave();

[Brendan]

Hi,

Turns out the logic doesn't work after all, and the reason is that the task can be scheduled on a new core before the old core loads a new thread.

It turns out that having massive concurrency/race condition bugs doesn't work after all (but the FPU/MMX/SSE/AVX state loading is fine if/when done correctly)?

Since loading/saving FPU state is likely part of the scheduling logic, wouldn't it be simpler to simply make (part of) the actual scheduling uninterruptible?

Typically there's "something" to make sure that the same thread isn't being executed by different CPUs at the same time. For an example, maybe you've got a queue of "ready to run" threads, where the scheduler has code to atomically add/remove threads onto/from the "ready to run" queue; and where if a thread is running (on any CPU) it's not on the "ready to run" queue and therefore no other CPUs can switch to that thread. Of course there's many other ways of doing it - the point is that there must be "something" to ensure that other CPUs won't switch to an already running thread.

Now; given that there must be "something" to ensure that other CPUs won't switch to an already running thread, that "something" can be used to ensure that no other CPU can switch to a thread until after all its state has been saved correctly (including its FPU/MMX/SSE/AVX state). Essentially, a thread is considered "running" until all of its state has been saved (and not just until part of its state has been saved).


Cheers,

Brendan

[rdos]

It turns out that having massive concurrency/race condition bugs doesn't work after all (but the FPU/MMX/SSE/AVX state loading is fine if/when done correctly)?

It's only because the state saving must be done before the old thread is put into a schedulable list, which it has to before a new thread is selected, so the FPU state cannot be saved when the next thread is loaded. It's simply too late.

Typically there's "something" to make sure that the same thread isn't being executed by different CPUs at the same time. For an example, maybe you've got a queue of "ready to run" threads, where the scheduler has code to atomically add/remove threads onto/from the "ready to run" queue; and where if a thread is running (on any CPU) it's not on the "ready to run" queue and therefore no other CPUs can switch to that thread. Of course there's many other ways of doing it - the point is that there must be "something" to ensure that other CPUs won't switch to an already running thread.

Now; given that there must be "something" to ensure that other CPUs won't switch to an already running thread, that "something" can be used to ensure that no other CPU can switch to a thread until after all its state has been saved correctly (including its FPU/MMX/SSE/AVX state). Essentially, a thread is considered "running" until all of its state has been saved (and not just until part of its state has been saved).

Not the same thing. It isn't running on multiple cores at the same time. It is the FPU state that is saved too late and thus the stale FPU state before saving is used as the save has not been done before the FPU is used again.

[rdos]

Since loading/saving FPU state is likely part of the scheduling logic, wouldn't it be simpler to simply make (part of) the actual scheduling uninterruptible?

Of course. Loading and saving is uninterruptable when done at the proper place. It's just that once the old task is saved, it is put into some queue, possibly the global scheduler queue where another core can take it from and execute. This can happen before the other core can load a new task.