The Place to Start for Operating System Developers
http://f.osdev.org/
To change from "plain 32-bit paging" to PAE you'd have to set the flag in CR4 (to enable PAE) and load CR3 with the address of a correct PDPT. You can't do both at the same time; and (without disabling paging) you can't do one before the other (e.g. enable PAE then crash because CR3 is wrong, or load CR3 then crash because PAE isn't enabled).626788149 wrote:Since my kernel boot,I use non-PAE paging mode simply maping 0~4MB to 0~4MB and 0xC0000000 ~ 0xC0000000 + 768MB to 0 ~ 768MB useing 4MB paging.
I have set kernel PDPTE up, But I just don't know how to change to PAE paging mode.
The right thing depends on the design of the boot code.kiznit wrote:The right thing to do is enable PAE when you turn on paging in the first place. Not disable existing paging and then re-enable it.
Systems with different CPUs? You've ever seen one of these? I always understood that all CPU packages on a given motherboard must be identical. Is that not true? How often does it happen in practice?Brendan wrote:(e.g. if the boot CPU supports PAE and other CPUs don't, then not all CPUs support PAE and PAE shouldn't be used)
How to you call back to UEFI (Runtime Services) after all of this? Or do you simply not?Brendan wrote:The funny thing is that when the firmware is 64-bit UEFI it's all the same - the BAL has to assume "80486 or later" and uses plain 32-bit paging (and the chosen "kernel setup module" may have to switch from plain 32-bit paging to PAE or long mode after) even when the boot loader uses 64-bit long mode.
This is actually Intel's advice (from their "MultiProcessor Specification"). Specifically:kiznit wrote:Systems with different CPUs? You've ever seen one of these? I always understood that all CPU packages on a given motherboard must be identical. Is that not true? How often does it happen in practice?Brendan wrote:(e.g. if the boot CPU supports PAE and other CPUs don't, then not all CPUs support PAE and PAE shouldn't be used)
Basically; at a minimum, an OS should get the features flags from all CPUs and AND them together to find the subset of features supported by all CPUs.Intel wrote:B.8 Supporting Unequal Processors
Some MP operating systems that exist today do not support processors of different types, speeds, or capabilities. However, as processor lifetimes increase and new generations of processors arrive, the potential for dissimilarity among processors increases. The MP specification addresses this potential by providing an MP configuration table to help the operating system configure itself. Operating system writers should factor in processor variations, such as processor type, family, model, and features, to arrive at a configuration that maximizes overall system performance. At a minimum, the MP operating system should remain operational and should support the common features of unequal processors.
I don't use run-time services after boot.kiznit wrote:How to you call back to UEFI (Runtime Services) after all of this? Or do you simply not?Brendan wrote:The funny thing is that when the firmware is 64-bit UEFI it's all the same - the BAL has to assume "80486 or later" and uses plain 32-bit paging (and the chosen "kernel setup module" may have to switch from plain 32-bit paging to PAE or long mode after) even when the boot loader uses 64-bit long mode.
There can be something like one PC and many motherboards plus some common memory. If an OS is going to support all those differences then it's going to be hard not only to design it, but also to test it and even to find a case where it really can be used.Brendan wrote:In practice most motherboards manufacturers only test a few variations (e.g. same family and clock frequency with different model numbers) because fully testing every possible combination can get time consuming/expensive; but that doesn't mean the motherboards won't work for other "not officially supported/tested" combinations. The only real constraint is that the externally visible signals are compatible (e.g. both chips are using the same speed and protocol for the quick-path/hyper-transport interconnect between them).
Obviously, then they just haven't implemented correct processor-processor interaction protocol. Because existing OSs are "likely broken" anyway. And, obviously, if there's no protocol, then what for such a motherboard can be used? Only some special cases. But how can you know the exact case?Brendan wrote:Because existing OSs are "likely broken" anyway, motherboards manufacturers don't really need to care about supporting/testing many combinations.
If most OSs do not support something, then how the "something" can work? It can work if manufacturer just spend some money for the special case handling. It can be a design, which supports some standards, but it also can be some special software. The latter is most probable. The former (as you have said) can be untested or even worse. And you can spend a lot of time to support cases that just aren't work.Brendan wrote:Basically; at a minimum an OS should get the features flags from all CPUs and AND them together to find the subset of features supported by all CPUs, and processes should never use CPUID directly (and should ask the OS for a sanitized "CPUID combination" to avoid bugs); but in practice most OSs fail to meet the minimum, and don't even provide a way for processes to get a sanitized "CPUID combination" (which is also bad for other reasons - just because a CPU says it supports a feature doesn't mean it's usable and not effected by errata).
In that case it'd be separate PC's with some common memory (e.g. using something like non-transparent bridges, which are typically just used as a point-to-point network device).embryo2 wrote:There can be something like one PC and many motherboards plus some common memory. If an OS is going to support all those differences then it's going to be hard not only to design it, but also to test it and even to find a case where it really can be used.Brendan wrote:In practice most motherboards manufacturers only test a few variations (e.g. same family and clock frequency with different model numbers) because fully testing every possible combination can get time consuming/expensive; but that doesn't mean the motherboards won't work for other "not officially supported/tested" combinations. The only real constraint is that the externally visible signals are compatible (e.g. both chips are using the same speed and protocol for the quick-path/hyper-transport interconnect between them).
For things like blade servers (which are mostly just separate computers on cards/blades that plug into a back-plane/chassis that provides power and networking) I'm not sure - it seems like there's lots of different "specifications" (e.g. several different ones from each vendor) where none of them are standard.embryo2 wrote:Out of curiosity, is there "multi-motherboard" specification? Or it's about special cases for supercomputing and something similar?
Currently; "multi-chip" 80x86 servers with (slightly) different CPUs are rare, and for "single chip" 80x86 systems all cores are identical (mostly because OSs/software doesn't support it properly and hardware manufacturer's hands are tied). In the future, who knows?embryo2 wrote:Obviously, then they just haven't implemented correct processor-processor interaction protocol. Because existing OSs are "likely broken" anyway. And, obviously, if there's no protocol, then what for such a motherboard can be used? Only some special cases. But how can you know the exact case?Brendan wrote:Because existing OSs are "likely broken" anyway, motherboards manufacturers don't really need to care about supporting/testing many combinations.
Basically, it seems highly unlikely to find a system with different CPUs, but designed to work according to widely accepted standards. So, the OS should support some special cases (is there documentation available?) or just assume there's no difference among processors.
What do you think about your OS and supercomputing? Many blades with bridges aren't seldom today. The specific case for supercomputing is the hardware capable of running an application server for the whole Internet (millions requests per second). Do you plan to address the scalability issue of such magnitude?Brendan wrote:In that case it'd be separate PC's with some common memory (e.g. using something like non-transparent bridges, which are typically just used as a point-to-point network device).
Processors are also connected via high speed "peer to peer" network. Even if there's some cache hierarchy, it's still the network of connected nodes. The only difference - the nodes are on the same PCB (or event on the same silicon board).Brendan wrote:Depending on how you define "computer"; super-computers aren't computers - they're many separate computers connected via. high speed networking.
Yes, the reason is there. But I see here an analogy with your computer farm with many old computers and the need to somehow use them all. First was the farm and next the need has came. Also the heterogeneous processors are the product of some extra silicon. It's just there and we can use it "somehow". Why not to use if it's already there? But there's the drawback - it's much more complicated thing to program. If things are identical then we can schedule our thread on every processor, but if things are different (like CPU and GPU cores) we need to recompile our program first just to be able to switch it between cores. And next follows the state exchange problem. We can get some extra speed after recompilation and tricky scheduling implementation, but why not to go a bit farther and not to split things in the arithmetic unit arrays and register files, for example? If there's no need for high load computation then just a few arithmetic units are used, but if we need some extra power then we can spread the load across many units. And the units are all the same, they are identical, there's no need for recompilation or too tricky scheduling. Just select independent units of work and assign an arithmetic unit per each unit of work. If there are more work units then more arithmetic units employed, if there are less - less arithmetic and less power consumed. Things like Green Arrays just ideally fit in such a picture.Brendan wrote:If you look at things like IBM's Cell or ARM's big.LITTLE you'll see different cores in the same chip. These exist because there's logical reasons (for performance and/or power consumption) to have one or more fast core/s (for things that need good single-thread performance) in addition to smaller slower cores (for things that benefit from many CPUs and/or for when the higher power consumption isn't justified).
The overhead of the task switch is relatively high. May be it can be implemented just as an equivalent of one pusha on one processor and next one popa on another, but the benefits here just the lesser caches and pipelines instead of flexible load distribution in case of many arithmetic units. The latter approach has greater potential, as I see it.Brendan wrote:For example; it's not entirely improbable that within the next 5 years we'll be able to get a normal "8 fast cores" Xeon chip and a "60+ slower cores" Xeon Phi chip and plug both into the same dual-socket motherboard
Well, then you can claim "it supports even martian computers!", regardless of whether or not it's ever actually useful in practice. It's not bad for marketing, but I see it's bad for reputation among the people who are a bit wired up. But of course, the marketing is up to you.Brendan wrote:Also note that being able to add "supports computers with extremely different CPUs" onto a list of reasons why my OS is better than existing OSs (Windows, OS X, Linux) is something I consider useful (for future marketing); regardless of whether or not it's ever actually useful in practice.
For super-computers (and possibly HPC in general) you've typically got the same (single) application running on each node, the application is custom designed for the hardware, and all you really want from an OS is for it to get out of the way. For this, my OS would mostly be inappropriate. I'm not even sure that any OS makes sense for this; and I'd be tempted to want bootable applications (with boot code and networking implemented as a library and statically linked directly into the application).embryo2 wrote:What do you think about your OS and supercomputing? Many blades with bridges aren't seldom today. The specific case for supercomputing is the hardware capable of running an application server for the whole Internet (millions requests per second). Do you plan to address the scalability issue of such magnitude?Brendan wrote:In that case it'd be separate PC's with some common memory (e.g. using something like non-transparent bridges, which are typically just used as a point-to-point network device).
The difference (at least for how I define "computer") is whether or not there's a single/shared physical address space. For NUMA systems (e.g. CPUs and memory connected by links) there is a single physical address space used by all the pieces; but for separate computers on a LAN, or blades or super-computers there isn't.embryo2 wrote:Processors are also connected via high speed "peer to peer" network. Even if there's some cache hierarchy, it's still the network of connected nodes. The only difference - the nodes are on the same PCB (or event on the same silicon board).Brendan wrote:Depending on how you define "computer"; super-computers aren't computers - they're many separate computers connected via. high speed networking.
I'm not interested in designing hardware (it would cost millions/billions of dollars to create hardware that has any hope of gaining market share). For a "software only" project, I think 80x86 is the best (first) choice, as there's lots of people with PCs and no need to do strange things (jail breaking, etc) just to install an OS. If Green Arrays ever get popular (and if they're ever used in standardised systems and not just "CPU with random non-standard pain" like systems containing ARM CPUs) they'd be worth considering.embryo2 wrote:Yes, the reason is there. But I see here an analogy with your computer farm with many old computers and the need to somehow use them all. First was the farm and next the need has came. Also the heterogeneous processors are the product of some extra silicon. It's just there and we can use it "somehow". Why not to use if it's already there? But there's the drawback - it's much more complicated thing to program. If things are identical then we can schedule our thread on every processor, but if things are different (like CPU and GPU cores) we need to recompile our program first just to be able to switch it between cores. And next follows the state exchange problem. We can get some extra speed after recompilation and tricky scheduling implementation, but why not to go a bit farther and not to split things in the arithmetic unit arrays and register files, for example? If there's no need for high load computation then just a few arithmetic units are used, but if we need some extra power then we can spread the load across many units. And the units are all the same, they are identical, there's no need for recompilation or too tricky scheduling. Just select independent units of work and assign an arithmetic unit per each unit of work. If there are more work units then more arithmetic units employed, if there are less - less arithmetic and less power consumed. Things like Green Arrays just ideally fit in such a picture.Brendan wrote:If you look at things like IBM's Cell or ARM's big.LITTLE you'll see different cores in the same chip. These exist because there's logical reasons (for performance and/or power consumption) to have one or more fast core/s (for things that need good single-thread performance) in addition to smaller slower cores (for things that benefit from many CPUs and/or for when the higher power consumption isn't justified).
Task switching wouldn't be much different to existing 80x86 (where you already want optimisations for locality in the scheduler).embryo2 wrote:The overhead of the task switch is relatively high. May be it can be implemented just as an equivalent of one pusha on one processor and next one popa on another, but the benefits here just the lesser caches and pipelines instead of flexible load distribution in case of many arithmetic units. The latter approach has greater potential, as I see it.Brendan wrote:For example; it's not entirely improbable that within the next 5 years we'll be able to get a normal "8 fast cores" Xeon chip and a "60+ slower cores" Xeon Phi chip and plug both into the same dual-socket motherboard
Bad for reputation is bad for marketing (and making false claims gets you sued). While it might not be used in practice, being able to make the (valid) claim that it does support extremely different CPUs gives consumers the impression that it's a well designed/very flexible system (which is good for marketing and reputation).embryo2 wrote:Well, then you can claim "it supports even martian computers!", regardless of whether or not it's ever actually useful in practice. It's not bad for marketing, but I see it's bad for reputation among the people who are a bit wired up. But of course, the marketing is up to you.Brendan wrote:Also note that being able to add "supports computers with extremely different CPUs" onto a list of reasons why my OS is better than existing OSs (Windows, OS X, Linux) is something I consider useful (for future marketing); regardless of whether or not it's ever actually useful in practice.
The common things for powerful computers include scheduler and process/thread support, drivers, bootloader, networking. May be it is a "specific OS", but it is changed too infrequent to keep it together with application logic. It's code should be isolated from applications just because of the useless complexity of many points of interaction. Application maintenance can be a problem if it includes the OS code. And statically linking things like scheduler looks a bit strange. Of course, there can be some modules and the whole thing can be called "the application", but if there's a protocol for interaction of application logic with general hardware related and execution organization functionality then it is the border where the OS starts.Brendan wrote:For super-computers (and possibly HPC in general) you've typically got the same (single) application running on each node, the application is custom designed for the hardware, and all you really want from an OS is for it to get out of the way. For this, my OS would mostly be inappropriate. I'm not even sure that any OS makes sense for this; and I'd be tempted to want bootable applications (with boot code and networking implemented as a library and statically linked directly into the application).
It's the hardware and drivers who are responsible for the processor-to-memory interaction. x86 requires little software for such interaction, but things like caches and NUMA enforce some constraints and it is more efficient just to pay attention to the constraints in software. For GPUs it is required to provide some software means for efficient memory management. And next step is transparent memory allocation over network, which is also implemented in software. Supercomputers have a need for cooperation and the need can be satisfied with the common memory approach.Brendan wrote:The difference (at least for how I define "computer") is whether or not there's a single/shared physical address space.
The jail breaking for Android is simplified to the level when many advanced users just install things like Cyanogen instead of the vendor's distribution with limited user rights and a lot of bloatware which is impossible to uninstall.Brendan wrote:For a "software only" project, I think 80x86 is the best (first) choice, as there's lots of people with PCs and no need to do strange things (jail breaking, etc) just to install an OS.
Yes, the time of the new hardware is still many years ahead. But the current way of silicon usage is inefficient, most of the silicon is idle most of the time. But every realistic OS, of course, should support x86.Brendan wrote:If Green Arrays ever get popular (and if they're ever used in standardised systems and not just "CPU with random non-standard pain" like systems containing ARM CPUs) they'd be worth considering.
But the locality optimizations only can not outperform things like AVX with it's 32 bytes or 16 words simultaneous operations, if your OS employs the lesser common denominator approach. And if it uses all available features then there's no way for using less feature rich parts of the heterogeneous system. So, much of the silicon is idle all the time. But yes, the task switching can be simple in this case. What about perfectionism?Brendan wrote:Task switching wouldn't be much different to existing 80x86 (where you already want optimisations for locality in the scheduler).
Nobody know what is a martian computer. So, you can safely claim your OS can manage it. If a proof is required then you can invent whatever you want and call it "the martian computer".Brendan wrote:Bad for reputation is bad for marketing (and making false claims gets you sued).
Yes, that's why you hate the marketing people. Aren't you? My thing supports whatever you want and there's no problem in the world where you just have no idea about what actually "whatever" means. It's not a problem if you should pay for the thing that supports "everything" because it "gives consumers the impression that it's a well designed/very flexible system".Brendan wrote:While it might not be used in practice, being able to make the (valid) claim that it does support extremely different CPUs gives consumers the impression that it's a well designed/very flexible system (which is good for marketing and reputation).
For a "one thread per CPU" model (where each thread/CPU is expected to be at 100% load all the time) you don't need a scheduler or much thread support (beyond maybe spinlocks). There'd only be one driver for the network card; and the networking stack could be very minimal (no need to support full TCP/IP, routing, etc - possibly just enough to implement MPI directly on top of the physical layer). There's also no need for security (including isolation between kernel and process). Yes; it would look very different to a typical general purpose OS, but that's because it's not a general purpose OS in the first place (e.g. no user/s, no file IO, no multi-tasking, etc).embryo2 wrote:The common things for powerful computers include scheduler and process/thread support, drivers, bootloader, networking. May be it is a "specific OS", but it is changed too infrequent to keep it together with application logic. It's code should be isolated from applications just because of the useless complexity of many points of interaction. Application maintenance can be a problem if it includes the OS code. And statically linking things like scheduler looks a bit strange. Of course, there can be some modules and the whole thing can be called "the application", but if there's a protocol for interaction of application logic with general hardware related and execution organization functionality then it is the border where the OS starts.Brendan wrote:For super-computers (and possibly HPC in general) you've typically got the same (single) application running on each node, the application is custom designed for the hardware, and all you really want from an OS is for it to get out of the way. For this, my OS would mostly be inappropriate. I'm not even sure that any OS makes sense for this; and I'd be tempted to want bootable applications (with boot code and networking implemented as a library and statically linked directly into the application).
For "common memory"; the additional networking needed for cache coherency alone would probably triple the cost of the super-computer (or more realistically, slash the number of nodes you can afford for the same budget). On top of that you're probably going to need to find CPUs that can actually support >64-bit addresses; which means forgetting about cheap/commodity hardware (e.g. CPUs and/or GPUs designed for normal workstation/server that have been recycled). Basically; it's not even close to being viable.embryo2 wrote:It's the hardware and drivers who are responsible for the processor-to-memory interaction. x86 requires little software for such interaction, but things like caches and NUMA enforce some constraints and it is more efficient just to pay attention to the constraints in software. For GPUs it is required to provide some software means for efficient memory management. And next step is transparent memory allocation over network, which is also implemented in software. Supercomputers have a need for cooperation and the need can be satisfied with the common memory approach.Brendan wrote:The difference (at least for how I define "computer") is whether or not there's a single/shared physical address space.
For current "80x86 NUMA" hardware, you typically want to run a process on the same NUMA domain so you can allocate memory that's "close" to that NUMA domain (and avoid the penalties caused by accessing memory that's further away from those CPUs). If you migrate a process from one NUMA domain to another you end up with "worst case NUMA penalties" (or re-allocating all the memory the process uses to avoid those penalties, which isn't cheap either); so you try very hard to make sure processes aren't migrated from one NUMA domain to another. Also; for most normal OS's there's a "CPU affinity" feature that allows a process to be "pinned"/restricted to a sub-set of CPUs.embryo2 wrote:But the locality optimizations only can not outperform things like AVX with it's 32 bytes or 16 words simultaneous operations, if your OS employs the lesser common denominator approach. And if it uses all available features then there's no way for using less feature rich parts of the heterogeneous system. So, much of the silicon is idle all the time. But yes, the task switching can be simple in this case. What about perfectionism?Brendan wrote:Task switching wouldn't be much different to existing 80x86 (where you already want optimisations for locality in the scheduler).
It's plausible that the "runs on some supercomputers" claim is valid and (assuming the claim is valid) an OS developer can prove that the claim is valid. It's not plausible that the "runs on some martian computers" claim is valid and impossible for an OS developer can prove that the claim is valid.embryo2 wrote:Nobody know what is a martian computer. So, you can safely claim your OS can manage it. If a proof is required then you can invent whatever you want and call it "the martian computer".Brendan wrote:Bad for reputation is bad for marketing (and making false claims gets you sued).
For ordinary people claims like "the OS runs on some supercomputers" is very similar to the claim "it runs on every martian computer".
The theory of capitalist systems is that better products/services succeed (are more profitable) and worse products/services don't, and this (combined with a desire for businesses/companies to make profit) causes better products/services for consumers. I personally think this theory fails in practice for multiple reasons (some that are prevented by law and some that aren't); and one of the reasons is that consumers can't easily make informed choices because they're fed advertising instead of unbiased/fair comparisons. Essentially; bad products/services with excellent advertising succeed and excellent products/services with bad advertising fail; and this erodes the foundation of capitalism.embryo2 wrote:Yes, that's why you hate the marketing people. Aren't you? My thing supports whatever you want and there's no problem in the world where you just have no idea about what actually "whatever" means. It's not a problem if you should pay for the thing that supports "everything" because it "gives consumers the impression that it's a well designed/very flexible system".Brendan wrote:While it might not be used in practice, being able to make the (valid) claim that it does support extremely different CPUs gives consumers the impression that it's a well designed/very flexible system (which is good for marketing and reputation).
Swiss knife is less popular than an ordinary kitchen knife.
I can't tell for all supercomputers, but many modern powerful clusters are used as a typical server when there is a queue of scheduled tasks and there is a need for some optimal task execution. Tasks are not sporadic and are planned by humans, but after they have a list of tasks all the rest is the server's responsibility. So, server needs a set of executables (tasks) and runs them in some order. People here need some means to control the task execution and the resources it needs. So, there is a console for admins (aren't they the missed users?) and there is an environment that runs the whole thing. And at least some clusters use Unix-like OSes as such environment. Even more, if a task has bugs it can crash all other tasks, lead to reboot and down time of a very expensive hardware. I see here a great need for an OS.Brendan wrote:For a "one thread per CPU" model (where each thread/CPU is expected to be at 100% load all the time) you don't need a scheduler or much thread support (beyond maybe spinlocks). There'd only be one driver for the network card; and the networking stack could be very minimal (no need to support full TCP/IP, routing, etc - possibly just enough to implement MPI directly on top of the physical layer). There's also no need for security (including isolation between kernel and process). Yes; it would look very different to a typical general purpose OS, but that's because it's not a general purpose OS in the first place (e.g. no user/s, no file IO, no multi-tasking, etc).
The common memory here is used for communication. It's not the goal to keep it on par with the processor's speed. It's like caches vs memory on x86.Brendan wrote:For "common memory"; the additional networking needed for cache coherency alone would probably triple the cost of the super-computer
Clusters are usually built around a lot of Intel's chips. And they deliver very high processing power. Specialized designs of supercomputers are not ready to compete with the raw numbers of similar processors. Intel's processors are fast and doubling the speed here is a big problem, so it's easier to create a cluster with slower processors, but used in great quantities. It's cheap and total performance can be as big as scalable is the task. And usually supercomputer's tasks are scalable because of efforts targeted for task parallelization. I doubt if there are ways for essential increase of the performance of a sequential task. Then the only problem left is the communication in cluster and here some special hardware can help. But it's not processing related thing.Brendan wrote:On top of that you're probably going to need to find CPUs that can actually support >64-bit addresses;
Is your OS capable of compiling a bytecode for different processors? Do you use AOT? How do you decide on the question of what processor should be used by which application? How do you know if application1 needs more processing power than application2? Or are you going to run the same application on different CPUs? And to have it compiled for every CPU. And to cope with it's thread interaction when threads are running on different CPUs? It isn't as simple as "I'd need to change anything at all".Brendan wrote:For current "80x86 NUMA" hardware, you typically want to run a process on the same NUMA domain so you can allocate memory that's "close" to that NUMA domain (and avoid the penalties caused by accessing memory that's further away from those CPUs). If you migrate a process from one NUMA domain to another you end up with "worst case NUMA penalties" (or re-allocating all the memory the process uses to avoid those penalties, which isn't cheap either); so you try very hard to make sure processes aren't migrated from one NUMA domain to another. Also; for most normal OS's there's a "CPU affinity" feature that allows a process to be "pinned"/restricted to a sub-set of CPUs.
If you add "CPU feature constraints" on top of that you end up with something approximating my OS design. To do this you just use feature's the process requires to pre-determine the CPU affinity for the process. E.g. if some CPUs support a feature (e.g. AVX-512) and some don't; then you setup the CPU affinity so that a process that uses that feature can't be run on a CPU that doesn't support it. Note: It's slightly more complicated than this because supporting different CPUs also effects the code that handles the "delayed FPU/MMX/SSE/AVX state saving" mechanism).
I do these things for normal 80x86; so for a theoretical/future "normal Xeon + Phi hybrid" system it'd be no different to what I already do and I doubt I'd need to change anything at all.
It's probably for the auto-delete forumBrendan wrote:The theory of capitalist systems is...
But yes, it fails, I agree.
If it's about rational assessment of your efforts then it's better to concentrate on something visually attractive or other "cool" stuff. But if it's about perfectionism then yes, you can try to justify your efforts with thoughts like above, but it's just for your personal consumption. Psychologically it helps, but [efforts]/[attracted users] ratio won't justify it.Brendan wrote:Mostly what I'm saying is that; using a fair/unbiased information only, I want (at least some) consumers/users to choose my OS at some point in the future; and to do that I need to create reasons why (using a fair/unbiased information only) at least some consumers/users would want to choose my OS instead of an alternative. Supporting different CPUs in the same system is just one more (relatively small/minor) way I can do that.
As far as I know supercomputers are mostly used for scientific modelling (e.g. protein folding) where there's a single task that requires a massive amount of mathematical calculations and "floating point operations per second" is the main design goal. It's these systems I was talking about, and it's for these systems I was proposing "bootable application".embryo2 wrote:I can't tell for all supercomputers, but many modern powerful clusters are used as a typical server when there is a queue of scheduled tasks and there is a need for some optimal task execution. Tasks are not sporadic and are planned by humans, but after they have a list of tasks all the rest is the server's responsibility. So, server needs a set of executables (tasks) and runs them in some order. People here need some means to control the task execution and the resources it needs. So, there is a console for admins (aren't they the missed users?) and there is an environment that runs the whole thing. And at least some clusters use Unix-like OSes as such environment. Even more, if a task has bugs it can crash all other tasks, lead to reboot and down time of a very expensive hardware. I see here a great need for an OS.Brendan wrote:For a "one thread per CPU" model (where each thread/CPU is expected to be at 100% load all the time) you don't need a scheduler or much thread support (beyond maybe spinlocks). There'd only be one driver for the network card; and the networking stack could be very minimal (no need to support full TCP/IP, routing, etc - possibly just enough to implement MPI directly on top of the physical layer). There's also no need for security (including isolation between kernel and process). Yes; it would look very different to a typical general purpose OS, but that's because it's not a general purpose OS in the first place (e.g. no user/s, no file IO, no multi-tasking, etc).
Another kind of clusters is something like Google or Facebook are using. The task there is always the same, but it's parts are very different, like is the case with every internet server. So they need some means to manage task's parts (application updates, for example) and task's distribution across available (clustered) computers. And because it's very hard to maintain one big application usually there are many applications. So, we have many processes and many threads distributed across many computers. What means of communication are available for the clustered computers? It's just the same TCP/IP. Where the data for every application is kept? It's the same RAID controllers with a lot of HDDs. So, again there is a need for OS. And not for something stripped down, but for something feature rich.
Then this is not "same physical address space shared by all nodes", and therefore doesn't meet my definition of "computer" (and is a cluster of many computers instead).embryo2 wrote:The common memory here is used for communication. It's not the goal to keep it on par with the processor's speed. It's like caches vs memory on x86.Brendan wrote:For "common memory"; the additional networking needed for cache coherency alone would probably triple the cost of the super-computer
Previous versions of my OS didn't use bytecode. For these, the executable file format included information about the CPU it was designed for (CPU type and a 256-bit bitfield of required/used CPU features) plus some other information (how much processing, memory and communication the executable does); where all this information (plus load statistics, etc) is used by the OS to determine where the executable should be run.embryo2 wrote:Is your OS capable of compiling a bytecode for different processors? Do you use AOT? How do you decide on the question of what processor should be used by which application? How do you know if application1 needs more processing power than application2? Or are you going to run the same application on different CPUs? And to have it compiled for every CPU.Brendan wrote:For current "80x86 NUMA" hardware, you typically want to run a process on the same NUMA domain so you can allocate memory that's "close" to that NUMA domain (and avoid the penalties caused by accessing memory that's further away from those CPUs). If you migrate a process from one NUMA domain to another you end up with "worst case NUMA penalties" (or re-allocating all the memory the process uses to avoid those penalties, which isn't cheap either); so you try very hard to make sure processes aren't migrated from one NUMA domain to another. Also; for most normal OS's there's a "CPU affinity" feature that allows a process to be "pinned"/restricted to a sub-set of CPUs.
If you add "CPU feature constraints" on top of that you end up with something approximating my OS design. To do this you just use feature's the process requires to pre-determine the CPU affinity for the process. E.g. if some CPUs support a feature (e.g. AVX-512) and some don't; then you setup the CPU affinity so that a process that uses that feature can't be run on a CPU that doesn't support it. Note: It's slightly more complicated than this because supporting different CPUs also effects the code that handles the "delayed FPU/MMX/SSE/AVX state saving" mechanism).
I do these things for normal 80x86; so for a theoretical/future "normal Xeon + Phi hybrid" system it'd be no different to what I already do and I doubt I'd need to change anything at all.
For my system an application is multiple cooperating processes. All of the stuff above (to determine where an executable/process should be run) only cares about processes, and not applications, and not threads. For a multi-threaded process, each of the process' threads use CPUs that match the process' requirements.embryo2 wrote:And to cope with it's thread interaction when threads are running on different CPUs? It isn't as simple as "I'd need to change anything at all".
You're under-estimating what would be required to compete favourably against existing OSs. If people start switching from existing OSs to your OS because your OS looks better; then within 6 months the existing OSs will improve their appearance and you'll be screwed.embryo2 wrote:If it's about rational assessment of your efforts then it's better to concentrate on something visually attractive or other "cool" stuff. But if it's about perfectionism then yes, you can try to justify your efforts with thoughts like above, but it's just for your personal consumption. Psychologically it helps, but [efforts]/[attracted users] ratio won't justify it.Brendan wrote:Mostly what I'm saying is that; using a fair/unbiased information only, I want (at least some) consumers/users to choose my OS at some point in the future; and to do that I need to create reasons why (using a fair/unbiased information only) at least some consumers/users would want to choose my OS instead of an alternative. Supporting different CPUs in the same system is just one more (relatively small/minor) way I can do that.