OSDev.org

Hi,

So many interesting posts!

I think most of you are confused about ARM a bit. ARM, as an architecture is pretty well documented, and easy to program (there are many high level language compilers available, C, Pascal, Rust whatever). Writing Assembly is a lot more difficult, because despite of the fact it supposed to be RISC (reduced instruction set), ARMv8 has more than 1200 instructions, lot more than Intel. But it is well documented, and specs are available for free.

On the other hand, you can't do anything with a CPU alone. You need some other circuitry as well, like a timer or an interrupt controller, or an uart to communicate with the user. These additional chips were somewhat standardized for the PC. PC is not just about the x86 CPU, but also about the 16550 (UART), MC146818 (RTC), i8259 (PIC) etc. compatibility. In the embedded world, these additional chips are usually shipped in a SoC (where the ARM CPU is just one little part of the chip). Now these SoCs are not standardized, they include various circuits with wide range of interfaces. They often contain chips that aren't documented at all. So for example, the fact that Raspberry Pi is booting on the Video Core, has nothing to do with ARM per se; that's a specialty of the Broadcom SoC. Likewise BCM2387 (RPi3) having a multicore QA7 interrupt controller and BCM2711 (RPi4) having a GIC has nothing to do with ARM itself. ARM based machines are not, and never were standardized as the PC

I'm pretty sure Apple will never give away a documentation on their SoC either (in which ARM is just one of the many), they will hide that and developers will be forced to use their bloaty XCode ecosystem. Same as how the low-level WinNT kernel interface isn't documented at all. They won't care about ARM, because their IP is in their SoC (which they manufacture themselves).

Korona wrote:The point is: ARM-based Macs will have a locked down bootloader, you cannot install other OSes on these things.

Is this any different to Intel based UEFI machines with forced Secure Boot? FYI, "Secure" here is just a buzzword, a Real Marketing Bullshit (TM) as it has nothing to do with actual security. You can only boot a loader on those PCs if you pay insanely huge amount of money to Microsoft to sign your executable. MS won't check the executable for malicious code or vulnurabilities, all that they care about if you've paid the signing fee or not.

Cheers,
bzt

bzt, WinNT interface is well documented, it's how different vendors provide drivers for their peripherals. they write drivers, using that interface. this is kernel exported functions, driver models, development kits etc. all is documented, samples are given. with the rather free license. A so called Native API, that one residing in ntdll.dll on the user mode side is intentionally not documented, but not for reasons you would think - it's because, it's an internal API, that way it's easier to change it and not break millions of third party programs. using it won't let you any advantage over WinAPI - a well documented interface for third party programs.
this is not how apple is doing with their ARM macs at all. apple, if really won't let run anything (not just by not providing the specification to their SoC, but locking the fw from loading any loader, other than their) is just purely p*ssy. I don't know if it's "anticompetitive" or not, but it makes them look like 4ssholes. however, it's not a big deal, since apple is not the only one who makes ARM chips. the whole buzz about it is because well, as always, - there is a lot of people that fell to worshipping the bitten apple. even if they are peeing them on the face.

about unifying ARM platform, it's all gonna be fine. ARM is getting ACPI/UEFI, so this is a temporary problem if at all. as for now, you either pick well documented SoCs, and 1) use the half assed device tree for device discovery (root enumeration) or write ACPI tables for the SoCs vendor. *trollface then your OS is ok, because everything is familiar. USB is USB, PCIe is PCIe etc. PCs also have peripherals, enthusiast developers have no way to get to them due to the lack of any documentation. they are even called the same on both platforms! read GPU, VPU, display controllers, wifi/blutooth. or ... program the one and only RPi. writing the code, that no other ARM machine could take advantage of. but hey, if you notice only RPi, you deserve what you get.

bzt wrote: Is this any different to Intel based UEFI machines with forced Secure Boot?

don't you see the difference? between "need to be signed for running" and "impossible due to the lock by fw"? come on.

bzt: your points on AMR's CPU vs. platform documentation are true. The CPU is well documented but the ecosystem is not.

bzt wrote:Is this any different to Intel based UEFI machines with forced Secure Boot? FYI, "Secure" here is just a buzzword, a Real Marketing Bullshit (TM) as it has nothing to do with actual security. You can only boot a loader on those PCs if you pay insanely huge amount of money to Microsoft to sign your executable. MS won't check the executable for malicious code or vulnurabilities, all that they care about if you've paid the signing fee or not.

There is no difference of course. But there is a crucial difference regarding compliance: in order to receive a "Certified for Windows" logo, OEMs must add the ability to disable secure boot and/or to add user-defined keys. That's one of the things that Microsoft did right here. No such provision exists on ARM systems.

By the way, while Microsoft will not audit your code but they will totally refuse to sign code that has known vulnerabilities that allow circumventing secure boot (such as loading kernel modules, writing /dev/mem or kexec()). Hence, there was an effort to get the Linux lockdown patchset into the mainline kernel and indeed, it has been merged in 5.4.

"Forced Secure Boot"

Is there such a thing on PCs? I can disable secure boot on all of my PCs, including the Microsoft Surface Book. Now, how do I boot my own OS on my iPad?

What about MIPS? Are they well documented? Is the Malta board well documented? bzt is right, ARM as a ISA is very well documented is very well documented. The ARM spec is 51MB in size! I think another reason why x86 is so documented is because we all have had years to tinker with these pieces if hardware. Overall, the only reason why you should use x86 for OSDev is not simplicity, nor it being a modern, lightweight architecture. The reason is because it is very well documented.

zaval wrote:bzt, WinNT interface is well documented

Nope, only the DLL interface is documented (USER32.DLL, SHELL32.DLL, KRNL32.DLL etc.) I was talking about the low level interface (the one between the DLLs and the actual kernel) and the NT internals. Those are not public at all. The best I could find is Tanenbaum's Modern Operating Systems book, 4th edition, Chapter 11 "Case Study 2: Windows 8". But it is not an official Microsoft document, not up to date, and it is pretty vague at certain points and only talks in general, simply because Tanenbaum himself couldn't get all the specs.


On the locked down topic: yes, for now you can disable secure boot. But what if a simple and single modification (through a forced update) removes that option from the menu? What can you do after that? The trend they trying to lock down ALL platforms is sadly happening. But I agree with you and I strongly believe users must boycot such hardware. I for one won't buy any hardware if I can't install my own keys on it, that's for sure. Thankfully the biggest manufacturers don't like US companies that much, so China will always provide Microsoft-free bootable machines Or someone will leak the private keys for singing (as it happened with PlayStation).

And yes, Macs don't have that option in the first place (it is disabled by default), but you can still hack your way in. It just doesn't worth it, because even if you manage to boot your code, all the environment is undocumented, so you don't know how to get the frame buffer, where to find the disk controller etc. This is a classic example of security through obscurity. But so far ALL STOs have been hacked

Cheers,
bzt

fresh news, nvidia considering arm. i am not fan of arm and sounds disheartening. https://www.forbes.com/sites/davealtavi ... ef84b75742

Read https://abcnews.go.com/Business/wireSto ... s-71974171. Sounds like the same story that produced x86 in the first place. Intel needs to be careful. AMD is going to develop a next gen chip like the AMD64 and Intel will just follow behind. AMD should be a big concern to Intel.

ggodw000 wrote:fresh news, nvidia considering arm. i am not fan of arm and sounds disheartening. https://www.forbes.com/sites/davealtavi ... ef84b75742

May I ask why you are not a fan of ARM? Anyway, the story itself is a non-issue. The ARM holding has always just licensed the Core IPs to actual manufacturers. All this means that in future, the profits from these licenses will go to the green-eyed monster instead of SoftBank. Oh, and they get to decide if someone should not get a license. One more reason to dislike them, but then the entire tech industry is like a pit of vipers where every single one is trying to eat all the others. Admittedly, nvidia is better at eating the competition that most of their competition.

Anyway, on the tech-front, this is a non-issue all around.

nexos wrote:Read https://abcnews.go.com/Business/wireSto ... s-71974171. Sounds like the same story that produced x86 in the first place. Intel needs to be careful. AMD is going to develop a next gen chip like the AMD64 and Intel will just follow behind. AMD should be a big concern to Intel.

AMD or Intel is like raspberries or strawberries. The difference isn't that big. And AMD is already a big concern for Intel, given they are the only surviving competition for x86 processors. But this news article may only mean that Intel becomes like the ARM holding, only licensing their cores to manufacturers instead of trying to sift the sand themselves. On the tech-front, again, a non-issue. I still think x86 is here to stay.

bzt wrote:Thankfully the biggest manufacturers don't like US companies that much, so China will always provide Microsoft-free bootable machines

I don't think the day is far off when everything coming out of China will allow you to boot absolutely any OS...

...provided that that OS is signed by the Chinese Communist Party.

And the price for getting your OS signed by the CCP? All the surveillance.

Paranoid much? lol.

nullplan wrote:

ggodw000 wrote:fresh news, nvidia considering arm. i am not fan of arm and sounds disheartening. https://www.forbes.com/sites/davealtavi ... ef84b75742

May I ask why you are not a fan of ARM? Anyway, the story itself is a non-issue. The ARM holding has always just licensed the Core IPs to actual manufacturers. All this means that in future, the profits from these licenses will go to the green-eyed monster instead of SoftBank. Oh, and they get to decide if someone should not get a license. One more reason to dislike them, but then the entire tech industry is like a pit of vipers where every single one is trying to eat all the others. Admittedly, nvidia is better at eating the competition that most of their competition.

Anyway, on the tech-front, this is a non-issue all around.

nexos wrote:Read https://abcnews.go.com/Business/wireSto ... s-71974171. Sounds like the same story that produced x86 in the first place. Intel needs to be careful. AMD is going to develop a next gen chip like the AMD64 and Intel will just follow behind. AMD should be a big concern to Intel.

AMD or Intel is like raspberries or strawberries. The difference isn't that big. And AMD is already a big concern for Intel, given they are the only surviving competition for x86 processors. But this news article may only mean that Intel becomes like the ARM holding, only licensing their cores to manufacturers instead of trying to sift the sand themselves. On the tech-front, again, a non-issue. I still think x86 is here to stay.

Nothing against the ARM, I am averse of studying new architecture after spent about two decades on x86.

ggodw000 wrote:Nothing against the ARM, I am averse of studying new architecture after spent about two decades on x86.

I would suggest you look at another architecture - any other architecture. It has been said that one does not know their mother tongue until one learns another language, and that is not just true for languages. If you really want to know why x86 is a hack of a hack, you need to see how things are done elsewhere. But, if you are looking for a simple architecture to study, might I suggest Microblaze? It is a softcore architecture for FPGAs, developed by Xilinx. It was my first foray into architectures beyond Intel, and it was eye-opening, how simple things can be, and you still basically have all you need for a desktop CPU. Privilege levels? There is a bit in some register for that. No descriptor table nonsense. Paging? Stolen from PowerPC. A bit weird, but you get used to it. And in some cases, you get TLB miss interrupts instead of automated page table walk, allowing you to implement whatever page table scheme you like.

Most of all, you probably should not be averse to learning new things that may prove useful at some point in the future. I would not want to be tethered to x86.

zaval wrote:A so called Native API, that one residing in ntdll.dll on the user mode side is intentionally not documented, but not for reasons you would think - it's because, it's an internal API, that way it's easier to change it and not break millions of third party programs. using it won't let you any advantage over WinAPI - a well documented interface for third party programs.

Except that native api doesn't actually change. And programs (e.g. google chrome) do depend on it. The syscall numbers do change, but ntdll doesn't. It's basically the equivalent of vdso on linux. The actual reason it's undocumented is that nt was originally supposed to be a polymorphic OS. There were multiple subsystems: win32 subsystem, posix subsystem, wince subsystem, etc. They wanted applications to be written targeting one of those subsystems, not the 'native api'. Today, the win32 subsystem is the only one that survives (unless you count uwp; I don't), so windows is synonymous with win32, and the win32/native api distinction isn't very meaningful.

linguofreak wrote:I don't think the day is far off when everything coming out of China will allow you to boot absolutely any OS...

...provided that that OS is signed by the Chinese Communist Party.

And the price for getting your OS signed by the CCP? All the surveillance.

Hahaha, you could be right! The problem is, there's no guarantee MS doesn't do surveillance. Sadly, Win10 is known to be spying on you, even if you choose the no telemetry option.

Then the only solution is to turn Secure Boot off, or only buy a machine which has a "setup mode" available (so you can install your own PK that you can actually trust).

Cheers,
bzt