[Microkernel] Granting access to protected resources

[jamesmintram]

I was curious how microkernels go about granting drivers (for example) access to protected resources.

My concrete problem is granting a framebuffer driver exclusive access to the region of memory that is used by the GPU (on a Raspberry Pi)

Basically I see it working in a way where the driver requests this region of memory from the kernel which will then map it into the driver's address space.

The bits I am not 100% sure about are:

- How do I restrict mapping this region to a single driver process?
I am thinking some sort of flags which prevent multiple processes to map a page frame - making the functionality generic. Also, a driver would need to be able to request a physical address to be mapped into its address space. Something I (probably) don't want user processes to do.

- How should I manage the permissions surrounding this? (I do not want non-driver code to access this)
Driver processes should run in EL0 (unprivileged) - should they then have some permissions that allow it to do the above? Should a driver process be fundamentally different from a user process? or is it just a user process with some additional privileges?

[OSwhatever]

I suggest that you read a little about the L4 kernel. In the L4 kernel the owner of the pages for hardware resources are user space processes. These pages can be mapped or "granted" as it is called for L4 into another address space of another process. If you want to ensure exclusive rights, then the user process must ensure that it doesn't grant any pages for multiple processes.

If you want to restrict it further, that the user of those pages don't pass the pages along to other processes, then you must had some protection in the kernel but it also depends if you allow user processes to map pages themselves.

There are several ways of doing this but in practice you have to in general you have go via some user process that tracks user and access rights. Usually putting this into a microkernel would add too much functionality and make it too complex.

You mention that you don't want "normal" user processes to access hardware resources, then you kind of throw ACLs in the mix. Then you usually need a server that handles the ACLs as well. One possibility is that process that from the beginning owns the resource, asks the ACL server about the access rights for the process that requests the resource before granting any pages.

[Brendan]

Hi,

The bits I am not 100% sure about are:

- How do I restrict mapping this region to a single driver process?

Typically (for micro-kernels) each driver is a different process and (like almost always) every process has its own separate virtual address space; so mapping a device's memory mapped IO into that driver's virtual address space (so that the driver can access it) doesn't cause it to be accessible form any other process' completely unrelated and separate virtual address space.

I am thinking some sort of flags which prevent multiple processes to map a page frame - making the functionality generic. Also, a driver would need to be able to request a physical address to be mapped into its address space. Something I (probably) don't want user processes to do.

Yes, you'd need some kind of "does this process have permission to access the thing it asked to map into its virtual address space" check; where (for frame buffer) only the video driver would pass the check. For 80x86 you can get all of the resources a device driver needs to access from the device's PCI config space before you start the device driver (while you're figuring out which device driver to use for the device) and then tell the kernel "start this executable/driver and let it have access to these areas when it asks for them", where kernel could just keep track of a list of "special areas for this process" and check that something is in that list (and where normal processes that aren't drivers would have an empty list because they're not allowed to access any special areas). For your case (no hardware enumeration built into the device/hardware) you're going to have to rely on something else to get the information (device tree? compile time settings?) but it's vaguely similar (e.g. kernel is told what the device uses and knows what the process should be permitted to access before/when the device's driver is started).

Should a driver process be fundamentally different from a user process? or is it just a user process with some additional privileges?

No, a driver shouldn't be fundamentally different to a user process - they'd both run at the lowest privilege level, both use the same kernel API, etc. The only differences are things like what the process is/isn't allowed to map into its virtual address space (and if the process is notified when which IRQ/s occur).


Cheers,

Brendan

[Schol-R-LEA]

Note that, as I understand it, Exception Level in the ARM security model is not quite the same as the CPU Privilege Level in x86 (and not simply because it is listed from lower to higher (EL0 to EL3) rather than higher to lower (CPL0 to CPL3).

As an aside, the intended x86 security model (going back to the 80286) called for drivers to run in CPL1 and CPL2, something that was typical for OS designs between 1966 and 1982. However, even when the 286 was taped out, this approach had generally been discarded as excessively complex for too little gain. For better or worse, the most common OSes running today - Windows and the various *nixes - use two level, all-or-nothing security models. Regardless of whether this makes sense or not, AFAICT no significant OS today uses CPL1 or CPL2 at all.

But that's x86, so it isn't really relevant to this discussion. Most, if not all, newer ISA security models - including ARM - assume two-level security within the OS itself (more on this later) from the outset.

So, ARM security models. Here is what I gather from what I have read:

To begin with, it sounds as if the EL values only apply to handling exceptions; it is not the overall privilege level. The actual Secure/Non-Secure system states only indirectly relate to EL.

Next, it is worth bearing in mind that, just as x86 has changed over time, so has ARM, and significantly, different models of RPi use SoCs with different revisions of the ISA. The original Raspberry Pi/Pi A/Pi B+ used the Broadcom BCM2835 SoC, which was based on the ARM11 core design for the ARMv6 series of cores. The earliest Raspberry Pi 2s (v.1.0 and v1.1) used the ARM Cortex-A7 (ARMv7)-based BCM2836, but when Broadcom ended production of that model, they went to the Cortex-A53 (ARMv8)-based BCM2837, which was underclocked to 900MHz for the RPi2 but clocked to its full 1.2GHz for the RPi 3. This means that, unlike the earlier RPis, the RPi 2 and later support the 64-bit Aarch64 extensions.

If I understand correctly, there are no system-wide 'AArch32' and 'AArch64' modes; rather, an ARMv8 can be either AArch32 or AArch64, and always runs as one or the other, and in both cases, user programs can (always?) run in Aarch32 mode. While the new ISA (A64) is based on the older ARM design, and most of the opcode and register encodings in the individual (32-bit) instructions are the same, the way conditional execution works is completely different under the hood (though the older instructions using it still work when running as Aarch32).

Originally, the ARM system model had several system modes, but Exception Levels weren't part of it. The modes in the ARMv2 (the first version actually used) were User, Supervisor, Interrupt, Fast Interrupt, Abort (mostly for handling bus exceptions), and Undefined (for handling undefined instruction exceptions). ARMv4 added a System mode, IIUC as a way to switch from one privileged (not user) mode to another without having to use an interrupt. ARMv6 added a Monitor mode to work with the 'TrustZone' security extensions, which ARMv7 added on to. ARMv7 also added a Hypervisor mode to support virtualization, but apparently, it did not enforce secure virtualization - my impression is that Monitor mode controls didn't apply to the hypervisor, but I could be wrong.

From what I am seeing here, Exception Levels were added in ARMv8 as a way of providing secure modes for Hypervisors, or at least, of better managing which mode the exception handling was in when an exception occurs. For this reason, it sounds to me like EL only applies to PI 3 and later.

The ARMv8 Exception Levels, as I understand them:

• EL0 - User programs
• EL1 - Guest OSes
• EL2 - Hypervisor
• EL3 - Secure Monitor, only available in Aarch64 CPUs.

We really need someone with more (and more accurate) knowledge of the ARM to correct - and elaborate on - this.

EDIT: I am going to continue to look up information on this, as it is relevant to a project I am prepping on right now. I did find this, this, and this, but I haven't had a chance to read them yet.

[zaval]

oh, well. I guess, the OP was asking something what Brendan has answered to. Not about this ARM mess.
Aarch32 and Aarch64 are "execution states" of the processor. Both are 32 bit wide instructions (except thumb/thumb2 for aarch32). A64 and A32 are ISAs. Basically, A32 is the old ARMv7 compatible ISA. Almost the same, with some additions and redefinitions. But highly compatible. A64 is more MIPS-ish. And the most significant change in the execution state (Aarch64), as for me, - the amount of general purpose registers not just doubled, but in the absolute number, it went from 16 to 32! It's huge. MIPS and PPC both had 32 registers even being 32 bit yet, x64 doubled as well, but in the absolute number, it added only 8 registers, and here - 16!

Every EL can be in any of these two execution states (but if I understood right - only in one of them, so, for example EL3 could be implemented either in the aarch32 or aarch64 execution state on a machine, but not in both). Still, execution states of different ELs are not totally independent and switching between ELs is a little bit tricky.
Namely, EL switching happens only on Reset or taking an exception/returning from it.
In the latter cases, if the transition is from ELx->ELy where y > x, then the execution state can only either:
remain the same or be aarch32->aarch64.
On return - the same, but in the reverse order.
If the exception doesn't change EL, then the execution state cannot change too.
And finally, exception cannot be taken into the lower EL.

ARMv8 Exception Levels are loosely an ARMv7 Privilege Levels evolvement. They are rather terminological redefinitions of the latter. In the attempt of making it clearer. yup. But in fact, if you are OK with 5842 pages of ARMv8 Architecture Reference Manual (ARM), it's comprehensible. with some effort. Section G4.1 "Execution privilege, Exception levels, and AArch32 Privilege levels", p. 4042, is a good point to try to get the idea of what they thought about PL/EL connection.
ELs are not exactly about Secure/Non-Secure state. Yes, EL2 could be only Nonsecure and EL3 is always Secure no matter what. But EL1 and EL0 could be both. Exception levels are not about "hypervisors" either, they really are mostly oriented at the execution state definition for the environment where exceptions are taken into. Some exceptions are architecturally assigned to be taken into predefined ELs, some are configured. EL1 is for OSes and not just "guest" OSes. EL0 is user mode. Basically every EL except EL0 is privileged.
EL3 controls everything. On ARMv7, it was Monitor Mode. Secure world has even a separate system address space. I cannot say about it much yet. Entering it is through a special software interrupt gateway instruction - SMC. On reset, CPU starts into EL3. How does this God Mode distinguish between Secure and Nonsecure system spaces - forgot completely.
EL2 is virtualization bullshit. just a noisy overhead. As EL3, does have its own, single virtual address space (for what?). Adds yet another stage of memory mapping for EL1 (turning it into VA<->IPA<->PA chain instead of normal VA<->PA and thus - introducing "stage 2 MMU" *rolleyes). Basically if you are not excited about HV (like me) - forget about it ASAP. Yes, it's Hypervisor Mode of ARMv7 and it's for hypervisors for intercepting real hardware interrupts etc. The only interesting thing for me is how to disable that thing. I found out that on a board I try to work with, uboot transfers control to my code in EL2. So, given all the clumsiness of EL switching and the fact, we cannot really do multiple VA spaces over here, we need to jump into a real OS level, EL1.
EL1, finally a normal OS level. Can be either Secure or Non Secure. Here, you have a real separation for process(or) address spaces - processes. You configure some pages (blocks) to be either EL0 accessible or not - nonprivileged access. Also, this is what we think of when thinking about memory "protection" between different processes. It is here. And it's almost the same as with x86.

From what I am seeing here, Exception Levels were added in ARMv8 as a way of providing secure modes for Hypervisors

No. Ironically, "hypervisors" are always Nonsecure on ARM. It's 2 totally different stuff - HV and Secure/Nonsecure split. The former, well, is HV bullshit. And the latter is making 1 machine to look like 2, but with really 2 independent HW resources. One is for "teh security". Different system address space, different OS, different peripheral set.
Monitor/EL3 thing is a way more interesting than HV. It challenges you to think out new things for the architecture it introduces really interesting things. But it's not always accessible.

or at least, of better managing which mode the exception handling was in when an exception occurs. For this reason, it sounds to me like EL only applies to PI 3 and later.

yes, exactly for this. It applies for any ARMv8 CPU SoC. Not necessarily 64 bit. after all, there are such things as Cortex-A35, a 32 bit only ARMv8 CPU IP. Unfortunately yet not found on the SBC range.

EL3 - Secure Monitor, only available in Aarch64 CPUs.

Wrong. can be available in both states. It's up to the implementer what to choose. And be sure they don't choose to be straightforward with this respect. they, being lazy, stay with the 32 bit one, introducing a great deal of mess and confusing for everybody so excited to run 64 bit on 64 bit machines (with me includingly). This is a sounding example, read the comment.