ACPI AML

[Candy]

Hi all,

So I decided I wanted to get a proper table to map PCI device interrupts to IO-APIC inputs. That requires parsing AML files and I figured, how hard can it be?

Actually, it's really hard because the spec is not entirely sane. So far I've found a few things:

- A minor claim that a 28-bit number can fit 0xFFFFFFFFF - somebody probably mistyped and it wasn't noticed.
- In the parsing language for AML, a Target can be either a SuperName or a NullName. A SuperName can be a SimpleName, which can also be a NullName. That's a basic shift-reduce conflict right there.
- Then if you look at SuperName, it can be a Type6Opcode, which can be a UserTermObj. That sounds important, but it's actually not defined in the entire document. That makes it hard to parse as AML isn't actually a language that you can parse without knowing what fields are (unlike most other file formats).

In short, anybody feel like joining the adventure?

[Owen]

I'm curious as to why you don't just piggyback on Intel's adventure?

[jnc100]

I started writing a parser in C# (because my OS doesn't support C libraries) but came up against most of the issues you noted. In particular, one of the problems I remember was that methods can be called before they are defined. Unfortunately, to correctly parse the call site signature, you need to know how many arguments the method has, which you only find out later in the stream. Thus you need to scan over first looking for all valid method definitions (bearing in mind that some may be defined twice in different IfElse blocks) to build up a database of definitions, then parse again.

My efforts are available here if you're interested. However, if you do support C then ACPICA is surely the way to go.

Regards,
John.

[Candy]

I'm curious as to why you don't just piggyback on Intel's adventure?

Stubbornness, unwillingness to integrate third-party code I don't know everything about into my kernel. I'm not making a kernel as a realistic target, I'm making it as a learning experience and as an all-self-written project. In particular for ACPI AML, I'd like to know where to hook in extra checks on what it's doing exactly.

I started writing a parser in C# (because my OS doesn't support C libraries) but came up against most of the issues you noted. In particular, one of the problems I remember was that methods can be called before they are defined. Unfortunately, to correctly parse the call site signature, you need to know how many arguments the method has, which you only find out later in the stream. Thus you need to scan over first looking for all valid method definitions (bearing in mind that some may be defined twice in different IfElse blocks) to build up a database of definitions, then parse again.

My efforts are available here if you're interested. However, if you do support C then ACPICA is surely the way to go.

Regards,
John.

Looks great. I was hoping that any such unparseable location could be skipped for a while using pkglength of its containing package. Also, for now I want to get the object hierarchy out first and then see if I want to run any calls. Thanks for the link though!

[mateuszb]

Definitely use ACPICA. Lots of it is pure error handling code and handling of improperly formatted AML and invalid/broken ACPI tables. They happen more often than you think.
That code is pretty much in every OS and nobody writes it. Linux uses ACPICA for example.

[SpyderTL]

I tried writing my own parser, but I ran into an issue where it was impossible to determine where one particular element (which contains sub-elements) ended, and the next one began.

It's been a few years, but I see if I can find that code.

[jnc100]

I was hoping that any such unparseable location could be skipped for a while using pkglength of its containing package.

The problem comes in parsing UserTermObj. The corresponding BNF (unrelated bits skipped with '...') is:

Code: Select all

TermList := Nothing | <TermObj TermList>
TermObj = ... | Type2Opcode                    (TermObj is essentially a statement - either in the global scope or within a method)
Type2Opcode = ... | UserTermObj
UserTermObj = NameString TermArgList           (UserTermObj is the encoding of a method call)
TermArgList = Nothing | <TermArg TermArgList>
TermArg = Type2Opcode | ...                    (TermArg is the argument to a method)

Without knowing the length of TermArgList, it is impossible to know whether any subsequent Type2Opcode is of type TermArg or of type TermObj (i.e. part of the method arguments or the next instruction in the stream), thus you may accidentally skip device definitions (which are encoded as Type2Opcodes) if you don't know the number arguments passed to a preceding method call.

Also, for now I want to get the object hierarchy out first and then see if I want to run any calls.

Unfortunately its not that simple - some devices are only defined within _INI methods, or dependent on the result of global IfElse statements (which may in turn require running a method first to evaluate).

Suffice to say I never managed to find a robust way to handle all these issues.

Regards,
John.

[Owen]

As far as I can tell, there exist a grand total of three (three!) production quality AML parsers:

• Microsoft's. You'll, obviously, find this in Windows
• OpenBSD's. You'll, obviously, find this in OpenBSD. As I understand this, it was kind of implemented out of stubbornness
• ACPICA. You'll find this in Linux, Solaris, Illumos, FreeBSD, NetBSD, DragonflyBSD, OS X, and pretty much everything else which has an ACPI implementation. People who value their time over stubbornness and who didn't already have an ACPI parser pick ACPICA

I don't really see why you would pick to write your own. AML is this horrible combination of a bytecode - i.e. not human readable, and a lanuguage - i.e. something which needs parsing by a complex stateful parser. I would characterize whoever designed it as not quite right in the head; as, really, I can't find any good reason for other absurdities of it (e.g. all names confined to 4 characters), except for that I'm pretty sure it was designed by committee, and therefore even if all the inhabitants of that committee were quite intelligent, the politics of said committee were obviously oriented towards providing grey goo, not a nice standard.

Of course, don't forget the aforementioned fun of invalid AML and tables (which work due to bugs in Windows parser and therefore are defacto valid because not booting on random machines is generally considered unacceptable). which ACPICA helps paper over thanks to the work of the contributors to the various other projects using ACPICA.

Also don't forget that AML is only a tiny corner of that huge spec. There is still an awful lot of complexity to ACPI. For your intended goal of getting PCI IRQ routing, remember that you need to get to a stage where you can invoke any and all AML, because they aren't declarative. You invoke _PIC(1) to tell the BIOS you're using APICs (as opposed to _PIC(0), the legacy 82C59 PICs), and then invoke the _PRT() method on each slot for each IRQ to figure out where the IRQ goes on the (A)PICs.

If ACPICA is cheating, remember this is a game where everybody else cheats, including the multi-billion dollar companies.

[Brynet-Inc]

ACPICA. You'll find this in Linux, Solaris, Illumos, FreeBSD, NetBSD, DragonflyBSD, OS X, and pretty much everything else which has an ACPI implementation..

It's really great that Intel has a permissively licensed reference implementation, but it's also very unfortunate that it's practically the only one in use.. and that makes it really difficult for anyone to use anything else.

Even OpenBSD developers agree that it's a BAD thing that OpenSSH is the de facto implementation in the world, there needs to be multiple implementations of a specification.

OpenBSD's. You'll, obviously, find this in OpenBSD. As I understand this, it was kind of implemented out of stubbornness

This is also true, working around buggy AML code and interpreting ambiguous documentation is almost an exercise in futility.. but unsurprisingly it mostly just works. Developers use it on their laptops, servers and routers. ACPI suspend resume (S3) has worked for years, even on systems that the ACPICA stack does not. ACPI hibernation (S4) is still a work-in-progress.. but the results are pretty damn impressive, no?

[tom9876543]

Linus Torvalds has described ACPI very well:

ACPI is a complete design disaster in every way. But we're kind of stuck with it. If any Intel people are listening to this and you had anything to do with ACPI, shoot yourself now, before you reproduce.

The fact that ACPI was designed by a group of monkeys high on LSD, and is some of the worst designs in the industry obviously makes running it at any point pretty damn ugly.

Intel has made a couple of extremely bad design decisions during its history. ACPI would be the number 1 biggest Intel f#$k up.

[Icee]

ACPI would be the number 1 biggest Intel f#$k up.

UEFI, anyone?

[Combuster]

ACPI would be the number 1 biggest Intel f#$k up.

UEFI, anyone?

Offtopic troll.

[Brendan]

Hi,

Intel has made a couple of extremely bad design decisions during its history. ACPI would be the number 1 biggest Intel f#$k up.

Note that ACPI is committee that's mostly controlled by Microsoft, not Intel.

The main complaints against ACPI are:

• It would've been better to standardise the hardware and avoid an over-complicated solution that's designed to support anything imaginable
• The ACPI specification itself is not very specific (a huge mess with missing information, conflicting information and wrong information)
• The OS has to identify itself so that AML can behave differently to suit whatever the OS says it is. This means that an alternative OS is screwed (no motherboard manufacturer is going to add support for "yourOS" or even "Linux" to their AML); and to avoid having various features locked out by AML an alternative OSs have to lie and pretend that it's a version of Windows.
• How Windows uses AML is mostly undocumented; so if your OS lies and pretends it's a version of Windows you're screwed.
• It's non-testable. What I mean here is that:
  • There's no guarantee that the firmware's AML is not buggy.
  • You can't test it on one machine with correct AML and assume it works on another machine with correct AML.
  • If (by luck or brute force) you manage to get it to work perfectly on a computer; something as simple as a firmware update can change the AML and break things on that computer.
  These things combined mean that it's impossible to claim that your OS works properly (you can only claim that it seems like it works on some computers sometimes).

Stubbornness, unwillingness to integrate third-party code I don't know everything about into my kernel.

ACPI's AML itself is "third-party code you can't know everything about" that you're integrating with your code. The fact that it's interpreted rather than native code doesn't make much difference - its still capable of doing anything it likes (including crippling the machine) and is still a huge security risk.

My plan is to use SMBIOS to get the motherboard vendor and product strings (where available), and get the PCI host controller's vendor ID and device ID (where available), and the PCI to LPC bridge's vendor ID and device ID (where available); and combine these things to form a "motherboard identifier". I'm also going to calculate a checksum of the firmware's AML. Then I'm going to have a set of "motherboard drivers" (as native code), where the OS uses the AML checksum and "motherboard identifier" to determine which motherboard driver to use. If there is no motherboard driver the OS will fall back to a "legacy mode" where there's no power management, etc (although it'd be possible to have a generic motherboard driver based on something like ACPICA that does use AML as a fall back, where "legacy mode" is used if there's no AML).

I'd also provide some sort of "motherboard driver template package" that includes all the boilerplate code, so that someone writing a motherboard driver only has to add the missing functions (based on information they get from disassembling the firmware's AML and/or chipset datasheets, etc). I'd also consider having some sort of tool that disassembles the AML and inserts it as comments into the correct places in the motherboard driver template. Note: The idea of having a tool to convert the AML into source code (e.g. an automated native motherboard driver generator) has occurred to me, but I want to ensure that developer/s have checked the motherboard driver's source code and don't think creating a tool like this would be worth the effort.


Cheers,

Brendan

[embryo]

My plan is to use SMBIOS to get the motherboard vendor and product strings (where available), and get the PCI host controller's vendor ID and device ID (where available), and the PCI to LPC bridge's vendor ID and device ID (where available); and combine these things to form a "motherboard identifier". I'm also going to calculate a checksum of the firmware's AML. Then I'm going to have a set of "motherboard drivers" (as native code), where the OS uses the AML checksum and "motherboard identifier" to determine which motherboard driver to use. If there is no motherboard driver the OS will fall back to a "legacy mode" where there's no power management, etc (although it'd be possible to have a generic motherboard driver based on something like ACPICA that does use AML as a fall back, where "legacy mode" is used if there's no AML).

But diversity of such approach is really great! I'm afraid you would end up with a set of about dozen motherboard drivers multiplied by a dozen AML firmware versions. And it will never be close to the diversity of the real world, where we can find thousands of motherboards multiplied by thousands AML firmware versions. It means you almost always will fall back to "legacy mode".

May be it is better just to use ACPICA without serious attempts to organize all existing hardware?

I'd also provide some sort of "motherboard driver template package" that includes all the boilerplate code, so that someone writing a motherboard driver only has to add the missing functions (based on information they get from disassembling the firmware's AML and/or chipset datasheets, etc). I'd also consider having some sort of tool that disassembles the AML and inserts it as comments into the correct places in the motherboard driver template.

The idea of such "open process", designed to cover as much of hardware as possible, is interesting. But it's widespread acceptance depends on a number of motherboards supported, until some threshold there will be no viable contribution. Such problem should be considered at the design stage of your approach.

But if you succeed - there can be really interesting alternative to AML

[Candy]

My plan is to use SMBIOS to get the motherboard vendor and product strings (where available), and get the PCI host controller's vendor ID and device ID (where available), and the PCI to LPC bridge's vendor ID and device ID (where available); and combine these things to form a "motherboard identifier". I'm also going to calculate a checksum of the firmware's AML. Then I'm going to have a set of "motherboard drivers" (as native code), where the OS uses the AML checksum and "motherboard identifier" to determine which motherboard driver to use.

Was also considering a thing like an "AML checksum" but mostly to download a patched version of the AML that's validated & bug-free. Using it to find out which mainboard driver to use instead is perhaps even a better idea... at least more portable & reliable than the actual AML.

I was also considering moving all the ACPI code to userspace to be able to limit it where possible. Not going to be too useful though without VT-d as you can't abstract out device access to hardware at all, so any malicious AML code could still write to any actual memory through a device... so no security benefits there...

Only secure way to use AML is to not actually use it. What's the alternative?