OSDev.org

I've looked at the OHCI-specification, and it cannot handle 64-bit physical addresses at all. It is fine for the scheduling lists (can "easily" be allocated below 4G), but there are issues with data. I suspect the only possibilty is to use "double buffering" and have buffers allocated below 4G that are used for data-transfers. It's a pain, but should work.

The EHCI chip is similar, but it has optional support for 64-bit physical addresses. I suppose a similar solution is required here, maybe complemented with the use of the 64-bit interface if it is available.

AHCI presents a similar problem. The specification defines 64-bit addresses for all fields, but it seems like support for 64-bit addressing is optional. I wonder if this means that if the hardware platform supports more than 4G, so would the AHCI controller, or maybe it could potentially mean that the machine might have more than 4G of memory, and an AHCI controller that cannot handle 64-bit bus-mastering. If the latter is never the case, it would be simple to just use the 64-bit addresses of buffers in the AHCI-controller, and don't worry about if the controller supports 64-bit or not. Otherwise, there would be a need to introduce new code in the disc buffer manager that makes sure that buffers passed to a particular disc never can be above 4G. I would rather not do the latter if it is uncommon. Especially since IDE never presents this problem as it always uses PIO-method.

So how do people that write long-mode OSes tackle these issues?

Generally a good physical memory manager will have different categories and priorities for allocations. I think a general setup for a 64-bit OS would be roughly three categories: Below and up to 16 MB for legacy DMA, addresses below the 32-bit line and then everything else. Any device that is 32 and possibly 64-bit capable will have to choose which pool to allocate from based upon feature support indicated by the device. As you said this would likely be in your disk buffer code and can be as simple as a lightweight wrapper which checks a flag and then chooses to use the 32 or 64 bit physical allocator. As for how common it is, who knows but the fact that at least two major host controller interfaces for USB and SATA require that functionality means you should really have such functionality in your driver API.

Yes, but the allocator cannot handle the situation where applications send buffers to USB. Memory in the application is typically allocated by pagefault handler which has no idea what it would be used for. This means that the USB driver cannot control how user-supplied buffers are allocated, but instead have to live with whatever it is passed.

For the AHCI-driver, the situation is somewhat different in that it is normally not passed buffers directly from applications, but from a kernel-mode driver that keeps the buffers. This means it could potentially allocate them only below 4G if it knows the driver cannot handle above 4G physical memory.

I forgot about modern network chips. Some of those also use in-memory structures, and are passed data from applications. Although neither my RTL8139 nor my RTL8169 driver suffer from this problem, but instead uses pre-allocated buffer-rings.

For small amounts of data (e.g. that transferred by USB control endpoints), it's almost always more efficient to just copy the data between address spaces, and so the USB driver can receive the data straight into memory allocated for device use

I'd require everyone to use specially allocated buffers - i.e everyone would allocate a "buffer handle" with a given amount of storage, and optionally pass a device handle indicating what device it is being allocated for (and therefore the address space constraints in play).

The device driver would then call the "map for DMA" method on the buffer, passing in its device handle. The buffer manager would then look at the memory backing the buffer, and if it was not aligned sufficiently for the device, or not in the address space region the device can handle, reallocate it such that it matched the device's constraints.

The device driver would return the buffer to its caller (since it's now setup correctly for the device's use - it makes sense to reuse memory which has (possibly expensively) been allocated specifically to match the needs of the device)