What to do if you need to allocate physical memory below 4gb

[devc1]

Some devices only support 32 bit addressing and optionally 64 bit (EHCI, AHCI...). In my OS, I just set a flag to the page allocator to search for memory below 4GB. Does Linux and other OSes do the same thing ?

[nullplan]

Basically. Linux has the idea of a "DMA mask". It has an extra DMA allocator to allocate device memory, and that allocator is constrained by the position of the device in the device tree. EHCI can only deal with 32-bit addresses (most of the time, at least), so it gets a mask of 32 bits. It can be constrained further if a specific device is detected that only supports 31 bits. Then most EHCI devices are PCI devices, so the memory they can access is constrained to the ability of the PCI host bridge to forward requests. On the PC, the PCI host bridge typically forwards requests to all physical memory and without offset, but that doesn't need to be the case.

In the end, all memory allocated to a device has three addresses: The virtual address in the CPU, the physical address from the point of view of the CPU, and the DMA address, which is the physical address of the memory from the point of view of the device. The last two needn't be equal. They are on the PC, but not everything is a PC.

[rdos]

Basically. Linux has the idea of a "DMA mask". It has an extra DMA allocator to allocate device memory, and that allocator is constrained by the position of the device in the device tree. EHCI can only deal with 32-bit addresses (most of the time, at least), so it gets a mask of 32 bits. It can be constrained further if a specific device is detected that only supports 31 bits. Then most EHCI devices are PCI devices, so the memory they can access is constrained to the ability of the PCI host bridge to forward requests. On the PC, the PCI host bridge typically forwards requests to all physical memory and without offset, but that doesn't need to be the case.

In the end, all memory allocated to a device has three addresses: The virtual address in the CPU, the physical address from the point of view of the CPU, and the DMA address, which is the physical address of the memory from the point of view of the device. The last two needn't be equal. They are on the PC, but not everything is a PC.

I don't use flags in the calls (don't like enums or flags-bits in APIs), but instead provide several allocator calls. I have variants that allocate memory below 4G (some 32-bit PCI devices) and variants that can allocate anywhere, but that allocates above 4G if there is free memory above 4G. I have variants that allocates one 4k page (typically used by page-fault handler), one that allocates one 2M page (used for large blocks), and a special function that can allocate any number of consecutive 4k pages (typically used by PCI devices that need large memory schedule blocks).

For easy translation between linear and physical addresses (and the reverse), I have an allocator built on a 4k page that is optimized for same-sized allocations. For large allocations, it will automatically add new 4k pages as needed. The header contains both the linear and the physical address, and it is quick to check if a particular address from a memory schedule is inside the allocator, and to return it's linear address.

I only support PC configurations, and assume devices see the same address space as the CPU.

[devc1]

So from what you say, Linux remaps DMA addresses to be accessible by the device using 32 bits for example. If the address is 0x10000000000, which is above 1gb the CPU accesses it as 0x10000000000 as physical address but we can make it accessible to the device using address 0x12000 for example.

How can I do that too ? Should I read this : https://wiki.osdev.org/ISA_DMA

[Octocontrabass]

So from what you say, Linux remaps DMA addresses to be accessible by the device using 32 bits for example.

No, Linux has a special allocator that chooses addresses that will be accessible by the device.

Linux is also capable of using an IOMMU to remap DMA addresses, but an IOMMU is not always available.

Should I read this : https://wiki.osdev.org/ISA_DMA

Probably not. Only ancient ISA devices use the ISA DMA controllers.

[devc1]

Ok, now I understand.