Is it really necessary to reserve memory for DMA?

deadmutex · Post by **deadmutex** » Fri Jun 22, 2007 1:25 am

I was thinking about my memory manager and how it needed to reserve addresses below the 16 MiB mark for DMA, but then I asked myself whether or not I really needed to do so. I've read that the motherboard DMA is slow and outdated. Most of the devices seem to rely on PCI and IDE bus mastering nowadays. If I don't plan on supporting legacy devices (floppy drive, ISA sound cards, etc.) do I still need to reserve the lower 16 MiB for something? Is the motherboard DMA still required for modern devices?

exkor · Post by **exkor** » Fri Jun 22, 2007 4:34 am

I heard some use DMA to speed up graphics (VESA VBE), otherwise no need for motherboard DMA I guess.

Aali · Post by **Aali** » Fri Jun 22, 2007 5:43 am

its usually enough to reserve regions within the first megabyte for DMA nowadays

its arguably good practice to reserve the first 16 MB though, since it does have a certain special purpose

all depends on what kind of hardware you're targeting really

deadmutex · Post by **deadmutex** » Fri Jun 22, 2007 8:48 am

Thanks for the replies.

its arguably good practice to reserve the first 16 MB though, since it does have a certain special purpose

all depends on what kind of hardware you're targeting really

What special purpose does it have? Could you give some examples? Thanks.

AJ · Post by AJ » Fri Jun 22, 2007 9:02 am

Hi,

This is probably an odd way of doing it, but my kernel reserves a different amount depending on the total system RAM. If you had 32MB total, for example, I reserve 1MB. If you had 4GB total, my kernel reserves the entire 16MB - there are various stages in between. Not that I do much with DMA yet...

Cheers,
Adam

Combuster · Post by **Combuster** » Fri Jun 22, 2007 11:23 am

I'm not convinced that not supporting the floppy drive for it's DMA needs is the right way. In fact many of us still use floppy disks for the real testing.

So much for legacy fading away since all my x86s have at least one.

deadmutex · Post by **deadmutex** » Fri Jun 22, 2007 1:03 pm

Well, I guess I can manage to reserve the first meg of physical memory for something... But do modern devices need the extra 15 MiB of 1:1 mapped memory?

Brendan · Post by **Brendan** » Fri Jun 22, 2007 2:20 pm

Hi,

deadmutex wrote:Well, I guess I can manage to reserve the first meg of physical memory for something... But do modern devices need the extra 15 MiB of 1:1 mapped memory?

Consider my memory manager/s...

For physical memory management I really like using "free page stacks" - they're fast. If I could I'd use free page stacks and nothing else, but this would make it impossible for any device driver (PCI or ISA) to ask for contiguous pages.

To get around this I use a bitmap (instead of free page stacks) to manage the lowest part of physical memory. This doesn't mean that this space is *reserved* for contiguous allocations - it just means that this space is the only space that can be used for contiguous allocations. For normal allocations (single pages where the phyical address doesn't matter) the physical memory manager tries to allocate other areas when it can, but if it can't it'll allocate the page from the area of lower memory.

This might mean that there isn't enough contiguous space left when a device driver wants it, but this is very unlikely - device drivers normally allocate their DMA buffers when they first start up (during boot), before other software gets a chance to use it all up.

The size of the area managed by the bitmap doesn't make much difference as long as it's "big enough". If it's too small device drivers may fail to initialize, but if it's too large then I end up with less space managed by faster free page stacks and more space managed by the slower bitmap - it reduces performance a little, but I doubt it'd be a huge difference. This means it's much better to have too much space managed by the bitmap than not enough...

I also don't 1:1 map anything. When a device driver says "give me N contiguous pages at FOO" then I allocate N contiguous pages and map those pages into their address space at "FOO". Until software allocates the physical pages the physical pages aren't mapped anywhere in any linear address space.

Cheers,

Brendan

exkor · Post by **exkor** » Fri Jun 22, 2007 8:48 pm

Combuster wrote:I'm not convinced that not supporting the floppy drive for it's DMA needs is the right way. In fact many of us still use floppy disks for the real testing.

You can use floppy for initial booting, but if floppy found on modern(your OS will probably be ready in about 2-10 years for extensive public use) computer then the floppy is deserved not to be working.
My point is: There is no need of support of any legacy devices such as COM mouses,modems,floppy, and with limited recourses available(human power,devices to test) you gotta aim for devices that are produced nowadays and not whose that out of production already.

Brendan · Post by **Brendan** » Fri Jun 22, 2007 9:34 pm

Hi,

exkor wrote:My point is: There is no need of support of any legacy devices such as COM mouses,modems,floppy, and with limited recourses available(human power,devices to test) you gotta aim for devices that are produced nowadays and not whose that out of production already.

Floppy drives, parallel ports and serial ports have all remained mostly unchanged for 20 years or more. Sound cards last for ages - even a 10 year old ISA card is "good enough" for most people.

Every 4 years or so there's a change to ATA/SATA/ATAPI, and video cards become obsolete in about 12 months. Network cards are a little better - they tend to remain used until the network itself is upgraded (cabling, switches, etc). I also wouldn't be surprised if a new "USB 3.0" standard is released in the next year or so, so you can expect scanners, printers, flash drives, etc to change too.

Out of all of this, I'd guess that floppy drives, parallel ports and serial ports are probably the most likely to remain unchanged between now and when your OS is finished...

The other thing is that they really aren't that hard to support, and because of this they're good device drivers to write before you get buried under things like PCI-Express, USB, power management and badly documented hardware.

Cheers,

Brendan

exkor · Post by **exkor** » Fri Jun 22, 2007 11:26 pm

And all old devices will remain unchanged for another 10 years. But majority don't used LPT printers, COM mouses, and PCs are sold without floppies. If somebody uses these then I suggest to switch to USB which atleast provides better transfer rates.

Supporting anything takes time and more time if there are many devices. Besides any programming job may yield programming mistakes in drivers. Nobody will need polished driver in several years people need them now and better start with nowadays devices and if all drivers written - have a life with your girlfriend.

After you boot your OS first drivers to write is hard drvice, usb, ps2 mouse & kbd, flash cards, network & one generic graphics driver. Manufactures will write more if they see that you OS is popular and if not then the lifetime will not be enough for you to support everything.

It is required to get deept into docs to write drivers and if not you'll be rewriting/copy linux drivers.

AJ · Post by AJ » Sat Jun 23, 2007 4:26 am

Personally, as a purely hobby OS developper, I think the best I can do is to start by supporting my own hardware and the emulated hardware found in the emulators I use. When I get to that stage, I will be happy to think about other hardware *if* anyone else wants to run my OS

deadmutex · Post by **deadmutex** » Sat Jun 23, 2007 5:31 pm

Hmm... well I wanted to somehow avoid DMA as much as practically possible, since it seems to be rather clunky. The whole point in using it is to transfer memory without having to go through the processor. However, sometimes there isn't an interrupt to notify when the transfer is done. That means that the processor has to poll to verify completion which defeats the entire purpose of using DMA in the first place, IMO.

Since I want to create a microkernel, I was wondering where to put something like DMA(if I need to use it). I'm guessing the kernel, because if DMA transfers physical memory, then there's a security/stability risk if a server manages it.

Man... trying to support all of these different devices can really ruin some good designs...

Kevin McGuire · Post by **Kevin McGuire** » Sat Jun 23, 2007 5:49 pm

You should be able to easily reserve memory needed by legacy DMA transaction by tinkering your memory manager to help you. The idea would be that during early kernel booting you need to specify all pages that legacy devices may use with a legacy flag. If you also use the standard method of having each process use virtual memory and map the kernel into each process then you can simply hand out all the legacy memory pages you want to.

When a driver requests legacy memory you see if there exists any free pages. If not then you find one that it used and swap it with a non-legacy page. It is as simple as copying the data from the legacy physical page to another free physical page, then changing the mapping for the process that was using the page. Also since the kernel executes in a virtual mode if it happened to be holding a virtual page that is mapped to this physical location you could also swap them out.

That method might allow you to add some flexibility so that no matter where a driver needs memory at you can provide it.

While, I just thought about it you should be able to force swap any physical page in the system with another as long as nothing has allocated that page as a physical page (only virtual). This might remove the need to even have to specify a page as being legacy. You would have to be careful about a page that might host a page directory or table in that before you start the force swap operation you verify and check and if it is you take the proper action or just deny it.

bewing · Post by **bewing** » Sun Jun 24, 2007 10:18 pm

There are 6 usable ISA DMA channels. 3 each of 8 bit, and 16 bit. An 8 bit channel maxes out at a 64K transfer. A 16 bit channel maxes out at a 128K transfer. Except for the floppy (8 bit) which shouldn't be asked to transfer more than 1 cylinder/head at a time. For a 2.88M floppy, that means a max of 18K per transfer, as I recall. If you want to be nice, and have 2 buffers for each DMA channel (so one can be DMAing while the other is being loaded/unloaded) then I figure that the first 1.75M is sufficient to handle all possible ISA DMA with enough space left over for minimal V8086 functionality. Then any driver acquires the 2 buffers when it acquires the DMA channel.

And I kinda figure that SoundBlaster cards are my main reason to want to support ISA DMA.