Reading sectors off flash drive in protected mode

[BMW]

Hi all,

I am writing an OS which is designed to run off a USB flash drive.

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)

[Combuster]

use int 0x13. Otherwise, no.

[Brendan]

Hi,

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)

First, normally there's a tree of device drivers and not just one. For example, you might have 4 different "USB controller" drivers (one for each different type of USB controller - UHCI, OHCI, EHCI, xHCI) that all talk to the hardware (in different ways) and provide some sort of interface for other software to use; and then you'd have drivers for a USB devices (e.g. a flash drives) that talk to the USB controller's driver using the interface that USB controllers provide. Of course this tree might be deeper - for example you might have a USB flash driver that talks to a USB hub driver that talks to another USB hub driver that talks to a USB host controller driver.

Near the top of the tree there's normally something to manage PCI buses, which is responsible detecting which PCI devices are present, managing resources that are assigned to different PCI devices, handling some power management (e.g. for PCI bridges), and starting the corresponding device drivers (and might also do other things, like handling the IOMMU if there is one and monitoring drivers and recovering/restarting them if they crash, etc). Below the drivers there's also more code - e.g. under an ethernet driver you might have several layers of code to manage things like bonding, routing, IP, TCP/UDP, etc; and under a USB flash driver you might have a piece of code to handle partitions then various pieces of code to handle different types of file systems.

The complete thing (for USB flash only) might look like this:

Code: Select all

  Device manager
   |__ USB controller driver
        |__ USB hub driver
                |__ USB flash driver
                     |__ Partition manager
                          |__ File system code
                          |__ File system code

Of course it's not that simple. Typically half of this talks to the OS's virtual file system layer (for e.g. the USB flash controller might be exposed as "/dev/sdd", the partition manager might be responsible for "/dev/sdd_p0" and "/dev/sdd_p1", and the various file systems might have mount points). In addition to this, everything depends on various parts of the kernel - things like IRQ handling, memory management, communication with processes, scheduling, timers, etc.

The thing is; all of the interfaces between all of these pieces can be very different for different OS's/kernels. The kernel API's will be different, the executable file format used for drivers may be different, the way pieces communicate (messages? direct calls?) may be different, the IO model may be different (asynchronous IO with IO priorities? first come first served synchronous IO?), how resources are handled may be different (does the driver tell the device manager which resources it needs, or does the device manager tell the driver which resources it should have?), IRQ handling may be different, the way critical sections and concurrency is handled can be different, the way scheduling is done may be very different, etc.

Basically; a driver that is designed for one OS will not work in another OS. It's like taking a dog's brain and putting it into a fish - nothing will be compatible.


Cheers,

Brendan

[Hoozim]

As Brenden said, USB driver is the only way. First you will need a PCI enumerator (though I suppose you could just use magic numbers but this is a terrible way of doing something). Then you will need an EHCI driver (assuming USB 2.0, there are a number of other controllers out there). Then on top of that, a USB2.0 driver. Above that you will need a Mass Storage Class driver. Finally, above that a file system manager/driver.

If you want, I can send you over to my code (git repo). It has USB2.0 support for Mass Storage Devices. It is portable (from my experiments) and well tested on several systems.

Hoozim

P.S. Mass storage support is great in my opinion. With it I can access all of my files and not have to worry about dated technology (floppy disk).

[Hoozim]

I should add I have no support for hubs. I designed it to be simple, but extensible to only some conditions (for simplicity). It does not require a specific machine (no magic numbers that way).

Hoozim

[BMW]

use int 0x13. Otherwise, no.

Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.

[BMW]

Hi,

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)

First, normally there's a tree of device drivers and not just one. For example, you might have 4 different "USB controller" drivers (one for each different type of USB controller - UHCI, OHCI, EHCI, xHCI) that all talk to the hardware (in different ways) and provide some sort of interface for other software to use; and then you'd have drivers for a USB devices (e.g. a flash drives) that talk to the USB controller's driver using the interface that USB controllers provide. Of course this tree might be deeper - for example you might have a USB flash driver that talks to a USB hub driver that talks to another USB hub driver that talks to a USB host controller driver.

Near the top of the tree there's normally something to manage PCI buses, which is responsible detecting which PCI devices are present, managing resources that are assigned to different PCI devices, handling some power management (e.g. for PCI bridges), and starting the corresponding device drivers (and might also do other things, like handling the IOMMU if there is one and monitoring drivers and recovering/restarting them if they crash, etc). Below the drivers there's also more code - e.g. under an ethernet driver you might have several layers of code to manage things like bonding, routing, IP, TCP/UDP, etc; and under a USB flash driver you might have a piece of code to handle partitions then various pieces of code to handle different types of file systems.

The complete thing (for USB flash only) might look like this:

Code: Select all

  Device manager
   |__ USB controller driver
        |__ USB hub driver
                |__ USB flash driver
                     |__ Partition manager
                          |__ File system code
                          |__ File system code

Of course it's not that simple. Typically half of this talks to the OS's virtual file system layer (for e.g. the USB flash controller might be exposed as "/dev/sdd", the partition manager might be responsible for "/dev/sdd_p0" and "/dev/sdd_p1", and the various file systems might have mount points). In addition to this, everything depends on various parts of the kernel - things like IRQ handling, memory management, communication with processes, scheduling, timers, etc.

The thing is; all of the interfaces between all of these pieces can be very different for different OS's/kernels. The kernel API's will be different, the executable file format used for drivers may be different, the way pieces communicate (messages? direct calls?) may be different, the IO model may be different (asynchronous IO with IO priorities? first come first served synchronous IO?), how resources are handled may be different (does the driver tell the device manager which resources it needs, or does the device manager tell the driver which resources it should have?), IRQ handling may be different, the way critical sections and concurrency is handled can be different, the way scheduling is done may be very different, etc.

Basically; a driver that is designed for one OS will not work in another OS. It's like taking a dog's brain and putting it into a fish - nothing will be compatible.


Cheers,

Brendan

Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?

As Brenden said, USB driver is the only way. First you will need a PCI enumerator (though I suppose you could just use magic numbers but this is a terrible way of doing something). Then you will need an EHCI driver (assuming USB 2.0, there are a number of other controllers out there). Then on top of that, a USB2.0 driver. Above that you will need a Mass Storage Class driver. Finally, above that a file system manager/driver.

If you want, I can send you over to my code (git repo). It has USB2.0 support for Mass Storage Devices. It is portable (from my experiments) and well tested on several systems.

Hoozim

P.S. Mass storage support is great in my opinion. With it I can access all of my files and not have to worry about dated technology (floppy disk).

That would be great thanks, I would love to have a look at your code. Link?

[Griwes]

use int 0x13. Otherwise, no.

Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.

If you bothered to read the answer, you would have noticed that "Otherwise, no." part. (Pro-tip: that "Otherwise, no." part was most probably talking about your "the code for it is probably so easy to write that there is plenty of such implementations; do you have any, because my searching skill sucks?" part).

Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?

lol

[Brendan]

Hi,

Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?

"Plug and Play" mostly just means that you can find out what the device is and which resources it uses; so that your OS can determine which driver is the correct driver for the device, and so that the driver can figure out how to talk to the device (which IO ports, IRQs, DMA channels, etc). Before "Plug and Play" there were things like ISA, where there's no way to determine if a device exists or not (or which IO ports, IRQs, DMA channels, etc. it uses if it does exist).

Plug and Play does not mean that the device has it's own driver (and doesn't mean that the device has the ability to magically transform generic code into something that actually suits your OS).


Cheers,

Brendan

[BMW]

use int 0x13. Otherwise, no.

Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.

If you bothered to read the answer, you would have noticed that "Otherwise, no." part. (Pro-tip: that "Otherwise, no." part was most probably talking about your "the code for it is probably so easy to write that there is plenty of such implementations; do you have any, because my searching skill sucks?" part).

Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?

lol

Instead of thinking you are the pro with all your so called "pro-tips" that are extremely irrelevant and not "pro" at all, why don't you invest in a good dictionary and spend some time reading it. The last time I checked, "Otherwise, no" meant: in all other cases, the answer is no.

Combuster said "use int 0x13. Otherwise, no."
This means that he is suggesting I use 0x13, and that there are no other drivers or anything that I could use apart from 0x13. However, this is completely wrong and irrelevant. For a start, I am talking about protected mode. Why would someone even suggest using interrupts in protected mode? Secondly, Hoozim has kindly offered his code. Therefore he is wrong with his "Otherwise, no" statement also.

You stupid "lol" comment about my question on plug and play was totally immature. I think you should strive to be helpful (like Brendan) and not a pain in the backside.

[SparrowOS]

Gene Wilder Meme:

Oh, so you're doing USB?
Tell me more.

[thepowersgang]

@BMW (I'm probably wasting my time, but we'll see)

The reason why combuster's post was so blunt, is because there is code that can just be dropped in for USB. It's a complex beast made up of many layers much like networking, and takes a robust scheduler and a skill at reading datasheets (some of which don't tell you how, just what - I'm looking at you, UHCI).

Given your previous posts on this forum, your code is very likely not ready for a full USB stack, and you wouldn't stand a chance at understanding the documentation (This will be my opinion until proven otherwise)

As for using BIOS interrupts in protected mode - it's possible. You can either write VM8086 monitor (not too difficult, but requires scheduler cooperation to an extent) or you can drop to real mode to do disk IO (very expensive, but works for loading a ramdisk).

[bluemoon]

Hi all,

I am writing an OS which is designed to run off a USB flash drive.

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)

After all, you don't need to read USB drive yourself in order to boot from an USB drive. Abuse the HDD emulation and load all the required sectors as a ram disk in your boot loader, or by grub.

[BMW]

@BMW (I'm probably wasting my time, but we'll see)

The reason why combuster's post was so blunt, is because there is code that can just be dropped in for USB. It's a complex beast made up of many layers much like networking, and takes a robust scheduler and a skill at reading datasheets (some of which don't tell you how, just what - I'm looking at you, UHCI).

Given your previous posts on this forum, your code is very likely not ready for a full USB stack, and you wouldn't stand a chance at understanding the documentation (This will be my opinion until proven otherwise)

As for using BIOS interrupts in protected mode - it's possible. You can either write VM8086 monitor (not too difficult, but requires scheduler cooperation to an extent) or you can drop to real mode to do disk IO (very expensive, but works for loading a ramdisk).

Ok... Thanks (At least your post contained some helpful content).

After all, you don't need to read USB drive yourself in order to boot from an USB drive. Abuse the HDD emulation and load all the required sectors as a ram disk in your boot loader, or by grub.

Good idea thanks... I think ill do that!

[Griwes]

So, in the end, you completely agreed with Combuster and me.

How surprising.


And that simple "lol" was to be read as "you need to start reading before asking, because no-one is gonna spoon-feed you here, and/or the question you've asked was so ridiculous and silly that there were three proper answers: flat "what.", "lol" and actual reply to the question, but most probably filled with sarcasm and inner laugh at your lack of reading and thinking skill" (seriously, if you did *any* OS-related research, you would know that there is currently exactly no way to create a driver that works under any possible OS, so your question was plainly silly).