OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Apr 18, 2024 12:11 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Reading sectors off flash drive in protected mode
PostPosted: Mon Nov 12, 2012 8:14 pm 
Offline
Member
Member
User avatar

Joined: Mon Nov 05, 2012 8:31 pm
Posts: 286
Location: New Zealand
Hi all,

I am writing an OS which is designed to run off a USB flash drive.

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)

_________________
Currently developing Lithium OS (LiOS).

Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Tue Nov 13, 2012 12:45 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 18, 2006 3:45 am
Posts: 9301
Location: On the balcony, where I can actually keep 1½m distance
use int 0x13. Otherwise, no.

_________________
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Tue Nov 13, 2012 6:30 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

BMW wrote:
Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)


First, normally there's a tree of device drivers and not just one. For example, you might have 4 different "USB controller" drivers (one for each different type of USB controller - UHCI, OHCI, EHCI, xHCI) that all talk to the hardware (in different ways) and provide some sort of interface for other software to use; and then you'd have drivers for a USB devices (e.g. a flash drives) that talk to the USB controller's driver using the interface that USB controllers provide. Of course this tree might be deeper - for example you might have a USB flash driver that talks to a USB hub driver that talks to another USB hub driver that talks to a USB host controller driver.

Near the top of the tree there's normally something to manage PCI buses, which is responsible detecting which PCI devices are present, managing resources that are assigned to different PCI devices, handling some power management (e.g. for PCI bridges), and starting the corresponding device drivers (and might also do other things, like handling the IOMMU if there is one and monitoring drivers and recovering/restarting them if they crash, etc). Below the drivers there's also more code - e.g. under an ethernet driver you might have several layers of code to manage things like bonding, routing, IP, TCP/UDP, etc; and under a USB flash driver you might have a piece of code to handle partitions then various pieces of code to handle different types of file systems.

The complete thing (for USB flash only) might look like this:

Code:
  Device manager
   |__ USB controller driver
        |__ USB hub driver
                |__ USB flash driver
                     |__ Partition manager
                          |__ File system code
                          |__ File system code


Of course it's not that simple. Typically half of this talks to the OS's virtual file system layer (for e.g. the USB flash controller might be exposed as "/dev/sdd", the partition manager might be responsible for "/dev/sdd_p0" and "/dev/sdd_p1", and the various file systems might have mount points). In addition to this, everything depends on various parts of the kernel - things like IRQ handling, memory management, communication with processes, scheduling, timers, etc.

The thing is; all of the interfaces between all of these pieces can be very different for different OS's/kernels. The kernel API's will be different, the executable file format used for drivers may be different, the way pieces communicate (messages? direct calls?) may be different, the IO model may be different (asynchronous IO with IO priorities? first come first served synchronous IO?), how resources are handled may be different (does the driver tell the device manager which resources it needs, or does the device manager tell the driver which resources it should have?), IRQ handling may be different, the way critical sections and concurrency is handled can be different, the way scheduling is done may be very different, etc.

Basically; a driver that is designed for one OS will not work in another OS. It's like taking a dog's brain and putting it into a fish - nothing will be compatible.


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Tue Nov 13, 2012 6:43 pm 
Offline
Member
Member

Joined: Fri Jul 23, 2010 8:26 am
Posts: 53
As Brenden said, USB driver is the only way. First you will need a PCI enumerator (though I suppose you could just use magic numbers but this is a terrible way of doing something). Then you will need an EHCI driver (assuming USB 2.0, there are a number of other controllers out there). Then on top of that, a USB2.0 driver. Above that you will need a Mass Storage Class driver. Finally, above that a file system manager/driver.

If you want, I can send you over to my code (git repo). It has USB2.0 support for Mass Storage Devices. It is portable (from my experiments) and well tested on several systems.

Hoozim

P.S. Mass storage support is great in my opinion. With it I can access all of my files and not have to worry about dated technology (floppy disk).


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Tue Nov 13, 2012 6:45 pm 
Offline
Member
Member

Joined: Fri Jul 23, 2010 8:26 am
Posts: 53
I should add I have no support for hubs. I designed it to be simple, but extensible to only some conditions (for simplicity). It does not require a specific machine (no magic numbers that way).

Hoozim


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 4:55 pm 
Offline
Member
Member
User avatar

Joined: Mon Nov 05, 2012 8:31 pm
Posts: 286
Location: New Zealand
Combuster wrote:
use int 0x13. Otherwise, no.


Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.

_________________
Currently developing Lithium OS (LiOS).

Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 4:57 pm 
Offline
Member
Member
User avatar

Joined: Mon Nov 05, 2012 8:31 pm
Posts: 286
Location: New Zealand
Brendan wrote:
Hi,

BMW wrote:
Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)


First, normally there's a tree of device drivers and not just one. For example, you might have 4 different "USB controller" drivers (one for each different type of USB controller - UHCI, OHCI, EHCI, xHCI) that all talk to the hardware (in different ways) and provide some sort of interface for other software to use; and then you'd have drivers for a USB devices (e.g. a flash drives) that talk to the USB controller's driver using the interface that USB controllers provide. Of course this tree might be deeper - for example you might have a USB flash driver that talks to a USB hub driver that talks to another USB hub driver that talks to a USB host controller driver.

Near the top of the tree there's normally something to manage PCI buses, which is responsible detecting which PCI devices are present, managing resources that are assigned to different PCI devices, handling some power management (e.g. for PCI bridges), and starting the corresponding device drivers (and might also do other things, like handling the IOMMU if there is one and monitoring drivers and recovering/restarting them if they crash, etc). Below the drivers there's also more code - e.g. under an ethernet driver you might have several layers of code to manage things like bonding, routing, IP, TCP/UDP, etc; and under a USB flash driver you might have a piece of code to handle partitions then various pieces of code to handle different types of file systems.

The complete thing (for USB flash only) might look like this:

Code:
  Device manager
   |__ USB controller driver
        |__ USB hub driver
                |__ USB flash driver
                     |__ Partition manager
                          |__ File system code
                          |__ File system code


Of course it's not that simple. Typically half of this talks to the OS's virtual file system layer (for e.g. the USB flash controller might be exposed as "/dev/sdd", the partition manager might be responsible for "/dev/sdd_p0" and "/dev/sdd_p1", and the various file systems might have mount points). In addition to this, everything depends on various parts of the kernel - things like IRQ handling, memory management, communication with processes, scheduling, timers, etc.

The thing is; all of the interfaces between all of these pieces can be very different for different OS's/kernels. The kernel API's will be different, the executable file format used for drivers may be different, the way pieces communicate (messages? direct calls?) may be different, the IO model may be different (asynchronous IO with IO priorities? first come first served synchronous IO?), how resources are handled may be different (does the driver tell the device manager which resources it needs, or does the device manager tell the driver which resources it should have?), IRQ handling may be different, the way critical sections and concurrency is handled can be different, the way scheduling is done may be very different, etc.

Basically; a driver that is designed for one OS will not work in another OS. It's like taking a dog's brain and putting it into a fish - nothing will be compatible.


Cheers,

Brendan


Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?

Hoozim wrote:
As Brenden said, USB driver is the only way. First you will need a PCI enumerator (though I suppose you could just use magic numbers but this is a terrible way of doing something). Then you will need an EHCI driver (assuming USB 2.0, there are a number of other controllers out there). Then on top of that, a USB2.0 driver. Above that you will need a Mass Storage Class driver. Finally, above that a file system manager/driver.

If you want, I can send you over to my code (git repo). It has USB2.0 support for Mass Storage Devices. It is portable (from my experiments) and well tested on several systems.

Hoozim

P.S. Mass storage support is great in my opinion. With it I can access all of my files and not have to worry about dated technology (floppy disk).


That would be great thanks, I would love to have a look at your code. Link?

_________________
Currently developing Lithium OS (LiOS).

Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 5:09 pm 
Offline
Member
Member
User avatar

Joined: Sat Jul 30, 2011 10:07 am
Posts: 374
Location: Wrocław/Racibórz, Poland
BMW wrote:
Combuster wrote:
use int 0x13. Otherwise, no.


Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.
If you bothered to read the answer, you would have noticed that "Otherwise, no." part. (Pro-tip: that "Otherwise, no." part was most probably talking about your "the code for it is probably so easy to write that there is plenty of such implementations; do you have any, because my searching skill sucks?" part).

Quote:
Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?
lol

_________________
Reaver Project :: Repository :: Ohloh project page
<klange> This is a horror story about what happens when you need a hammer and all you have is the skulls of the damned.
<drake1> as long as the lock is read and modified by atomic operations


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 5:39 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

BMW wrote:
Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?


"Plug and Play" mostly just means that you can find out what the device is and which resources it uses; so that your OS can determine which driver is the correct driver for the device, and so that the driver can figure out how to talk to the device (which IO ports, IRQs, DMA channels, etc). Before "Plug and Play" there were things like ISA, where there's no way to determine if a device exists or not (or which IO ports, IRQs, DMA channels, etc. it uses if it does exist).

Plug and Play does not mean that the device has it's own driver (and doesn't mean that the device has the ability to magically transform generic code into something that actually suits your OS).


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 8:04 pm 
Offline
Member
Member
User avatar

Joined: Mon Nov 05, 2012 8:31 pm
Posts: 286
Location: New Zealand
Griwes wrote:
BMW wrote:
Combuster wrote:
use int 0x13. Otherwise, no.


Hi,

If you bothered to read the title, you would have noticed that I am doing this in protected mode. I hope you know that interrupts are not available in protected mode.
If you bothered to read the answer, you would have noticed that "Otherwise, no." part. (Pro-tip: that "Otherwise, no." part was most probably talking about your "the code for it is probably so easy to write that there is plenty of such implementations; do you have any, because my searching skill sucks?" part).

Quote:
Thanks for the in depth answer. How do plug and play devices work? Do they have a driver stored on them which the OS utilizes?
lol

Instead of thinking you are the pro with all your so called "pro-tips" that are extremely irrelevant and not "pro" at all, why don't you invest in a good dictionary and spend some time reading it. The last time I checked, "Otherwise, no" meant: in all other cases, the answer is no.

Combuster said "use int 0x13. Otherwise, no."
This means that he is suggesting I use 0x13, and that there are no other drivers or anything that I could use apart from 0x13. However, this is completely wrong and irrelevant. For a start, I am talking about protected mode. Why would someone even suggest using interrupts in protected mode? Secondly, Hoozim has kindly offered his code. Therefore he is wrong with his "Otherwise, no" statement also.

You stupid "lol" comment about my question on plug and play was totally immature. I think you should strive to be helpful (like Brendan) and not a pain in the backside.

_________________
Currently developing Lithium OS (LiOS).

Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 8:29 pm 
Offline
Member
Member
User avatar

Joined: Wed Nov 14, 2012 5:22 pm
Posts: 72
Location: Vegas
Gene Wilder Meme:
Image
Oh, so you're doing USB?
Tell me more.


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 9:57 pm 
Offline
Member
Member
User avatar

Joined: Tue Dec 25, 2007 6:03 am
Posts: 734
Location: Perth, Western Australia
@BMW (I'm probably wasting my time, but we'll see)

The reason why combuster's post was so blunt, is because there is code that can just be dropped in for USB. It's a complex beast made up of many layers much like networking, and takes a robust scheduler and a skill at reading datasheets (some of which don't tell you how, just what - I'm looking at you, UHCI).

Given your previous posts on this forum, your code is very likely not ready for a full USB stack, and you wouldn't stand a chance at understanding the documentation (This will be my opinion until proven otherwise)

As for using BIOS interrupts in protected mode - it's possible. You can either write VM8086 monitor (not too difficult, but requires scheduler cooperation to an extent) or you can drop to real mode to do disk IO (very expensive, but works for loading a ramdisk).

_________________
Kernel Development, It's the brain surgery of programming.
Acess2 OS (c) | Tifflin OS (rust) | mrustc - Rust compiler
Currently Working on: mrustc


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Wed Nov 14, 2012 11:51 pm 
Offline
Member
Member
User avatar

Joined: Wed Dec 01, 2010 3:41 am
Posts: 1761
Location: Hong Kong
BMW wrote:
Hi all,

I am writing an OS which is designed to run off a USB flash drive.

Has anyone got any drivers made in NASM for reading/writing to USB flash drives?? (I don't really want to spend ages writing one when there are probably plenty around)


After all, you don't need to read USB drive yourself in order to boot from an USB drive. Abuse the HDD emulation and load all the required sectors as a ram disk in your boot loader, or by grub.


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Thu Nov 15, 2012 5:19 am 
Offline
Member
Member
User avatar

Joined: Mon Nov 05, 2012 8:31 pm
Posts: 286
Location: New Zealand
thepowersgang wrote:
@BMW (I'm probably wasting my time, but we'll see)

The reason why combuster's post was so blunt, is because there is code that can just be dropped in for USB. It's a complex beast made up of many layers much like networking, and takes a robust scheduler and a skill at reading datasheets (some of which don't tell you how, just what - I'm looking at you, UHCI).

Given your previous posts on this forum, your code is very likely not ready for a full USB stack, and you wouldn't stand a chance at understanding the documentation (This will be my opinion until proven otherwise)

As for using BIOS interrupts in protected mode - it's possible. You can either write VM8086 monitor (not too difficult, but requires scheduler cooperation to an extent) or you can drop to real mode to do disk IO (very expensive, but works for loading a ramdisk).


Ok... Thanks (At least your post contained some helpful content).

bluemoon wrote:
After all, you don't need to read USB drive yourself in order to boot from an USB drive. Abuse the HDD emulation and load all the required sectors as a ram disk in your boot loader, or by grub.


Good idea thanks... I think ill do that!

_________________
Currently developing Lithium OS (LiOS).

Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."


Top
 Profile  
 
 Post subject: Re: Reading sectors off flash drive in protected mode
PostPosted: Thu Nov 15, 2012 6:14 am 
Offline
Member
Member
User avatar

Joined: Sat Jul 30, 2011 10:07 am
Posts: 374
Location: Wrocław/Racibórz, Poland
So, in the end, you completely agreed with Combuster and me.

How surprising.


And that simple "lol" was to be read as "you need to start reading before asking, because no-one is gonna spoon-feed you here, and/or the question you've asked was so ridiculous and silly that there were three proper answers: flat "what.", "lol" and actual reply to the question, but most probably filled with sarcasm and inner laugh at your lack of reading and thinking skill" (seriously, if you did *any* OS-related research, you would know that there is currently exactly no way to create a driver that works under any possible OS, so your question was plainly silly).

_________________
Reaver Project :: Repository :: Ohloh project page
<klange> This is a horror story about what happens when you need a hammer and all you have is the skulls of the damned.
<drake1> as long as the lock is read and modified by atomic operations


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 230 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group