Booting from USB drive (SOLVED)

[bigbob]

I completely forgot that I installed Debian Linux on my Nextbook from a USB-CD-drive.
So it should work. I still have no idea.

Your MBR: on the D820 I could only read the "Space/Return"-text, because in the middle of the screen (or a little bit above the middle) there was "trash" instead of a correct partition-selection-text.
You mentioned that you only tested it on older computers (from the WinXP-era).

If I just change the bootable-byte (to be 0x80) in the original MBR of the pen-drive fromatted to FAT32 (so I don't overwrite it with my own MBR), and get the BPB of the original VBR(its address is in the partition-entry, of course), and put that BPB in my own VBR and then write my own VBR to the pendrive then it works (if I plug it in, Linux opens a window and I can copy files to it). What I mean is, that the original MBR passes the pointer to the active partition-entry in DS:SI, and the drivenumber in DL, so I don't even need to write an MBR. So, I can boot my OS from this pendrive and also easily read/write files from/to the pendrive with Linux. The only difficulty occurs, when somebody else wants to boot my OS (from USB) because he has to update my custom-VBR with his own BPB (from his own pendrive) and that can be inconvenient.
If I create my own MBR, then not only do I have to get the BPB for my own VBR, but I also have to fill the first partition-entry in my own MBR from the original-MBR of the pendrive.
Without writing code that formats the pendrive, I see no other way to get the values of the partition.
In other words: how do you fill the partition-entries in your own MBR?
If you add it manually, then the users of your OS will have to do that too.

In my opinion, just for booting my OS from a pendrive with one partition, there is no need for a custom-MBR.
By the way, I saw smbmbr.asm by John S. Fine, nice, especially the code of relocating.

[KemyLand]

The vague and broad solution, write an MBR generator .

[Octocontrabass]

I completely forgot that I installed Debian Linux on my Nextbook from a USB-CD-drive.
So it should work. I still have no idea.

Try partitioning the disk using diskmgmt.msc in Windows. Normally it won't let you do this, but if you give it an empty partition table with a 55 AA signature, it will let you create one simple volume on the disk. You can then take the partition table generated by Windows and tweak it until you find the values your netbook doesn't like.


As you've seen, BIOSes are pretty picky about what's in the MBR. You must have exactly one bootable partition. You can't have partitions that overlap each other or the MBR. You can't have anything but zero bytes in unused partition table entries. You have to calculate CHS values using 255 heads and 63 sectors, and mostly they need to match the LBA values. (If the CHS values would point beyond the last whole cylinder on the disk, Windows uses the last sector of the last whole cylinder instead of the correct conversion.) You must have a 55 AA signature at the end of the MBR. These are just limitations I've found through testing, not a complete list!

I've seen at least one BIOS that also examines the contents of the bootable partition's VBR during boot, and if the VBR looks like a FAT filesystem, it will skip the MBR and directly load the VBR. So, it's probably a good idea to have a 55 AA signature in your VBR as well, in case the BIOS also verifies that.

[DavidCooper]

Your MBR: on the D820 I could only read the "Space/Return"-text, because in the middle of the screen (or a little bit above the middle) there was "trash" instead of a correct partition-selection-text.

If the partition table data points to actual partitions with VBRs in them, instead of displaying trash it should write a list of bootable partitions for the user to choose from, lifting names straight out of the VBRs, so it might say MSB-OS or MS-DOS, though it might still be random junk if it's a strange VBR format, but there will still be a number 1, 2, 3 or 4 in front of it.

The only difficulty occurs, when somebody else wants to boot my OS (from USB) because he has to update my custom-VBR with his own BPB (from his own pendrive) and that can be inconvenient.

A lot of flash drives don't have any MBR code in them, so changing the first partition byte to 80h isn't always going to be enough, and because partition locations vary greatly, they will have to edit the BPB. You need to write a guide for them to follow.

If I create my own MBR, then not only do I have to get the BPB for my own VBR, but I also have to fill the first partition-entry in my own MBR from the original-MBR of the pendrive.

If you do it through a hex editor you can just copy the MBR code in up to the partition table without covering the partition table. It's much safer to work that way anyway rather than expecting users to risk using more dangerous tools like dd.

In other words: how do you fill the partition-entries in your own MBR?
If you add it manually, then the users of your OS will have to do that too.

I just copy, paste and edit with a hex editor. I wrote a guide to help people install my OS and its MBR onto a flash drive, and in that guide I have a JavaScript program which helps them add a new partition to it as well. No one's ever used this guide yet (apart from me) because the flash drive compatible version of the OS isn't available yet (too many unfinished parts of it which could cause users problems), but the guide is here: http://www.magicschoolbook.com/computin ... -partition, and I use it whenever I put my OS on a new drive. You obviously won't need to go to such lengths for your OS as users won't need to create a new partition or modify the size or location of the existing one, but anyone who wants to use your OS should be capable of following instructions and doing a little bit of editing to get it installed onto a drive. If they aren't, you could sell them a flash drive that already has your OS on it, and include a program in your OS to help them install it onto other flash drives.

In my opinion, just for booting my OS from a pendrive with one partition, there is no need for a custom-MBR.

Indeed. I only wrote my own because I have to be able to boot my OS, then use it to modify and save a more advanced version of itself to the same drive, and then it has to be able to boot the new version while retaining the option of booting the previous version (or any older version) if the new version has a fault in it that prevents it booting.

[bigbob]

Try partitioning the disk using diskmgmt.msc in Windows.

Everybody uses Linux in the family, so it wouldn't be easy to get a computer with Windows, but there are several similar programs for Linux, so I am sure I can do it with Linux too.

As you've seen, BIOSes are pretty picky about what's in the MBR. ...
I've seen at least one BIOS that also examines the contents of the bootable partition's VBR during boot, and if the VBR looks like a FAT filesystem, it will skip the MBR and directly load the VBR. So, it's probably a good idea to have a 55 AA signature in your VBR as well, in case the BIOS also verifies that.

You mentioned several interesting things regarding what BIOSes check in the MBR.
I also tried to use the original MBR of the pen-drive formatted with FAT32 (the one and only partition was originally set to be bootable), and I think that the original MBR contains correct partition-entry(/ies).
I tried it with at least two pendrives.
I have the 55 AA signiture in my VBR, so that can't be the problem. It has always been there.
For me, booting from USB on my netbook is not urgent/important, because the netbook is my main computer. I am developing my OS on it but I test it on my D820.
Nevertheless it is strange (that the netbook ignores the pen-drive) and I am simply interested in figuring out the why.

[FallenAvatar]

I would just like to point out that Windows (and some/most BIOS might do the same) assumes that a flash drive has only 1 partition on it. (Where a flash drive is different than a USB Hard Drive) And that could lead to some of the symptoms you are seeing.

- Monk

[bigbob]

If the partition table data points to actual partitions with VBRs in them, instead of displaying trash it should write a list of bootable partitions for the user to choose from, lifting names straight out of the VBRs, so it might say MSB-OS or MS-DOS, though it might still be random junk if it's a strange VBR format, but there will still be a number 1, 2, 3 or 4 in front of it.

It's possible that I forgot to copy the correct VBR onto the partition during the test, I don't remember.

A lot of flash drives don't have any MBR code in them, so changing the first partition byte to 80h isn't always going to be enough, and because partition locations vary greatly, they will have to edit the BPB. You need to write a guide for them to follow.

EDIT: Probably you mean that there is an MBR but no code to load the VBR !? Since a partition-entry is necessary.

I just copy, paste and edit with a hex editor. I wrote a guide to help people install my OS and its MBR onto a flash drive, and in that guide I have a JavaScript program which helps them add a new partition to it as well. No one's ever used this guide yet (apart from me) because the flash drive compatible version of the OS isn't available yet (too many unfinished parts of it which could cause users problems), but the guide is here: http://www.magicschoolbook.com/computin ... -partition, and I use it whenever I put my OS on a new drive. You obviously won't need to go to such lengths for your OS as users won't need to create a new partition or modify the size or location of the existing one, but anyone who wants to use your OS should be capable of following instructions and doing a little bit of editing to get it installed onto a drive. If they aren't, you could sell them a flash drive that already has your OS on it, and include a program in your OS to help them install it onto other flash drives.

This is what I was interested about. Let's say somebody has a 4GB-pendrive and he/she just wants to test your (USB-bootable)OS with keeping the files that are already on the pendrive. It would be good to have a LiveUSB, just like a LiveCD that doesn't change the winchester . The process of creating the LiveUSB should also be automated somehow, but compared to a LiveCD, the user should be able to restore the original state of the pendrive. Either by restoring the original filesystem on the pendrive or at least allowing the user to use his/her modified pendrive for storing his/her files on it as before.
I know it's not possible to do it as simple as a LiveCD, so it's just much ado about nothing.

An anecdote:
VBR: there is a 3-byte "hole" right before the BPB for a long jump (3-bytes) or a short jump(2-bytes) with a NOP.
Originally I had a long jump there but I reorganized my code, so that it became a short-jump and I forgot to add the NOP to be 3-bytes.
Linux didn't recognize the file-system on the pendrive and it took me some time to figure out that the NOP was missing.

[bigbob]

I would just like to point out that Windows (and some/most BIOS might do the same) assumes that a flash drive has only 1 partition on it. (Where a flash drive is different than a USB Hard Drive) And that could lead to some of the symptoms you are seeing.

- Monk

It is good to know, but there is only one partition on my pendrive.
So, I don't think that's the problem.

[DavidCooper]

Probably you mean that there is an MBR but no code to load the VBR !? Since a partition-entry is necessary.

I wasn't counting the partition table as code - the MBR code stops before the table. The drives I've put my OS on certainly didn't come with any MBR code on them, but then I've only used two different types for that and I haven't looked at the MBR on most of my other drives.

Let's say somebody has a 4GB-pendrive and he/she just wants to test your (USB-bootable)OS with keeping the files that are already on the pendrive.

I don't think many people would want to put someone's obscure operating system on a drive with important data on it, so realistically they're either going to use a new one or an old one which they've cleared for the purpose. They're inexpensive enough for it not to be any great problem just to use a new one. In your case it's easier though: your OS will just be saved in the FAT32 partition as a normal file or set of files (though it may need to go in the root directory), and all the user needs to do after that is use a hex editor to add MBR code (without wiping out the original partition table), mark the partition as bootable, and then add your VBR code (without wiping out the existing BPB). Either DS:SI or the BPB can then be used by the VBR to work out where the FAT32 partition is, and the VBR should then be able to hunt for the file which it is to load (which may be the whole OS or a small loader which can find and load all the other files that make up the OS) - there should be just enough room in a VBR to do this without having to reserve a special location to hold the OS. If you want to make it really easy for the user, you could write programs that can be run under Windows/Linux/etc. to install your OS onto a flash drive for them, but it'll probably be a long time before your OS attracts the kind of users who would need so much help - to begin with, most of them will be happy to do a bit of direct editing of sectors.

[bigbob]

I don't think many people would want to put someone's obscure operating system on a drive with important data on it, so realistically they're either going to use a new one or an old one which they've cleared for the purpose.

I wrote my first bootloader (VBR) with code to load from FAT32 today (I booted from CD with floppy-emulation before). It is 428 bytes longer than 1 sector. And it ignores LongFilename-entries (just checks the short ones). It's not easy to have code that loads from FAT32 in just 512 bytes.
The code of MenuetOS looks good, I learned a lot from it:
http://www.menuetos.org/appl/bootmf32.asm

Now I realize that what I was saying in the previous posts was nonsense. The code must be very tight, the files (loader and kernel) have to be in the first sector (or cluster!?) of the root-directory, in order to be found quickly.

[DavidCooper]

I wrote my first bootloader (VBR) with code to load from FAT32 today (I booted from CD with floppy-emulation before). It is 428 bytes longer than 1 sector. And it ignores LongFilename-entries (just checks the short ones). It's not easy to have code that loads from FAT32 in just 512 bytes.
The code of MenuetOS looks good, I learned a lot from it:
http://www.menuetos.org/appl/bootmf32.asm

Now I realize that what I was saying in the previous posts was nonsense. The code must be very tight, the files (loader and kernel) have to be in the first sector (or cluster!?) of the root-directory, in order to be found quickly.

There is an alternative to writing very compact (and possibly inadequate) code, and that's to take advantage of the many spare sectors which follow the VBR (or the ones ahead of the partition - there is often a megabyte of unused space sitting there, and sometimes more). You could write more extensive loader code to fit into perhaps three or four of those sectors for loading the OS, and you could allow the user to decide where to write them depending on where they find empty space. They would then only have to edit one byte of your VBR to tell its code where these extra sectors of code have been written. If the four sectors ahead of the VBR are chosen, for example, the value would be 252 (= -4), or if the four sectors after the VBR are used it would be 1. This creates a bit more work for the user and requires them to be intelligent enough to identify several consecutive free sectors (and to count how far away from the VBR they begin), but the gain is that they could just save your OS file(s) onto the drive without worrying about where either it or its directory entry will end up. Trying to rearrange the data in the first root directory cluster to make room for the new entry could be a nightmare on a drive that's already got a lot of files on it, unless there's already a good piece of software available for doing that job.

If the drive is new or has just been formatted, it isn't a problem as you can simply save the OS as the first file on it and then the loading task becomes simple. So, you could have two different methods of installation: one with a VBR loading the first file on the disk, and another with a VBR loading a more advanced loader from free sectors. Which one the user chooses would depend on whether (s)he wants to install it on a new/newly-formatted drive or an old one that's already full of junk. Most of them will likely use a new or newly-formatted drive, so the easiest solution is just to ditch the option of them putting it on an old drive that's already half full. Then again though, you may also want them to be able to replace the OS with a newer, bulkier version which can't simply overwrite the old one, so the other option of using a loader stored in a few free sectors might yet be more attractive.

There's no clear winner, but both options are viable.

[Octocontrabass]

There's no clear winner, but both options are viable.

I, personally, have found that being able to copy replacement files onto the disk without reformatting it every time makes development much faster. That's a clear winner for my needs.

the files (loader and kernel) have to be in the first sector (or cluster!?) of the root-directory, in order to be found quickly.

It's not about being found quickly; the author of the MenuetOS boot code simply didn't feel the need to write a bootloader capable of parsing the entire root directory. You don't have to limit yourself that way if you don't want to.

[bigbob]

Thanks guys!
I will use some of the free sectors.

[onlyonemac]

You may just have an unusually awkward machine. Mine won't boot from USB if they're being booted up after hibernating so I have to shut Windows right down first. That won't be the issue on your machine, but it does show that there is at least one hidden setting somewhere that can make the BIOS ignore a USB device if Windows decides to be unhelpful.

That behaviour is not an "awkward" machine but is common to almost all computers when used with hibernation. There is a flag in the CMOS RAM that is set by most operating systems when the machine is put into hibernation, which tells the BIOS that on the next power-up it must not allow booting from another device (or changing of the CMOS settings). This flag is to prevent dumb people from corrupting their filesystem by booting up another operating system and then accessing a device which was in use when the system was hibernated (and which therefore is in an inconsistent state and should not be used), and while there should be a way to override it by pressing some kind of "yes I really do know what I'm doing and I want to boot from another device anyway" in my experience there never is. One trick though that works in my situation is that when I have hibernated my Linux system then I boot Windows and shut it down again to reset the flag (but obviously that wouldn't work if you only have one operating system on the hard drive; then you will have to resume the hibernated system and shut it down fully).

[bigbob]

That behaviour is not an "awkward" machine but is common to almost all computers when used with hibernation.

I forgot to comment on "hibernation". I always properly shutdown my computers and have never used hibernation.