OSDev.org

If I may answer if quok hasn't done so up to now...

I think many of your questions may be answered by stating: "Do not assume anything except that the medium is not bigger than 8 GB, a file named "OSLOADER" is present, the size of this OSLOADER is below or equal to 128 kB, it has to be loaded somewhere in the first megabyte (though I don't think you have to load it into the ROM ) and OSLOADER is a statically linked ELF32 file." (I think that's what quok means)

So let's go through each of your points.

technik3k wrote:would it be safe to assume that static ELF needed to be supported is like the hello.asm from fasm for linux examples, where the file doesn't have any section headers and only two program headers (one for each code and data segments)?

Do not assume anything.
I'm pretty sure it will have section headers and it may also contain more than two program headers.

technik3k wrote:What is the linking alignment? Do we expect to relocate data section relative to the code section or they are already properly spaced?

You won't know anything about the alignment. But what do you want to say with that relocation?

technik3k wrote:Do you want a long jump to e_entry or to the first p_vaddr?

quok wrote:Jump to the ELF entry point.

So it's e_entry.

technik3k wrote:Do you guarantee that is is going to be paragraph aligned?

Sorry if it's either my English or my knowledge (or both of them) that's that bad: But what's a paragraph?

technik3k wrote:BIOS/MBR calls this boot sector code at 0:7C00h with correct boot drive number passed in DL

I think this one is safe to assume.

technik3k wrote:BIOS int 13h CHS calls should be used when Cyl<1024 and LBA calls should be used otherwise

How you read that sectors is up to you. You might even write a floppy disk driver, a USB driver, a HDD driver and a CDROM/DVD driver but I hardly believe all of those fit into 512 bytes.

technik3k wrote:This boot code needs to support real floppies and therefore needs to do a reset operation.

Of course it has to support real floppies. But I can't tell you anything about that reset.

technik3k wrote:FAT BPB has correct sector size (for this media) and it is one of the following values: {512, 1024, 2048, 4096}

As far as I know, that BPB will be correct in all points. Hence this sector size will be correct, though I don't know if it's forced to be one of those values.

technik3k wrote:FAT cluster size has one of the following values: { 1, 2, 4, 8, 16, 32, 64 }

128 is also possible (4 GB partition).

technik3k wrote:Number of FAT copies is either 1 or 2 and they have reasonable size (i.e. cluster#2 is within 64k sectors from the beginning of the partition, so 32-bit arithmetic is not necessary there)

I think the number of FAT copies isn't limited. (I'll tell something about those 32 bit arithmetic later on)

technik3k wrote:FileSystemID field is correct and could only be either "FAT12 " or "FAT16 "

quok personally stated to me that this is safe to assume though it is wrong according to the specification.

technik3k wrote:Finally, since you didn't mention anything about error messages I am going to assume that calling "int 18h" in case of an error is good enough to meet the official rules.

Error messages? Who ever said anything about those?


Now something about 32 bit arithmetic when using sectors...
quok stated to me that one of his tests will be a 4 GB partition and one will be on an 8 GB medium (I think with the relevant partition as a partition which starts after several GB). Those huge sector numbers won't fit into 16 bit integers. I think you don't get around using 32 bit arithmetic if you want to accomplish all goals, but it's your choice if you want to take a chance.

XanClic, thank you for taking time answering my questions.

I am, obviously, new on OSDev.

As almost anyone around here, I have a boot sector code of mine for years that was designed to boot DOS, Win9x, or NT from partitions above 2G. Eventually it evolved to boot whatever from FAT-12 or FAT-16 partition off a floppy or anywhere on HD up to 2Tb in size. It takes about 400 bytes (excluding error messages) and fully resides in the boot sector. Essentially once you put it into boot sector you can simply copy system files (IO.SYS and MSDOS.SYS) for DOS 5, 6, Win 95, 98 or NTLDR onto partition and it will correctly boot them. In addition, you could name any binary up to 500kb "NTLDR" and it will load and run it as long as conventional memory permits. Variable sector sizes are handled as well (though never tested).

OSNews had a post about this contest few days ago and (since I had few bytes left) I decided to take a good look.

Of course, rather than doing any real work , I would be happy to convince quok and bewing that if they are looking for "copy and boot" capability, then:

having a single boot code that supports CHS, LBA, floppies, HDDs up to 2Tb, variable sector size, files up to 600k, several executable format's for entry points, etc...

far outweighs benefits of fully parsing ELF headers in the boot sector at the expense of other features. You would need another boot code if you need to bootstrap flat binary or 'MZ'/'PE'. Right?

I personally would rather have freedom to choose between different executable formats and deal with a fixed load address, than having a freedom of any load address and be limited to ELF located under 8G !

In respect to additional goal, wouldn't it be more flexible handling switching to protected mode in OSLOADER rather then trying to cram it into boot sector? If you want to be compatible with older hardware I seriously doubt that you can reliably enable A20 and detect extended memory in a few bytes that might be left.

XanClic wrote:

technik3k wrote:What is the linking alignment? Do we expect to relocate data section relative to the code section or they are already properly spaced?

You won't know anything about the alignment. But what do you want to say with that relocation?

I meant to ask if sections are aligned such that disk image is identical to the memory image. For example, by default, data section could follow code section immediately in the file, but you would need to "insert" zeros when loading it into memory to get beginning of data section aligned on 4k page boundary.

XanClic wrote:

technik3k wrote:Do you guarantee that is is going to be paragraph aligned?

Sorry if it's either my English or my knowledge (or both of them) that's that bad: But what's a paragraph?

Oh, well, this is old-speak for alignment on 16 byte boundary, i.e. "segment para" in the MASM world. (OMG I am old )

XanClic wrote:

technik3k wrote:This boot code needs to support real floppies and therefore needs to do a reset operation.

Of course it has to support real floppies. But I can't tell you anything about that reset.

Real floppies and zip drives would occasionally return read errors due to dirty surface, slight misalignment between devices that formated, wrote, and now reading the media, etc...
Issuing int 13h (ah=0) disk reset would send reading head back to track 0 sensor and force recalibration / reset of the drive hardware (or whatever it needs to do).
Doing three resets before reporting an error was done in all DOS utilities and you can find it in quok and bewing's fat_boot.S as well.

XanClic wrote: Now something about 32 bit arithmetic when using sectors...
quok stated to me that one of his tests will be a 4 GB partition and one will be on an 8 GB medium (I think with the relevant partition as a partition which starts after several GB). Those huge sector numbers won't fit into 16 bit integers. I think you don't get around using 32 bit arithmetic if you want to accomplish all goals, but it's your choice if you want to take a chance.

Eh, I could have saved another 5 bytes there...

Thanks,
technik3k

technik3k wrote:I meant to ask if sections are aligned such that disk image is identical to the memory image.

I wouldn't rely on that.

technik3k wrote:Real floppies and zip drives would occasionally return read errors due to dirty surface, slight misalignment between devices that formated, wrote, and now reading the media, etc...
Issuing int 13h (ah=0) disk reset would send reading head back to track 0 sensor and force recalibration / reset of the drive hardware (or whatever it needs to do).

If that's necessary you have to do it because it has to work on real hardware. If it works without reset you may skip that of course.

Well, now it's me who has a question about what's reliable and what isn't.

I have obviously a problem with loading sectors from above 4 GB on real hardware, while the code works in VMs and also on a floppy disk on real hardware. So it might turn out that the computers' BIOS I have here doesn't support reading sectors from over 504 MB, 2 GB, 4 GB or whatever border there might be. But the INT 13h specification clearly states (at least as far as I know, which might be not very far ) that reading up to 8 GB should be possible (as said, the VMs work properly). So may I rely on that the BIOS does what it's supposed to do (i. e., supporting reading sectors from up to 8 GB via CHS) or am I forced to use LBA if it doesn't?

I know, I stated very clearly that nothing, really nothing, should be assumed. But I think this is not a real "assumption" in a way that something might be somehow while there is no problem of being otherwise. I simply want to assume that things are working properly.

(Of course it's also very likely that my code simply turns out as being totally broken and it's funny that VMs take it as it is—I just want to make sure.)

PS: A reason for asking this question is quok's limitation to 8 GB, which is exactly the limit for CHS—by chance?

XanClic wrote:I have obviously a problem with loading sectors from above 4 GB on real hardware, while the code works in VMs and also on a floppy disk on real hardware. So it might turn out that the computers' BIOS I have here doesn't support reading sectors from over 504 MB, 2 GB, 4 GB or whatever border there might be. But the INT 13h specification clearly states (at least as far as I know, which might be not very far ) that reading up to 8 GB should be possible (as said, the VMs work properly). So may I rely on that the BIOS does what it's supposed to do (i. e., supporting reading sectors from up to 8 GB via CHS) or am I forced to use LBA if it doesn't?

Yes. It is correct that with BIOS translation then you can read locations up to 8G using only CHS. VMWare does that by default, but the real one might need "Large disk mode" option to be enabled.
When PATA disks report their geometry to BIOS they are limited to 1023 Cyl x 16 Heads x 63 Sectors (504 MB). If BIOS can emulate "Large disk mode" it reports 255 heads and does translation of CHS addresses to proper ATA commands giving you access up to 8G. Commonly mentioned 2G limit have nothing to do with BIOS but was due to the sloppy code in the MS-DOS boot sector. Once you fix that DOS has no problem booting and accessing anything under 8G. Win 98 was released fully aware of LBA mode.

If you have problems around 4G it is likely a bug in your code. But you still should check that disk geometry reported by BIOS is for the large mode (i.e. 255 Heads). Linux and Windows talk directly to disk controller, so you should go outside. The easiest way is to boot the "Ultimate Boot CD" and run some Disk Editor or Partition Manager tool to see what geometry BIOS reports.

Hope this helps,
technik3k

Yes, I know about that translation problem but the funny thing here is I'm using a USB stick, those use LBA (afaik), so there should be no such problem... And a bug in my code... Well, that's of course very likely, but why should it work in VMs if I have overflows or anything like that?

XanClic wrote:Yes, I know about that translation problem but the funny thing here is I'm using a USB stick, those use LBA (afaik), so there should be no such problem... And a bug in my code... Well, that's of course very likely, but why should it work in VMs if I have overflows or anything like that?

How do you format USB and get your boot sector onto it vs how do you run it under VMware?

The reason I am asking is that when booting off USB drive some BIOSes first read boot sector BPB and then decide to either emulate it as a floppy (standard size?) or as a hard disk with or without MBR.

I guess the first thing to do is to compare initial environment and reported disk geometries under VMWare and the real thing. The easiest way to debug it would be to pusha all input registers then call int 13, ah=8, pusha again and call a routine that prints 16 words from the top of the stack. You may add to that LBA installation check int 13, ah=41h,bx=55aah as well.

Let us know what you see,
technik3k

The USB stick I'm talking about has a size of 8 GB. There are two partitions on it, one formatted with ext2 which is 4 GB in size and another one formatted with fat16 which is also 4 GB in size. The MBR is occupied by GRUB which then chainloads my code which lies in the boot sector of the fat16 partition. I formatted that partition with mkdosfs, than copied the BPB out of it, copied my boot code on it and copied the BPB again into the boot record.

technik3k wrote:I guess the first thing to do is to compare initial environment and reported disk geometries under VMWare and the real thing.

Hey, I did exactly that.
And this might solve the problems (though I use qemu): The geometries are different (i. e., 63 sectors per track in qemu and 32 on the real hardware). So I now don't trust in the BPB anymore and get the limits (sectors per track, heads, etc.) by a call of int 13h with AH = 8h. Didn't try that on a real computer until now but it at least works in qemu.

Thanks for your reply!

XanClic wrote:The geometries are different (i. e., 63 sectors per track in qemu and 32 on the real hardware).

This is the problem - there is no way to address up to 8G using CHS unless you max out all dimentions (1024 Cyl x 255 Heads, 63 Sectors). If you cannot get BIOS to emulate USB with 1024x255x63 CHS geometry you have to use LBA.

In case BIOS looks at the first partition record in MBR to figure out disk geometry you may try to remove ext2 partition record and/or make sure that both partition records end on head 255 sector 63. I assume it passes DL=80h when booting it, right?

technik3k wrote:I assume it passes DL=80h when booting it, right?

I haven't looked at that value yet, but I figure it'll passing something with bit 7 set.

The partitions don't end on those boundaries, so I'll try that... Thanks in advance!

EDIT: Hm, still doesn't work... So my original question remains: Will the BIOS work properly and support devices up to 8 GB with CHS only?

XanClic wrote:PS: A reason for asking this question is quok's limitation to 8 GB, which is exactly the limit for CHS—by chance?

Yes, I chose 8GB for the maximum disk image size so you wouldn't have to worry about testing for and using EDD functions when available. Good catch.

XanClic wrote:And this might solve the problems (though I use qemu): The geometries are different (i. e., 63 sectors per track in qemu and 32 on the real hardware). So I know don't trust in the BPB anymore and get the limits (sectors per track, heads, etc.) by a call of int 13h with AH = 8h. Didn't try that on a real computer until now but it at least works in qemu.

Another good catch. I found this when testing my fat12/16 bootsector many months ago. Kept forgetting about adding it to the wiki's FAT article though. Thanks for the reminder! I hope I didn't mis-state in paste posts that you can assume ALL the BPB fields are valid. You may assume the BPB fields relating to the filesystem metadata is valid, otherwise the filesystem would be corrupt. But you can't necessarily rely on the geometry fields.

XanClic wrote:I know about that translation problem but the funny thing here is I'm using a USB stick, those use LBA (afaik)

I have a USB stick that works quite well using CHS.

XanClic wrote:Will the BIOS work properly and support devices up to 8 GB with CHS only?

About the test images, you can assume that I won't do anything crazy like try to boot a 2GB partition at the end of a 8GB drive on an i386. Tests that would require large drive support in the BIOS will be done on machines that have such support.

quok wrote:HOW TO ENTER
Publicly reply to this post with your entry as an attachment. Do not send your entry privately (via PM or email), or provide a link to another site that hosts your entry. Make sure your entry contains compilation instructions as well as a means of contacting you if your entry should be a winner.

quok wrote:THE TWIST

* The FIRST entry submitted that accomplishes goals 1 through 7 and passes all tests will be declared the winner. A submitted entry that does not pass all tests may be modified and resubmitted.

This makes conflict with basic rules of contest. As I understand it correctly. When I write code I must post it on forum for public view. Then it may occur that its not accomplishes all goals and I have a chance to correct it and post again. But everyone who reads my code could take my solutions and implement it in their own code. So my proposition is to make code public after contest is over.

Only submit when you're confident your code meets the standards set by this contest. If you fail to do so, then it serves you right to then have to race against your own mess-up to get an updated entry in.

quok wrote:and the second winner will be announced after community voting has closed.

What exactly will the community be looking for? The goals are very strict and there's not much room for creative freedom if at all. So if two bootsectors complete all the goals - how am I suppose to judge the better of the two?

MessiahAndrw wrote:

quok wrote:and the second winner will be announced after community voting has closed.

What exactly will the community be looking for? The goals are very strict and there's not much room for creative freedom if at all. So if two bootsectors complete all the goals - how am I suppose to judge the better of the two?

Which does it in less space? Which checks more potential error conditions? Maybe one adds support for FAT32 as well? Maybe the other has LBA and[/a] CHS in the same binary?