Zaval's issue with GPT and ESP

[bzt]

I've moved this from the other topic, because this has absolutely nothing to do with BOOTBOOT.

that modifying ESP in some circumstances is harder than an ordinary FAT volume.

Harder how and when? You are talking to someone who has implemented an ESP creator, so unlike you, I know what ESP is and isn't. This has nothing to do with BOOTBOOT.

GPT may be unavailable for some (virtual) disk sizes.

Interesting, show me such a case! This has nothing to do with BOOTBOOT either.

SD cards cannot be formatted with it, without breaking their standard.

Pure lie, no partitioning tool nor formater refuses to do so, you can format SD cards with GPT no probs. I for one use GPT partitioned SD cards on my Pi machines all the time, all works fine. Nobody ever reported non-functioning SD card with GPT. NEVER. Also, this has nothing to do with BOOTBOOT.

wrong! it's such a BS, that I even don't know what to say against.

You should say nothing. You even have quoted the part of the spec that contradicts you.

The following is the order in which a block device must be scanned to determine if it contains partitions. When a check for a valid partitioning scheme succeeds, the search terminates.
1. Check for GUID Partition Table Headers.
2. Follow ISO-9660 specification to search for ISO-9660 volume structures on the magic LBA.
3. Check for an “El Torito” volume extension and follow the “El Torito” CD-ROM specification.
4. If none of the above, check LBA 0 for a legacy MBR partition table.
5. No partition found on device.

Meaning if your storage has GPT (point 1), then EFI shouldn't and won't parse the legacy tables (point 4). Nobody cares if you have another disk which is not the boot disk.
Furthermore, BOOTBOOT uses GPT, so you can't use MBR tables (this is third time I've written down this), therefore MBR tables have nothing to do with BOOTBOOT.

Cheers,
bzt

[PeterX]

I personally think that Zaval is wrong about his kind of crusade against the GPT. But that discussion has reached a point where everything is said and it leads nowhere. Bzt says: Use GPT. Zaval says: Use MBR. OK, all said. Newbies have to decide which way to follow, that may be a problem, but apart from that...

Greetings
Peter

[BenLunt]

I agree. There is nothing that says I *must* follow some specification to boot my OS.

I currently have a hard drive image that has both MBR *and* GPT on it. Yes, it doesn't exactly follow both the MBR or the GPT specifications, but it boots both legacy *and* UEFI on all systems that I have tried it on, both emulated and real hardware. That is all I care about. This is a hobby, something I do for fun.

The PMBR (Protected MBR) of the GPT was for backward compatibility anyway, to keep (correctly written) legacy operating systems from accessing any sectors the new partition scheme encompasses. However, this doesn't mean that I can't write a MBR entry that accesses a partition within the disk that just so happens to be the same partition pointed to be the GPT, knowing that the GPT entry must be less than 0xFFFFFFFF + 1 sectors from start of the disk *and* must be less than 0xFFFFFFFF + 1 sectors is size.

This idea stated above may not 100% follow the GPT specification, I understand that and except that. This is my project, this is my hobby, this is my decision to do so. I did it to see if I could write a Legacy MBR, Partition Boot Record, and legacy Loader, to boot the image *as well as* have UEFI boot the same image. It does, so I feel I have accomplished that task, and I have moved on.

Anyway, I believe it is up to the author to choose what he/she would like to do, follow a specification or not to follow a specification. I would hope that none of us here, including myself, is doing this as anything more than a hobby and/or a learning experience, so what is the problem if we color outside the lines, right?

Well, hope I haven't fed the flames, it was not my intent to do so.

Ben

[Octocontrabass]

so what is the problem if we color outside the lines, right?

SD cards are an unusual case because unlike most other storage media, the SD card itself is allowed to change its contents outside of whatever reading and writing you do. If you don't follow the specifications, you risk confusing the card into corrupting your data.

There are also performance considerations: SD cards may not be able to reach their rated performance if you don't read and write them as the manufacturer intended.

You can probably still get away with a hybrid MBR+GPT scheme (as long as there's no backup GPT), since the specification doesn't explicitly say what you can and can't put in the gap between the MBR and the start of the partition.

[bzt]

I agree. There is nothing that says I *must* follow some specification to boot my OS.

Generally yes, but this was argued in the BOOTBOOT topic. Now that does not use MBR tables any more, therefore if you want to use BOOTBOOT, you'll have to use GPT, simple as that. It was a design choice for me to use only a single common partitioning format on all systems and on all platforms with all boot specifications.

For the records, I think (and I must put emphasis on IMHO), there's no point in advocating MBR in 2020, because disks with more than 2^32 sectors are already in mass-production, and all mainstream OSes have switched to GPT already (even the ones that were never MBR-compatible in the first place, like BSDs and Solaris). But I can accept if someone thinks otherwise, as long as they are not trying to convince me to cripple my loader from GPT to MBR. They are free to write their own MBR-only loader, I wouldn't mind that at all!

I currently have a hard drive image that has both MBR *and* GPT on it. Yes, it doesn't exactly follow both the MBR or the GPT specifications, but it boots both legacy *and* UEFI on all systems that I have tried it on, both emulated and real hardware. That is all I care about. This is a hobby, something I do for fun.

Yes, this works. I've used the same trick to allow Raspberry Pi to locate its firmware files on the ESP, because RPi firmware does not understand GPT. For legacy BIOS this doesn't matter, because BIOS didn't parsed the MBR partitions in the first place, the MBR code did.

This idea stated above may not 100% follow the GPT specification

There's nothing in the UEFI spec that would say you can't map the same partition in the GPT and in the MBR table too. Actually many OS did that a few years ago when they were in the transition to GPT. (Apple (and Bootcamp) did that automatically, Windows has the MBR2GPT tool, and both fdisk and gdisk has an option to sync the tables).

Anyway, I believe it is up to the author to choose what he/she would like to do, follow a specification or not to follow a specification. I would hope that none of us here, including myself, is doing this as anything more than a hobby and/or a learning experience, so what is the problem if we color outside the lines, right?

Absolutely, except firmware usually do follow the specification, and you can't change the firmware, so there's not much point in fighting windmills (you'll never be able to get an UEFI to read the ESP from Solaris slices for example). But once you're out of dragon land, it is totally up to the OS developer.

Well, hope I haven't fed the flames, it was not my intent to do so.

No, not at all. Your posts are always welcome.

Cheers,
bzt

[bzt]

SD cards are an unusual case because unlike most other storage media, the SD card itself is allowed to change its contents outside of whatever reading and writing you do. If you don't follow the specifications, you risk confusing the card into corrupting your data.

Yes, so I've heard. But nobody ever experienced such corruption, and believe me, there are lots and lots of people using ext file systems on SD cards. (FYI, the card does not interpret the data, it's receiving lists from the driver to categorize sectors as meta-data, directory, file content etc. So the drivers are allowed to tell the card whatever they want.)

There are also performance considerations: SD cards may not be able to reach their rated performance if you don't read and write them as the manufacturer intended.

This is just half truth. Yes, for full performance the drivers have to send sector lists to the card. But drivers can send lists for other file systems as well, it is most certainly possible without performance penalty. Of course, Microsoft don't want you to know this, and they don't want you to use anything else than their FAT, but you surely safely can.

Think about this: Raspberry Pi alone sold already more than 30 million boards in 2019, all running on SD cards with ext4 file system, and none reported corruption nor performance loss. It's interesting, don't you think? And there are many more boards running Linux with ext4 on SD cards.

Cheers,
bzt

[Octocontrabass]

FYI, the card does not interpret the data,

Some of them do. WiFi SD cards are the most obvious example, but you can't tell what ordinary SD cards are doing inside their firmware.

[PeterX]

FYI, the card does not interpret the data,

Some of them do. WiFi SD cards are the most obvious example, but you can't tell what ordinary SD cards are doing inside their firmware.

Yes, theoretically in the future SD cards might do fancy stuff, but

a) according to Wikipedia SD cards are just memory cards
b) the interface to access them is straight forward, like a HD (EDIT: No matter what the card does inside)

I know SD cards from photographing, and I think, they don't interpret data.

Greetings Peter

[Octocontrabass]

Yes, theoretically in the future SD cards might do fancy stuff, but

Who said anything theoretical? You can buy a WiFi SD card right now.

a) according to Wikipedia SD cards are just memory cards

According to Wikipedia, no they're not.

b) the interface to access them is straight forward, like a HD

It is, which is why you must be aware that SD cards are not straightforward when you're messing with details the ordinary user never sees.

[PeterX]

Yes, theoretically in the future SD cards might do fancy stuff, but

Who said anything theoretical? You can buy a WiFi SD card right now.

a) according to Wikipedia SD cards are just memory cards

According to Wikipedia, no they're not.

Secure Digital, officially abbreviated as SD, is a proprietary non-volatile memory card format developed by the SD Association (SDA) for use in portable devices.

from here:
https://en.wikipedia.org/wiki/SD_card
The Wifi SD cards you mention are like radio cassettes and similar devices. They are not normal cards and they behave like normal ones on their interface.

b) the interface to access them is straight forward, like a HD

It is, which is why you must be aware that SD cards are not straightforward when you're messing with details the ordinary user never sees.

Yes, true! But that's always true. But if you don't mess with them, you don't have that problem.

[zaval]

I've moved this from the other topic, because this has absolutely nothing to do with BOOTBOOT.

rename this BS and remove my nickname from it. you may behave as much as crackpot as you want, just don't f&cking use my nickname in your clowning as you did here.

that modifying ESP in some circumstances is harder than an ordinary FAT volume.

Harder how and when? You are talking to someone who has implemented an ESP creator, so unlike you, I know what ESP is and isn't. This has nothing to do with BOOTBOOT.

a utility may reject to delete it because it's ESP. for example. and no, you have zero clue judging by your BS.

GPT may be unavailable for some (virtual) disk sizes.

Interesting, show me such a case! This has nothing to do with BOOTBOOT either.

for small size it can be true. if one makes a small virtual HD, some utilities reject creating GPT with it.

SD cards cannot be formatted with it, without breaking their standard.

Pure lie, no partitioning tool nor formater refuses to do so, you can format SD cards with GPT no probs. I for one use GPT partitioned SD cards on my Pi machines all the time, all works fine. Nobody ever reported non-functioning SD card with GPT. NEVER. Also, this has nothing to do with BOOTBOOT.

official SD Formatter lets you do this? GPT with non-FAT volume(s)? pure lie is your BS. I am tired to point you to the 2nd part of SD specification, where it's specified.

wrong! it's such a BS, that I even don't know what to say against.

You should say nothing. You even have quoted the part of the spec that contradicts you.

The following is the order in which a block device must be scanned to determine if it contains partitions. When a check for a valid partitioning scheme succeeds, the search terminates.
1. Check for GUID Partition Table Headers.
2. Follow ISO-9660 specification to search for ISO-9660 volume structures on the magic LBA.
3. Check for an “El Torito” volume extension and follow the “El Torito” CD-ROM specification.
4. If none of the above, check LBA 0 for a legacy MBR partition table.
5. No partition found on device.

Meaning if your storage has GPT (point 1), then EFI shouldn't and won't parse the legacy tables (point 4). Nobody cares if you have another disk which is not the boot disk.
Furthermore, BOOTBOOT uses GPT, so you can't use MBR tables (this is third time I've written down this), therefore MBR tables have nothing to do with BOOTBOOT.

Seriously? You would better pretend to be an idiot instead of admitting you got the text wrong? Okay. "nobody" cares, let's leave this to that "nobody".
There is no such a term as "boot" disk in UEFI. all disks MUST be inspected because any of them may contain a valid Load Option target or be accessed by the FW/OSLs during boot time. the partition discovery algorithm mandates a compliant FW, to first check every disk for GPT scheme and stop discovery FOR THIS DISK if the scheme is found, then proceed WITH THE SAME DISK if not found with checking for other schemes. this procedure is being applied FOR EVERY DISK. if you found "Nobody cares if you have another disk which is not the boot disk." in the spec. then either show this line or you are hallucinating or making things up on the fly to still look a smart aleck. just try to read the quote you are waving - it literally sais:

The following is the order in which A BLOCK DEVICE must be scanned to determine if it contains partitions.

just one of any. no "boot disks", no "special ones". for every. attach to qemu several vhds with GPT and MBR and then look at "mounted" devices listed, genius.

I personally think that Zaval is wrong about his kind of crusade against the GPT.

you just have a strong need to talk in every thread, do you? I started in that thread just by pointing out, that bzt as always sais wrong about what's allowed and what's not. and suggested the user to try an ordinary (non ESP) FAT volume with MBR (with his own UEFI OSL, which is always way better, than the lamery from some loonies) to overcome issues he had. and also made a note about the fallacy of abusing the resort path of \efi\boot\bootXXX.efi especially on non-removable storages.
my first post there:

just for the note, despite bzt scares you, that you are required to mark a FAT partition as ESP and use only GPT, it's not true. you can create MBR with an ordinary FAT partition and be sure - it will be recognized by UEFI. since you are only at the start, I'd go this way (it's easier and maybe you'd have less troubles)

it's as a "crusade" against GPT as bzt's bootboot worth trying, honestly.

[PeterX]

Zaval, please stop your rants! I give nothing on your hate of Bootboot, GPT and what else.
And why do you throw out words like BS (everybody understands that means "bullshit").

Your writing isn't rational nor amusing anymore. Please play bad guy elsewhere. EDIT: This doesn't mean "bugger off" It just means don't play bad guy here anymore. If you can't control yourself THEN be better off. If you can, everthing is fine and I like to hear from your OS dev wisdom.

[zaval]

Zaval, please stop your rants! I give nothing on your hate of Bootboot, GPT and what else.
And why do you throw out words like BS (everybody understands that means "bullshit").

Your writing isn't rational nor amusing anymore. Please play bad guy elsewhere. EDIT: This doesn't mean "bugger off" It just means don't play bad guy here anymore. If you can't control yourself THEN be better off. If you can, everthing is fine and I like to hear from your OS dev wisdom.

did you imagine yourself a moderator? for commanding me what I should stop. I responded to the thread, using my nickname in the title and quoting my comments followed by wrong statements I considered required to answer to (and I called them BS because they are such and being repeated endlessly, they beg to have been called as they are). if you see violations of the forum rules on my side, report them, but don't command me what to do or moreover "be off" as if you are in authority to do so.

last, but not least, I am not an anime girl to amuse you.

[PeterX]

Zaval, please stop your rants! I give nothing on your hate of Bootboot, GPT and what else.
And why do you throw out words like BS (everybody understands that means "bullshit").

Your writing isn't rational nor amusing anymore. Please play bad guy elsewhere. EDIT: This doesn't mean "bugger off" It just means don't play bad guy here anymore. If you can't control yourself THEN be better off. If you can, everthing is fine and I like to hear from your OS dev wisdom.

did you imagine yourself a moderator? for commanding me what I should stop. I responded to the thread, using my nickname in the title and quoting my comments followed by wrong statements I considered required to answer to. if you see violations of the forum rules on my side, report them, but don't command me what to do or moreover "be off" as if you are in authority to do so.

last, but not least, I am not an anime girl to amuse you.

I did not pretend to be a moderator, but I did tell what ticks me off and what I personally don't like to read here. That's all to it. I still say you haven't good, rational arguments anymore. It's more a general rant biting reflexively at everything others (especially bzt) say.

Maybe I did speak with an authority which I (of course) don't have and maybe it's wrong to say "Leave if you can't stop your rants". Sorry then, it was never my intention to play moderator or God. But I am not sorry for telling "I don't wanna hear that stuff anymore". What you and the moderators do with that is up to you and them.

You may not be here to amuse but you indeed are here to give everybody a good time. This is a place where people want to talk about OS dev and other fun stuff. With an emphasize on fun.

And I DID read your postings and I can't see good arguments. It's all said several times now. It's starting to turn into a circle.

[zaval]

maybe pictures can convince that stubborn dude to get finally, he was wrong?

Nobody cares if you have another disk which is not the boot disk.

but UEFI does. and if you read and thought, you'd get it as well. here is a screenshot, showing a setup in which all the storage attached to the machine is 2 hard disks; they are 2 vhds - one partitioned with MBR, having one FAT volume (fs0) and another with GPT, having one FAT volume (fs1) and an unformatted space (blk3) of the almost equal size with fs1. so the system has 2 volumes.

- both volumes are detected by the FW and hence - listed by the EFI shell at its start.
- both volumes contain copies of my OSL and they may be run from there
- even as a Load Option, hence - may have been registered with the FW.
- NONE of these are marked as ESP