well, I know how to load it if it is in a certain place of the disk (hardcoded sector block into MBR code), but what if I would like it to be in /MyOS_dir/Kernel.bin of the system drive?
I know I can use a second stage, but I can't understand how a filesystem work?
different os:s using different aproaches of mapping physical drives, linux/*nix will mount them to a map and Windows map them to a drive letter AmigaOS (if I don't remember wrong) map then to some unique name, e.g. FD0 for floppy
so lets say my OS uses my own implementation similar to the AmigaOS then the kernel could be in FD0:/MyOS_dir/Kernel.bin or HD0:/MyOS_dir/Kernel.bin
how do I map physical drives to names like that?
should the second stage know about those names?
how to load a kernel using a filesystem?
Re:how to load a kernel using a filesystem?
Hm. How to explain this?
If you want to use a file system, you have to know it. The mapping of a file name to a physical location on disc is part of what constitutes a file system. inodes, directory entries, FAT tables, drive mappings, it all depends on which file system you use, and is documented together with the file system in question.
Either your bootloader does know the file system - i.e. how the file system works - and can do the mapping allright.
Or it doesn't, and has to rely on hardcoded sector numbers.
Does that help a bit?
If you want to use a file system, you have to know it. The mapping of a file name to a physical location on disc is part of what constitutes a file system. inodes, directory entries, FAT tables, drive mappings, it all depends on which file system you use, and is documented together with the file system in question.
Either your bootloader does know the file system - i.e. how the file system works - and can do the mapping allright.
Or it doesn't, and has to rely on hardcoded sector numbers.
Does that help a bit?
Every good solution is obvious once you've found it.
-
McZ
Re:how to load a kernel using a filesystem?
so then the stage two is hardcoded for that particular filesystem that I use, then I can have like 100 stage two ready for different filesystem, if I want the OS to be able to run in different filesystems then I only needd to put the correct stage two part (e.g. st2ext2.bin, st2fat32.bin etc. ) in the correct place on the bootdrive.
-
aladdin
Re:how to load a kernel using a filesystem?
if the file system is simple (FAT12/16/32) you can load your kernel or second stage loader directly from the boot sector (like DOS).
an other solution is to create a special partition (a boot partition) with a simple file system so it can be interpreted from your boot sector which will load a second stage loader and this one will detect what kind of filesystem is installed and decide which interpreter to use (what you called st2xxx.bin).
an other solution is to create a special partition (a boot partition) with a simple file system so it can be interpreted from your boot sector which will load a second stage loader and this one will detect what kind of filesystem is installed and decide which interpreter to use (what you called st2xxx.bin).
Re:how to load a kernel using a filesystem?
Hi,
Most people here seem to think that when the computer starts the BIOS should load a boot sector, which should load the kernel, which has to handle everything during boot. IMHO this thinking is wrong (unless kernel is capable of containing every device driver imaginable).
Let me illistrate an (again IMHO) much better way of booting - I'll explain each step after...
A) Somehow a "boot image" gets into memory
B) Some startup code is copied from the boot image into memory
C) The startup code is started
D) The startup code copies the kernel from the boot image into memory
E) The startup code starts the kernel
Ok, at first glance it looks like a lot of work for nothing, but...
Step A involves many different little utilities that all load the boot image into memory (I call these little utilities "boot loaders"). One for 1.44 Mb floppies, another for raw DOS, another for hard-drives, one for ethernet/TFTP, one for GRUB, one for serial ports/SLIP, one for bootable CD-ROM, etc. So here's the first 2 benefits: the OS doesn't have to care how the boot image gets into memory, and the OS can boot from ANYTHING.
The boot image contains FILES, where each file has a header that keeps track of the the file size, name, access flags, etc. This means that once the boot image is in memory all boot code can use it to "load" any files needed for boot. This includes the startup code, kernel/s, any device drivers, scripts, CLI/GUI, etc. This gives additional benefits: the kernel doesn't have to include every device driver imaginable, the boot image can contain multiple kernels, and the OS can work without ANY device drivers.
Step C (the startup code) gives the OS the ability to run code in real mode before the OS starts. Benefits include: sucking heaps of info out of the BIOS/s, handling a boot menu, setting up a video mode properly, letting the user change settings, selecting which kernel to boot (automatically or by user selection).
After the kernel has started, the boot image remains in memory. As soon as the virtual file system is working files are transfered from the boot image into the VFS cache, which behaves like a ram disk. Now you've got the kernel and full file IO working - the OS can continue booting while loading files like normal (even though there's still no device drivers). At this stage the OS might load a configuration script, a device manager, some generic device drivers, or seomthing else. Because any/all of these would be coming from RAM it makes the OS boot much faster
. Sooner or later the OS will load devices drivers to enable it to access files that aren't in the boot image (e.g. networking or hard-disk or something).
Once the OS has booted (and the storage device drivers, file systems, etc are running), the VFS code can write the cached files from the boot image to disk/network/whatever (if they aren't already there). This simplifies OS installation, as you can have a boot image that contains multiple kernels and heaps of device drivers which all end up automatically copied to disk (or wherever). Once this is done the OS can create a customized boot image (containing only the files needed to boot a specific computer), which can be used specifically for that computer in conjunction with any of the boot loaders mentioned in "step A".
It also makes it easy to produce a "demo disk" which allows a potential end-user to try out the OS without installing it, or an "application specific disk" which allows an end-user to use a specific application without installing the OS (e.g. someone could write a game for the OS and distribute the OS and game on a bootable CD - put it in the CD-ROM drive and restart your computer to play).
Cheers,
Brendan
PS. sorry for posting a long rant that doesn't actually answer the question asked...
Most people here seem to think that when the computer starts the BIOS should load a boot sector, which should load the kernel, which has to handle everything during boot. IMHO this thinking is wrong (unless kernel is capable of containing every device driver imaginable).
Let me illistrate an (again IMHO) much better way of booting - I'll explain each step after...
A) Somehow a "boot image" gets into memory
B) Some startup code is copied from the boot image into memory
C) The startup code is started
D) The startup code copies the kernel from the boot image into memory
E) The startup code starts the kernel
Ok, at first glance it looks like a lot of work for nothing, but...
Step A involves many different little utilities that all load the boot image into memory (I call these little utilities "boot loaders"). One for 1.44 Mb floppies, another for raw DOS, another for hard-drives, one for ethernet/TFTP, one for GRUB, one for serial ports/SLIP, one for bootable CD-ROM, etc. So here's the first 2 benefits: the OS doesn't have to care how the boot image gets into memory, and the OS can boot from ANYTHING.
The boot image contains FILES, where each file has a header that keeps track of the the file size, name, access flags, etc. This means that once the boot image is in memory all boot code can use it to "load" any files needed for boot. This includes the startup code, kernel/s, any device drivers, scripts, CLI/GUI, etc. This gives additional benefits: the kernel doesn't have to include every device driver imaginable, the boot image can contain multiple kernels, and the OS can work without ANY device drivers.
Step C (the startup code) gives the OS the ability to run code in real mode before the OS starts. Benefits include: sucking heaps of info out of the BIOS/s, handling a boot menu, setting up a video mode properly, letting the user change settings, selecting which kernel to boot (automatically or by user selection).
After the kernel has started, the boot image remains in memory. As soon as the virtual file system is working files are transfered from the boot image into the VFS cache, which behaves like a ram disk. Now you've got the kernel and full file IO working - the OS can continue booting while loading files like normal (even though there's still no device drivers). At this stage the OS might load a configuration script, a device manager, some generic device drivers, or seomthing else. Because any/all of these would be coming from RAM it makes the OS boot much faster
. Sooner or later the OS will load devices drivers to enable it to access files that aren't in the boot image (e.g. networking or hard-disk or something).Once the OS has booted (and the storage device drivers, file systems, etc are running), the VFS code can write the cached files from the boot image to disk/network/whatever (if they aren't already there). This simplifies OS installation, as you can have a boot image that contains multiple kernels and heaps of device drivers which all end up automatically copied to disk (or wherever). Once this is done the OS can create a customized boot image (containing only the files needed to boot a specific computer), which can be used specifically for that computer in conjunction with any of the boot loaders mentioned in "step A".
It also makes it easy to produce a "demo disk" which allows a potential end-user to try out the OS without installing it, or an "application specific disk" which allows an end-user to use a specific application without installing the OS (e.g. someone could write a game for the OS and distribute the OS and game on a bootable CD - put it in the CD-ROM drive and restart your computer to play).
Cheers,
Brendan
PS. sorry for posting a long rant that doesn't actually answer the question asked...
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re:how to load a kernel using a filesystem?
@ Brendan:
BIOS loads the MBR and executes the contained bootloader. You can't change that.
And what McZ is talking about is basically the details of what you call "step A".
BIOS loads the MBR and executes the contained bootloader. You can't change that.
And what McZ is talking about is basically the details of what you call "step A".
Every good solution is obvious once you've found it.
Re:how to load a kernel using a filesystem?
Hi,
A) boot ROM in an ethernet card gets boot code from remote server (diskless workstation) - normal system BIOS boot code bypassed entirely.
B) BIOS loads MBR which loads boot sector which loads DOS which loads windows which gets "shutdown to DOS", and then the user types "boot" (where "boot.com" is the OS's boot loader for DOS).
C) Embedded CPU starts custom ROM code (on power-up or reset) which copies boot image from the same ROM into RAM and starts the OS's start-up code (PDA, or even an internet ready refrigerator??).
D) BIOS loads MBR from floppy which loads DOS which loads a device driver for USB flash memory which contains "boot.com" (and a boot image) that is started by DOS's "autoexec.bat".
Please note that my post was not specifically directed to McZ, but was intended as an alternative for all of the OS developers who think the MBR/boot sector should load the kernel directly (and by "kernel" I mean the binary that contains the scheduler). I'm not specifically advocating the method suggested, but merely want people to consider alternatives rather than following others without fore-thought or innovation.
Cheers,
Brendan
Usually, but not always. Examples:Solar wrote: BIOS loads the MBR and executes the contained boot loader. You can't change that.
A) boot ROM in an ethernet card gets boot code from remote server (diskless workstation) - normal system BIOS boot code bypassed entirely.
B) BIOS loads MBR which loads boot sector which loads DOS which loads windows which gets "shutdown to DOS", and then the user types "boot" (where "boot.com" is the OS's boot loader for DOS).
C) Embedded CPU starts custom ROM code (on power-up or reset) which copies boot image from the same ROM into RAM and starts the OS's start-up code (PDA, or even an internet ready refrigerator??).
D) BIOS loads MBR from floppy which loads DOS which loads a device driver for USB flash memory which contains "boot.com" (and a boot image) that is started by DOS's "autoexec.bat".
McZ is asking about loading a kernel from a file system, where in the boot method I described the kernel (or the boot image) may be loaded from something that does not have a file system (e.g. raw floppy disk, ROM, ethernet), by file system code that is part of another OS (e.g. MS-DOS), or directly from a file system (via a boot loader that understands the file system or perhaps via GRUB), or any other (existing or future) source.Solar wrote: And what McZ is talking about is basically the details of what you call "step A".
Please note that my post was not specifically directed to McZ, but was intended as an alternative for all of the OS developers who think the MBR/boot sector should load the kernel directly (and by "kernel" I mean the binary that contains the scheduler). I'm not specifically advocating the method suggested, but merely want people to consider alternatives rather than following others without fore-thought or innovation.
Cheers,
Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Re:how to load a kernel using a filesystem?
A) has nothing to do with a filesystem.
B) includes the very step we're talking about.
C) is highly platform-dependent, and I seriously doubt McZ meant that.
D) includes the very step we're talking about.
B) includes the very step we're talking about.
C) is highly platform-dependent, and I seriously doubt McZ meant that.
D) includes the very step we're talking about.
Yet still it doesn't answer, circumvent, or otherwise solve the question of how to load a kernel from a filesystem.McZ is asking about loading a kernel from a file system, where in the boot method I described the kernel (or the boot image) may be loaded from something that does not have a file system (e.g. raw floppy disk, ROM, ethernet), by file system code that is part of another OS (e.g. MS-DOS), or directly from a file system (via a boot loader that understands the file system or perhaps via GRUB), or any other (existing or future) source.
No offense intended. But I think your suggestion basically copies what GRUB is about.Please note that my post was not specifically directed to McZ, but was intended as an alternative for all of the OS developers who think the MBR/boot sector should load the kernel directly (and by "kernel" I mean the binary that contains the scheduler). I'm not specifically advocating the method suggested, but merely want people to consider alternatives rather than following others without fore-thought or innovation.
Every good solution is obvious once you've found it.
-
administrator2004guest
Re:how to load a kernel using a filesystem?
I think we can implement the file system as a high-level service which run in user space[i don't talked about details,so,just tell me your opinion]not to think about details{{{It's concept,I mean}}}.
bye
bye
-
aladdin
Re:how to load a kernel using a filesystem?
the main problem, is : how to load a kernel using a filesystem ?I think we can implement the file system as a high-level service which run in user space[i don't talked about details,so,just tell me your opinion]not to think about details{{{It's concept,I mean}}}.
1 - we consider that the kernel is not yet loaded, and stored on a filesystem , not in a known secteor on the disk (like these kernel stored just after the boot sector making floppy disk unusable).
2 - we want to load it from the boot sector (or using a second stage loader).
McZ want to have some solutions to do this, not to implement high level filesystem interpreters .
-
distantvoices
- Member
- Posts: 1600
- Joined: Wed Oct 18, 2006 11:59 am
- Location: Vienna/Austria
- Contact:
Re:how to load a kernel using a filesystem?
@MrZ:
I suggest two methods, depending on what requirements you have:
first, you put some fat12-table parsing into the bootloader itself. This is achieveable in 512 bytes as far as i know - fetch the fat into memory and scan it for the kernel - named "kernel" f. ex. Then have the bootloader fetch the blocks the fat entry for kernel indicates into memory, provide some GDT entries, activate protected mode and jump to the thing.
second, you can have some second stage loader: the bootloader itself just knows an array of disk blocks - the second stage loader. It fetches it into memory, jumps to it and lets the second stage loader do the rest: say:
file system parsing (reading suffices), loading of additional modules to convenient locations and building some info structure bout them, activating the a20 line, fetching the memory information from bios - and have it communicate this to the kernel in a well known structure. Depending on the requirements, you can have the secondstage loader activate some video mode or collect vbe information.
Keeping a second stage bootloader around is especially recommended for filesystems with quite an amount of bookkeeping data, ext2 f. ex.
Well - With my words I 've described roughly, what GRUB is doing. But maybe you find the one or other inspiration.
I suggest two methods, depending on what requirements you have:
first, you put some fat12-table parsing into the bootloader itself. This is achieveable in 512 bytes as far as i know - fetch the fat into memory and scan it for the kernel - named "kernel" f. ex. Then have the bootloader fetch the blocks the fat entry for kernel indicates into memory, provide some GDT entries, activate protected mode and jump to the thing.
second, you can have some second stage loader: the bootloader itself just knows an array of disk blocks - the second stage loader. It fetches it into memory, jumps to it and lets the second stage loader do the rest: say:
file system parsing (reading suffices), loading of additional modules to convenient locations and building some info structure bout them, activating the a20 line, fetching the memory information from bios - and have it communicate this to the kernel in a well known structure. Depending on the requirements, you can have the secondstage loader activate some video mode or collect vbe information.
Keeping a second stage bootloader around is especially recommended for filesystems with quite an amount of bookkeeping data, ext2 f. ex.
Well - With my words I 've described roughly, what GRUB is doing. But maybe you find the one or other inspiration.
... the osdever formerly known as beyond infinity ...
BlueillusionOS iso image
BlueillusionOS iso image
-
McZ
Re:how to load a kernel using a filesystem?
Thanks for all answers ::) and different ideas.
I think I'll go for the stage2 aproach, or something like that.. this way I can have one kernel and different stage2's for different filesystems.
altough I don't really know where to begin with all this.. everything is only at planing stage at the moment
I think I'll go for the stage2 aproach, or something like that.. this way I can have one kernel and different stage2's for different filesystems.
altough I don't really know where to begin with all this.. everything is only at planing stage at the moment
-
distantvoices
- Member
- Posts: 1600
- Joined: Wed Oct 18, 2006 11:59 am
- Location: Vienna/Austria
- Contact:
Re:how to load a kernel using a filesystem?
May I give a suggestion?
Start with the most simple thing: write some program to the boot sector and have your bochs/vmware/test computer load it into memory and execute it.
Then proceed with *fetching blocks from disk via bios interrupts*
This can also include some fat/ext2 parsing, it is up to you.
Split a huge task into small tasks you can overview and test with ease. No need to hurry.
Stay safe
Start with the most simple thing: write some program to the boot sector and have your bochs/vmware/test computer load it into memory and execute it.
Then proceed with *fetching blocks from disk via bios interrupts*
This can also include some fat/ext2 parsing, it is up to you.
Split a huge task into small tasks you can overview and test with ease. No need to hurry.
Stay safe
... the osdever formerly known as beyond infinity ...
BlueillusionOS iso image
BlueillusionOS iso image
-
McZ
Re:how to load a kernel using a filesystem?
thank you. I have written a simple bootsector code with nasm (well compiled with nasm) a simple "hello world" bootsector thing, altough I have a problem with the FAT12 Bios Parameter Block.
I have read alot of different texts about this and I found that I can write in the assembler file a Fat12 Bios Parameter block like this:
;*******************************************************************
;* Fat 12 BIOS Parameter Block ( not working )
;*
nop
???DB???'MyOS '??????; 8-byte system id?????????( offset: 0x03 size: 8 bytes )
??????????????????;?????????????????????Offset is now 0x03 + offset
???DW???512????????????; sector size in bytes??????( offset: 0x00 size: 2 bytes )
???DB???1????????????; sectors per cluster??????( offset: 0x02 size: 1 bytes )
???DW???1????????????; reserved clusters?????????( offset: 0x03 size: 2 bytes )
???DB???2????????????; number of fats?????????( offset: 0x05 size: 1 bytes )
???DW???224????????????; root directory entries???( offset: 0x06 size: 2 bytes )
???DW???2880?????????; total sectors????????????( offset: 0x08 size: 2 bytes )
???DB???0F0h?????????; format id???????????????( offset: 0x0a size: 1 bytes )
???DW???9????????????; sectors per fat?????????( offset: 0x0b size: 2 bytes )
???DW???18????????????; sectors per track?????????( offset: 0x0d size: 2 bytes )
???DW???2????????????; sides??????????????????( offset: 0x0f size: 2 bytes )
???DD???0????????????; special hidden sectors???( offset: 0x11 size: 4 bytes )
???DD???0????????????; more sectors????????????( offset: 0x15 size: 4 bytes )
???DB???0????????????; drive number????????????( offset: 0x19 size: 1 bytes )
???DB???0????????????; reserved???????????????( offset: 0x1a size: 1 bytes )
???DB???41????????????; boot signature?????????( offset: 0x1b size: 1 bytes )
???DD???00000001??????; volume serial number??????( offset: 0x1c size: 4 bytes )
???DB???"My OS "???; volume label (11 bytes)???( offset: 0x20 size: 11 bytes )
???DB???"FAT12 "??????; file system (8 bytes)??????( offset: 0x2b size: 8 bytes )
;*******************************************************************
altough when I use partcopy to write my bootsector.bin to the disk it discards the Fat12 info stuff so Windows wants to Format the disk.
but if I do like this it will not write the Fat12 info stuff then it works
nasm -f bin -o BootSctr.bin BootSctr.asm
partcopy BootSctr.bin 0 3 -f0
partcopy BootSctr.bin 3e 1c2 -f0 3e
EDIT:
just another thing, how do I make my Stage2.bin to be in a certain place that the BootSctr.bin knows about? without trashing any Fat12(or other filsystem table info)
I have read alot of different texts about this and I found that I can write in the assembler file a Fat12 Bios Parameter block like this:
;*******************************************************************
;* Fat 12 BIOS Parameter Block ( not working )
;*
nop
???DB???'MyOS '??????; 8-byte system id?????????( offset: 0x03 size: 8 bytes )
??????????????????;?????????????????????Offset is now 0x03 + offset
???DW???512????????????; sector size in bytes??????( offset: 0x00 size: 2 bytes )
???DB???1????????????; sectors per cluster??????( offset: 0x02 size: 1 bytes )
???DW???1????????????; reserved clusters?????????( offset: 0x03 size: 2 bytes )
???DB???2????????????; number of fats?????????( offset: 0x05 size: 1 bytes )
???DW???224????????????; root directory entries???( offset: 0x06 size: 2 bytes )
???DW???2880?????????; total sectors????????????( offset: 0x08 size: 2 bytes )
???DB???0F0h?????????; format id???????????????( offset: 0x0a size: 1 bytes )
???DW???9????????????; sectors per fat?????????( offset: 0x0b size: 2 bytes )
???DW???18????????????; sectors per track?????????( offset: 0x0d size: 2 bytes )
???DW???2????????????; sides??????????????????( offset: 0x0f size: 2 bytes )
???DD???0????????????; special hidden sectors???( offset: 0x11 size: 4 bytes )
???DD???0????????????; more sectors????????????( offset: 0x15 size: 4 bytes )
???DB???0????????????; drive number????????????( offset: 0x19 size: 1 bytes )
???DB???0????????????; reserved???????????????( offset: 0x1a size: 1 bytes )
???DB???41????????????; boot signature?????????( offset: 0x1b size: 1 bytes )
???DD???00000001??????; volume serial number??????( offset: 0x1c size: 4 bytes )
???DB???"My OS "???; volume label (11 bytes)???( offset: 0x20 size: 11 bytes )
???DB???"FAT12 "??????; file system (8 bytes)??????( offset: 0x2b size: 8 bytes )
;*******************************************************************
altough when I use partcopy to write my bootsector.bin to the disk it discards the Fat12 info stuff so Windows wants to Format the disk.
but if I do like this it will not write the Fat12 info stuff then it works
nasm -f bin -o BootSctr.bin BootSctr.asm
partcopy BootSctr.bin 0 3 -f0
partcopy BootSctr.bin 3e 1c2 -f0 3e
EDIT:
just another thing, how do I make my Stage2.bin to be in a certain place that the BootSctr.bin knows about? without trashing any Fat12(or other filsystem table info)