How the kernel is loaded into memory from USB flash?

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

How the kernel is loaded into memory from USB flash?

Post by wy »

Sorry, my question seems to be incorrect , I described the problem again in order to clarify the expression of my purpose.
===========================================================================
Hey, guys
I wrote a kernel that does not support file system.
This simple kernel is elf64 file, it can be booted through grub2, it can also be booted by bootloader I have written.
My purpose is to test the kernel on bare metal.I will observe whether it can perfectly run on bare metal.
So I want to boot by external USB flash, and then load the kernel to the memory.
How the kernel is loaded into memory from external (USB) flash?
Last edited by wy on Fri Feb 05, 2016 9:58 pm, edited 10 times in total.
User avatar
Nutterts
Member
Member
Posts: 159
Joined: Wed Aug 05, 2015 5:33 pm
Libera.chat IRC: Nutterts
Location: Drenthe, Netherlands

Re: How the kernel is installed on the U disk?

Post by Nutterts »

wy wrote:The kernel.bin which should be placed in the folder in the U disk?
I don't have a clue what a U disk is but generally speaking, yes. You'd generally put your kernel, say kernel.bin, in /boot. Then you'd edit /boot/grub/grub.cfg to add a line like menuentry "myos" { multiboot /boot/kernel.bin }. Basicly the same config as if you'd made an iso image like in the barebones,
wy wrote:This simple kernel final form is a raw binary file which is generated by the objcopy
Why use a binary and not ELF if you're using grub?
"Always code as if the guy who ends up maintaining it will be a violent psychopath who knows where you live." - John F. Woods

Failed project: GoOS - https://github.com/nutterts/GoOS
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

Re: How the kernel is installed on the U disk?

Post by wy »

Nutterts wrote:
wy wrote:The kernel.bin which should be placed in the folder in the U disk?
I don't have a clue what a U disk is but generally speaking, yes. You'd generally put your kernel, say kernel.bin, in /boot. Then you'd edit /boot/grub/grub.cfg to add a line like menuentry "myos" { multiboot /boot/kernel.bin }. Basicly the same config as if you'd made an iso image like in the barebones,
wy wrote:This simple kernel final form is a raw binary file which is generated by the objcopy
Why use a binary and not ELF if you're using grub?
I'm sorry , my expression is not correct, I described the problem again.
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: How do make a 'Live USB' for a elf kernel ?

Post by Brendan »

Hi,
wy wrote:I wrote a kernel that does not support file system.
This simple kernel is elf64 file, it can be booted through grub2, it can also be booted by bootloader I have written.
How do I make a 'myKernel Live USB' like Linux Live USB?
So I can get rid of the hard disk , I can test my kernel on any computer that supports USB boot.
You'll probably need to be a little more specific - are you having trouble creating a disk image, or having trouble with boot loader/s, or ... ?

There isn't much difference between booting from internal (PATA, SATA, SCSI, SAS) hard disk, booting from external (USB) hard disk, and booting from external (USB) flash. It's the same boot code with the same challenges (partitions, providing device drivers that work after kernel takes control of hardware, finding out which device you booted from after kernel takes over, etc).

The main difference is "live" vs. "installed for one specific computer"; where the user plugs it into any computer they feel like and you have to cope with "not known in advance" hardware; and can't (e.g.) have an OS installer that only install things that one computer needs, and can't (e.g.) rely on settings/configuration that only suits one specific computer. This could mean supporting/using "hybrid GPT+MBR" partitioning, having one boot loader for MBR and another boot loader for UEFI that both start the same OS, relying on auto-detection/auto-configuration instead of using rigid/fragile methods (e.g. kernel compile time options), etc.

Of course all of the things that are necessary for "live" are beneficial for "installed for one specific computer" anyway.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
onlyonemac
Member
Member
Posts: 1146
Joined: Sat Mar 01, 2014 2:59 pm

Re: How do make a 'Live USB' for a elf kernel ?

Post by onlyonemac »

If your kernel can be booted by GRUB, then just copy your kernel onto a flash drive and install GRUB to the flash drive.
When you start writing an OS you do the minimum possible to get the x86 processor in a usable state, then you try to get as far away from it as possible.

Syntax checkup:
Wrong: OS's, IRQ's, zero'ing
Right: OSes, IRQs, zeroing
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

Re: How do make a 'Live USB' for a elf kernel ?

Post by wy »

onlyonemac wrote:If your kernel can be booted by GRUB, then just copy your kernel onto a flash drive and install GRUB to the flash drive.
grub2 be installed to the external USB flash , with the following command:

Code: Select all

# Suppose USB flash is / dev / sdb1
sudo mount /dev/sdb1 /mnt
sudo grub-install --force --no-floppy --boot-directory = /mnt/boot /dev/sdb
But I did not find grub.cfg.
I created a grub.cfg file in the /mnt/boot/grub directory
The contents of the configuration file is:

Code: Select all

set default=0
set timeout=10

insmod vbe
insmod vga

menuentry 'MyKernel 0.1.0' --class mykernel --class os {
echo 'Loading kernel.bin'
multiboot /boot/kernel.bin
} 
Then the kernel is copied into the /mnt/boot/ directory
Last run:

Code: Select all

sudo umount /mnt
Then I set the U disk boot in the BIOS, the result is "grub> _" on the screen, '_' is a cursor.
Last edited by wy on Fri Feb 05, 2016 9:59 pm, edited 3 times in total.
User avatar
Nutterts
Member
Member
Posts: 159
Joined: Wed Aug 05, 2015 5:33 pm
Libera.chat IRC: Nutterts
Location: Drenthe, Netherlands

Re: How do make a 'Live USB' for a elf kernel ?

Post by Nutterts »

Try adding this to your grub.cfg:

Code: Select all

insmod usbms
insmod ehci
Btw, I don't think you need --force or --no-floppy. Worth a try if the above doesn't fix it because with those modules it should work.

Also, you might want to reconsider loading from usb. Sure, it'll load fine and you can load modules alongside with it. But if you want to have your kernel read from the usb filesystem some day then you'll have to implement or port a usb stack. It's certainly not the path of least resistance.
Last edited by Nutterts on Fri Feb 05, 2016 6:16 pm, edited 1 time in total.
"Always code as if the guy who ends up maintaining it will be a violent psychopath who knows where you live." - John F. Woods

Failed project: GoOS - https://github.com/nutterts/GoOS
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

Re: How do make a 'Live USB' for a elf kernel ?

Post by wy »

Brendan wrote:Hi,
wy wrote:I wrote a kernel that does not support file system.
This simple kernel is elf64 file, it can be booted through grub2, it can also be booted by bootloader I have written.
How do I make a 'myKernel Live USB' like Linux Live USB?
So I can get rid of the hard disk , I can test my kernel on any computer that supports USB boot.
You'll probably need to be a little more specific - are you having trouble creating a disk image, or having trouble with boot loader/s, or ... ?

There isn't much difference between booting from internal (PATA, SATA, SCSI, SAS) hard disk, booting from external (USB) hard disk, and booting from external (USB) flash. It's the same boot code with the same challenges (partitions, providing device drivers that work after kernel takes control of hardware, finding out which device you booted from after kernel takes over, etc).

The main difference is "live" vs. "installed for one specific computer"; where the user plugs it into any computer they feel like and you have to cope with "not known in advance" hardware; and can't (e.g.) have an OS installer that only install things that one computer needs, and can't (e.g.) rely on settings/configuration that only suits one specific computer. This could mean supporting/using "hybrid GPT+MBR" partitioning, having one boot loader for MBR and another boot loader for UEFI that both start the same OS, relying on auto-detection/auto-configuration instead of using rigid/fragile methods (e.g. kernel compile time options), etc.

Of course all of the things that are necessary for "live" are beneficial for "installed for one specific computer" anyway.


Cheers,

Brendan
This is a micro-kernel, it does not support any file system, currently only contains some basic functions such as IPC, Memory Management, Scheduler, Synchronization.
My purpose is to test the kernel on bare metal.I will observe whether it can perfectly run on bare metal.
So I want to boot by external USB flash, and then load the kernel to the memory, This test will not be related to the internal (PATA, SATA, SCSI, SAS) hard disk from beginning to end .
:)
Last edited by wy on Fri Feb 05, 2016 9:50 pm, edited 1 time in total.
User avatar
Nutterts
Member
Member
Posts: 159
Joined: Wed Aug 05, 2015 5:33 pm
Libera.chat IRC: Nutterts
Location: Drenthe, Netherlands

Re: How do make a 'Live USB' for a elf kernel ?

Post by Nutterts »

wy wrote:This is a micro-kernel ..... This test will not be related to the internal (PATA, SATA, SCSI, SAS) hard disk from beginning to end. :)
Oh never mind then. :) I love micro-kernels, let me know if adding those modules I suggested works.
"Always code as if the guy who ends up maintaining it will be a violent psychopath who knows where you live." - John F. Woods

Failed project: GoOS - https://github.com/nutterts/GoOS
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

Re: How do make a 'Live USB' for a elf kernel ?

Post by wy »

Nutterts wrote:
wy wrote:This is a micro-kernel ..... This test will not be related to the internal (PATA, SATA, SCSI, SAS) hard disk from beginning to end. :)
Oh never mind then. :) I love micro-kernels, let me know if adding those modules I suggested works.
Grub.cfg file contents are:

Code: Select all

set default=0
set timeout=10

insmod usbms
insmod ehci
insmod vbe
insmod vga

menuentry 'MyKernel 0.1.0' --class mykernel --class os {
echo 'Loading kernel.elf'
multiboot /boot/kernel.elf
} 
grub boot interface appears:

Code: Select all

                                          GNU GRUB  VERSION 2.02~beta2-9ubuntu1
*Mykernel 0.1.0
error:disk 'hd0,msdos1' not found.
error:disk 'hd0,msdos1' not found.
error:disk 'hd0,msdos1' not found.
then,

Code: Select all

error:disk 'hd0,msdos1' not found.
error:disk 'hd0,msdos1' not found.
Press any key to continue...
finally,

Code: Select all

                                          GNU GRUB  VERSION 2.02~beta2-9ubuntu1
*Mykernel 0.1.0
Stop here can not continue, only to restart.
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: How the kernel is loaded into memory from U flash?

Post by gerryg400 »

What's a U disk ?
If a trainstation is where trains stop, what is a workstation ?
User avatar
Nutterts
Member
Member
Posts: 159
Joined: Wed Aug 05, 2015 5:33 pm
Libera.chat IRC: Nutterts
Location: Drenthe, Netherlands

Re: How the kernel is loaded into memory from U flash?

Post by Nutterts »

Mmm, weird. Probably a stupid question but do you happen to have your usb plugged into usb3 port? If so try a usb2 port. Just to rule out a buggy bios maybe try changing the boot sequence in the bios if your using something like F12 to select the usbdrive. But honestly, I don't have a clue.

Another wild guess, try a smaller usbdrive if your using something larger then 4GB.
Last edited by Nutterts on Fri Feb 05, 2016 9:28 pm, edited 2 times in total.
"Always code as if the guy who ends up maintaining it will be a violent psychopath who knows where you live." - John F. Woods

Failed project: GoOS - https://github.com/nutterts/GoOS
User avatar
wy
Posts: 9
Joined: Mon Jan 11, 2016 11:35 pm

Re: How the kernel is loaded into memory from U flash?

Post by wy »

gerryg400 wrote:What's a U disk ?
U disk is equivalent to:
USB flash disk
USB drive
flash drive
usb stick
pen drive
User avatar
Nutterts
Member
Member
Posts: 159
Joined: Wed Aug 05, 2015 5:33 pm
Libera.chat IRC: Nutterts
Location: Drenthe, Netherlands

Re: How the kernel is loaded into memory from U flash?

Post by Nutterts »

Another thing popped in my mind. Did you format the drive fat32 or ext2? If it's ext2 you'll also need to "insmod ext2". Hell, add that and "insmod fat32", what harm could it do. ;) Must be something simple like that.
Last edited by Nutterts on Fri Feb 05, 2016 9:35 pm, edited 2 times in total.
"Always code as if the guy who ends up maintaining it will be a violent psychopath who knows where you live." - John F. Woods

Failed project: GoOS - https://github.com/nutterts/GoOS
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: How the kernel is loaded into memory from U flash?

Post by gerryg400 »

wy wrote:
gerryg400 wrote:What's a U disk ?
U disk is equivalent to:
USB flash disk
USB drive
flash drive
usb stick
pen drive
There's no need to create a new name for something that already has many names.
If a trainstation is where trains stop, what is a workstation ?
Post Reply