How to get my os on hard disk

[wonde]

Hi ya all !! I am in a kind of strange problem, i have developed a kernel called kernel.bin and also i have a boot loader called boot.bin I have used partcopy to create bootable floppy disk and it works quite good ,the problem is i want to do the same thing that i have done on the floppy ,on my testbed pc hard disk ,I know all the instructions are similar with that of the floppy (pratcopy xxxxx stuff) if there is windows OS installed in that computer but what if ,if there are no any operating system running on that machine how could i get my kernel and my boot loader get copied on the hard disk?

[Dex]

You could for example use a bootloader like grubs or bootprog. Bootprog can load a exe or bin or com file off a floppy or fat16 formated hdd.
It come with a range of tools to help you do this, you can use these tool from a window startup disk etc.

http://alexfru.chat.ru/epm.html#bootprog

Ps: I use it to boot my OS.

[xyzzy]

First, don't post your entire message in bold. It won't help it get noticed.

Why do you need to run it off the hard disk? It's perfectly OK to run it off the floppy while in early stages of development. I still have absolutely no need to run my OS off a hard disk, I simply boot from a CD, Floppy or network.

[piranha]

I am in a kind of strange problem, i have developed a kernel called kernel.bin and also i have a boot loader called boot.bin I have used partcopy to create bootable floppy disk and it works quite good ,the problem is i want to do the same thing that i have done on the floppy ,on my testbed pc hard disk ,I know all the instructions are similar with that of the floppy (pratcopy xxxxx stuff) if there is windows OS installed in that computer but what if ,if there are no any operating system running on that machine how could i get my kernel and my boot loader get copied on the hard disk?

Man, turn off the bold! Edit it out of your post!
And you do realize that that was all one sentence? That is really annoying!

I agree with Alex, you don't need to boot off a HD yet, but if you really want to you could make a disk image and copy it to that.

-JL

[bewing]

On a blank hard disk, you need an MBR that recognizes the disk format, with a partition table, stored in LBA 0 (which you don't have yet).
You need a slightly modified boot sector program (almost the same thing you have for your floppy boot sector), stored as the first sector of your kernel.bin's partition.
The boot sector program needs to be able to read the filesystem stored on the hard disk. It is likely to be a little different from the one on your floppy.
In your kernel, you will need to initialize the devices on the ATA bus. You wil need to send IDENTIFY commands to those devices. You will need to read the responses to the IDENTIFY commands. And you will need a singletasking 28bit PIO mode disk driver, which will read/write sectors from/to the hard disk.

There are many utility programs that will allow you to write sectors of data to a physical disk on your computer. I personally like Norton Diskedit the best.

[Ready4Dis]

Well, I disagree, what is wrong with booting from the hard-disk? it's faster, much more space, and is a good test that your code works on more than just a floppy. Personally, I wrote my own tools to write out information to my hard disk. I also wrote my own MBR that allows me to select a partition to boot from, and write's out the Active partition flag so windows doesn't spaz out . I multi-boot windows xp, dos and my OS on 3 seperate partitions. My partition and Dos are both Fat, which my OS can read easily. I can copy files to my partition straight from windows, boot my OS, update my kernel/bootsector by booting to my dos partition and running my utilities. I am also writing some windows utilities to be able to write directly to partitions (as long as you have admin rights). Another idea is to write an installer, basically a bootsector that loads a copy program. You don't even have to leave real-mode, and can use bios int's very easily to copy your OS to a partition from a floppy disk. If you are not comfortable writing tools, or understand how reading/writing to a partition or MBR will affect your computer, I suggest sticking with the floppy until you are a bit more seasoned, because playing with those things can hose a system pretty quickly. I tested my MBR a bit before writing it to my working hard drive, since I actually USE the computer I was messing with . Most of my tools were written in Turbo C/C++ for Dos, and run from my dos boot. I do plan on writing a bunch of tools (fdisk type, partcopy, etc), and a standalone installer, if there is enough interest I may try to make them more generic rather than tailored to my specific OS. It'd basically be a very small r-mode OS that could view the parition info, and copy files from a fat12 partition to raw sectors on a hard disk. Since most bios' support LBA (I think up to 64-bit is reserved in the bios function?), I would probably write an LBA only version (not that CHS would be hard to add). Anyways, my OS boots much faster from the hard drive, than my floppy, and I don't have to worry about my laptop with doesn't have a floppy disk (which was my main dev machine!), which was a pretty big reason for getting my hd booting code/utilities done faster.

[zaleschiemilgabriel]

Here's a quickie (if you don't want to write new tools):
* There's a bootable CD around the internetz called Universal Boot CD (or UBCD for shorts). Google it, download it, burn it to a CD and then:
* Use the MBR editor on the UBCD to create your partition on the HDD. Give it a partition type you like (I used 72h, because I'm designing my own file system and 72h was not used)
* Remember all the values (starting & ending LBA / CHS) that are shown in your partition's entry
* reboot and then use one of the DOS/Linux boot disks
* To use DOS: on a separate floppy/CD or on another HDD partition supported by DOS, copy the RAWRITE tool and use it to write first your boot sector to the partition you created before, then your kernel.
* If your partition is DOS-compatible (i.e., FAT), you may need to reboot after writing the boot sector, so that you can use the DOS copy command to copy the kernel. If not, you have to find another way to write the kernel (it depends on the type of partition your OS supports).

[wonde]

Thank you! I think now I have some clues, but let me to raise another question how can I configure stage1 and stage2 GRUB boot loader files to MBR so that GRUB will display all OSs, that are available in my pc, this is because if I copy my boot sector to MBR I am replacing all the MBR record so then I will not be able to load any other OS in my pc.

Have you ever noticed when you install Ubuntu or any other Linux family it installs GRUB on the MBR I just want to do the same thing when I install my OS

[bewing]

GRUB uses a standard MBR on hard disk. You do not need to overwrite the MBR with your own.

[Ready4Dis]

Thank you! I think now I have some clues, but let me to raise another question how can I configure stage1 and stage2 GRUB boot loader files to MBR so that GRUB will display all OSs, that are available in my pc, this is because if I copy my boot sector to MBR I am replacing all the MBR record so then I will not be able to load any other OS in my pc.

Have you ever noticed when you install Ubuntu or any other Linux family it installs GRUB on the MBR I just want to do the same thing when I install my OS

I apologize, but I am not familiar with GRUB, only my custom MBR. I know a lot of people use grub, and that it can boot other paritions without messing them up (so you won't lose say your windows partition). My utility reads the current MBR, and stores the partition table, then reads my MBR from disk, and fills in the table (also filling in the default boot partition), so it doesn't wipe out any partition data. Then when it boots, my MBR is called, copies itself out of the way (for later) and it reads the partition table, and displays it with an option to boot from any partition it finds, but it defaults to whatever you set the default to (in my utliity). My MBR then waits 10 seconds (or however long you set it for in the utility) and if it receives no input, boots the default. If it receives input, it checks if it's valid. Once it knows which partition to boot, it writes itself back out with an updated Active Partition flag, telling it which partition was the boot partition (windows XP actually checks this flag, i found that out the hard way) then loads the bootsector of that partition, and copies it to 0x7c00 (this is why it copies itself out of the way earlier) and boots whatever you told it to. My entire MBR fits in 512 bytes, unlike grub that takes many more. It works with linux, windows, dos, my os, and i'm sure others as well. GRUB is WAY more advanced (which is why it takes up so much more space), in that it can be used to actually boot your OS rather than just be an MBR that calls a bootsector (although, for most OS's, that's all it does is load an boot sector to 0x7c00 and then jump to it so it thinks it was just called by the bios). I am sure GRUB works in a similar way, but has many more options (like loading an OS with a specific file system, giving your kernel a multiboot header, enabling A20, start up a graphics mode before transferring control, etc). I do OS dev for 'fun', so using tools like GRUB for me is cheating myself out of learning something, not that I will never use it, but I like writing my own tools to know how everything works/interacts before using something else. your mileage may vary, and GRUB is open source, so you can easily see what is going on.

[binutils]

IMHO, usb flash memory is better place than HDD(only if it has 2 partitions)/FDD

[zaleschiemilgabriel]

IMHO, usb flash memory is better place than HDD(only if it has 2 partitions)/FDD

That means you need to have a USB flash driver. I don't know much about those things, but maybe it's easy to write one (maybe it's just plain old ATA ).

[Ready4Dis]

IMHO, usb flash memory is better place than HDD(only if it has 2 partitions)/FDD

That means you need to have a USB flash driver. I don't know much about those things, but maybe it's easy to write one (maybe it's just plain old ATA ).

Well, USB, that's an interesting topic for OS dev... useful, because it's easy and portable, much faster than a floppy, and has more disk space. It's bootable if your BIOS supports it (which most newer bios' do), and you can access it using the standard HD interrupts... and that's about where the friendlyness ends. After switching to p-mode, you can no-longer use bios int's, and you must write 3 drivers, PCI enumerator, USB Enumerator/interface and Mass Storage driver. Where, ATA is very easy to do using standard I/O ports, USB and a Mass Storage driver are a lot more complex (I've read the USB specs, and would take a long time to turn that into an implementation, and it's not PC specific, more geared towards USB hardware). So, the fact that you can have a 256mb flash drive for $10 at most stores to play with is great, and if you either A.) have a kernel image that is fully loaded on boot-up, or B.) have usb mass storage drivers written, then it's a great platform. If you don't have those, and you are working on a file system or disk drivers, I don't think USB mass storage drivers are the best place to start learning, much simpler to imlpement a fdc or hdc controller. Might I suggest a bootable CD (can use a cd-rw to be able to write it many times), it uses ATAPI (runs through ATA, so HD code is similar), and the CD-Rom file system is pretty easy to read from, so your first I/O drivers will be pretty simple and give you a good learning point. A cd allows you 650-700mb, and is easily distributed and tested on multiple machines. I don't know many bios's that don't support booting from CD (way more than those that support USB). Anyways, I don't see a reason to not use a Hard disk, just make sure you don't a.) mess up the MBR, or b.) mess up a partition used by another OS.

[Dex]

Good points Ready4Dis, but i would just add that it is possible to read/write usb, by going to and from realmode, i have a pmode floppy and a bios driver in my OS.
Because if you go back to realmode from pmode emulation works again, this lets you read/write to a USB fob, also it lets you use a USB floppy drives etc or read the boot image of a CD uder emulation.

Now when you go to and from pmode to realmode for floppy driver, it is not any slower than using it in realmode.
But when you load from USB under emulation, even in realmode, it is much slower than floppy loading, i would say as much as 4 times slower than a floppy.

I have started coding a USB stack, so far it was not that hard, its just very hard to get any info.

As for CD, you can code a driver, but they are read only, my problem in my OS is write, as i have a fasm port, i can assemble the code, but it is so slow if you use floppy, you can not use CD, so that leaves hdd which is fine, but my OS only read/writes to fat16/32, which most are NTFS .
I am thinking about using a floppy image to get around this, as i can much easier code a driver to load and write back a image file eg: vfat.ima to a NTFS partion than read/write files in the normal way to NTFS, as the image will stay the same size.

[Ready4Dis]

Good points Ready4Dis, but i would just add that it is possible to read/write usb, by going to and from realmode, i have a pmode floppy and a bios driver in my OS.
Because if you go back to realmode from pmode emulation works again, this lets you read/write to a USB fob, also it lets you use a USB floppy drives etc or read the boot image of a CD uder emulation.

Now when you go to and from pmode to realmode for floppy driver, it is not any slower than using it in realmode.
But when you load from USB under emulation, even in realmode, it is much slower than floppy loading, i would say as much as 4 times slower than a floppy.

I have started coding a USB stack, so far it was not that hard, its just very hard to get any info.

As for CD, you can code a driver, but they are read only, my problem in my OS is write, as i have a fasm port, i can assemble the code, but it is so slow if you use floppy, you can not use CD, so that leaves hdd which is fine, but my OS only read/writes to fat16/32, which most are NTFS .
I am thinking about using a floppy image to get around this, as i can much easier code a driver to load and write back a image file eg: vfat.ima to a NTFS partion than read/write files in the normal way to NTFS, as the image will stay the same size.

Well, that's why I have a Fat32 partition, I have a bootsector for fat12/fat16/fat32, and utilities I wrote to format, and write files out to each type (yes, I write a LOT of utilities myself, but then, I can have them do anything I want also, like writing contiguous files rather than scattered for things like my kernel image). Anyways, yes CD's are read only, which is a problem when you are trying to self host and would like to be able to compile, etc. To get around the slow floppy, most people store an image of the floppy in memory, and read/write to it, then once all the updates are done, they commit the 'dirty' pages back to disk (and if done in a background thread, the main application can exit before the data is actually done writing, but you know when it's done writing because the light on the disk will go out). It's similar in concept to a ram-disk or memory mapped file (except you are mapping the disk, not just a file). Your driver for the floppy would be coded in such a way that when reading/writing it does so from memory (which is ALWAYS current, so you know it's good), where the driver stores a list of pages that have been modified. It would then have a thread that would write the data to disk in the background, so it can return to the user application as soon as the memory is updated (a few MS), which can significantly increase your application performance, even if you must wait for the light to extinguish to turn off your PC or reboot, at least you can do other things (compile more files?) while waiting for the disk to be updated. It wastes a mb of ram, but I think it's a fair tradeoff, especially if you use the floppy a bunch, if not, and you have a good memory manager with virtual memory (swap disk/file), read/writing to the floppy will read/write from memory, which would be stored on your HD in most cased (paged out), then it would write to floppy when updated, and be paged back out not to much later (depending on your manager). Now, me personally, I like having a spare partition setup for booting, it just makes things 'simple' . Yes, I am aware of going back to real-mode and reading/writing, you can do that for FDC, HDC, CD, VESA, etc.. but chances are, you will miss a LOT of interrupts if you're not in v86, multitasking will 'pause', anything relying on timers goes to crap, etc. And reading large files is a big no-no, unless you are using voodoo mode/flat real mode/unreal mode, but then you still don't have paging in real-mode, so you would need to pass it a list of physical pages, which gets very combersom... of course you could limit transfers to a smaller size, and copy them from the pmode driver, minimizing the time spent in real-mode for each read (so you won't miss quite as many interrupts, just 99% of them), and multi-tasking will only appear very sluggish instead of paused completely, and your I/O rates (which are already dismal at best) will be completely shot, so it's far from a solution, although it would be a workaround while you are working on your USB drivers. I have not tried writing USB drivers yet, but that is a goal so I can boot from my thumb-drive, until then I will stick with my HDD (which was the original question, and I still recommend if you have a spare one, or spare partition). A few things to always do when working on a hard drive.

1.) Read before writing -> always get your file reading code working before attempting to write anything to disk, if you read from another partition by accident, no big deal, if you write to it, plan on re-installing an OS.
2.) Use a function specifically designed for writing to partitions -> Especially while still developing have a function that uses the partition start LBA as a variable of some sort (whether it's passed in or global), and always check to make sure it's not 0 before issuing the write command. If it's zero, throw an error, and crap out so you know something tried to f*ck up your HD
3.) Always make sure you back-up anything important before playing with a HD that has data you want to keep, accidents happen.
4.) Test in an emulator -> Download free-dos or another small OS to play with, make an image file with multiple partitions, and test your code, make sure it doesn't hose the alternate OS, never try your code on a real pc unless your a.) never make mistakes, b.) have spare hardware and it doesn't matter if you mess up, or c.) like running scandisk and trying to recover lost file segments .

Anyways, I write most of my own utilities, and I know my HD pretty well now for doing so, but I had to do a lot of testing and trial and error before everything worked (know how many times I booted to bochs, and ran chkdisk before I got my fat code to not throw any errors, stupid things too, like the fact that windows doesn't recognize a fat partition if anything but MSWIN4.1 is in the FAT table (OEM Field)). I tried writing another string in there, and it just wouldn't recognize my drive, changed it to MSWIN4.1 like MS uses, and bam it worked.