BOOTCFG - Startup Configuration Tricks and OS Loader for DOS

[~]

Download the current source code:
LowEST-Kernel-LEVEL-1-base--2012-06-21--src.zip

And a video showing how to assemble and use it:



Contains the LowEST Kernel.

BOOTCFG is a set of snippet programs in constant increase in number, quality and amount of specific tricks.

These snippet programs are designed to leave some aspect of the computer configured or detected when the program finishes.

While the programs are native to DOS, the tricks contained in the source code apply to any operating system we want to develop as they are details that are specific to the machine, not to DOS, although tricks to make use of the DOS services and resources exist too because they are necessary to implemet BOOTCFG.

Ultimately, after running the set of configuration snippets (as if it was a bootloader built from a lot of separate programs that call each other like parent/child functions), an operating system other than DOS can be launched from DOS, although a specific bootloader capable of running under DOS and of loading our specific operating system needs to be written.

The tricks included range from detecting the CPU type (8086/8088, 286, 386, 386+), detecting whether the A20 line is enabled with pure 16-bit code for full portability, enabling Unreal Mode, and anything you can think of learning and doing at boot time, and more if you need to (just like the well-known PC code demos from Programmer's Heaven).

And the code is written in x86 Portable NASM, so the same code can be directly ported with little to no rewrites to 16, 32 and 64-bit x86 targets.

This is a key component of my operating system, but it can be used anywhere else because it's designed to excel over time at documentation and teaching final tricks, that is, providing full working source code tricks that a developer will always unavoidably come down to because they are intrinsic to the implementation of a hardware, firmware or software system, so infinite tricks exist that need to be included, explained and turned into configuring and detection snippets.

DOS and Windows 9x are the most representative and specific OSes native to the standard x86 PC, even more than Windows XP despite their age, and for which most fully-native programs that specifically make use of PC development details has been written and keeps to be written (for more portable programs, less specific to the standard PC as a whole, we could point to any other multiplatform-targeted OS, but BOOTCFG is fully specific to the PC).

For me and my PC-specific projects, the PC is like the fully capable, fully standard calculator of the computers, so I consider that a fully standard PC will always be as necessary as a good calculator, apart from the rest of platfoms which to this day continue to lack the degree of standardization even of a 386, with all imaginable peripheral cards and devices, so a fully open hardware and open software defined as open specifications from the already known x86 PC (16, 32, 64-bit, BIOS...) foundation will always be desired.



_____________________________________________
_____________________________________________
_____________________________________________
_____________________________________________

About the LowEST Kernel

Video showing how to assemble, run and run a simple external program all from floppy:

http://youtube.com/watch?v=kKQpa1eQXgU


The LowEST kernel is aimed at serving as a platform to perform the simplest tests, to drive software and hardware through console commands and external programs to make any functionality mature from manual batch-like sequences to formal libraries, subsystems and programs.

It's a 32-bit system written in NASM.

This is the final revision of this version of the kernel. I have revised it to make sure that external programs read from floppy are loaded and executed stably. However, if there are media read errors the system will most likely crash when trying to execute the badly loaded program.

I have only included what is really needed to run. Any error check will be left for later versions. It's been done so to ensure that functions are robust in themselves without the help of exception code, and now regular code and error-handling code will be equally strong on their own.

____________________________________________
____________________________________________
____________________________________________
____________________________________________
As is, the kernel has a text console, a few internal commands, PS/2 keyboard, can load external programs from floppy A:\ which must always be written in their raw, low level 8.3 file name form as stored on disk. The programs can import any functions that the kernel exports, there is a sample program that shows how to do it. Everything about the current kernel is at the lowest level of abstraction yet it is directly accessible.


What is Left
Currently I need to play around with this kernel by running programs from Programmer's Heaven, Denthor of Asphyxia, Alexei Frounze, Geezer, OSDever.net, among many others, to take it to its limits. Any functions that I find myself unable to handle with the current kernel codebase or with external snippet programs will be implemented into the kernel.

It's very similar to an absolutely minimal barebones, but with the difference that I made it myself and is public domain.

It's actually ultimately more adequate as a third stage bootloader with low level testing capabilities.

So now I have a console with a simple binary program loader with full hardware access that will allow me to test different code to expand the needed functions as I get to understand them. It will really allow me to perform tests in a way that is really comfortable, allowing me to integrate code gradually as I find out how to pack it into reusable units.

What isn't convenient or clean to test under DOS, will be under this system.

[iansjack]

That link doesn't work.

[~]

You might need to try several times, or later. You will be able to download then.

My server is a tiny BenQ S6 UMPC with Windows XP and often resets while I'm asleep or away and I need to manually reboot it, maybe once or twice a day, with a maximum upload bandwidth of 1Mbps, so you need to be insistent, use a download manager, etc...

It's like very old Internet access.

[ronsor]

You might need to try several times, or later. You will be able to download then.

My server is a tiny BenQ S6 UMPC with Windows XP and often resets while I'm asleep or away and I need to manually reboot it, maybe once or twice a day, with a maximum upload bandwidth of 1Mbps, so you need to be insistent, use a download manager, etc...

It's like very old Internet access.

Windows XP. Server.

May I tell you how bad that is?

[Kazinsal]

I honestly wonder who cares about the stupid DOS/Win9x fetishism the OP posts.

It's not OS dev at all, is completely uninteresting to anyone who is actually interested in the design and development of new operating systems, and is constantly just an exercise in "how annoyingly obsessive over purely obsolete tech and bad designs can the OP be".

DOS and Windows 9x are the most representative and specific OSes native to the standard x86 PC

No. No they are not. You're an idiot if you genuinely believe this.

[zaval]

You might need to try several times, or later. You will be able to download then.

My server is a tiny BenQ S6 UMPC with Windows XP and often resets while I'm asleep or away and I need to manually reboot it, maybe once or twice a day, with a maximum upload bandwidth of 1Mbps, so you need to be insistent, use a download manager, etc...

It's like very old Internet access.

Windows XP. Server.

May I tell you how bad that is?

Oh, that crappux arrogance.
There is nothing wrong with Windows XP used as a server.
The only question, haven't you noticed that "BenQ S6 UMPC" string just before "Windows XP" trigged you? Here, look at it. This is a freaking ancient tablet!11 with half of GB of ram and 800MHz cpu. I am confused how it can have been used as a server at all.

[Kazinsal]

Windows XP was declared unfit for being put on the public-facing internet as a *client* in 2014. It has not received updates for vulnerabilities except for a few core ones since then. I say this as a Windows user who would in almost any situation rather use Windows as an operating system: Windows XP is not fit to be a server on any network connected to the internet, no matter how much firewalling and NAT there is in the way.

This isn't 2007. We aren't forced to choose between XP SP2 or RTM Vista here. There have been four (realistically five due to the sheer amount of under-the-hood changes between 8 and 8.1) major releases of Windows since XP. It should have died completely 7 years ago, but here we are, with people still using it for some godawful braindamaged reason or another.

[iansjack]

You might need to try several times, or later. You will be able to download then.
...
It's like very old Internet access.

Well, thank you for the information but - honestly! If you can't even maintain a server that works to distribute this material I don't think I'm going to bother retrying. It doesn't bode well for any value in the information.

[onlyonemac]

It's not like the server's very well configured either. Last time I checked, I could access a bunch of temporary files that are probably supposed to be private. I'm sure there are loads of vulnerabilities too if anyone should care to exploit them (personally I've got better things to do).

He also dumps stuff on the Internet Archive.

[ronsor]

You might need to try several times, or later. You will be able to download then.

My server is a tiny BenQ S6 UMPC with Windows XP and often resets while I'm asleep or away and I need to manually reboot it, maybe once or twice a day, with a maximum upload bandwidth of 1Mbps, so you need to be insistent, use a download manager, etc...

It's like very old Internet access.

Windows XP. Server.

May I tell you how bad that is?

Oh, that crappux arrogance.
There is nothing wrong with Windows XP used as a server.
The only question, haven't you noticed that "BenQ S6 UMPC" string just before "Windows XP" trigged you? Here, look at it. This is a freaking ancient tablet!11 with half of GB of ram and 800MHz cpu. I am confused how it can have been used as a server at all.

There are still decent, stable Linux distros that would support it, such as Tinycore

[~]

The idea is to write an OS and a loader that can be booted from an existing disk with existing partitions and an OS, without having to change absolutely anything, just like a regular DOS application, or just like one of those disk management utilities from Seagate or Western Digital with Caldera DR-DOS. No changes to the existing system must be done to run this tiny OS so that users actually can, know how to, and are encouraged to, test for the sake of OS development as free as when OSes were low-level and simple enough to allow for this a few years ago. Now such accessibility without having to alter a machine to install, to test a good custom OS, needs to be reimplemented for the new generation of machines.

I have updated the code to add this kernel loader for running from DOS that I have called "/BOOTCFG/LDR/4/PATIO.COM":
BOOTCFG__v2017-06-16.zip


I have written a loader for my kernel to boot it from a DOS-like OS. Now what I'm left next is to implement an "exittodos" command and functions in the kernel and this loader so that I can freely enter and exit my own stand-alone OS (tiny and extremely incipient so far), from hard disk or floppy disks that DOS recognizes:


I'm thinking about developing a good multitasking structure to save all of the register values once the kernel is loaded, IF it was booted from DOS, so that it can return to DOS. In this way when I return to the area of the 16-bit COM kernel loader program from my kernel (right after the 32-bit call into the kernel) I can restore any relevant CPU registers as in a simplified task switch.

The kernel can now be loaded from a hard disk with DOS without making any changes at all to a computer, PC or laptop, but external programs still need to be read from the first floppy. I need to add code to handle IDE hard disks, partitions and FAT32 file systems as well as selecting the current disk/directory, as a minimum.


I see that with all this I have done rather less than by implementing a DPMI program, but I want to be able to have a dual OS setup, from DOS to:

- My OS.
- To ReactOS.
- To MenuetOS.
- To VisOpSys.
- Any interesting OS I or others can manage to write a kernel/system DOS loader for.


It can become interesting if we add ATA, SATA and USB mass storage drivers to DOS and the OS we are to load.


Current code of the PATIO.ASM kernel loader:

Code: Select all

PATIO_COM:
;PATIO.COM
;
;         This program will load our 32-bit kernel
;         into physical address 1048576.
;
;         version 2017-06-17
;;

bits 16
org 100h



;Print program identification:
;;
 mov ah,9
 mov dx,str_kernelloader
 int 21h




;Open file read only:
;;
 mov ax,3D00h
 mov dx,kernel_bin
 int 21h
 jnc .noerr

  ;If we are here, print error:
  ;;
   mov ah,9
   mov dx,str_fileopenerr
   int 21h

 .noerr:
 mov [file_handle],ax

mov ah,9
mov dx,str_openkernfile
int 21h




.readkernloop:

mov ah,9
mov dx,str_readingkernfromdos
int 21h


 mov ah,3Fh            ;Service -.- read from file
 mov bx,[file_handle]  ;File handle by AH==3Dh
 mov dx,data_cache     ;Our temporary file read buffer
; mov cx,65535          ;Number of bytes to read
 mov cx,512            ;Number of bytes to read
 int 21h               ;Execute
 jc .kernreaderr       ;If CF==1, there was a read error


  ;;;INIT: Copy the kernel to its final destination
  ;;;INIT: Copy the kernel to its final destination
  ;;;INIT: Copy the kernel to its final destination
  ;;;INIT: Copy the kernel to its final destination

   cli
   push ds
   push es
   push cx
   push ax
    mov esi,data_cache
    xor ebp,ebp
    mov bp,ds
    shl ebp,4
    add esi,ebp

    mov edi,[final_kern_buff_ptr]

    mov ecx,512
    ;mov ecx,65535

    xor eax,eax
    mov ds,ax
    mov es,ax

    a32 o32 rep movsb
   pop ax
   pop cx
   pop es
   pop ds
    ;mov [final_kern_buff_ptr],edi
    add dword[final_kern_buff_ptr],512
   sti


  ;;;END:  Copy the kernel to its final destination
  ;;;END:  Copy the kernel to its final destination
  ;;;END:  Copy the kernel to its final destination
  ;;;END:  Copy the kernel to its final destination




 ;See if we already read all of the file.
 ;If AX and CX are different, then it means
 ;that we alreade read it all:
 ;;
  cmp ax,cx
  jne .kernreadend

jmp .readkernloop



.kernreaderr:
;If we are here, there was a kernel read error:
;;
 mov ah,9
 mov dx,str_fileopenerrread
 int 21h
 jmp .END


.kernreadend:
;;;INIT: Enable Protected Mode with local program GDT and jump to it
;;;INIT: Enable Protected Mode with local program GDT and jump to it
;;;INIT: Enable Protected Mode with local program GDT and jump to it
;;;INIT: Enable Protected Mode with local program GDT and jump to it

;Print that we are finally jumping into the kernel:
;;
 mov ah,9
 mov dx,str_kernjump
 int 21h

       ;Adjust the actual pointer of the GDT
       ;to where DOS has actually loaded this program:
       ;;
        mov eax,[GDT_actualptr]
        xor ebp,ebp
        mov bp,ds
        shl ebp,4
        add eax,ebp
        mov [GDT_actualptr],eax



        lgdt    [ds:GDT]              ;Load GDT. This is how we would
                                      ;exactly access the GDT in
                                      ;real mode

        cli     ;v2012-05-31
	mov	eax, CR0		;Turn on protected mode
	or	eax, 0x00000001         ;v2012-05-31
	mov	CR0, eax
        mov     ax, SELDat32            ;Setup ds and es
	mov	ds, ax
	mov	es, ax

       ;Indicate in EAX that we have loaded
       ;our kernel from a flavor of DOS, and thus
       ;we can exit our kernel back to it.
       ;We just need to learn to be increasingly
       ;careful to leave all of the hardware
       ;and CPU environment as it was before we
       ;loaded our OS, or as compatible as possible
       ;to avoid DOS from freezing.
       ;;
        mov eax,"DOSL"
        mov edx,"OAD1"


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;        jmp dword SELCod32:0xFF800000   ;Go
;     a32   jmp dword SELCod32:0x100000   ;Go
     a32   call dword SELCod32:0x100000   ;Go


 ;We will return to this point if we use
 ;the "exittodos" command of our
 ;LowEST Kernel console.
 ;
 ;We will need to learn a lot from DPMI
 ;to know how to properly disable paging
 ;and any other thing, as well as to use
 ;newly-developed multitasking structures
 ;whose first genuine task will be to save
 ;the machine state right after entering
 ;the kernel, for being able to return
 ;to our previous OS, more commonly DOS,
 ;or being able to return to the BIOS boot-level
 ;to execute INT 19h and load any other OS
 ;without rebooting the machine, which are things
 ;that will help us better understand the fine
 ;details of the machine and manage them
 ;by knowing that they are there, which will
 ;yield tiny hacking-grade tricks for low-level
 ;programming:
 ;;
  cli

 ;Disable Protected Mode:
 ;;
  mov eax,cr0
  dec eax
  mov cr0,eax


;;;END:  Enable Protected Mode with local program GDT and jump to it
;;;END:  Enable Protected Mode with local program GDT and jump to it
;;;END:  Enable Protected Mode with local program GDT and jump to it
;;;END:  Enable Protected Mode with local program GDT and jump to it






.END:
;Return to DOS:
;;
 sti
 mov ax,4C00h
 int 21h







kernel_bin db "kernel.bin",0
file_handle dw 0

final_kern_buff_ptr dd 1048576



str_kernelloader db "32-Bit LowEST Kernel Loader for DOS",10,13,"$"

str_openkernfile db "Opening KERNEL.BIN file for reading...",10,13,"$"

str_fileopenerr db "Error opening file KERNEL.BIN",10,13,0

str_readingkernfromdos db "Reading kernel file from DOS...",10,13,"$"

str_fileopenerrread db "Read error while reading KERNEL.BIN",10,13,0

str_kernjump db "Jumping into 32-bit kernel at 0x100000...",10,13,"$"









 ;INIT: GDT
 ;INIT: GDT
 ;INIT: GDT
    GDT:
   _SELNull equ 0   ;WARNING: This selector, besides being the pointer to the GDT
        GDT_size:   ;         is the null selector.
        dw GDTsize
        GDT_actualptr:
        dd GDT
       dw 0x0000

   _SELCod32 equ 8
     dw 0FFFFh       ; bits 0-15 length
     dw 00000h       ; bits 0-15 base address
     db 0            ; bits 16-23 base address
     db 10011010b    ; bits P,DPL,DT and type
     db 11001111b    ; bits G,D and bits 16-19 length
     db 0            ; bits 24-31 base address

   _SELDat32 equ 16  ;this is the "plain data selector"
     dw 0FFFFh       ; bits 0-15 length
     dw 00000h       ; bits 0-15 base address
     db 0            ; bits 16-23 base address
     db 10010010b    ; bits P,DPL,DT and type
     db 11001111b    ; bits G,D and bits 16-19 length
     db 0            ; bits 24-31 base address
   GDT_end:

   GDTsize equ (GDT_end-GDT)-1


  SELNull  equ 0
  SELCod32 equ 8
  SELDat32 equ 16
 ;END:  GDT
 ;END:  GDT
 ;END:  GDT







data_cache times 512 db 0


;EOF

[~]

I have finally managed to complete the basic loader for DOS (PATIO.COM, which loads and enters KERNEL.BIN from the DOS console).

I have also finally implemented the "extittodos" command for the kernel console. Now it allows me to freely enter and exit DOS and my kernel back and forth. Of course, to exit to DOS we need to reconfigure the PICs, the PIT, the IDT, reenable scancode translation from set 2 to set 1 so that DOS isn't left without the keyboard. In the future more things will have to be reconfigured for DOS, specially if we enter 64-bit mode, for returning to 16-bit mode. The command will not work if we didn't come from DOS in the first place, and we know that because PATIO.COM passes a signature to the kernel in EAX and EDX and the kernel records if the EAX="DOSL" EDX="OAD1" signature is present. If not, the command will do nothing, for example, if the kernel is booted stand-alone.




The current code is still located at:
BOOTCFG__v2017-06-16.zip

Now as long as no kernel components or external programs are buggy so as to destroy the sane configuration of the system, we can exit to DOS as if the kernel was a DPMI program. We can also boot it from floppy, but it's more valuable if used as a regular protected mode DOS program because it will give us a reference to know whether our code is buggy enough to make even DOS and the BIOS unstable when we return execution control to them, or which things make loading our kernel fail when used from DOS to take them into account being able to pinpoint them accurately to then learn how to handle them for more valuable PC tricks.

The base kernel is designed to run with or without paging, so we should still be able to return to DOS by disabling paging and using only the base kernel code.

I will put massive updates in video, text, source code, binaries and others in the following root URL. I will put the most important updates here at OSDev.org as I achieve them. Please use the language flags , , to switch to translations of the web pages if available.
I might add additional languages in the future. Translations are not identical, they just contain all of the vital details in each language:

http://devel.archefire.org/forum/viewforum.php?f=5 (Operating Systems Forum)

BOOTCFG:
http://devel.archefire.org/forum/viewto ... 2274&hl=en

LowEST Kernel:
http://devel.archefire.org/forum/viewfo ... =113&hl=en

[onlyonemac]

TL;DR You're using DOS as a glorified (and outdated) bootloader and pre-execution environment. Stop doing this.

[iansjack]

I will put the most important updates here at OSDev.org as I achieve them.

For the love of God - please don't.

[onlyonemac]

I will put the most important updates here at OSDev.org as I achieve them.

For the love of Dog - please don't.

Seconded.

Also I wonder if he realises that his server returns a 404 when I view the link in his signature and is apparently empty if I view the root. (If you're looking for the *really* bad server it's fortunately still online at http://devel.archefire.org/ for your viewing pleasure.)