Year 2024 Project - Studying Drivers

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
~
Member
Member
Posts: 1228
Joined: Tue Mar 06, 2007 11:17 am
Libera.chat IRC: ArcheFire

Year 2024 Project - Studying Drivers

Post by ~ »

In this year I was studying how to make and load Win9x drivers,
ring 0 32-bit VxD, 16-bit DLL/DRV/SYS drivers. experimenting heavily with
the bit flags of those files' headers, realizing how these drivers
can call BIOS interrupts and DPMI services.

YouTube playlists:
-----------------------------

Example drivers and header configuration internals, built by hand with NASM
(Pending, all information is still generic instead of in a single generator with all options manually selectable)

Simple x86 CPU emulator manual
https://www.youtube.com/watch?v=Jbuz-0m ... JJOUvattDC


Miscellaneous driver information
https://www.youtube.com/watch?v=w3md8LI ... ZD_tdmtEMW
https://www.youtube.com/watch?v=QZrqu5v ... Dw81QjDlrp
https://www.youtube.com/watch?v=0hsr1qR ... yn_HhwpB-l
https://www.youtube.com/watch?v=V5l6R8_ ... 3itUY7ygWN
https://www.youtube.com/watch?v=fHAmmnz ... hhsOM7F1Gt
https://www.youtube.com/watch?v=Ms99-yd ... NmjZ2zwkVP
https://www.youtube.com/watch?v=vW6Tjcl ... 4drq_VV3_X


Source code:
http://master.dl.sourceforge.net/projec ... ip?viasf=1
Clean code of the emulator kernel that aims to capture the execution of the native software/hardware

http://master.dl.sourceforge.net/projec ... ip?viasf=1
Full accumulated code


I have seen that switching modes by hand is not so easy, My graphics
switching code doesn't work if I first enter graphics mode with the BIOS
and then I try to switch to another mode or to text mode with my code.
It just doesn't work.

This code, mainly the x86 CPU emulator and VxD/NE EXE driver file skeletons
in NASM, without needing the DDKs, special linkers, etc., is the code I was
working on in 2024, most of the time.





Short x86 CPU Emulator Manual
------------------------------
After adding the NASM code and selecting if it will compile for
16, 32 or 64-bit with _x86_Portable__PLATFORMBITS_, you have to
configure the virtual CS:IP, DS, ES, FS, GS, SS and SP, so that
it can run some code:

mov widedi,x86CPU

push word 0x7C0
pop word[widedi+x86_CPU_State_SS]
mov word[widedi+x86_CPU_State_SP],0x8000

push word[0x10*4+2]
push word[0x10*4+2]
push word[0x10*4+2]
push word[0x10*4+2]
push word[0x10*4+2]
pop word[widedi+x86_CPU_State_CS]
pop word[widedi+x86_CPU_State_DS]
pop word[widedi+x86_CPU_State_ES]
pop word[widedi+x86_CPU_State_FS]
pop word[widedi+x86_CPU_State_GS]

mov word[widedi+x86_CPU_State_CS],0
mov word[widedi+x86_CPU_State_RIP],0

mov word[widedi+x86_CPU_State_AX],3h

mov byte[widedi+x86_CPU_State_IPadd_state+1],1






Set the size of the virtual RAM, although it should be 0
if you let the native hardware to validate the RAM
address space and manage it properly from the emulated code:

mov dword[widedi+x86_CPU_State_emuRAMSZ],1048576*4 ;Si es 0, es TODO el espacio
mov dword[widedi+x86_CPU_State_emuRAM],0
mov dword[widedi+x86_CPU_State_loopsPerTick],1






You have to mark the CPU state as running:

mov byte[widedi+x86_CPU_State_IPadd_state+1],1





A test program:

mov byte[0],0xCD
mov byte[1],0x10
mov byte[2],0xFA
mov byte[3],0xF4





The execution loop, with the initial cdecl parameter
containing a pointer to the virtual CPU structure:

push widedi
.lle:
call OPCODE__emu86__C__CPU__emulate16

cmp byte[widedi+x86_CPU_State_IPadd_state+1],1
je .lle






Declare the virtual CPU structure:

align 16
x86CPU times x86_vrCPU_SZ db 0





To get the emulator kernel, a raw 32-bit binary by now:
D:\apiclean\prg\emukern

To get the loader for MS-DOS, that only loads 32-bit raw binaries by now:
D:\apiclean\prg\kernels

As you can see, the test kernel only enters mode 3h by now
with the emulator and writes 2 colored bytes in the kernel loop.
It runs at least in Bochs and maybe some old Thinkpad laptops.
The emulator needs a lot of work to make it run as fast as the
native CPU, and adding full 16/32-bit protected mode, FPU,
protection, emulated v86, paging, and later 64-bit long mode
for being able to load and emulate the whole system, existing
drivers native to the machine where it is currently running,
and full operating systems like DOS, Windows 3.0 (in 16-bit
at least), Windows 9x, and older Linux, to make access to
hardware that needs custom drivers possible.

Completing this emulator in this fashion will take several
years probably, along with learning to load drivers and their
executables of all kinds, but without that, fully understanding
and taking advantage of the workings of the x86 PC ecosystem
is just not possible with this way of developing a kernel,
that in this case will be more brief, mostly only a CPU emulator
and key register/resources manager for ensuring it keeps full control
of the emulated state at all times.


----------------------------------------------------------------

In the full source code, in the VGA section, there are programs
to save the palette and VGA registers, which can be used to debug
which registers are well programmed and which not, after running
a VGA register programming sequence.

Most of this emulator source code has been obtained with generators
that just copy the entire code to text or that dynamically generate
each part of the code, to ensure that the code is always fresh,
as you always need to update the code or get another copy through
the generators, so they will always be more up to date than the rest
of copies of the code.



First I need to enumerate the things I need to achieve so far
and that maybe someone else could help me with code. It takes me
too much to write very little code, and in a year I only manage
to do the very basic of a project:

- Make a full emulator (16/32/64-bit with full protection, paging, modern features,
optimization to make it run as fast as the native CPU, capability to trace and
extract the interesting parts of any program, driver, hardware sequences) that I can
simply add to any program, with a port that commands the start of capturing any
instructions in general or of a memory range, and a final instruction in a range,
like ret or iret, that disables capturing instructions.

- Build and load all sort of drivers (16/32-bit DLL/DRV, VxD, SYS, Linux, DOS, etc.)
for all operating systems. In this way I can load them in my own kernel environment
as if the hardware was fully standard/transparent.

- Get the information to make all types of drivers, Win16/32/64, DOS, Linux,
Mac, from the CDs that come with the hardware (video, audio, network, TV,
USB, to load existing or custom drivers in any format).

- Make a kernel that is capable to run a kernel containing an emulator
capable of fully trapping the machine from real mode upwards through
my own emulation so that everything becomes more reusable and fully
understand everything that happens in each computer and how it works
with the OS we will emulate along with the native hardware with the emulator
(native hardware except what is necessary to trap the machine and fully
emulated OS/software).



I have been studying BIOS and Win9x drivers and making some simple ones by hand
(16-bit DRV/DLL and 32-bit VxDs).

I have also been making an Unreal Mode emulator for basic 16 and 32-bit instructions,
to try to get to the point where I can load and run them from DOS for example.
By now I can only run and extract things like video BIOS sequences (switching modes)
and simple COM file running.

Simple x86 emulator explanation:
https://www.youtube.com/watch?v=ztw1AGm ... ZmxGm4SDLQ

Full source code of the emulator and all my projects:
http://master.dl.sourceforge.net/projec ... ip?viasf=1

My YouTube channel with programming playlists:
https://www.youtube.com/@AltComp126/playlists

Donate to PayPal or post more code if you want to help me:
https://www.paypal.com/donate/?hosted_b ... QS2YTW3V64

If you want to help me more/daily, you can write me to [email protected]
-------------------------------------------------------------
-------------------------------------------------------------
-------------------------------------------------------------
-------------------------------------------------------------

By now I will explain how the emulator can be added to a project.

It is written in NASM, it is compatible with unreal mode, 32 and 64 bits
(this last one is untested but it should compile).

Having a full x86 emulator is a key milestone in addition to be able to run
with V86, VT-x, or other third party emulation sources.


So far I have seen that it is easy to replicate drivers like diskio.dll,
but other drivers like graphics, even Win9x VGA, have a much more unknown structure
that needs to be discovered for such a starting-level project with not much information.

Declare in NASM:

global _OPCODE__emu86__C__CPU__emulate16

From the C code, just call OPCODE__emu86__C__CPU__emulate16(vrCPU);

That will execute instructions one by one (normally, I will prepare it to
execute more than an instruction at a time for speed in the full
x86/x64/x87 emulator). The parameter it takes is a virtual CPU structure.
You just need to configure CS:IP and other registers before starting.
The CPU will not stop executing in a while loop until:

while(*(unsigned char*)(vrCPU+x86_CPU_State_IPadd_state+1)==x86_CPU_status__running)



You have this file to generate a stand-alone COM emulator for DOS or for 32/64-bit
for your own use:

C:\api\CPU\x86\emu\NASM\x86emu16.asm__src.html

It will generate x86emu16.asm. This is the unreal mode emulator, without any
protected mode or protection, only basic unreal mode with 16/32-bit data.


Here you have a demonstration of a DJGPP emulator, just run test.asm.
You have main.asm, x86emu16.asm and main.c, and it will generate emu.exe:

C:\api\PC\BIOS\emu\DJGPP

You can run:

tsrtest.com
rmpmur.com -- just in case you run unreal mode code
emu.exe src.asm


It will generate an .asm file, by now containing the in/out instruction sequence
with the values in EAX to reuse things like entering standard or VESA modes for
any specific video card that you can later accomodate and run as native code
to manipulate the video on your own based exactly on what the emulator ran.






VGA BIOS Drivers

There is the vgaregs.c/vgaregs.exe program that extracts registers:

vgaregs 3 mode3h.c
vgaregs 13 mode13h.c


myfile.c can be compiled into a program that switches into a standard
VGA mode, with all of the registers you recorded when you entered the
standard mode with vgaregs.exe.
Post Reply