Issues while implementing VM86 support

[Pype.Clicker]

Hi all ... I've been recently checking out OSLib (just to compare with Flux OSKit) and then i saw it had "vm86 support for BIOS VBE mode changes" ... no need to say that was highly interresting me since that's something i try to have for about 2 years now, without having time for doing "research" on the subject
<ranting>
i had a working implementation of VM86 in clicker 0.0.0, but that code was ugly, crappy and whatever other thing you could imagine. So i wasn't feeling like re-writing something from scratch
</ranting>.

About the OSLib
Despite a few compilations issues, the OSLib is simple to set up and compile for those who have a Linux environment (and almost trivial if you still have a GCC 2.95.3 somewhere). I didn't gave examples a try in a bochs, though and went directly to porting vm86 code.

The only thing i'd say "disturbing" about the OSLib is their kl/ll/xl thing: you cannot figure at a first sight where a given component will be. "vm86" was in "Low Level" library. well. "find" did found it rather quickly.

With the help of CSCOPE to retrieve definitions of symbols met, it was quite straightforward to replace OSLib definitions into Clicker definitions (mainly thanks to the fact all the intel-defined structures were ALL_CAPS names). I even was surprised that their "context switch" thing co-exists seamlessly with clicker's scheduler
So the "rewrite" took only a couple of hour and i had then the VM86 module for Clicker ...

What was less funny to discover that OSLib has no apparent support for paging (at least not in the vm86 code) so i had "debug sessions darker than the night" including crawling through bochs sources to find out why there were still tripple-faulting GPF even before any vm86 instruction was executed...
(For the curious ones, that was eventually due to the fact i left the "TSS.cr3 = 0" instruction in the code, which lead to ?ber-chaos, invalid IDT table etc.)

Finally, by sunday 22:00 pm, i see bochs window shrunking and start the Dance of Victory not for too long ... It appears that the screen is filled with garbage and that the CPU has transferred execution to the "Krash Task" of Clicker (meaning a double fault has occured somewhere), leaving me blindfolded for later debugging (oh, the niceties of graphic modes )

You might find interrest in the fact that the first thing int 0x10 does with BOCHS bios is calling int 0x6d which also was a nuisance since i had no handler for 0x6d in my IDT (no, even with IOPL=3, software interrupts are not automagically deferred to real-mode IVT and i instead get a wrongly-handled GPF )

So i've printed the OSLib-provided code and wait for a few hours to read it tonight, hoping to figure out what i still have to handle.

[Pype.Clicker]

Hi, Great Yoda Of All, May The Schwartz Be With You *bows*

What is it doing if you issue a mode_info call to vga bios - to query some vesa mode?

I'd rather pick some bios command which not necessarily switches to graphics mode ];->. Then you can backtrace the whereabouts of your gpf handler quite nicely. Oh I remember the dark nights I have spent debugging it.

Yep, that's probably what i'll be doing right tonight (or i'll just try to switch to mode 0x03 -- which is the current mode ::) )

If it works, there needs not be a crash the reason of garbage on the screen(my english prof 'd kill me for this one), but rather the garbage that is munged into the Framebuffer. Already tried to draw something? *gg*

Actually, yes. I first tried to draw something, but i got garbage ... so i said to myself "maybe i just don't see it's drawn, let's clear the screen first", but there was still garbage. So i hit CTRL+C and saw the CPU was actually in a "for(;;)" loop at the end of "crashTask" code

Oh ... welcome to debug sessions darker than night *hihihi*

was sure you'd notice it ;D

Btw: How are you handling it? some small 16 bits stub loaded as process with vm86 properties and all?

For now it's just a service. There's a vm86 TSS that can be used to do some INT nn using structures to fill/save registers from the TSS ... If i want it to be in some special process later, i'll simply not call it from other processes ... i'll probably will allow user-access to table 0 in the process that initiated it alone ...

btwII: How could you read that code with that odd indentation? *rofl* Hope your eyes arent twisted.

i guess i have re-indented it quickly (M-x indent-region or something) ... i had a couple of #defines too ...

Code: Select all

#define LIN_ADDR vaddr_t
typedef short CONTEXT;
#define DWORD dword
#define WORD word
#define BYTE byte
#define TSS ia32Tss
#define get_TR() kCurrentTask()
#ifdef __DEBUG__
#define __LL_DEBUG__
#endif
#define message anykey(); kprint
#define lmempeekd(_a) *((DWORD *)a2l(_a))
#define FREE_TSS386 IATYPE_TSS
#define GRAN_16 IAFLAG_GRAN1B
#define GDT_read(sel,lim_ptr, acc_ptr, gran_ptr) segGetBase(sel)
#define GDT_place(_sel, _base, _size, _type, _gran) \
(segSet(0,(_sel=segSelect(0)),_type,l2a(_base),_size, _gran))
#define lmemcpy memcpy

btwIII: have found _x_biosCALL() which is a call to a call gate. Does this call some 16 bits asm code which issues the int XX calls? Or is that something completely different(tm)?

dunno ... i'll get a look at this ...

[Pype.Clicker]

Damn! there was a bug in the bochs ... It may not affect all V8086-related code since it seems related to expand-down stacks (and not many ppl actually use expand-down segments) but still, it's there: exception.cc, around line 653

Code: Select all

          load_cs(&cs_selector, &cs_descriptor, cs_descriptor.dpl);
          EIP = gate_dest_offset;

          if (gate_descriptor.type>=14) { // 386 int/trap gate
            if (v8086_mode()) {

         BX_INFO(("interrupt from virtual mode\n"
                       " +- push on                          %4x:%8x\n"
             " +- initial stack:                   %4x:%8x\n"
             " +- vector %2x, INTnn? %c  errcode?%c    %8x\n",
             ss_selector.value ,ESP,old_SS, old_ESP,
             vector, is_INT?'y':'n', is_error_code?'y':'n',
             error_code));

         /* <PYPE: imho, we need this or 'push_32' will hickup ...> */
         BX_CPU_THIS_PTR protectedMode=true;
         /* </PYPE> */
              push_32(BX_CPU_THIS_PTR sregs[BX_SEG_REG_GS].selector.value);
              push_32(BX_CPU_THIS_PTR sregs[BX_SEG_REG_FS].selector.value);
...
              BX_CPU_THIS_PTR sregs[BX_SEG_REG_ES].selector.value = 0;
         /* <PYPE> */
         BX_CPU_THIS_PTR v8086Mode=false;
         /* </PYPE> */
         BX_INFO(("now in protected mode with v8086 return on stack"));
       }

Without that fix, Bochs wil apply real-mode rules to the protected-mode stack while pushing interrupted state ...

Now i can get to VM86 mode and back. I still have no workaround for the "int 0x6d" thing except calling int 0x6d instead of int 0x10 (which is of course auwful), but at least, i get a "regular" GPF when "int 0x6d" is encountered in BIOS code ...

[Pype.Clicker]

another 'trick' i had to use was CLicker's stack contains crucial informations at their bottom (about printing buffer, current thread and the like). The OSLib wasn't clicker-friendly since it was using a static char stack[N] for exception-handling in v8086 mode, so i simply made that array a local variable of V86_callBIOS ;D

[Pype.Clicker]

it worked nice a couple of times with the "int 0x6d" hack and then i added support for "int nn" in a GPF handler. That works fine aswell...

I've identified some weird issue in OSLib's code that seems to trash some registers around the call (%ebx, which was VRAM in my case)

obviously, GCC assumes more will be preserved than what is *actually* preserved. I haven't figured out what's the cause so far.

[Pype.Clicker]

okay. Neither OSLib nor GCC were responsible of "%ebx" bug ... that was just me, myself and i ...

stupidly called "cpuid()" to check whether VirtualModeExtension was there (and it is not on Bochs ...) but i forgot to say "cpuid" instruction could trash %ecx and %ebx to gcc ...


** vjhydgklg ** (<-- slamming forehead on keyboard)

here you get a snapshot of work-in-progress

[mystran]

If you found a Bochs bug, you should definitely report it to upstream vendor (= Bochs project ).

[bubach]

Nice to see that you are doing some progress, and not only help out other people on this forum..
So, when can I expect to see the FutureShell 0.01 in Clicker?

/ Christoffer

[Pype.Clicker]

@mystran: i reported it, but i don't expect it to be patched in bochs before a few monthes (i once reported a bug in NASM aswell, but it took years before a debugged version of nasm appear ... welcome to Open Source)

@bubach: yeah, actually, that's because i had a couple of new ideas for FutureShell that i decided to push VBE ... Now i must contain myself so that i keep on track and don't start doing fancy stuff like moving sines while the kernel is loading, etc

btw, if you have PD or GPL font redering code to share, you'll get FutureShell 0.01 earlier ;D

a PD font for you to play with

[Pype.Clicker]

yup. Several VM86 modules work that way (including TRAN's start32/pmode extender, which clicker 0.0.0 was based on). Actually, your scheme is more complete (e.g. you're not limited to BIOS services), but also involves more overhead (e.g. being able to send/receive generic message from the VM thread, keep switching between VM86 task and ordinary task, load code into DOS memory, etc.

Now you get why i couldn't decide myself to start porting this (plus its german-variables addiction)

[bubach]

Maybe you could port/steal something from: http://www.minigui.org/ ?
I don't know much about, just found it, but as it's GPL:ed someone may find intrest in it?

[EDIT] Other links I found, which may or may not be of any value...

http://enchantia.com/software/graphapp/
http://cvs.reactos.com/cgi-bin/cvsweb.cgi/
hmm... This list may get bigger.. (with junk)

/ Christoffer

[Candy]

btw, if you have PD or GPL font redering code to share, you'll get FutureShell 0.01 earlier ;D

a PD font for you to play with

don't know whether I have used that font in specific, but I kind of felt called by the ability to help with FutureShell. So, I've been hacking the evening away at this font rendering code for allegro, and atm it works very nicely on my windows box rendering anything it wants to. Allegro has its quirks here, so I'm not going to guarantee it working, but if it doesn't segfault before the main function begins it should work.

Note for others, if you know how in gods name it can segfault before main starts, tell me . Same goes for after it, which happens on my linux box.

Stuff attached, Bubach, will upload it to pd.asmhackers.net

Yes, that means it's PD. The fonts aren't however, believe one is PD other two are GPL'd. Screenshots will be posted as well, this is like a hack-together-page.

The code is a "mess" which means that I can read it all pretty quickly, but don't expect lots of tactical comments. I commented the render code. The font generator code has two defines indicating the size of the characters and atm assumes 16 chars/row. Changing any of these or introducing variable font width is all future stuff. Might hack on them tomorrow or sth, but this should be enough for FutureShell.

It supports translucency btw, as can be seen in the render code. Not sure it uses the background, it might just not care. They're both black atm, so not much for testing. Might hack on fixing that tomorrow, then again, probably not before next week.

Enjoy this fine release by h4x0ring t34m c4|\|c|y ... ehm... Candy.

temporary upload to http://candy.dublet.org/fr.html in about 30 min. Gnite all!




On the topic of a shell:

I've worked on a shell a while ago. You can probably use the lower levels of it (the job combining) for a quick start-up for the FutureShell project. You will probably need to adjust the lowest-level job class though.

and next edit, include the url. It's at www.sf.net/projects/cdsh. The OS-independent startup is at www.sf.net/projects/ncsh, I'm admin and can add people to it as developer, should you really have the time & interest to do that.

[mystran]

@mystran: i reported it, but i don't expect it to be patched in bochs before a few monthes (i once reported a bug in NASM aswell, but it took years before a debugged version of nasm appear ... welcome to Open Source)

Yeah, I know. It's even funnier when you provide a nice (and manually cleaned) patch and everything, and it still won't make it.

btw, if you have PD or GPL font redering code to share, you'll get FutureShell 0.01 earlier ;D

Urgh... What exactly is wrong with FreeType? I mean, that's portable, works nice, and is available with either a BSD (with credit) or GPL.

[Pype.Clicker]

Urgh... What exactly is wrong with FreeType? I mean, that's portable, works nice, and is available with either a BSD (with credit) or GPL.

nothing wrong, except i'm not sure i'll be able to integrate it right atm. it might be a bit larger than what i need and can afford (the whole stuff still need to fit 512KB atm).

[Pype.Clicker]

@Freetype: it took me about 1 day to integrate Candy's font engine which was fairly simple software, btw ... that gives you an idea of the very crude environment my userprograms get. no malloc, no printf ... using "*(int*)0xdeadbeef=1;" statement to exit with error codes ...

That being said, i now have a working "video server" but it looks like on some hardware (my development platform, among others), i get weird "opcode 0xEE unsupported by the monitor" (which is some I/O operations, according to the knowledge collected in the FAQ).

What seems weird to me is that i thought i filled the IObitmap to avoid such things.

Need a deeper debug session.