bewing's complete bochs rewrite: test request

[Brynet-Inc]

wxWidgets doesn't have a Qt backend. Qt, on the other hand, does come with QGtkStyle - or, in other words, if it detects a Gtk using desktop (GNOME, XFCE), it will automatically load gdk and extract the style information it needs from it. Gtk, on the other hand, does not have a builtin style for drawing widgets natively on Qt desktops, and the only option which exists is pathetically buggy.

Right, sorry, for some reason I thought there was a Qt backend.. seems to be only a X11, Motif and GTK+ backend on Unix.

@Brynet: Your 2nd major crashing issue should be resolved now. If you would be willing to verify that, it would be helpful.

Yes, that's how I had patched it locally.. it works, but as stated, once back at the text menu I cannot return to ReBochs for some reason (..maybe it's a local issue, not dismissing it..).

Code: Select all

Please select an option: [2] 2
1. Quit now
2. Return to simulation
3. Load from ...
4. Save to ...

Please select an option: [1] 2
Sizing init error = 0
Xlib: unexpected async reply (sequence 0x8f)!
The program 'rebochs' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadShmSeg (invalid shared segment parameter)'.
  (Details: serial 19 error_code 149 request_code 142 minor_code 3)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the --sync command line
   option to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)

Sometimes it's just the first line, "Sizing init error = 0", and I get a blank white box and ReBochs loops indefinitely, the sequence number on the Xlib error also changes.

Anyway, have fun.

[bewing]

Actually, I meant the other error -- the one you said you didn't want to touch. Basically, it shouldn't be giving you any more segmentation violations (or stack errors).

And yes, restarting a sim is a feature that hasn't been tested yet. Thanks for the heads-up that it isn't working. (I forgot about that sizing error thing on restarts, I'll fix that immediately -- it's simple.)

[Neolander]

Sorry for the off-topic, but that...

IWhat I find amazing is that some crazy people managed to write complete Ubuntu desktops and apps using GTK. I'm not sure how many people died in the process, but I'm sure it was a few.

...is exactly, word for word, the reason why my long-term highly hypothetical plans are to try writing a full GUI desktop OS.
It probably won't work, and I don't care since I'm already happy hacking the kernel. But GUI implementations in Linux (especially) and OS X suck so bad that it makes me want to bang my head against a wall. GDI on windows was slighty better, if I remember well, but it's deprecated anyway. I don't understand why is it so hard to think about GUI toolkit design before implementation, but I want to try to code something that respects basic usability and device-independence laws and uses perfectly clean code in order to find out...

/off-topic

Anyway, it's great to see a Bochs rewrite, especially if it leads it to run faster and stop using 100% CPU all the time. Sadly, I've got nothing stable to test it on and I'm not knowledgeable enough about very low-level hardware operation to debug it. But keep up the good work !

[Creature]

One one hand I like the idea of a Bochs rewrite (more stable, more features, faster, ...), but on the other hand, it could turn out crappy as well: suppose the regular Bochs keeps adding good features (that rebochs will need to adapt, or worse, doesn't adapt), then we will need to switch between one more emulator all the time to test certain features. On one hand: more features, more testing = better, but on the other hand = more effort. There is already so much competition in emulators and such, that I guess this emulator should outstand (in comparison to Bochs) by far, or else it will be one of those cases where you have to test your kernel with 5 different emulators to see the result (and of course: real hardware). The variaty of emulators however is kinda fitting: there is so much real hardware that can act so differently, that many emulators may be good as well.

[Neolander]

One one hand I like the idea of a Bochs rewrite (more stable, more features, faster, ...), but on the other hand, it could turn out crappy as well: suppose the regular Bochs keeps adding good features (that rebochs will need to adapt, or worse, doesn't adapt), then we will need to switch between one more emulator all the time to test certain features.

It's true. At some point, when reBochs is mature enough for everyday use, legacy Bochs must be flagged as deprecated, all its useful features must be ported to reBochs, and reBochs must take its place on Bochs's main page

On one hand: more features, more testing = better, but on the other hand = more effort. There is already so much competition in emulators and such, that I guess this emulator should outstand (in comparison to Bochs) by far, or else it will be one of those cases where you have to test your kernel with 5 different emulators to see the result (and of course: real hardware).

I wonder : does one really need more features than what legacy Bochs currently offers for OS testing purposes ? At some point, it's good to stop adding major features and start to write on polishing (new virtual devices, bugfixes, and so on).

About competition... I decided to go with bochs as my main emulator because of its debugger, range of emulated virtual hardware, and focus on rigorous emulation (ie outputting a byte when you should be outputting a long will lead to an error). AFAIK, no other emulator provides that, so I don't see much competition here, except Bochs itself...

[bewing]

I wouldn't have even started writing it if I didn't have a clear vision of the end result being significantly better than bochs, qemu, or any other emulator around. And if this becomes (in time) a community OSDev project -- while bochs is only maintained by Stanislav and Volker -- we can add more and better features faster than they can (especially more accurate hardware models). If you want to adopt a "wait and see" attitude, go right ahead. You will see.

[Creature]

... If you want to adopt a "wait and see" attitude, go right ahead. You will see.

I hope I will .

[stlw]

It's true. At some point, when reBochs is mature enough for everyday use, legacy Bochs must be flagged as deprecated, all its useful features must be ported to reBochs, and reBochs must take its place on Bochs's main page

Probably now it is my time for some critisizm

I guess you gonna lose all BeBochs advantages of being smaller very qickly. All we are good until we grow up
Once you add FPU/MMX/SSE/SSE2/SSE3/SSE4/VMX, you will see BeBochs binary growing and growing with every feature.
Also expect your decoder to go more complicated when you add more different crap to decode.
Expect your paging stuff to become more complicated when you add more paging modes to support.
And more complicated = slower.

For more readable code advantage - not sure your CPU code is more readable. And it is not just my optinion - I asked few students which worked on Bochs last year.
Also don't think Bochs won't steal some good ideas from BeBochs
So probably finally we will end with one project, better and faster Bochs

Stanislav

[Neolander]

I guess you gonna lose all BeBochs advantages of being smaller very qickly. All we are good until we grow up
Once you add FPU/MMX/SSE/SSE2/SSE3/SSE4/VMX, you will see BeBochs binary growing and growing with every feature.
Also expect your decoder to go more complicated when you add more different crap to decode.
Expect your paging stuff to become more complicated when you add more paging modes to support.
And more complicated = slower.

I agree with you on the speed and binary size thing, one is going to slow down and the other is going to bloat up. But consider the following option : when the original Bochs source was made, its design did not plan introduction of many feature that have been introduced since, hence requiring many hacks since modifying a source code after leaving it for several months always leads to hack (tm).
If ReBochs' design takes account of those feature from the ground up, final binary size and speed may still be better than Bochs' one, due to cleaner code.

Anyway, if Bochs steals some ideas from ReBochs, this effectively leads to the same result as my proposal : a better "official" Bochs Because since Bochs is unmaintainable as is, they would have to rewrite it too.

[Brynet-Inc]

Hi bewing,

Since the debate about the toolkit has been ongoing, one idea that wasn't proposed is using a graphical toolkit that builds upon SDL directly.

There is a toolkit called Agar, BSD licenced, and it boasts a comprehensive set of "standard widgets", this would remove the platform specific debuggers and reduce code duplication.

This way ReBochs appears as a single window, and perhaps switching between VGA output and the debugger can be done using keyboard shortcuts? like in QEMU?

http://www.libagar.org/

Any comments on this?

[bewing]

Yes, Stanislav -- I expect my code to double in size from what it is now with everything I still need to add. It'll still be less than half the size of bochs.

I have been careful with my planning, so that adding all those features should not slow my code down TOO much. But it will slow down some, I agree. The main point is to keep the primary logic pathway in cpu_loop as short as possible.

Go ahead and adapt ideas into bochs. Bochs is not very adaptable, but good luck trying.

[bewing]

@Brynet -- hmmmm. One thing that scares me about SDL is that it doesn't look like the basic message loop has the ability to block, waiting for an incoming GDI message. From all the documentation I read, it looked like it is built exclusively to poll -- because SDL is designed as a singlethreaded game platform, so they don't care about cpu-hogging.

I certainly prefer the basic structure of SDL to GTK. (Especially with GLib and GDK all mixed into the mess.) I hate the piling of libraries on top of libraries on top of libraries, in general. It's an extremely bad design mantra. That's why I like working directly with the win32 GDI. If I could program my interface directly on X and forget ALL the libraries, I would -- but that's beyond me, and is way too bug-prone and unmaintainable anyway. But Agar (built on top of SDL, build on top of X) would be better than what I have now, I agree. The problem is that the GUI debugger window really is quite complex in the details of the window manipulation. 1800 lines of Win32 code = 2100 lines of GTK code in my current toolkit routines is not trivial. I'm quite confident that WxWidgets simply cannot do what I'm doing in GTK and Win32. I'd be surprised if Agar could do it. TreeViews, and ListViews, and mouse captures, and special painting procedures, and split mouse/keyboard focus, and reparenting widgets, and autoscrolling, and multithreading. And more, I'm sure. I did a lot of stuff. You really can't see it all with just a quick intro to that debugger -- it's much more complete than you would guess.

As far as merging the GUI debugger window together with the VGA window ... the idea has some advantages, but some drawbacks. It solves the singlethreadedness issues with SDL, GTK, etc, as you say. It kind of messes up some of my design ideas, though. The GUI debugger is supposed to be optional. But if you turn it off, and just use a textmode debugger instead -- you still need the VGA window. Which basically means the two windows need to be separate. And I think there are some small debugging advantages in being able to see both at the same time. That is, having the VGA window visible, and remaining current, as you step through breakpoints in the debugger.

Which doesn't mean that I'm rejecting your idea. I will look into Agar, and think about it.

[stlw]

I agree with you on the speed and binary size thing, one is going to slow down and the other is going to bloat up. But consider the following option : when the original Bochs source was made, its design did not plan introduction of many feature that have been introduced since, hence requiring many hacks since modifying a source code after leaving it for several months always leads to hack (tm).
If ReBochs' design takes account of those feature from the ground up, final binary size and speed may still be better than Bochs' one, due to cleaner code.

Anyway, if Bochs steals some ideas from ReBochs, this effectively leads to the same result as my proposal : a better "official" Bochs Because since Bochs is unmaintainable as is, they would have to rewrite it too.

Neolander, "They" - is me
And I actually planned up-front all CPU related changes that were introduced in Bochs in last few years.
Some rewrite will be needed only for parallel Bochs (doing SMP in many threads), but this not ground up rewrite.
I am ready for it, all CVS changes taking it into account.

Stanislav

[stlw]

Go ahead and adapt ideas into bochs. Bochs is not very adaptable, but good luck trying.

BTW, answers for some of your questions from the code:

in rmode, if I have an instruction that goes off the end of a 64k page -- what happens?
-- does the CPU wrap to 0 when extracting instruction bytes?

it will #GP(0)

Does an operand size prefix do anything to an 8bit instruction? "undefined"?

Well defined. Do nothing.

Other prefixes (ie. lock) on unsupported opcodes? Nothing? Undefined?

keep #UD.

UNREAL MODE: verify the stack operation re: d/b bit -- d/b is overridden? -- 4b push/pop ever does 2b stack move?
-- pushad uses [esp], not [sp]? Others?
-- What the hell is this Rmode (32bit) stuff? Can you really exit straight from 32b mode to rmode, and have things run?

When you exit from protected mode to real mode the hidden part of the segment remains.
No matter what it was inside - which limit you had (no necessary 4G), which permissions and etc. Only base will be overwritten.
So push/pop /pushad will use whatever current SS.D_B defines.

If you POPF a value that does not have the 2 bit set, what happens? (haven't checked the manual yet)

bit 2 is hardwired to '1 - if you write '0 into it, it still remains '1.

BTW, till now I wasn't able to boot any OS with BeBochs so speed comparison is delayed
Will wait for next more stable build.

Stanislav

[bewing]

Thanks for the answers. Yes, the only way to do a simple speed test currently is to time a small loop in an rmode or pmode bootsector. There are many small fixes needed before any OS could boot completely. And the test would be unfair to bochs on Windows currently, since bochs runs at reduced priority on Windows.