OSDev.org

Please let's not start a C vs. C++ flame war ok? Let's focus on the debugger stuff please!


JAL

WindowsNT wrote: ...
unsigned __int64 PhysicalToLinear(unsigned __int64 a);
...
unsigned int GetCPUMode(); // Say Bit 0 for PM , Bit 1 for paging, Bit 2 for Long

I suspect the first is impossible. And I doubt that bochs keeps track of, or can calculate the second any better than we can. But as a class, it looks basically workable.

WindowsNT wrote:Putting the new stuff into a class again:


class DEBUG
{
public:
// ...
void GetRegs(CONTEXT*);
void SetRegs(const CONTEXT*);
void GetMemory(unsigned __int64 From,unsigned __int64 Bytes,char* dst,bool IsAddressLinear);
void SetMemory(unsigned __int64 From,unsigned __int64 Bytes,const char* dst,bool IsAddressLinear);
unsigned int CpuNum();
unsigned __int64 PhysicalToLinear(unsigned __int64 a);
unsigned __int64 LinearToPhysicalunsigned __int64 a);
void SetBreakNotification(void (*)(DEBUG*,unsigned __int64 lParam),lParam);
void Break();
void Die();
void GetInstructionDissassembly(unsigned __int64 AtAddr, bool IsAddressLinear,char* rslt);
void GetBreakpoints(vector<BREAKPOINT>& brks);
void SetBreakpoint(const BREAKPOINT&); // a BREAKPOINT can also be a watch point
unsigned int GetCPUMode(); // Say Bit 0 for PM , Bit 1 for paging, Bit 2 for Long
void GetMemoryInformation(MEMORY_INFORMATION*); // Should include total physical, virtual etc
void Step();
void StepInfo();
void StepOut(); // to exit from function calls


// .....

};

Guys, I see you taking Bochs debugger in extreme and see it ONLY as you could type a string in debugger window and parse the output.
This is not right, Bochs already has internal debugger and debugger interface.
The internal debugger is written in pure-C, it is text-based and accepts commands only through command line.
The debugger has some state like a breakpoints or current CPU being debugged and the state is NOT in the interface.
The debugger interface is a set of functions which responding on commands that user type in the debugger command promt.
And the interface is connecting between Bochs emulated CPUs/devices and debugger itself. The interface is described in bx_debug/debug.h and it is working interface that you could use.
The problem that the Bochs internal debugger is currently part of Bochs and it is not the best debugger ever so I would like to be able to separate it from Bochs and have an option to attach another debugger, GUI one or new command line one. So the first step might be would be to go over bx_debug folder and understand what is debugger implementation and what is debugger interface.
But anyway, assuming you now writing a GUI to existing debugger - you are allowed to use its interfaces - everything that is in debug.h
If you writing external debugger you don't need functions like GetBreakpoints on your interface - the breakpoints are NOT inside the CPU in current interface !
Bochs CPU tells debugger about any instruction executed and about any memory access executed and debugger handles its breakpoints and watchpoints alone.
This is example of current interface.
We could make it better but first we should decide which kinds of services we want to have from CPU object.
Currently CPU provides the following services for debugger:
- Access registers, in future will be ONLY through param_tree
- Callback on ANY memory access
- Instruction breakpoints handling
- Back to debugger on demand from CPU

Stanislav

and see it ONLY as you could type a string in debugger window and parse the output

Not quite -- it is just that, for all we know, everything that we are not doing is inside the simulator. We do not know bochs well enough to know the difference of what features we use are inside the current debugger, and what is inside the sim. We are just giving you a list of all the things that our GUIs cannot currently do for themselves -- since the sim calls the debugger, and never calls us.

So yes, that is what we need you to tell us: what features that we need are inside the current debugger, and what features are in the sim.

If you writing external debugger you don't need functions like GetBreakpoints on your interface - the breakpoints are NOT inside the CPU in current interface !

Do I understand you to mean that in the NEW interface that you will create, all breakpoints and watchpoints will be controlled by the debugger only? And you will be removing all of that support code from the sim?

Currently CPU provides the following services for debugger:
- Access registers, in future will be ONLY through param_tree
- Callback on ANY memory access
- Instruction breakpoints handling
- Back to debugger on demand from CPU

Except that the current debugger calls memory.cpp directly to set and read physical memory. And makes many direct calls to cpu.cpp.
In dbg_main, it calls:
BX_CPU(i)->guard_found
BX_CPU(i)->prev_rip;
BX_CPU(i)->get_laddr(BX_SEG_REG_CS, BX_CPU(i)->prev_rip);
BX_CPU(i)->get_cpu_mode()
SIM->refresh_ci();
SIM->set_display_mode(DISP_MODE_CONFIG);
SIM->has_debug_gui()
SIM->debug_get_next_command() -- for when the user interface is done through the DOS window
SIM->is_sim_thread()
BX_CPU(dbg_cpu)->trace
BX_CPU(cpu)->trace_mem
bx_pc_system.time_ticks()
BX_CPU(dbg_cpu)->mode_break
BX_CPU(which_cpu)->dbg_xlate_linear2phy(laddr, &paddr);
BX_CPU(dbg_cpu)->protected_mode()
BX_CPU(dbg_cpu)->show_flag
BX_CPU(cpu)->cpu_loop
BX_CPU(cpu)->stop_reason

And I only went halfway through it. These all have to be part of the interface between the sim and the debugger, too -- unless you can figure out a better interface that can eliminate a few of these calls. But the current debugger will still need to run, so it will still need all this information from the sim.

Unless what you WANT, is for the debugger to make all these calls to memory.cpp and cpu.cpp directly.

(And yes, WindowsNT, it looks like you were right, and I was wrong about whether bochs knows what mode the CPU is in.)

OK, so after thinking about what you said, I think my suggestion for the interface looks more like:

Whichever debugger the user chooses must be compiled into bochs, and has access to all the config.h compile parameters.
So the max number of CPUs is BX_SMP_PROCESSORS, for example, and the debugger knows what is being emulated and what is not.


1. A function to generate a runtime enumeration of all valid param strings.
2. The standard param_tree access functions.
3. Synchronous memory callback before every mem access (linear mem only?).
-- In debug.h there is a second callback for physical mem accesses? I don't see any need for that?
-- separate callback registered for each SMP thread
-- Note: to provide the functionality that's in the internal debugger for instruction breakpoints, it seems to me that the callback could have an additional "segment" field, for "virtual memory" watchpoints. It doesn't need to pass a segment register name or anything -- just the value of whatever segment was used to generate the linear address? Hopefully, this would be an easy change. It's not an important feature.
4. Synchronous instruction callback before every simulated instruction.
-- (unsigned cpu, Bit64u ptime, bx_address lin, Bit16u CsSegRegValue, (, cpumode)(, Bit8u *data) )
-- separate callback registered for each SMP thread
-- Note: If the cpu mode is stored in a variable that is easy to access, it would be really really nice if that value could be passed in the callback function, too! And 16 bytes of instruction opcodes (just a pointer into the instruction buffer?), so I don't have to fetch them from physmem, myself.
-- Note2: On return from the callback, is it possible that you could please check if the debugger changed the value of EIP/RIP during the call, and reload the instruction queue if EIP/RIP changed? Being able to change EIP/RIP is EXTREMELY useful in a debugger!
5. A function to translate linear into physical addresses for a particular CPU (or this could be done with a direct call to cpu.cpp).
6. The total number of simulated cores = SMP threads must be in the param_tree.
7. The current "mode state" of each cpu must be in the param_tree.
8. The total amount of emulated physical memory must be in the param_tree.
9. The standard method to force the simulator to quit.

The simulator uses the VGAW to handle all "event" interaction with the user.

The debugger will access memory.cpp directly to do all physical memory reads and writes.
The debugger will access the disassembler directly to do all opcode disassembly.
-- The disassembler interface will need to be slightly modified.

*** When you search the param tree for a name that doesn't currently exist, right now it panics Bochs. This is VERY annoying. Please modify it so that it just returns a NULL!

Plus, as I said before, the interface still needs to include anything else that the internal debugger requires (keyboard input, mostly).

Edit: I changed some details of my suggested interface. If you are going by my original post, you may want to look at it again.

@WindowsNT:
Attached is my heavily modified preliminary version of your stuff.h code. It still uses parsing. Some of the changes: compiles/runs on Pre-XP Windows, change in disassembly window handling, bochs memory accesses are direct (no parsing), some speedups, some fixes.

Considering how much I changed it, I doubt that you will want to use any of my code. This version is still compatible with yours, and looks almost the same.

If you want to compile it, you need to make one change. In win32dialog.cpp, you want to get rid of #include "textconfig.h" and #include "siminterface.h" -- and you want to add #include "bochs.h". After you make this change, YOUR version of stuff.h will still compile and run normally.

Some bugs and issues that I saw and remember:
*Pre-XP versions of Win can't handle groupID's. Old compilers also don't have some of the
groupID functions/defines. With #ifdef's your code will compile and run on pre-XP.

*When bochs is compiled without 64b support, the EFER register is never displayed --
which blows up your state machine (ProcessDebuggerUpdate), because the Creg update is never completed.

*Some compilers have trouble with the "for (int i=0;" redefining i at the same bracket
depth multiple times (merely a compiler compatibility issue).

*In WM_SIZE: the GDT, IDT, and PAGE windows have a width of (float)rc.right*0.85f,
instead of (float)rc.right*0.45f
EDIT: woops! I forgot. I had moved these windows sideways. Your original size was correct for your original placement, the problem was all mine, created by me.

I added a bunch of comments, too.

____________

I was completely not careful about handling wchars. So I do not know if it will work properly on XP/Vista. If you would be willing to compile it and try it out -- especially the AskText dialog box, I think -- I'd be grateful.

EDIT: removed obsolete download

bewing wrote:@WindowsNT:
Attached is my heavily modified preliminary version of your stuff.h code. It still uses parsing. Some of the changes: compiles/runs on Pre-XP Windows, change in disassembly window handling, bochs memory accesses are direct (no parsing), some speedups, some fixes.

Considering how much I changed it, I doubt that you will want to use any of my code. This version is still compatible with yours, and looks almost the same.

Hi guys,

I hope we also converging because I would like to take some version and integrate into Bochs once ..
I failed to do so with previous version by WindowsNT but just because I didn't invest in it too much time. I dream it supposed to be plug-and-make

Thanks,
Stanislav

Yes, I hope we are converging, too. I posted the updated interface that I would like from you, two posts ago.
If you can write the interface, I can integrate my GUI to bochs in less than a day.
Do you want me to create a "patch" for you so you can try the current GUI out on the current bochs version?
Or do you want me to rewrite a big piece of bochs for you, and show you what I want the bochs interface to look like? I could do that!


The instruction callback function that we talked about, is what I most need:

4. Synchronous instruction callback before every simulated instruction:
-- (unsigned cpu, Bit64u ptime, bx_address lin, Bit16u CsSegRegValue, cpumode, Bit8u *data)
-- separate callback registered for each SMP thread
-- Note2: On return from the callback, is it possible that you could please check if the debugger changed the value of EIP/RIP during the call, and reload the instruction queue if EIP/RIP changed? Being able to change EIP/RIP is EXTREMELY useful in a debugger!

All the rest of what I asked for mostly exists, or is not as urgent.

I need you to tell me how to register my callback function.
It would help a lot if you could modify the interface to the disassembler.
It would help if you could modify the SIM->get_param_num() function to return a NULL instead of a panic.

bewing wrote:Yes, I hope we are converging, too. I posted the updated interface that I would like from you, two posts ago.
If you can write the interface, I can integrate my GUI to bochs in less than a day.
Do you want me to create a "patch" for you so you can try the current GUI out on the current bochs version?
Or do you want me to rewrite a big piece of bochs for you, and show you what I want the bochs interface to look like? I could do that!


The instruction callback function that we talked about, is what I most need:

4. Synchronous instruction callback before every simulated instruction:
-- (unsigned cpu, Bit64u ptime, bx_address lin, Bit16u CsSegRegValue, cpumode, Bit8u *data)
-- separate callback registered for each SMP thread
-- Note2: On return from the callback, is it possible that you could please check if the debugger changed the value of EIP/RIP during the call, and reload the instruction queue if EIP/RIP changed? Being able to change EIP/RIP is EXTREMELY useful in a debugger!

All the rest of what I asked for mostly exists, or is not as urgent.

I need you to tell me how to register my callback function.
It would help a lot if you could modify the interface to the disassembler.
It would help if you could modify the SIM->get_param_num() function to return a NULL instead of a panic.

Yeah, the patch for current version with current interface would be perfect. At least I will upload it into patches folder and it will be biggest step toward main code integration.
If you want ot go over the code and show me how do you want to see the inetrface - do it !
I am always better in inetgration others ideas that in doing something from scratch myself

About your inetrface questions:

- Current Bochs debugger works a bit differently with CPU than you probably see, You see the emulator as a manager which runs and talks with debugger.
But in current Bochs debugger the debugger FULLY controls the smulation, it chooses where and when to start the CPU loop and how many instructions to keep inside.
he same with many CPUs. In theory current Bochs debugger could schedule your CPUs and even control time. The CPU is only a slave which knows to run and give
some informnation to debugger, for example about breakpoints being hit (both code and data). I really like it and would like to keep it.
This is why I kind of dislike your instruction callback and don't want to do it.
For me the debugger has CPU_loop method for each CPU which is equivalent to step/stepN/c commands and could check param tree about the state.

- In current implementation if you changed EIP using set $eip=value command in the debugger - Bochs will take it and handle it correctly.
You only change CS without problems.
BTW recently you got command to work with param tree directly from the debugger, in particular to save/restore it - inetersting which usages you can find for it.

- Disasm is easy thing. I wrote it and it is completely stand alone and even state less module. It could be changed in any way you want
Please supply your wish list !

- get_param_num changed in CVS.

Stanislav

You say:

If you writing external debugger you don't need functions like GetBreakpoints on your interface - the breakpoints are NOT inside the CPU in current interface !
Bochs CPU tells debugger about any instruction executed and about any memory access executed and debugger handles its breakpoints and watchpoints alone.

Then you say:

The CPU is only a slave which knows to run and give some information to debugger, for example about breakpoints being hit (both code and data). I really like it and would like to keep it.

I just thought from your first comment that you wanted to modify the CPU to take all the breakpoint detection out of the CPU.

But OK! If you want to keep it all, I will try to use it this way.

It will take a few more days to get you some code, now. But it is very close. I do not know how many compilers will be able to build it. Probably most of them. But you may need to make it a little more portable.

Bruce

bewing wrote:@WindowsNT:
Attached is my heavily modified preliminary version of your stuff.h code. It still uses parsing. Some of the changes: compiles/runs on Pre-XP Windows, change in disassembly window handling, bochs memory accesses are direct (no parsing), some speedups, some fixes.

Considering how much I changed it, I doubt that you will want to use any of my code. This version is still compatible with yours, and looks almost the same.

If you want to compile it, you need to make one change. In win32dialog.cpp, you want to get rid of #include "textconfig.h" and #include "siminterface.h" -- and you want to add #include "bochs.h". After you make this change, YOUR version of stuff.h will still compile and run normally.

Some bugs and issues that I saw and remember:
*Pre-XP versions of Win can't handle groupID's. Old compilers also don't have some of the
groupID functions/defines. With #ifdef's your code will compile and run on pre-XP.

*When bochs is compiled without 64b support, the EFER register is never displayed --
which blows up your state machine (ProcessDebuggerUpdate), because the Creg update is never completed.

*Some compilers have trouble with the "for (int i=0;" redefining i at the same bracket
depth multiple times (merely a compiler compatibility issue).

*In WM_SIZE: the GDT, IDT, and PAGE windows have a width of (float)rc.right*0.85f,
instead of (float)rc.right*0.45f
EDIT: woops! I forgot. I had moved these windows sideways. Your original size was correct for your original placement, the problem was all mine, created by me.

I added a bunch of comments, too.

____________

I was completely not careful about handling wchars. So I do not know if it will work properly on XP/Vista. If you would be willing to compile it and try it out -- especially the AskText dialog box, I think -- I'd be grateful.

I 've seen it. Some issues I want to discuss:

1. The work is generally very good. You still keep some of my vectors<> .

2. I see too many comments. Remember this code is not for educational purposes, so I find it somewhat reduntant to comment each line of code like if it would be sent to a beginners' class.

3. I haven't yet tried to test it because I am rather busy, I will tell you If i had compilation and/or runtime issues.

4. I don't like anything GNU-related applied to my code. That means that, upon releasing the source, I am happy to allow anyone to reuse it in anyway without being forced to comply with any form of GNU - related license agreement, in any open-source or closed-source, free or commercial project. I don't know if Bochs sources are under GPL or GLPL and therefore if all of its code should be under the same license, but (due to quite different status in Windows - you may excuse me I am a Windows developer!) , I would prefer it out of anything I can contribute.

1.

2. I usually write in ASM -- where it's important to comment a lot. It's a habit. So, I'll try to uncomment a bit of what I added.

3. It's OK. I'm about to post another prelim version, anyway -- that has absolutely no parsing at all. Then I'm going to modify the menus to add some options -- then my version unfortunately won't be compatible with your 2.rc file anymore. And then I'm going to get rid of the Input & Output windows, and the entire bochs internal debugger, and control the sim directly -- the way Stanislav is saying. So my version is going to end up very different than the way it is now.

4. I understand. However, Stanislav asked you if you if you were content releasing your code under LGPL, and you said "yes", so I did that to push the issue. All of Bochs is under LGPL. If you want to place the code under something less restrictive/other than LGPL, please say so. BSD? PD? If I don't stick a licence on it, Stanislav certainly will, when he releases it to 50 thousand users.

After posting a final version that is still compatible with yours, I will take the next version of the modified code, and post it on a different osdev thread, and let you have this thread back for yourself, OK?

bewing wrote:1.

2. I usually write in ASM -- where it's important to comment a lot. It's a habit. So, I'll try to uncomment a bit of what I added.

3. It's OK. I'm about to post another prelim version, anyway -- that has absolutely no parsing at all. Then I'm going to modify the menus to add some options -- then my version unfortunately won't be compatible with your 2.rc file anymore. And then I'm going to get rid of the Input & Output windows, and the entire bochs internal debugger, and control the sim directly -- the way Stanislav is saying. So my version is going to end up very different than the way it is now.

4. I understand. However, Stanislav asked you if you if you were content releasing your code under LGPL, and you said "yes", so I did that to push the issue. All of Bochs is under LGPL. If you want to place the code under something less restrictive/other than LGPL, please say so. BSD? PD? If I don't stick a licence on it, Stanislav certainly will, when he releases it to 50 thousand users.

After posting a final version that is still compatible with yours, I will take the next version of the modified code, and post it on a different osdev thread, and let you have this thread back for yourself, OK?

BTW, how about the requirements from disasm module you mentioned ?
AS I already told the disasm interface is stand alone and very easy to change - if you could avoid some hard work of parsing and etc with some disasm change - I would do it.

Thanks,
Stanislav

After looking into it a little further (but not completely, yet) it looks like disasm works very well for me.
The only disasm function I use so far is bx_disassemble.disasm(). (I named my object bx_disassemble, obviously, like in the internal debugger).

The internal debugger sets arguments 3 and 4 to (bx_address)(-1), so I do that also. I would like you to tell me that it is always OK to do that. I do not want to ever set arguments 3 and 4.

It is a slightly sad thing if I have to instantiate an entire disassembler object, just to use one member function -- because it's a fairly big object. But it looks like that is necessary?