OSDev.org

Hi again.
In that case I trust you to use your modifications in the best way and perhaps the Bochs team will consider using your variation because it seems to be C- oriented (as most *nix projects).

bewing wrote: @Stanislav:

I tried to turn off the A20 stuff in bochs -- but there is a problem in cpu.cpp when doing that. The A20ADDR function is not defined, but on lines 660 and 663, the function is used with no #ifdef around it.

A20aDDR macro is always defined, no matter if A20 supported or not:

#if BX_SUPPORT_A20
# define A20ADDR(x) (bx_phy_address(x) & bx_pc_system.a20_mask)
#else
# define A20ADDR(x) (bx_phy_address(x))
#endif

Stanislav

WindowsNT wrote:Actually, what is needed is to find a class-based communication to query the CPU state. Currently, my code parses bochs output, which means that the slightest change in the output format will render my code obsolete. I need a function that will return the entire cpu state in a big class; and then everything will be OK.

For example, return a CONTEXT (winnt.h) to describe the CPU state, and provide a way to query or set a memory location. Then everything will be much simpler and portable.

You already have a 'class' or more correctly 'a tree' of ALL Bochs internals.
Bochs internal debugger has an object bx_list_c *dbg_cpu_list which contains ALL info about currently debugged CPU.
This list is only subtree of the bigger tree, containing data about ALL bochs CPUs and devices.
I suggested you from the begginning to have a tree view of the registers in the registers view window instead - this way your Window updates itself automaticallt when I add some register or bitfield to emulator.
Functions to quary and set memory location are also pretty styandart. If you running with Bochs debugger - you are allowed to use its interfaces dbg_set_mem and bx_dbg_read_linear.
Sounds like you could make your code 10x simpler now

Thanks,
Stanislav

bewing wrote:Not directly. However, it accepts any command that the text debugger accepts. And one of those commands is for spooling commands from a file, AFAIK.

@WinNt: Yes, I agree. It looks like you have the beginnings of that with your SIM-> etc. stuff. But parsing is good enough for now, and I am simplifying the parsing code. If Stanislav will create a nice interface into the CPU state (especially Mode, and register values), and a nice interface into the physical/linear memory models (allowing read & write) then this code will be simpler and nicer.

I think I'll have a modified version (with expanded features, and a bugfix) by this weekend. I have a lot of testing to do, to make sure I haven't broken anything -- and I do not currently have the ability to test Long Mode in bochs.

Hi All,

When all this thread just started I told that this is very good opportunity for all you to define Bochs debugger inetrfaces. It is possible now to open any inerfaces and add accessors to the CPU and devices state and memory. I accept any proposal noww. If your proposal is good enough - it is going to be integrated into Bochs code very soon.
BTW, if you will have updated version, could you please make it kind of standard 'patch' format - add new filees and for every existsing modified files procide 'diff -up' kind of patch. This will significantly speed up Bochs code integration.
Once the patch is available I'll post it into Bochs mailing list and Bochs patches tracker.

Thanks,
Stanislav

To post a bochs patch, we would have to agree on the exact version of the bochs software that we were applying the patch to, wouldn't we? Or would you promise to stop making any changes in win32dialog.cpp for a little while?

I will try to post my modifications for test purposes tomorrow -- especially to see if WindowsNT will decide not to integrate my code into his. This version of the code will still use parsing, I think.

Yes, Stanislav, I have seen that much of the CPU state is easily available from the tree, and my next version (of my version of WindowsNT's code ) will stop using parsing to recover register values. However, I currently do not understand how to distinguish the number of bytes and format that are stored for the tree version of each register. I also need to look further into seeing how to recover blocks of emulated memory with the function you specified.

But parsing will still be necessary for disassembly dumps, AFAIK. And perhaps also useful for breakpoint dumps.

bewing wrote:To post a bochs patch, we would have to agree on the exact version of the bochs software that we were applying the patch to, wouldn't we? Or would you promise to stop making any changes in win32dialog.cpp for a little while?

I will try to post my modifications for test purposes tomorrow -- especially to see if WindowsNT will decide not to integrate my code into his. This version of the code will still use parsing, I think.

Yes, Stanislav, I have seen that much of the CPU state is easily available from the tree, and my next version (of my version of WindowsNT's code ) will stop using parsing to recover register values. However, I currently do not understand how to distinguish the number of bytes and format that are stored for the tree version of each register. I also need to look further into seeing how to recover blocks of emulated memory with the function you specified.

But parsing will still be necessary for disassembly dumps, AFAIK. And perhaps also useful for breakpoint dumps.

I suggest to take recent CVS snapshot
But if talking about GUI and files like win32dialog.cpp - they didn't changed AT ALL since last 2.3.7 release and, yes, I promise not to change them - I anyway don't know how to that
BTW, current CVS contains some buggy not working piece of code called win32debug as well, I suggest you to clean it out from the code.

I also hope you'll agree on the implementation and post the updated patch soon !

P.S> About printing the param tree you could look onto print_tree function in main.cc.
About parsing of disasm data - it is also possible to add accessors to the 'insn' structure and avoid unnecessary parsing. You are welcome to define interfaces !

Stanislav,

stlw wrote: Bochs internal debugger has an object bx_list_c *dbg_cpu_list which contains ALL info about currently debugged CPU.
This list is only subtree of the bigger tree, containing data about ALL bochs CPUs and devices.

Yes, but the correct "parameter_id"s for the interface are extremely hard to find.

stlw wrote: Functions to query and set memory location are also pretty standard. If you running with Bochs debugger - you are allowed to use its interfaces dbg_set_mem and bx_dbg_read_linear.

To the best that I can tell, I am NOT allowed to use those interfaces. If I try to compile either of those functions into the GUI code, the compiler says that they are unknown external references.

I am fairly certain that I am supposed to use the siminterface, somehow, to do the memory query and set functions. But I have no idea what the parameter_id's are for the functions to set and get physical memory addresses, or for the function to convert laddr's to paddr's for the current cpu.


Edit: OK, I cheated really bad and compiled bochs.h into the CI file win32dialog.cpp -- so now

BX_CPU(dbg_cpu)->dbg_xlate_linear2phy(laddr, &paddr);
and
BX_MEM(0)->dbg_fetch_mem(BX_CPU(dbg_cpu), paddr, len, buf);

seem to work. But as far as I can tell from the explanation in siminterface.h, there should not be any includes of bochs.h in any CI file (such as dbg_main.cpp).
-- When you ask us to describe the interface that we want, are you talking about rewriting siminterface? I certainly wouldn't mind if you did ... it is not nice to use. But it would sure be a lot of work!

Thanks,
Bruce

Well, the "correct" Win32 way to do it is to define a class and export it through either a global function or through COM. I don't really see why you are obsessed with C, except perhaps that the Gnu compiler is a good C compiler but a very bad C++ one. I feel miserable when I see code like this:

BX_CPU(dbg_cpu)->dbg_xlate_linear2phy(laddr, &paddr);

It might be because I am a Windows programmer of course. If I had to create a debugging interface, I would create it from scratch in the following, really more efficient and simple way:

class DEBUG
{
public:
// ...
void GetRegs(CONTEXT*);
void SetRegs(const CONTEXT*);
void GetMemory(unsigned __int64 From,unsigned __int64 Bytes,char* dst,bool IsAddressLinear);
void SetMemory(unsigned __int64 From,unsigned __int64 Bytes,const char* dst,bool IsAddressLinear);
// .....
};

DEBUG* GetDebugInterface()
{
// ...
}

WindowsNT wrote:Well, the "correct" Win32 way to do it is to define a class and export it through either a global function or through COM. I don't really see why you are obsessed with C, except perhaps that the Gnu compiler is a good C compiler but a very bad C++ one. I feel miserable when I see code like this:

BX_CPU(dbg_cpu)->dbg_xlate_linear2phy(laddr, &paddr);

It might be because I am a Windows programmer of course. If I had to create a debugging interface, I would create it from scratch in the following, really more efficient and simple way:

class DEBUG
{
public:
// ...
void GetRegs(CONTEXT*);
void SetRegs(const CONTEXT*);
void GetMemory(unsigned __int64 From,unsigned __int64 Bytes,char* dst,bool IsAddressLinear);
void SetMemory(unsigned __int64 From,unsigned __int64 Bytes,const char* dst,bool IsAddressLinear);
// .....
};

DEBUG* GetDebugInterface()
{
// ...
}

You, I mean to redefine the interface.
But not in 'WINDOWS-friendly' way but in portable way. Such that the same inetrface would be good for connecting any external debugger into Bochs.
But as I already told, I am not an expert in connecting external interfaces and even not sure I know that their needs.
But I know very well all Bochs CPU/MEMORY and debugger and could code the complete interface in a few days once it is defined.
Yes, I would take a part in new inetrfaces definition but certainly could not define them alone. You should be the drivers for this !
Please start discussion of that do you need from the inetrfaces and why and we will reach something useful very shortly.
If you keep it blindly as today - you will get January Bochs 2.3.8 release with no changes in the debugger.

Thanks,
Stanislav

There is nothing weird. I need exactly the class I posted.

1 function to read all registers, 1 function to write registers, 1 function to read memory and 1 function to write memory.
The registers structure is a collection of the registers BOCHS support. For registers over 64-bit, there can be a data type.

struct DATA_8
{
char d[8];
}; // etc

struct CONTEXT
{
unsigned __int64 RAX;
...
DATA_8 GDT;
DATA_16 LDT;
DATA_32 IDT;

};

@Stanislav: Heh. First, I have to get my version finished, to see what it is that I actually need.

The first big question is whether the debugger will always be compiled together with Bochs into the bochs executable?
If so, the code can use #ifdefs regarding whether optional features like MMX, SSE, FPU, 64bit, etc are actually supported in the build. If the debugger needs to test each of those things from the interface, it needs more things than I listed.

But basically, it seems that I need:
1. A function to read and set physical memory addresses.
-- Note: it is best if it can copy a large number of bytes (without crossing a 4K page boundary, of course).
-- It would be acceptable (but painful) if the copy process had to be done 1, 4, or 8 bytes at a time.
2. The CPU index of the CPU that is currently being simulated.
3. The total number of CPUs.
4. A function to convert a linear address to a physical address for a particular CPU.
5. A signal from the simulator, every time bochs gets to a "break" of any kind.
6. A way to force the simulator to break.
7. A way to force the simulator to quit.
8. A function to disassemble the next mnemonic at a linear address -- it should return the bytelength, and the disassembly as a string. I can pass a buffer for the string. The disassembly function would need to handle all the "guard" stuff ITSELF.
9. A count of the total maximum number of breakpoints -- so I can preallocate a buffer.
10. An array or enumeration of all the breakpoints -- addresses, type, index number.
11. A count of the total maximum number of watchpoints -- so I can preallocate a buffer.
12. An array or enumeration of all the watchpoints -- addresses, length, type, index number.
13. A function to add and delete breakpoints.
14. A function to add and delete watchpoints.
15. A tree (or whatever) that will tell me all the available emulated registers.
16. Functions that will read 32bit and 64bit registers, for a particular CPU.
-- Note: if possible, I want to get a function pointer, or something, that I can store and use repeatedly to get/set the value of a register. I do not want to have to search the tree every single time I try to read a register.
17. Functions that will set any registers that are allowed to be set, for a particular CPU.
18. Something that will tell me if I should be using 16, 32, or 64 bit addressing for a particular CPU. (It changes my formatting.)
19. Total amount of emulated physical mem.
20. If textconfig is still going to be part of bochs, I need a function that will send user commands to it.
21. A signal from textconfig, when it is ready to accept a new command.
22. Again with textconfig -- if it creates output, I need a signal that there is output to read, and a way to get it.
23. ptime
24. proceed, step (#), and continue commands
25. A function that will convert CS:RIP/EIP into a linear address for a particular CPU -- to handle unReal/Real mode craziness.


Optional things that would be nice:
1. A flag on each CPU to let me know if its mode had changed since the previous break.
2. A flag on memory to let me know if memory had been modified since the last break.
3. A ping function that would let me know that the sim has not crashed/gone into an infinite loop.

-- That's all I can think of right off the top of my head. Maybe enough to start a discussion.


@WindowsNT: The code that I listed is not mine -- it was lifted directly from debug_main.cpp. I think it is just as ugly as you do.
However, you should know that when a C++ compiler does its magic on your C++ classes -- the kind of C that I listed is EXACTLY what the compiler turns your code into. You may choose to hide from that fact, if you wish.

Also, this code is what already works with bochs. We are too late to affect that. And the siminterface is ten times worse.
And why do I prefer C so much over C++? Because it's 10 to 50 times faster with typical coding. It's not nice to waste all of a user's CPU power, like your program is the only one that will ever run on their machine.

bewing wrote:@Stanislav: Heh. First, I have to get my version finished, to see what it is that I actually need.

The first big question is whether the debugger will always be compiled together with Bochs into the bochs executable?
If so, the code can use #ifdefs regarding whether optional features like MMX, SSE, FPU, 64bit, etc are actually supported in the build. If the debugger needs to test each of those things from the interface, it needs more things than I listed.

But basically, it seems that I need:
1. A function to read and set physical memory addresses.
-- Note: it is best if it can copy a large number of bytes (without crossing a 4K page boundary, of course).
-- It would be acceptable (but painful) if the copy process had to be done 1, 4, or 8 bytes at a time.
2. The CPU index of the CPU that is currently being simulated.
3. The total number of CPUs.
4. A function to convert a linear address to a physical address for a particular CPU.
5. A signal from the simulator, every time bochs gets to a "break" of any kind.
6. A way to force the simulator to break.
7. A way to force the simulator to quit.
8. A function to disassemble the next mnemonic at a linear address -- it should return the bytelength, and the disassembly as a string. I can pass a buffer for the string. The disassembly function would need to handle all the "guard" stuff ITSELF.
9. A count of the total maximum number of breakpoints -- so I can preallocate a buffer.
10. An array of all the breakpoints -- addresses, type, index number.
11. A count of the total maximum number of watchpoints -- so I can preallocate a buffer.
12. An array of all the watchpoints -- addresses, length, type, index number.
13. A function to add and delete breakpoints.
14. A function to add and delete watchpoints.
15. A tree (or whatever) that will tell me all the available emulated registers.
16. Functions that will read 32bit and 64bit registers, for a particular CPU.
-- Note: if possible, I want to get a function pointer, or something, that I can store and use repeatedly to get/set the value of a register. I do not want to have to search the tree every single time I try to read a register.
17. Functions that will set any registers that are allowed to be set, for a particular CPU.
18. Something that will tell me if I should be using 16, 32, or 64 bit addressing for a particular CPU. (It changes my formatting.)
19. Total amount of emulated physical mem.
20. If textconfig is still going to be part of bochs, I need a function that will send user commands to it.
21. A signal from textconfig, when it is ready to accept a new command.
22. Again with textconfig -- if it creates output, I need a signal that there is output to read, and a way to get it.
23. ptime
24. proceed, step (#), and continue commands


Optional things that would be nice:
1. A flag on each CPU to let me know if its mode had changed since the previous break.
2. A flag on memory to let me know if memory had been modified since the last break.
3. A ping function that would let me know that the sim has not crashed/gone into an infinite loop.

-- That's all I can think of right off the top of my head. Maybe enough to start a discussion.


@WindowsNT: The code that I listed is not mine -- it was lifted directly from debug_main.cpp. I think it is just as ugly as you do.
However, you should know that when a C++ compiler does its magic on your C++ classes -- the kind of C that I listed is EXACTLY what the compiler turns your code into. You may choose to hide from that fact, if you wish.

Also, this code is what already works with bochs. We are too late to affect that. And the siminterface is ten times worse.
And why do I prefer C so much over C++? Because it's 10 to 50 times faster with typical coding. It's not nice to waste all of a user's CPU power, like your program is the only one that will ever run on their machine.

All your suggestions are good. But I am not going to argue about using C++ or C, you have your opinion, I have mine. It is pointless to continue because we are used in quite different operating systems and compilers (C++ is a superset of C and it has exactly the same speed if the same code is used, and if classes are used, then the header overload is equal to the C's overload in parameter passing. If your compiler generates slower C++ code, that's a problem of the compiler, not a problem of the language). As for the class exporting, who said anything about separated processes ? Bochs can load a plugin ( a DLL in case of Windows) and provide the class pointer.

You may just want to try speed testing our two versions against each other, if you think C++ is efficient. Or, just try singlestepping your way through some C++ code that does a copy of a szCharstring into a vector<string>. And remember that a call and ret slow down your CPU a loooooong way, because the instruction queue gets trashed. You will step through 25 or 50 unnecessary call/rets.

Oh, and you should REALLY read through the text at the beginning of siminterface.h -- before we go much farther about whether the debugger will be separated from the sim.

bewing wrote:You may just want to try speed testing our two versions against each other, if you think C++ is efficient. Or, just try singlestepping your way through some C++ code that does a copy of a szCharstring into a vector<string>. And remember that a call and ret slow down your CPU a loooooong way, because the instruction queue gets trashed. You will step through 25 or 50 unnecessary call/rets.

Oh, and you should REALLY read through the text at the beginning of siminterface.h -- before we go much farther about whether the debugger will be separated from the sim.

I never said that the copy of a char* into a string won't take time. Of course it will take time. Of course, I do not expect compilers such as GNU C to be good in C++, because they are not C++ compilers and therefore you can expect the result to be bad (If you are comparing C code with C++ code with the GNU compiler, the comparison is totally unreliable). But for that time spent in a decent C++ compiler, you get quite other benefits (STL) that we need not discuss here. If you think that avoiding STL for a big project just to make it 0,001% faster will allow you to end up in a good result, I am sorry but I do not agree. Experience has proven that, if an application is slow, its the programmer's fault and not because its written in C++ instead of C. And, if you want to use the char*, you have no problem because the char is also C++. C++ is not just C with classes or STL - it is way more.

Also, I am really interested in the end result , and the end result is that I created a gui with C++ (which actually uses classes only for vector<>, the interface itself is not any class-based, it is Win32) that works and makes my life (and hopefully a lot more people's) easier. If there was a C++ interface in the way I described it, or with the features u added, that would make my code /3 in size.

If you claim that it would be easier and/or faster to make it in C, why didn't I see any C-style GUI yet ? Why lack of a Linux version? Before saying that implementation of something in X is bad, you must prove it by implementing it in Y.

And note that all this discussion is not to be offensive or criticize what you are saying, because here all users should be experienced and exchanging opinions in OS development. But I really care only for the end result.

Putting the new stuff into a class again:


class DEBUG
{
public:
// ...
void GetRegs(CONTEXT*);
void SetRegs(const CONTEXT*);
void GetMemory(unsigned __int64 From,unsigned __int64 Bytes,char* dst,bool IsAddressLinear);
void SetMemory(unsigned __int64 From,unsigned __int64 Bytes,const char* dst,bool IsAddressLinear);
unsigned int CpuNum();
unsigned __int64 PhysicalToLinear(unsigned __int64 a);
unsigned __int64 LinearToPhysicalunsigned __int64 a);
void SetBreakNotification(void (*)(DEBUG*,unsigned __int64 lParam),lParam);
void Break();
void Die();
void GetInstructionDissassembly(unsigned __int64 AtAddr, bool IsAddressLinear,char* rslt);
void GetBreakpoints(vector<BREAKPOINT>& brks);
void SetBreakpoint(const BREAKPOINT&); // a BREAKPOINT can also be a watch point
unsigned int GetCPUMode(); // Say Bit 0 for PM , Bit 1 for paging, Bit 2 for Long
void GetMemoryInformation(MEMORY_INFORMATION*); // Should include total physical, virtual etc
void Step();
void StepInfo();
void StepOut(); // to exit from function calls


// .....

};