Create programming language to develop emulators

[ves]

Hi all.

I plan to start such task, and next write an emulator. Advantage of such language is, that it will help to write emulators easily and fast, and it will be easily to improve next.
Note the following:
- C or assembler are not good choices, since the result product will be fast, but not easy to develop!
- VHDL is also not suitable for explained aim. Main future of presented language is the it can be easily to develop, not to be easily to implement in real system.

I will give one demonstration of my idea, give some demo code:

Code: Select all

cmd add b45; // command add has opcode byte value 45
reg acc b;   // define register accumulator
ref flags b;

template fl {zero_f:1;over_f:1; :6.}

func add
{arg1=get b;   // Read 1 byte after IP (instruction pointer)
 rez=arg1+acc; // Apply operation.
 acc=rez[1b];  // Store in accumulator register l.byte of rez
 flags.fl[over_f]|=rez[1b:1] // Set flag, if overflow
 if(0==rez) flags.fl[zero_f]=1;
}

Of course this is only pre-version of my language. Note that local variables as "arg1" and "rez" not require to declare, and also they are of "unlimited" size. If you implement the code in C you must caring about size of variables!

If someone want to help me to develop this language, please send me private message. Of course he/she must know x86 assembler (but here I hope that everybody know this).

[Solar]

Generic "new great project idea" flak. Don't take this personally: It's standard procedure to rattle a new idea to determine if it holds value or falls apart.

So let's see...

You want to replace the complexity of developing an emulator in an existing language to the complexity of developing a new language.

The net complexity will increase. C compilers, for example, are incredibly well-tested, well-documented and well-understood. None of these advantages will hold for your EmulatorLanguage.

There are well-established emulators around. How big is the demand for another one, written in your new language or not?

The existing emulator projects are not using your language. Since I don't see any of those projects doing a complete rewrite, your emulator language will not only be a highly specialized niche language, but even then only appeal to a tiny subsection of that niche market.

Most professional software engineers always have an eye on the ROI (return on investment) of a new technology: Will the experience of working with this technology increase my chances to keep my job, get a next / better one, getting a pay raise?

Contrary to your claim, building an emulator in C is not hard. Certainly no harder than developing a compiler (most likely in C) for compiling a language into an emulator.

What's the time-to-market for an emulator written in your language, showing that the whole concept is worthwhile? I estimate > 1 man-year to develop the compiler, and > 1 man-year to develop and debug the emulator. Assuming full-time work by professional-grade developers.

Public interest in your language will be very small, reducing the manpower available for development and testing to... well, pretty much yourself, I fear. That makes 2 man-years for you alone, that's what, 5+ years to-market? Not very enticing for others to join you in that endeavour.

Feel free to field counter-arguments. But this isn't a flamewar or one-upmanship or "you don't see the light, you suck". This is about proving why your project would be worthwhile.

[ves]

You want to replace the complexity of developing an emulator in an existing language to the complexity of developing a new language.

Yes true. If you have better tolls you can find advantage in the future, when it is necessary to develop many different emulators, and update them very fast. Of course to create language will be difficult and slowly, but very interesting thing.

The net complexity will increase. C compilers, for example, are incredibly well-tested, well-documented and well-understood. None of these advantages will hold for your EmulatorLanguage.

Yes you are right, about this point. It is not possible my idea to have only advantages!

There are well-established emulators around. How big is the demand for another one, written in your new language or not?

These emulators are final product. You can not improve them easily! For example can you told me an emulator, which support AMD secure virtual machine commands?

The existing emulator projects are not using your language. Since I don't see any of those projects doing a complete rewrite, your emulator language will not only be a highly specialized niche language, but even then only appeal to a tiny subsection of that niche market.

You are right it will be "tiny subsection". And this is planed in this project. Emulators with my language will be used for OS developers, where it will be needed to add some specific new functionality (this include some special statistics).

Most professional software engineers always have an eye on the ROI (return on investment) of a new technology: Will the experience of working with this technology increase my chances to keep my job, get a next / better one, getting a pay raise?

ROI will be good for use the language interpreter.

Contrary to your claim, building an emulator in C is not hard. Certainly no harder than developing a compiler (most likely in C) for compiling a language into an emulator.

It will be better to develop interpreter for my language. Constructed emulators will not be concurrent with existing emulators. Aims are completely different. Well know emulators must be effective and fast, and allays emulate existing systems. Emulators for my language will be used only to implement rearly used functionality (for example qemu does not support DRi x86 registers - just there is no OS which use these registers, and so this is not implemented (I check this about 3-4 years ago, but I suppose, that situation now is the same)).

What's the time-to-market for an emulator written in your language, showing that the whole concept is worthwhile? I estimate > 1 man-year to develop the compiler, and > 1 man-year to develop and debug the emulator. Assuming full-time work by professional-grade developers.

I suppose that it will be 3-4 man-year to develop the language (since this is interesting). And 1-2 (or max about 3) man-year to develop interpreter. Next the same 1-2 man-year to write suitable x86 emulator.

Public interest in your language will be very small, reducing the manpower available for development and testing to... well, pretty much yourself, I fear. That makes 2 man-years for you alone, that's what, 5+ years to-market? Not very enticing for others to join you in that endeavour.

I hope that I write in right place (this forum), where can find 3-4 peoples interested in my idea. If I try to write in another place - you are right - I can not find even one person!

Feel free to field counter-arguments. But this isn't a flamewar or one-upmanship or "you don't see the light, you suck". This is about proving why your project would be worthwhile.

Thank you for you replay. Such arguments as yours are very essential to demonstrate the idea of my project. Yes it is obvious, that business model is not so good (and I know this), but for some specific areas of develop this language is better solution.

There is many people, which like to develop you own OS (which usually read this forum) - they know that his products will be unusable, but they develop them. If one day someone of them decide that he/she want to improve assembler functionality of hit toy, then they can use emulator, with interpreter from my language. This will be the easy method add new "hardware" functionality (since my language will be developed exactly to do this easily), and almost only possible method to test his "improved" OS.

Only one essential problem will be that new language must be learn. That is why I plan to develop easy to learn language, and also to write many interesting tutorials.

[Solar]

We're getting close to the core now.

I don't see a need for "many different emulators", nor a need to "update them very fast". CPU architectures do not get updated very fast, and all that is really needed is one working emulator per architecture.

Which brings us to the next point.

For example can you told me an emulator, which support AMD secure virtual machine commands?

[...]

qemu does not support DRi x86 registers - just there is no OS which use these registers, and so this is not implemented (I check this about 3-4 years ago, but I suppose, that situation now is the same)).

I seriously doubt that it is more difficult to implement the two features you named in e.g. qemu, with the help of the people already on the project, who have the experience with the existing architecture of the emulator, with the language and the toolchain involved, than creating a language, toolchain, and developer base from scratch - than get to where qemu already is, and then implement those two features. Not to mention that your project would then have to continue to play catch-up with whatever improvements have been made in the years it took you to reinvent the wheel.

If you find it too difficult to extend qemu in these two regards, rest assured that creating a new language / toolchain / emulator is much more difficult than that, and in consequence, just as well beyond your reach.

And, trying not to put too fine a point to it, I've read two posts from you so far, and there is some doubt about the appropriateness of your existing skill set, foremost your ability to cope with technical English (i.e., reading books on compiler design / CPU architecture, writing documentation / tutorials for your new language / toolchain, etc.)

I don't want to discourage you or be a pain in your backside. I just want to give you every chance to realize that you might be about to bite off more than you can chew, before you sink much work into it and become really frustrated.

I will shut up now. No, I won't join your project - compiler construction is not my strong side, I don't find the project attractive, and anyway, like 99% of the people here, I have my own projects which are already eating all the free keyboard time I can spare.

[ves]

AMD secure virtual machine (SVM) is not just some feature to implement. This is big set of commands, with many specific futures in action. I read (and understand) AMD documentation, about SVM - it is written really well. So do not worry, about "my ability to cope with technical English".

Note, that you even not read my posts precisely. I plan to write interpreter not compiler. The idea of emulator is to be possible to use only for OS development. Hardware functionality development is also thing, which is interested to me, and easy to develop emulator is only possible option for me. I can not produce the real hardware.

I will give simple example. "Applying" stack overflow it is possible to change return address after ret instruction. If mark specific (d)word in stack in special way, then attempt to change this (d)word will be impossible, and will result on exception (not wrong return execution point). This can be implemented in some hypothetical processor, and test with emulator how good it work.

[Solar]

Note, that you even not read my posts precisely. I plan to write interpreter not compiler.

From the implementation standpoint, and for the sake of our discussion, this makes next to no difference. The complexity of your project doesn't change a bit. I would daresay, an interpreter has even less appeal due to performance considerations.

Hardware functionality development is also thing, which is interested to me, and easy to develop emulator is only possible option for me. I can not produce the real hardware.

Are you actually into hardware development, or is this just something you find interesting?

Ah damn, I said I would shut up...

[fronty]

Note, that you even not read my posts precisely. I plan to write interpreter not compiler.

From the implementation standpoint, and for the sake of our discussion, this makes next to no difference. The complexity of your project doesn't change a bit.

Let me disagree with this a bit. Writing an interpreter is definitely easier than writing an optimizing compiler. Writing an interpreter won't be so much easier than a simple compiler, but IMO there still is a difference, possible even a difference big enough to make some difference from the implementation standpoint.

[Solar]

Writing an interpreter is definitely easier than writing an optimizing compiler.

Yes, because you introduced the word "optimizing". Is the interpreter still easier if you add some JIT / hotspot optimizing so that it does not crawl?

[Tosi]

Your language is just a C-like pseudocode. Most programmers writing an emulator would prefer C in the first place because they're more familiar with it and it gives them more control. And writing an emulator is not that hard, whether in C or assembly. For a system like the NES I would say it's vastly simpler than writing an operating system or a compiler.

Most of the well-established emulators are open source. If you want to, you can modify the source to do what you want to and send the project maintainers a patch. If they like it they will apply it, if not, you at least have a version of the emulator that does what you want. As for adding support for rarely used functionality, why add it since its rarely used? The time could be better spent on improving the most commonly used features.

For emulating anything more complicated than the Sega Genesis, dynamic recompilation is almost a necessity. Modern systems might be able to handle an interpreted PSX or 8086 emulator, but that's about it. Adding features like variables of unlimited size increases the overhead and will make the emulation bloated and slow.

Also, how do you intend to handle operating system and architecture-dependent functionality such as video output, audio, and input? Even if your language is for x86 only, what about handling differences between the different operating systems in use and in development?

[ves]

Your language is just a C-like pseudocode. Most programmers writing an emulator would prefer C in the first place because they're more familiar with it and it gives them more control. And writing an emulator is not that hard, whether in C or assembly. For a system like the NES I would say it's vastly simpler than writing an operating system or a compiler.

Yes it is not so hard, but it is not very simple. If you want to extend assembler functionality often, then it is better, if you can do this action in very simple way. Just look carefully on my example - it is NOT pseudocode, since syntax and semantic are well defined!

Most of the well-established emulators are open source. If you want to, you can modify the source to do what you want to and send the project maintainers a patch. If they like it they will apply it, if not, you at least have a version of the emulator that does what you want. As for adding support for rarely used functionality, why add it since its rarely used? The time could be better spent on improving the most commonly used features.

It is not essential will my improvement used or no. It is essential to simplify my development process. For example I can start to write source for "AMD secure virtual machine" in my language immediately, just after read documentation (I can give example who is interested). If I do this for qemu, I must add many different places of qemu C source - and it is not so simple.
Another essential reason. If my language interpreter is improved, then it is possible for example to count CPU cycles for commands. So, in computer program for emulators must just include (in suitable syntax) any command how many cycles used. Very easy. If I want to do for qemu - OK it is also simple, but I must study the source. If someone want for another emulator - again he/she must study the source.

For emulating anything more complicated than the Sega Genesis, dynamic recompilation is almost a necessity. Modern systems might be able to handle an interpreted PSX or 8086 emulator, but that's about it. Adding features like variables of unlimited size increases the overhead and will make the emulation bloated and slow.

Again it seems, that you NOT read my post. Aim of emulator, which will be used with my language are only for OS development. Simple development is priority, not the speed of execution! This is the same to explain me - why to produce the truck, since cars are obviously faster speed and cheaper to produce.

Also, how do you intend to handle operating system and architecture-dependent functionality such as video output, audio, and input? Even if your language is for x86 only, what about handling differences between the different operating systems in use and in development?

You are right in this point. Hardware functionality (and also BIOS) must be implemented separately. These modules will be written in C. For example it is possible to use qemu code. So in my language must be planed suitable interface about this. This will be similar, as C language used external libraries (some of them are written in assembler). About operating systems - there is no problem. OS just used hardware, not modify it.

[eddyb]

Well, didn't read the whole thread word by word, but I think I have some advice:
Did you try C++ with some syntactic sugar? It's sweet, believe me. All those goodies ((variadic) templates, classes, (implicit) conversions, operators etc.) .

[ves]

Well, didn't read the whole thread word by word, but I think I have some advice:
Did you try C++ with some syntactic sugar? It's sweet, believe me. All those goodies ((variadic) templates, classes, (implicit) conversions, operators etc.) .

This is really good idea. So it will not require to write interpreter. So, the problem will be only to write C++ library. But I am not so familiar with C++. Do you want to help me to change syntax from my language to C++ style. First I will try to write specification about my language.
Using C++ framework is better, than require people to study specific language.

[Solar]

Oi, oi, oi...

Never set out to learn any language X while implementing YourBigProject.

Start small. Come up with a pet project, something small and manageable, and use it as a "playground" to learn the new language.

Then do the same with a bigger, less trivial, yet still "pet" project.

Continue until you can say that you have "mastered" the language (i.e., use it with confidence and competence).

Then start YourBigProject.

Believe me. Diving head-first into unknown depths will result in failure and frustration.

(At the very least, you should come up with some architecture plan for your project. You cannot come up with a good architecture without having an idea of what the implementation language offers you. Yes I know I just kicked the shins of many Software Architects out there. Hint: You deserve to be kicked. )

[ves]

Ah damn, I said I would shut up...

Solar please stop to spam anymore. Nobody ask you about your "intelligent" opinion. Only your first replay has any meaning!!!

[ves]

Yeah, don't listen to Solar, ves. Jump straight ahead.

I'm asking about only one favour though: post your thoughts during development process on some blog (not here, please). Should make a good laugh for me from time to time.

I already do this. But I will not told you the URL. Laugh is type of entertainment, and so it is not for free .