OSDev.org

(Tangental info so we don't get derailed challenging the analogy: its now accepted that narrow roads and narrow lanes are much safer than wider ones in (sub)urban settings. http://www.citylab.com/design/2014/10/w ... ow/381117/ is a popular science intro to the topic but there are serious papers backing it all up too.)

Hi,

embryo wrote:

Brendan wrote:It's because developer time is less important than end user time.

Yes, Brendan, in an ideal world it is. But in our "not so ideal" world there is a need for automation of every possible developer's move, even if it costs much more for end users.

You still fail to see that "unmanaged" can provide the same automation, with far lower cost for end users.

Also note that we are not web developers who are forced to use javascript because that's the only thing that all web browsers support, we are not games developers who are forced to choose between DirectX and OpenGL because they're the only things the target platform provides. We are not maintaining old banking software that was written 40 years ago and forced to keep using COBOL. We are not iPhone application developers forced to use Objective-C because that's what the OS developer wants (or Android application developers forced to use Java).

We are OS developers. We create our own worlds.

embryo wrote:And in essence we are talking about automation limits. You limit it by the phrase "no performance loss". But the world still wants unlimited automation, would it cost some performance or not. And the progress in languages and software environments just reflects the demand. It creates new software with better automation and cares too little about performance. For example the very popular JavaScript framework node.js is just a horrible performance killer (even if it is claimed as very fast), but it's use is so popular for high traffic sites, that it already has outperformed Google's back-end.

There is nothing that "managed" can automate that "unmanaged" can't; so this is all pure nonsense.

embryo wrote:May be sometime you will invent a way for old styled developers to catch the market share and show the world the best possible quality, but the world looks scornfully at such efforts and it is up to you to fight the Goliath, if you have enough time and courage.

If all the incompetent programmers were tricked into going to a special island in the middle of the Pacific Ocean where they're prevented from doing any programming that matters (and only able to do disposable trash - e.g. web sites and smartphone apps), then that'd be awesome. The single best thing about managed languages is that it has a similar effect.

The problem is that it's still possible for these programmers to escape from the locked down enclosures and attempt things like OS development. It's sort of cute in a way - like watching a child trying to build a real building with a toy hammer made of rubber and left over cardboard boxes - until that child finds their way into a real construction site where safety is still important but "100% safe" is impossible due to the nature of the work.


Cheers,

Brendan

MessiahAndrw wrote:In my experience, bugs tend to be more frequent in managed languages because they're perceived to be safer.

Have you ever participated in any complex C-based project?

MessiahAndrw wrote:Yet which one is safer? The one that is perceived to be unsafe, because people are driving slow and cautiously because they're expecting the unexpected (like people crossing or a child running out).

Your "safety" approach is about prohibiting all roads. It will be as safe as nobody ever dreamed. But it will require you to go into a forest and chop some fuel. And next the fuel should be taken to your house, without any road, of course, because they aren't safe.

Now can you tell us, which way is "safer"?

Brendan wrote:You still fail to see that "unmanaged" can provide the same automation, with far lower cost for end users.

You still fail to see that "unmanaged" can provide the same automation by closing the gap between managed and unmanaged. And lower cost for end users is not the actual goal of any business (surprise?), because our world in not an ideal.

Brendan wrote:Also note that we are not web developers who are forced to use javascript because that's the only thing that all web browsers support

There are compilers. And they compile a lot of languages into a form of JavaScript. Also there are Java applets. Next there is ActionScript from Adobe (supported by it's Flash engine). And finally there are Microsoft's ActiveX and other C-based browser extensions. So, web developers had the choice and had made the decision - just to forget about any ActiveX or anything similar.

The performance problem is not in the script itself, but in the developer's habits. And such habits are required by business, just because such developers can deliver a product quickly (much faster and cheaper, than if compared with C-developers). And it is the most important truth you have missed.

Brendan wrote:We are OS developers. We create our own worlds.

Yes. We create. And we can even talk about how cool our world is. But does anybody care? We can spend years and deliver really wonderful OS, next we can spend even more years and deliver a lot of talks about wonderful OS. But does anybody care?

The market share is telling us the truth - managed environments win, simplification matters, fast time to market matters, users are brainwashed with heavy advertising and buy things, that "old style" developers call "crap".

And I'm not about the beauty of such world, but I'm about your touch with reality.

Brendan wrote:There is nothing that "managed" can automate that "unmanaged" can't; so this is all pure nonsense.

Well, then automate anything at runtime without managed environment. And please, don't call your OS unmanaged, if it is able to manage code at runtime.

Brendan wrote:If all the incompetent programmers were tricked into going to a special island...

Oh yes, if everything in the world were just as we wish...

Brendan wrote:The problem is that it's still possible for these programmers to escape from the locked down enclosures and attempt things like OS development

But it seems to me that the problem is in a very sophisticated imagination, that is able to put all "bad" things "to a special island", but just can't imagine itself in a real world.

embryo wrote:

MessiahAndrw wrote:In my experience, bugs tend to be more frequent in managed languages because they're perceived to be safer.

Have you ever participated in any complex C-based project?

Yes, does VLC count? You can write clean modular code in any language.

embryo wrote:

MessiahAndrw wrote:Yet which one is safer? The one that is perceived to be unsafe, because people are driving slow and cautiously because they're expecting the unexpected (like people crossing or a child running out).

Your "safety" approach is about prohibiting all roads. It will be as safe as nobody ever dreamed. But it will require you to go into a forest and chop some fuel. And next the fuel should be taken to your house, without any road, of course, because they aren't safe.

Now can you tell us, which way is "safer"?

Huh? Where did I ever talk about abandoning all modern advancements? You can have GUI designers and step-by-step debuggers and all of those cool things in an unmanaged environment.

Delphi is a popular IDE based on Object Pascal - compiles to a native unmanaged environment:


Want an unmananged language with a large standard library? Try Go.

Do you like having a debugger that lets you connect to a program running on a machine over the network and tinker with values? Visual Studio's C++ debugger:


Visual Studio even offers REPL for C++, that you can execute at any point while a program is running:


Oh yeah, there's even Edit and Continue for C++. No one's making you go back to the stone age.

To explain my analogy: the unmanaged environment is safer because it's perceived to be unsafe, so we proceed cautiously expecting the unexpected. (In the streets example these unexpected things can be pedestrians, children playing, people in wheelchairs, bicyclists, segways, etc.)

As a programmer, we know there's no bounds checking, so we get into the habit of validating our data ourselves - in ways that go beyond simply bounds checking our arrays.

If most of your errors are memory leaks and out of bounds array indices then I'm afraid you're getting stuck on beginner bugs and you're not as good of a programmer as you think.

The worst thing that destroys productivity is not the language (unless you're purposely using some esoteric language), but the use of anti-patterns. Here are some examples I deal with:
1) I see so much lasagna code. This one project has 7 layers of interfaces between clicking a button in the GUI and actually writing data to the database, including calling multiple webservices. What better way to demotivate a programmer than one simple client requested change like "make a dialog pop up notifying the user if they're trying to create two of the same items, with an option to jump to other item" means taking a week to pull apart half the program to figure out what the heck it's doing.
2) Instead of simply saying "if(age < 25) return false;" you'll create a 'validator' class that implements IValidator, and creating a new IValidator class means updating the web service's interface. And if you want to change that logic to "if(age < 25 && gender == Gender.Male) return false;" you need to create a seperate AgeAndGenderValidator class and rebuild all 6 libraries that reference your core library that contains the Validator classes.

Yes, you can write code that's clean and easy to work with in a managed language, and you can write horrible code in an unmanaged language. Yes, anti-patterns aren't unique to managed languages, but in my daily experience in the corporate world, they seem to be far more prevalent in managed languages. But, anti-patterns slow down my productivity a whole lot more than simple "out of bounds" bugs, and often anti-patterns that result in action at a distance makes it a lot harder to find bugs when they do occur.

For the record, I personally like C#.

I'd say that the best way to summarize my answer to this would be Ousterhout's dichotomy. My opinion is that there are actually 3 types of languages: systems programming languages, applications programming languages, and scripting languages.


Details
-------
Systems programming languages should be statically typed, provide some interface for manual memory management, and can interact with hardware.

Applications programming languages can be statically or dynamically typed, do not require manual memory management, and should be abstracted from the system. Loosely speaking, if the code is managed or running in a virtual machine (eg. languages like VB, C#, or Java) than it is probably a good applications programming language.

Scripting languages should be dynamically typed, not allow manual memory management, and be capable of embedding into applications. Scripting languages should be used for light work only. If you need to perform a lot of work, then you should probably be writing whatever it is you're doing in an applications programming language; and using a scripting language to glue the components together if needed.
-------


You should use all 3 types of languages for writing operating systems. Picking the right tool for the job is important. A systems programming language like C for interacting with hardware, an applications programming language like C# for your tools, and a scipting language like TCL for connecting your tools together.


Edit
Something I forgot to mention, while I did say that there are 3 types, it's really more of a spectrum. For example, C++ falls between systems and applications and Python falls between applications and scripting. While these languages are often more flexible, the lack of specialization leads to certain complications. C++ requires you to have extern C stubs to do certain systems oriented things, and adds a lot of complexity with reference and pointer types to bridge the gap between systems and applications programming. Python is too dynamic to write large applications in and too large to be a suitable scripting language.

Don't get me wrong, they're both great languages. But, to keep your projects as clean as possible, only use systems programming languages when you need to interact with hardware or use platform specfic code and use applications programming languages for most of anything else so your code is platform independent without having to rely on complicated configuration setups. Only use scripting languages to automate the use of applications code.

Here are some things that would increase my productivity as a developer (regardless of language, or if it's in a managed or unmanaged environment) in an applications language:

- A way to test out small snippets of code and experiment with algorithms I have in my head quickly. Being able to compile and run a single file with all of my IDE tools (the graphical debugger) without the IDE creating a whole project directory structure and making me mess around with libraries (.jar files, download/extract/build/include libraries etc.) just to run a small bit of code once!

- Edit and continue. If I see an issue in my long running program, I should be able to correct the bad variables and the code, and continue running my program without restarting it.

- A debugger that understands complex types (This object is a bitmap image, can I visualize it? This object is a sound bite, can I play it?) Some libraries could come with an IDE plugin that understands how to view the types that come with the library.

- REPL feature while the executing code is frozen, in the local scope of where the code is frozen.

- A profiler that can show me what parts of my code took the longest to run and what functions were called the most.

- An IDE that doesn't get in my way while typing. If I'm typing out some source code, the functions I'm referencing might not exist yet (because I'm about to write them), so don't start auto-correcting my input as I type! I don't mind it underlining errors as I type or code completion when I CTRL+Space, as long as it doesn't start doing stuff to interfere with me if I'm just typing normally.

- Some kind of package manager for searching for, downloading, and including the most popular libraries for the language. It shouldn't randomly break with a compile error while attempting to import a library.

- If I've never used the language before, the IDE should have the option to come bundled with the tools needed to build 99% of projects written in that language out of the box (and it can automatically resolve most library dependencies by downloading them if needed).

MessiahAndrw wrote:Yes, does VLC count?

Depends on participation value.

MessiahAndrw wrote:You can write clean modular code in any language.

For only small or simple programs.

MessiahAndrw wrote:Where did I ever talk about abandoning all modern advancements?

You can reread your previous message.

MessiahAndrw wrote:You can have GUI designers and step-by-step debuggers and all of those cool things in an unmanaged environment.

"all of those cool things" include runtime code change, for example. Please, show me how it is accomplished in case of unmanaged language.

MessiahAndrw wrote:the unmanaged environment is safer because it's perceived to be unsafe, so we proceed cautiously expecting the unexpected

If a car can go 100 miles per hour and a distance is 200 miles, then how soon I can get to the desired location? It depends on how cautiously I will proceed. On a highway I'm sure that I can get where I want in 2 hours. But if in a city, then... In short - do you want to be where you want much quicker? And if you are so scared of "unsafeness" of a highway, then nobody prevents you from driving at 10 km/h (but don't forget to turn on warning lights).

It is the same for languages. If you want some kind of a quality, then you can write your code with caution. And if you write your code in a careless manner, then why do you blame the language, but not yourself?

MessiahAndrw wrote:As a programmer, we know there's no bounds checking, so we get into the habit of validating our data ourselves - in ways that go beyond simply bounds checking our arrays.

If you do not have exactly the same habit when use a managed language, then you are a bad programmer. Just because your programs always will need to be tested more extensively than programs of other programmers. And after tests your programs should be changed more often. And finally your performance is very weak comparing to other programmers, so, your salary will reflect it.

It means you just point to a wrong problem. Instead of pointing to a difference you are pointing to a similarity. It's like if I point that both, the Java and C programs, are written by programmers. Is it a useful information?

MessiahAndrw wrote:If most of your errors are memory leaks and out of bounds array indices then I'm afraid you're getting stuck on beginner bugs and you're not as good of a programmer as you think.

If most of your thoughts are about a bad code, then I'm afraid you still do not understand programming. Most of my thoughts are about how to speed up the process of writing a good program. Writing a program is much more complex activity than typing some signs using a keyboard (a monkey level efforts).

MessiahAndrw wrote:The worst thing that destroys productivity is not the language (unless you're purposely using some esoteric language), but the use of anti-patterns.

There are two things, in fact. First is the programmer's experience. It allows some people to use any suitable pattern efficiently (even if it is called with the prefix "anti"). The second is the help from some automation. The managed vs unmanaged is about the second, but not about the first, as you wrongly suggest.

MessiahAndrw wrote:Instead of simply saying "if(age < 25) return false;" you'll create a 'validator' class that implements IValidator, and creating a new IValidator class means updating the web service's interface. And if you want to change that logic to "if(age < 25 && gender == Gender.Male) return false;" you need to create a seperate AgeAndGenderValidator class and rebuild all 6 libraries that reference your core library that contains the Validator classes.

The program quality includes a lot of things. The programming is a complex process. And when you see some examples that make you feel bad, it can mean two things. First - you do not understand the real purpose of the features you are enforced to use. Second - the complexity of the programming process became visible to you in some form and you are eager to claim that you see how to make it better. But exactly the same thought was excited an architect, which had designed the architecture you are disliking now. And it is very improbable that if you take a position of an architect, the programming process will be less complex and your disliked program will not break in some other place.

So, you have to create something really useful first, and only after many people have decided that your architecture is really useful, only then you can claim that "Instead of simply saying 'if(age < 25) return false;'..." is about something viable.

MessiahAndrw wrote:anti-patterns aren't unique to managed languages, but in my daily experience in the corporate world, they seem to be far more prevalent in managed languages.

And in my experience useless efficiency hunting was a biggest problem of C/C++ developers. It was too expensive in terms of time and money.

Low level teaches you, that the efficiency is important, but it doesn't teach you where it is require to stop your hunting process.

embryo wrote:

MessiahAndrw wrote:You can have GUI designers and step-by-step debuggers and all of those cool things in an unmanaged environment.

"all of those cool things" include runtime code change, for example. Please, show me how it is accomplished in case of unmanaged language.

I just happened to come across this article earlier today.


EDIT:
Aside from that, there's also DDD and Insight. Insight in particular is an amazing tool that for seemingly no reason is fading into obscurity.

SoulofDeity wrote:My opinion is that there are actually 3 types of languages: systems programming languages, applications programming languages, and scripting languages.

There are actually some helpful things and some other stuff that is bothering and time consuming. And the problem is how to retain just helpful things and to remove all bothering. One promising direction is about automatic management of things for a programmer's advantage.

SoulofDeity wrote:I just happened to come across this article earlier today.

There's just some advertising buzz and nothing more. Some "cool" thing was awarded with some well known award... But what exactly is the thing about?

SoulofDeity wrote:there's also DDD and Insight. Insight in particular is an amazing tool that for seemingly no reason is fading into obscurity.

There are 274 000 000 results from Google for the word "Insight".

embryo wrote:Please, show me how it is accomplished in case of unmanaged language.

before that, MessiahAndrw wrote:Oh yeah, there's even Edit and Continue for C++. No one's making you go back to the stone age.

You might want to click that link and do your own research instead of attempting to shift burden of proof for no good reason. Arguments aren't magically invalidated by laziness on your end.

embryo wrote:

SoulofDeity wrote:there's also DDD and Insight. Insight in particular is an amazing tool that for seemingly no reason is fading into obscurity.

There are 274 000 000 results from Google for the word "Insight".

Lol, that might have something to do with it. The tools I use for debugging most of my C/C++ code are gdb, gcov, gprof, DDD, Insight, and DejaGnu. The last one in that list is another one of those obscure gems; used for unit testing.

embryo wrote:

MessiahAndrw wrote:You can have GUI designers and step-by-step debuggers and all of those cool things in an unmanaged environment.

"all of those cool things" include runtime code change, for example. Please, show me how it is accomplished in case of unmanaged language.

MessiahAndrw's Edit-and-Continue algorithm:

1. Programmer pauses execution and enters the IDE.
2. Programmer changes the code inside of a function.
3. The programmer presses 'Continue'.
4. The compiler recompiles the source file.
5. The linker compares the old and new object files, and notices the code inside one of the functions has changed.
6. The linker keeps a memory map from each time the program is compiled, and sees where in memory the old version of the function was, and frees that memory in the running process.
7. The linker then allocates some memory in the running process to fit the new version of the function in and loads that.
8. The linker then knows everywhere this function is referenced, and updates the address.
9. If the instruction pointer of any thread was in the old function, update it to the closest equivilent position in the new function.
10. Continue executing the process.

Some other features:
- Array bounds checking in an unmanaged environment: Generate a small snippet of machine code before array accesses.
- Garbage collection in a unmanaged environment: Exactly the same as garbage collection in a JIT'ed environment.
- GUI designer in an unmanaged environment: Exactly the same as a GUI designer in a managed environment - it just generates a source code file that constructs the GUI you designed (similar to InitializeComponent() in .Net when using WinForms), or a layout file (similar to .xaml in .Net when using XAML). None of that requires a managed environment.
- Debugger that let's you singlestep: keep a record of what instruction points to what line of code and the stack/registers - virtually every language.
- A large standard library: Write one.
- REPL: Compile and run a small snippet of code, but also tell the compile where in the source code the instruction pointer is so that it can compiler that snippet within the context of that scope, so your snippet of code can access local variables.

Personally, in my OS, I am going to distribute my applications as bytecode, but I'm thinking about invoking bytecode->native-code compilation only during the first execution or when a referenced library changes. Despite being bytecode, my language will allow 'unsafe' things like pointers. Would you call this managed? Does this fit your definition of requiring a virtual machine?

MessiahAndrw wrote:Personally, in my OS, I am going to distribute my applications as bytecode, but I'm thinking about invoking bytecode->native-code compilation only during the first execution or when a referenced library changes. Despite being bytecode, my language will allow 'unsafe' things like pointers. Would you call this managed? Does this fit your definition of requiring a virtual machine?

One of the things I would love to have is a throttled kernel-like MIPS IV VM with simulated memory-mapped hardware interfaces. Like a cross-platform sandbox with no limitations were I can play around with doings you normally can only do on embedded systems.