We (OSDev.org) are in the wrong direction, guys...

[KemyLand]

I've already expressed this, and I do know this will most likely start a great flamewar, but let's accept it. Most of the recent posts are about how to program, not how to OSDev. All but two moderators, Brendan and Combuster, are somewhat active (sortie and chase are apart, special cases). This time, I do have some proof of what's happening, for those that argue it's not.

List of irrelevant threads in the View active topics page (not to offend anyone, BTW):
http://f.osdev.org/viewtopic.php?f=1&t=29271
http://f.osdev.org/viewtopic.php?f=1&t=29269
http://f.osdev.org/viewtopic.php?f=1&t=29248
http://f.osdev.org/viewtopic.php?f=1&t=29265
http://f.osdev.org/viewtopic.php?f=1&t=29264
http://f.osdev.org/viewtopic.php?f=1&t=29260
http://f.osdev.org/viewtopic.php?f=1&t=29258
http://f.osdev.org/viewtopic.php?f=1&t=29252
http://f.osdev.org/viewtopic.php?f=1&t=29242
http://f.osdev.org/viewtopic.php?f=13&t=29241
http://f.osdev.org/viewtopic.php?f=15&t=29250

And for topping, there's this thread. Although it originally was a irrelevant question like all others above, it did resulted in documentation reading by the OP, which ended in a great wiki contribution.

Actually, we've been better this week. On previous ones, you could see the View active topics page fully loaded with these threads.

And about the moderators, we do already know for a long time now that there are few of them active. When was the last post you saw from thepowersgang, JamesM, or chase? There's actually a special case with the later one. He shall be somehow active, as otherwise the site would not be online. I did also mentioned above that there's a special case with sortie. What I meant is that he's the third most active moderator after [Combuster, Brendan], but at my opinion, not active enough.

We're loosing the point of this site. For beginners, the wiki looks pretty misleading and data-lacking after Bare Bones/Meaty Skeleton/JamesM's tutorial/Stuff in http://www.osdever.net (difunct since January 29, 2007). When it comes to interrupts, they will find a problem, and search the Holy and Sacred Wiki for answers. They'll get into the I Can't Get Interrupts Working article; a very misleading and anti-intuitive one in particular for the purpose it has. Then they come to the forums, and you know what follows...

The most advanced OSes in this site appear to be privative. Their code isn't shared in any way in the forums/wiki. You'll never find how they do get insert some function here to work, unless you try to find (the hardest part) and then analize their source code. This same event has hard effects over the existing projects. As their mantainers can't get something to work, the project is abandoned.

Let's remember something. How were Linux/GNU/BSD/A lot of Unices/Micro$oft Windows/M$-DOS/Whatever you wanna to put in here/'s codebases created? Either from other ones' code, or other ones' knowledge! Another bitter cup of tea no one here can reject is the fact that specifications are strangely confusing and misleading by nature.

I don't want to offend no one here, but it's true. In the way we are, lots of potential projects will be lost along with already defunct ones, and with them, this site's reason to exist.

[no92]

I agree with you on most points. I spent some time thinking about that topic; I honestly don't know how to fix this completely. A fix for that would be to check every page twice, by two different members for mistakes etc. With this, two issues arise: not everybody can check everything (I've got no clue about networking) and we're not enough active users to check 630+ pages detailed enough to fix most mistakes.
Maybe we should get rid of old unnecessary information and fill the wiki with new; for example a proper UEFI Bare Bones, possibly with a custom-built UEFI implementation.

These are just my worthless 2 cents, summarized in a mess of thoughts.

@KemyLand: what's your suggestion to fix this?

[KemyLand]

You've just said it. We're not enough users to fix this properly, and I can't come with a proper solution.

BTW, everything would be easier if this whole community was about a single project, but we'ld loose the community's purpose (i.e: you, anyone can create his/her own O.S.). But this has rather damaged us, preventing the first objective to work in the first place.

[iansjack]

I believe that whatever faults there are with this site lie in the opposite direction. There is too much trivial information, particularly "Bare Bones" tutorials and the like.

I think the site was originally set up for experienced amateurs to share the meagre knowledge that was available. Now it is, to a large extent, being used as a cut-and-paste resource by those who don't know how to program in C and assembler and can't be bothered to read the manuals. The last thing needed is a more detailed "how to program a network card" sections with reams of code to be blindly copied. We need information, not trivial examples.

If there were to be any overhaul I would recommend weeding out the "Bare Bones tutorial using GW Basic" type tutorials and adding more links to documentation. But, whatever, it's not something to post "somebody should do ..." topics on. If you think the Wiki needs fixing then be somebody and fix it.

[Combuster]

There will always be users not worth spending a word on because they're lazy leechers. There will always be users not worth spending a word on because they don't need to ask. Then there remains a group of people that does honestly need help. Whatever trend you try to set with passive means only shifts where the boundaries between the groups are - you will never get rid of them. And as such, there is no fix you can think of just as you can't do away with a country's police.


If you really want to make osdev.org a better place, promote it, be active, and most importantly, try to be nice. I know from personal experience that that is very hard.

[Brendan]

Hi,

List of irrelevant threads in the View active topics page (not to offend anyone, BTW):

Let's analyze!

http://f.osdev.org/viewtopic.php?f=1&t=29271. Someone having trouble with an interface provided by firmware to the OS. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29269. Someone having trouble with booting from USB using GRUB. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29248. Someone having trouble with implementing paging. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29265. Someone having trouble with an interface provided by firmware/hardware to the OS. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29264. Someone having trouble understanding "virtual to physical" conversion and/or an interface provided by hardware. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29260. Someone having trouble with hardware (PIC chips) and IRQ handling. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29258. Someone having trouble with IRQ handling. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29252. Someone having trouble with the character set provided by VGA (and how it relates to ASCII/UTF-8). An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=1&t=29242. Someone having trouble with IRQ handling. An OS development question and not a "how to program" question.

http://f.osdev.org/viewtopic.php?f=13&t=29241. A question about ideas for getting Python programming experience. Not OS development question; but in the "General Programming" sub-forum that's explicitly intended for general programming questions.

http://f.osdev.org/viewtopic.php?f=15&t=29250. Someone exploring ideas on scheduler/OS design. An OS development question and not a "how to program" question.

Mostly; this site's purpose is to help OS developers. You can split OS developers (rather crudely) into 2 categories: experienced developers and beginners. The experienced developers rarely need any help; and if they post it's typically either helping others, high level theoretical pondering or showing off their work. Naturally; the vast majority of posts are beginners.

For beginners...

There has always been a kind of "culture shock" when someone moves from normal programming (with IDEs, nice debuggers, libraries, APIs that work, etc) into OS development. As time has passed "normal programming" has become higher and higher level (more abstractions, libraries, etc; and less focus on lower lever concepts like pointers, cache behaviour and the specifics of CPUs and hardware). For an example, I'd be very surprised if a modern experienced Java programmer (e.g. someone who's been programming in Java professionally for years) can figure out how to implement a basic linked list (not because they're bad programmers, but because Java has built in collections). Also over time, the hardware has been getting progressively more convoluted - the introduction of APICs, ACPI, multi-CPU, 64-bit, NUMA, UEFI, etc, have all added to the "burden of knowledge" needed. Finally, the goal has also shifted - once upon a time (a long long time ago) something like MS-DOS was all users really expected, but now they expect far more in terms of OS features and capabilities.

Putting all this together, that "culture shock" (that used to be more like a small hill) has become more like a tall mountain. I'd estimate that at least half of the beginner posts are effected by this. I mean, they do know they have to do things themselves and can't just (e.g.) include some standard library and have terminal input and output working, but the true nature of "bare metal" hasn't become ingrained yet. I'd also estimate that it takes an average of about 2 months for a good/experienced application developer to fully make the transition.

Let's have a look at one of these beginners. More specifically, let's look at this forum topic. What do you see? I see an application developer who wanted to use relatively high level concept (garbage collection) in a very low level environment (kernel) using a very low level language (C). That is a reasonable example of the "culture shock" I'm talking about.

So, think of it like this: The forum's purpose is to help OS developers; most of the people that need help are beginners, and most of the beginners are suffering from (natural/inevitable) "culture shock".


Cheers,

Brendan

[iansjack]

I'm sorry, Brendan, but if you think that some of those questions have a place in an OS Development forum then maybe this is the wrong place for us. The first question, for example, shows a fundamental misunderstanding of how to use gcc inline assembler plus a lack of knowledge of AT&T assembler syntax. These are not, primarily, OS development questions, even if asked in an OS development context. And they demonstrate a lack of research; this is not information that is difficult to find using Google. There is a sticky about "Required Knowldege" for a good reason.

I firmly believe that a lot of the problems people have is because "Bare Bones" tutorials give the impression that OS development is easy; a quick cut and paste and then a post starting "I have written a kernel...." with a question, "why doesn't it work?". Do you really contend that this is the level that these forums are aimed at? I firmly believe that if improvements in the Wiki are needed it would help to get rid of a lot of the detailed code and concentrate on teaching the principles, and the practice of subjects such as debugging, rather than trying to act as a cookbook.

[Kevin]

We're loosing the point of this site.

Perhaps you should start with a definition of the point of this site. I'm not sure it's the same for everyone.

As for the "stupid" beginner questions, I would see it as a sign that the wiki has quality problems. Of course, there are always those that don't even look at the wiki, but for the rest of them it means that either the information they need isn't on the wiki, it is well hidden so they can't find it, or something that looks relevant is there, but it's not actually helpful.

To be honest, if I were new to OS development and looked at the wiki, I would probably be impressed by a long table of contents on the main page, but confused what I actually need to do. Sure, I would get a "Hello World" message because there is this nice copy&paste Bare Bones tutorial. And then? Perhaps the Meaty Skeleton, but that's only worse - "Here's the code, I won't explain it. Have fun."

At Lowlevel, the quality of the questions was improved a lot after we had abandoned the old "copy this code" style tutorials for individual topics (actually, they are still around, but much less prominent) and guided people towards a new coherent series of good tutorials. "Good" means that they don't just offer code (they do, but usually it's incomplete so that you have to think a bit and can't just c&p), but explain what options you have, where to get information on each of them and how you can integrate the pieces best. In the first part, the provided code is rather complete, and as the series progresses, it becomes less and you have to fill in more by yourself using the links that are given. This way it didn't duplicate the topic-specific articles, but is more like a "guided tour" through the wiki.

Perhaps something like this is what's missing in the osdev.org wiki.

BTW, everything would be easier if this whole community was about a single project

Yes, in the sense that a dead community is easy to handle. You would end up with like three active developers, five more people who want to contribute and don't actually do anything and a few others that watch the project. The rest wanders off to somewhere else where they can talk about their own projects. That would result in a place totally unlike today's osdev.org.

[KemyLand]

Let's have a look at one of these beginners. More specifically, let's look at this forum topic. What do you see? I see an application developer who wanted to use relatively high level concept (garbage collection) in a very low level environment (kernel) using a very low level language (C). That is a reasonable example of the "culture shock" I'm talking about.

That's not that "culture shock" I talked about. Although it's somewhat true, I did just though of it for a time and didn't really put too much attention on that irregular idea. If you do say so, then you didn't understood my post, as many others. I didn't intended was is normally called a "garbage collector", but rather a sequence of pointers to be recollected by a dedicated thread, so to allow complex (no buckets!) memory allocation techniques that I though. The problem with those ideas is that they do make the freeing process expensive. Anyway, I don't consider a "good programmer" to be someone of those C-fans who reject any high-level idea. BTW, I use C++, not C .

Personally, I don't accept any of your opinions. That "culture shock" concept is discriminating for those programmers that come from a high-level context. Saying that "it'll be amazing to see a Java programmer create a well-done linked list" is just silly and, begin almost the only base infrastructure of your whole argument of "culture shock," the later can be simpl discarded.

Secondly, your arguments about those links are provided are, as others have mentioned, invalid. You are ignoring some important points I made, specifically "that topics have already been covered", "they show low technical knowledge", and "they don't have nothing to do with OSDev". I argue the later due to the fact that you're not investigating, but rather asking others to do the hard work.

I never though this topic would start such a big inter-forums problem .

[sortie]

This is a silly topic. I've seen a dozen of these. They all condemn the state of things and accomplish nothing. If you want to make a change, go seriously improve the wiki, find suitable mentors for aspiring developers, give very high quality responses to threads, try to overall raise success and rationality of the average osdever, get rid of sources of bad influence on newbies, or whatever actually manages what you want. Instead of bickering here. (Of course, not everyone should do this, newcomers helping newcomers usually won't help well, as experience shows.)

In osdev self-reliance is a prerequisite to success. There's those that make it and those that won't. All the ones-that-will-definitely-make-it need is a good start and advise. They are going to make it as far as they want without much handholding. Any further wiki improvements will be of diminishing influence on whether they succeed or not. However, we are suffering from bad tutorials elsewhere that badly influence newcomers with bad advise and code. Like "copy-paste" development or not, we need to supply proper replacements for common troubled tutorials. Otherwise new osdevers with actual potential might follow them and get off to a bad start. I did. Many did.

Your vision of the forum is an illusion. The people asking for help here is correlated with those that are not self-reliant (self-reliant people still ask for help, as an optimized way of solving the problem, after they fail at first). What do you expect? No newcomers asking questions that are a waste of time? Established developers in serious technical debates that reach a consensus? Those are not ever going to happen.

There's an exponential decay along the curve of "Let's make an operating system" to "I made a self-hosting operating system". It then follows there's a lot more newcomers that will never make it compared to those that do. The newcomers that do make it tend to be more self-reliant, and don't ask for help. Most newcomers posts will then be come from those that will never make it (and that's not to say such people are unworthy, not at all, osdev is just that hard) rather than those that do make it. We can then predict most newcomer posts aren't they pleasing to established developers. That's what we see.

Established developers are generally self-reliant and on their own can come to a conclusion on a technical matter. If they do make a technical debate on the forum, it likely won't produce any insights the OP couldn't have reached with some personal research with a search engine and some soul searching. Instead, what often happen is that the subject is very much according to personal taste. The repliers to the thread doesn't want to reach a consensus but just push their own agenda. These topics quickly into into ten page long quote fests where established and semi-established members mostly agree, but manage to bicker on and on about minutia. A healthy debate requires all the participants to be willing to admit they viewpoint is wrong and actually reach a consensus.

I spent almost all of my time on this hobby working on my project rather than on the forums. In contrast, there's a suspiciously large number of established community members that we never see actually osdev. These people just hang around and push their own agenda. It doesn't help that a large number of community members are crazy.


So: 1) This topic is silly. 2) The vision for a golden era of osdev forum is silly. 3) We should focus on getting the osdevers with potential to a good start. That means improving the wiki and getting rid of the bad influences on newcomers. 4) Go make a difference instead of bickering here, too.

Go use #osdev on freenode IRC instead. Topics that disinterest you are soon gone in the back log, and you got instant access to a lot of established peoples. The same illnesses that plague the community here is also there, but the fact there's a scrollback makes the place seem much less spammy. On the forums, a displeasing topic fills as much space as a pleasing now.

[SpyderTL]

One idea would be to remove, or reduce the amount of code blocks in wiki articles. Since I decided to write my own os dev language, these code blocks are useless to me from a copy/paste standpoint. I have to convert them in my head to values and instructions, and then use those to build my own code.

Removing code blocks from the wiki would force others to do the same. Instead of C/ASM code blocks, you could use tables and descriptive paragraphs describing which values to use, and when. There are several non-C, non-ASM projects going on right now, and I think tables full of values and memory locations are more useful for cross-language development, anyway.

Toolchain support is a completely different problem, though. And I think GCC questions make up a pretty significant part of the forum posts. Those may be a little off topic for this site, but since most os dev is done with GCC, and most "getting started" tutorials use them as well, I don't see any way to avoid them, short of just prohibiting them. GCC/Nasm/Masm aren't exactly user-friendly, and can be just as complicated as any CPU or hardware interface.

So, reduce code blocks and either prohibit toolchain discussion, or at least move them to their own sections. I'm not sure if this would make the site better or worse, but I think it would address the "problem" at hand. Just an idea...

[sortie]

I'm generally in favour of revising the code blocks on the wiki. I do agree I occasionally find them useless. In particular, the assembly parts you occasionally see are often particularly useless. Operating systems written in assembly also tends to use their own calling conventions, so even using such code is generally impossible. The C code pastes can be useful, as they convey algorithmic information considerably better than raw assembly and carry more information. They are sometimes useless, though, those code pastes needs to be identified. Another problem is that the C code pastes tend to use use different coding conventions and non-standard types (like DWORD). I've been generally reformatting such pastes systematically for a while to use standard <stdint.h> types and be simple, generic C code.

I'll be starting a new topic on wiki cleanup. Identifying particular problems and dealing with them in their own threads is a more efficient way to get the solved, rather than discussing all the problems at once.

Toolchain issues are not going away. Murphy's law. A way to ease this is identify what trouble newcomers have and try to solve them on the wiki. We've already done this and I think this is as good as it gets. Further ways to improve the situation could include pre-built toolchains and environments that newcomers can install. However, this introduces trust issues (was that prebuilt compiler actually a virus?), complexity issues (as its hard to make compilers for all the various systems people might run), maintenance issues (as someone needs to keep all those compilers up to date), and missed opportunities to learn. If you go even further and make a osdev IDE, then there's even more issues that one size does not fit all. An interesting alternative approach is to get prebuilt compilers into Linux distributions (and cygwin?) as native packages. Such distributions are already packaging gcc, it should be manageable for such distros to ship cross-compilers too (and some distros do ship mingw cross-compiler for instance). Vendor patches could be a problem, like enabling stack protection (which is cool) but isn't on by default in a default i686-elf compiler.

Prohibiting toolchain discussion is a really poor solution. People will still have problems, and the result will be rejecting more osdevers with potential. A major concern in many issues is that people failed to read the wiki instructions and actually follow them. It's important to learn your personal folly and learn to read instructions and actually follow them.

[no92]

GCC/Nasm/Masm aren't exactly user-friendly, and can be just as complicated as any CPU or hardware interface.

Clang has expressive diagnostics. It's error messages are way more user-friendly than gcc's. If you mistyped a type name, it suggests the one it thinks matches the best. Also, when using -> to reference a struct member when you really should use ., it's stating it directly, not like gcc does.

[max]

In my opinion, it is not a good idea to remove all code snippets from the wiki, or make the wiki so complicated that only advanced users can take any information from it. Sure, this will lead to people just copy-pasting stuff. But OS development is still just a hobby that should bring fun to anyone who is interested. If it helps the newbies to just copy code & they even manage to run it, they will learn from that code. They will sooner or later see that they have to fully understand what they copied there, anyway.

You will always have new users that post in the forum about trivial problems or show lack of programming skill; but then you should tell them in a polite way that they need to do some work & learn more before they can dive into the world of OS development. If they really want to do it and are interested, then they will keep on trying and once come back to the forum asking on a higher level. To the blog posts you mentioned: about half of them are on a level that I consider acceptable for this forum.

About toolchain issues & the setup tutorials: if you remove these, it will only lead to more people loosing interest in osdeving. Its a very hard part of work to get a proper toolchain running (and I'm not sure if I want to see the toolchain setup of some users here) but even having a toolchain that is prebuilt and just works, or building one with low effort, will help beginners to start out easier. As it is with copying code; once they truly understand what they are doing, they will want to make their own OS specific toolchain.

Closing the door for beginners makes this community even more elitist; and theres really no need to do so.

[iansjack]

Closing the door for beginners makes this community even more elitist; and theres really no need to do so.

Allowing too many trivial threads has exactly the same effect, of limiting the appeal of the forums, just in a different direction. I guess it's a question of whether the site is aimed at experienced programmers, who are prepared to do research but need some resources to help them with OS development, or beginners who don't understand the way that strings work in C and can't be bothered to read the Intel manuals to determine how the stack behaves with different sized operands (and different processor modes).

I may have misjudged the aims of this web site.