OSDev.org

Esselamu aleikum all, I am currently a student in university. I am in my third year Elhamdulillah, so I need to plan my graduating project. I am planning to choose developing an OS. I have asked some of my assistants about developing an OS and they said that you shouldn't try such a complex thing, instead you should make a project which includes machine learning etc. Since I am planning to choose cyber security as my field, I want to dive into OS development(I actually don't much know how much it will really help me). I have made some research about developing an OS, it seems like there are 3 way for me:
1-Develop OS from scratch through resources on the internet.

Advantage:
-Understand how do OS'es really work.
Disadvantage:
-It requires more time and there is a possibility(inshaALLAH) that OS will even may not have a GUI which is not good for our project representation.

2-Use lfs(linux from scratch) to develop an OS.

Advantage:
-It requires less time and there is a possibility(inshaALLAH) that OS may have a nice GUI and it may even be used by someone else.

Disadvantage:
-Since the linux kernel is already written, I may lost the chance(I hope I don't inshaALLAH) to understand how do kernel works.

What is your advice to me? In which part I am right, which part I am wrong?

How much time do you have for your project? Because Andrew Tanenbaum started Minix more than 30 years ago as an OS to teach students about OSes, and he's still at it. Writing an OS for this purpose might be biting of more than you can chew, is what I am saying.

Also, just because Linux is already written does not mean that you cannot understand it. Linux is a pretty complex piece of software, but most of that is in the drivers. In fact, I would argue that that is also an idea for your project: an in-depth analysis of how the Linux kernel actually works (how is the kernel entered, how are system calls handled, how does the VFS work, how does the scheduler work, and the IO scheduler, etc. pp.). But you might want to talk that over with the people qualified to give you advice.

For instance, I was wondering how system call tracing works. In Linux, you can let another process run until it starts a system call, then let it continue until it ends the system call. At each point, the traced task is stopped, and the registers can be examined. This is what allows strace(1) to work. I was wondering how the entire mechanism functions, because I'd thought about how I would do it myself. And the way I would do it, is to check for a syscall trace on entry to the syscall, and if set suspend the task with the registers from the system call entry. Then, when it is continued, it starts the system call over, only to suspend at the system call exit now. But this would require two different ptrace() subcalls, two different states for the system call tracer to be in. Then I actually read that part of the source code, and I learned that if the system call tracer is in effect, then on entry to the system call, the task will be suspended with state "TASK_TRACED", and then schedule itself out, right there in the system call handler. And the registers you can manipulate with ptrace() are actually not the ones on the top of the stack, but at the bottom of it. Because whenever the kernel is entered from userspace, it saves registers in exactly the same way. My mind was blown, and I was so furiously proud of having found out about that, that I had to go brag about it to total strangers on the Internet.

I would reconsider this plan. As the wiki says, trying to develop an OS for a thesis project is likely to end up getting bogged down, and unless you can devote an extended time to the topic - a year or more - chances are you will overrun your project deadline. This isn't a matter of knowledge or skill; effective OS dev simply takes time, period.

To be honest, I would say that if your interest is primarily in security, a deep dive into OS dev and kernel structure is counterproductive. You certain want to learn the basic concepts, yes, with a heavy focus on the systems you're likely to actually be applying this to (which is likely to mean either iOS, Android, small-device variants of Linux, or Windows; plain-vanilla Linux for desktop is generally too small a target for most attackers to bother with, while server Linux is - one hopes - better managed than consumer systems). But in the end, the kernel isn't where you need to worry.

In fact, what you really want to study isn't technology at all; it's user psychology. Almost every penetration starts with Social Engineering, and most never need anything more. Apply Murphy's Law in its original form ("If there are two ways to do something, and one will lead to disaster, one of the technicians [users, in this context] will do it that way") and try to find ways to reduce the windows of attack, by reducing the chance that the users - whether they are kids with a smartphone or experienced sysadmins who just happen to lose their focus for a moment - will unintentionally do something which compromises the system.

While you may want to learn OS dev for other reasons - it is a fascinating topic, after all, and an appealing hobby for some - as a matter of learning security, it is focusing on the wrong problems.

Hi,

Machine learning is becoming more and more important. But many people go that road. So becoming an operating system expert may be a smaller niche, perhaps. I personally would love to code everything including games. But of course a mortal person has to focus on few things, not getting lost in trying to do everything. So I concentrate on OS development, because that really excites and interests me. I would recommend you choose what you love most. That way you might later earn money with your hobby! And you are more active and better at what you love than at any other thing.

I wouldn't recommend LFS. But neither other Linux distributions. I would like to add something to the Linux software (both kernel and higher level system parts). But I'm not good enough yet to do so. So I write an OS from scratch to increase my skills. You can learn how kernel, compiler, object files, hardware etc. work. That turns out to be very useful for many computer areas, including application writing and computer security. So my recommendation is go for developing a kernel. Later you can extend your coding.

Oh, and you could help improving open source anti-virus software (ClamAV) and other security software. That could be counted as system development, too, and it's in your interest about security.

Happy hacking
Peter

nullplan wrote:How much time do you have for your project? Because Andrew Tanenbaum started Minix more than 30 years ago as an OS to teach students about OSes, and he's still at it. Writing an OS for this purpose might be biting of more than you can chew, is what I am saying.

Also, just because Linux is already written does not mean that you cannot understand it. Linux is a pretty complex piece of software, but most of that is in the drivers. In fact, I would argue that that is also an idea for your project: an in-depth analysis of how the Linux kernel actually works (how is the kernel entered, how are system calls handled, how does the VFS work, how does the scheduler work, and the IO scheduler, etc. pp.). But you might want to talk that over with the people qualified to give you advice.

For instance, I was wondering how system call tracing works. In Linux, you can let another process run until it starts a system call, then let it continue until it ends the system call. At each point, the traced task is stopped, and the registers can be examined. This is what allows strace(1) to work. I was wondering how the entire mechanism functions, because I'd thought about how I would do it myself. And the way I would do it, is to check for a syscall trace on entry to the syscall, and if set suspend the task with the registers from the system call entry. Then, when it is continued, it starts the system call over, only to suspend at the system call exit now. But this would require two different ptrace() subcalls, two different states for the system call tracer to be in. Then I actually read that part of the source code, and I learned that if the system call tracer is in effect, then on entry to the system call, the task will be suspended with state "TASK_TRACED", and then schedule itself out, right there in the system call handler. And the registers you can manipulate with ptrace() are actually not the ones on the top of the stack, but at the bottom of it. Because whenever the kernel is entered from userspace, it saves registers in exactly the same way. My mind was blown, and I was so furiously proud of having found out about that, that I had to go brag about it to total strangers on the Internet.

I have 1.5 year for that project alongside my lessons ELHAMDULILLAH. I know that it is not much important to dive into security with going into the deeps of os. It is generally enough to use the programs written by other experienced programmers, but the main problem is that the people which write that kind of programs are generally the ones which understand how computers do actually work(or what I think it is like that). There is also one other reason I go from that way is I want to see whether is it the right way to go for. I will consider your idea INSHAALLAH, thank you.

I think, Spectre/Meltdown is a good example that low level computer knowledge is indeed security relevant.

Regards
Peter