OSDev.org

I very much enjoy the fact that I don't need special tools to write a new version of the kernel binary to disk, and in fact can do so on any machine (like, even one that doesn't have "dd" at hand). Windows, Linux, MacOS, doesn't matter.

Compatibility is a feature. It's not the only one, but sometimes it's a good enough one.

FAT12, althought vastly outdated compared to EXT/2|3 and NTFS, is extremely simple to use and is supported by virturally every majour operating system out there.

This makes it simple for a devlopment enviroment. Otherwise you will find yourself having to write extra software to allow you to easily write files to your new filesystem.

So my opinionated list of file systems for floppies goes as follows:
1. RAW (really is supported by everything; obviously)
2. FAT12 (easiest to copy files to because of large support)
3. EXT/whatever (Supported by Linux and somehow with Windows (havent figuered that out yet but i have heard it done) (even worth using it on a floppy?) )
4. Your own FS here (least support, but you wrote it, damnit, so it must be good)

Hi,

AR wrote:The existing file systems are well tested with well defined advantages/disadvantages. Inventing your own will only yield an immature and potentially unreliable filesystem that will require rigourous testing before gaining any sort of acceptance [which is required for information interchange with other Operating Systems].

The same can be said for the entire OS: "The existing OSs are well tested with well defined advantages/disadvantages. Inventing your own will only yield an immature and potentially unreliable OS that will require rigourous testing before gaining any sort of acceptance.".

IMHO existing file systems can be categorized into "public" and "secure", where a public file system has no security features at all (DOS, ISO9660, etc), and a secure file system attempts to provide some form of security regardless of how successful these attempts are (NTFS, ReiserFS, ext2, etc).

I fully intend to support public file systems, but for legacy purposes only (so files can be interchanged between my OS and other OSs). I refuse to allow my OSs system files, passwords, user data, etc to be stored on any of these public file systems for obvious reasons (security). I also refuse to allow my OS to read from other "secure" file systems, so that my OS can't be used to break the security of other OSs (much like a Knoppix boot-CD can be used to gain full access to an otherwise secure computer running any version of Linux).


Cheers,

Brendan

Just want to make sure you are aware of the implications:

Brendan wrote: I refuse to allow my OSs system files, passwords, user data, etc to be stored on any of these public file systems for obvious reasons (security).

You just took away the ability to store any "user data" on the USB stick my co-worker just gave me, to burn a data CD for a friend using Win98, and a couple other very valid uses of legacy file systems. After all, you will allow users to store data using FTP, SCP, SMB et al., will you?

Or does "user data" refer to data about a user?

Anyway, you also made it a requirement that your OS be installed in a partition of its own (unless your bootloader is capable of reading an image file). While I understand the security concerns, I also consider that a serious drawback.

I also refuse to allow my OS to read from other "secure" file systems, so that my OS can't be used to break the security of other OSs (much like a Knoppix boot-CD can be used to gain full access to an otherwise secure computer running any version of Linux).

So people using your OS in a dual-boot setup won't be able to read from any kind of shared data partition, unless they use a public filesystem, fortfeiting any security?

Unless, of course, that other OS implements drivers for your FS...

Laudable security concerns, but I don't think it's really thought through...

Hi,

Solar wrote:

I refuse to allow my OSs system files, passwords, user data, etc to be stored on any of these public file systems for obvious reasons (security).

You just took away the ability to store any "user data" on the USB stick my co-worker just gave me, to burn a data CD for a friend using Win98, and a couple other very valid uses of legacy file systems. After all, you will allow users to store data using FTP, SCP, SMB et al., will you?

The basic idea is that there's a native file system, and legacy file systems. Combined you get a directory structure that looks something like this:

[tt]/ (the VFS root)
|_#FDA
| |_autoexec.bat
| |_config.sys
| |_command.com
|_#USB02
| |_readme.txt
| |_<a directory>
| |_something.jpg
|_sys
|_programs
|_home
|_Brendan
|_Solar[/tt]

Any top level directory that statrs with a '#' is a mounted legacy file system. On the native file system all of the OS's files are in the "sys" directory, all application's files are in the "programs" directory, and all of the user's files are in the corresponding "home" directory. The native file system is considered secure, while any file systems mounted at a '#' mount point isn't.

Now, if you want to copy files from "/home/Solar/myProject" to "/#USB02", then the OS knows that an authorised user is shifting thier own data to an unsecure file system and allows it. If I log in and try it I get "access denied".

If a user wants to copy files from "/sys" or "/programs" to a legacy file system then the user must have adequate authority (e.g. the root user), otherwise the OS refuses. Even then there's a warning - "WARNING: You are about to copy secure data to an insecure device or file system. Continue (Y or N)?".

It would be possible for flash memory, a floppy disk, etc to be formatted with the native file system. In this case the OS considers it secure and it's not mounted at a '#' mount point, and users can copy data to it without restrictions or warnings.

To complicate things, my OS will also refuse to read from any native file system that belongs to a different "cluster of computers". This means that if you and I were part of the same cluster, I could format a native floppy disk and send it to you in the snail mail and no-one between here and there could take the floppy and read it (regardless of which OS they use). It also means that if I've got a laptop that gets stolen the thief won't be able to login as me and won't be able to read my files without logging in as me (they could fdisk and re-install any OS and/or sell the hardware, but my data remains mine).

Anyway, you also made it a requirement that your OS be installed in a partition of its own (unless your bootloader is capable of reading an image file). While I understand the security concerns, I also consider that a serious drawback.

My bootloaders all load a single "boot image" containing the kernel, all the device drivers and things needed to get file systems, networking, etc working (IMHO it's standard practice for micro-kernels). This boot image contains files complete with directory information, but also only contains binary code that can be downloaded freely from the internet.

So people using your OS in a dual-boot setup won't be able to read from any kind of shared data partition, unless they use a public filesystem, fortfeiting any security?

That's correct. Have you tried doing the same with Linux and Windows? The problem is that each file system stores a "user ID" and "group ID" which is only sane for one of the OSs. For example, if Windows tried accessing data that is on a Linux ext2 file system how is it meant to interpret the "user ID" and "group ID"? There's ways around this, but none that I'd consider acceptable - put simply, pigs do fly, but only if you use enough force (and pigs are heavy, so you'd need a lot of force) .

Solar wrote:Unless, of course, that other OS implements drivers for your FS...

I wouldn't consider this likely as I'll be doing everything possible to prevent it (encrypted file system structures, no public documentation, no public source code, etc).


Cheers,

Brendan

Brendan wrote: Now, if you want to copy files from "/home/Solar/myProject" to "/#USB02", then the OS knows that an authorised user is shifting thier own data to an unsecure file system and allows it. If I log in and try it I get "access denied".

OK, that made more sense to me.

To complicate things, my OS will also refuse to read from any native file system that belongs to a different "cluster of computers".

Make sure you cater for backups that might have to be installed into a new "cluster of computers" because the old "cluster" catastrophically ceased to exist.

My bootloaders all load a single "boot image" containing the kernel, all the device drivers and things needed to get file systems, networking, etc working (IMHO it's standard practice for micro-kernels).

Nicely done.

So people using your OS in a dual-boot setup won't be able to read from any kind of shared data partition, unless they use a public filesystem, fortfeiting any security?

That's correct. Have you tried doing the same with Linux and Windows?

Yes, using Samba. Worked for me.

Solar wrote:Unless, of course, that other OS implements drivers for your FS...

I wouldn't consider this likely as I'll be doing everything possible to prevent it (encrypted file system structures, no public documentation, no public source code, etc).

Ah... so even if it would be technically possible to have a secure shared data partition, you placed a bolt in front of it in best Microsoft tradition... nah, don't like it. But it's your OS.

Hi,

Solar wrote:

So people using your OS in a dual-boot setup won't be able to read from any kind of shared data partition, unless they use a public filesystem, fortfeiting any security?

That's correct. Have you tried doing the same with Linux and Windows?

Yes, using Samba. Worked for me.

I've used Samba, NFS, FTP, etc too, but we're talking of a dual-boot setup where only one of the OSs can be running at the same time (at least without a hypervisor).

Of course it would be possible to run a samba server and a samba client on the same computer, with the non-native file system running under the samba server only. In this way the OS would ask the samba client for a file, the samba client would ask the samba server and the samba server would ask the file system. Also in this case the samba server would assign the same access restrictions to all files within the non-native filesystem, bypassing all security within that non-native file system (which AFAIK occurs for normal samba over different computers too, except the samba server itself needs to be able to access the local file in order to provide remote access).

In any case we'd still need the "not-running" OS's "/etc/passwd" file, even if all you're doing is trying to figure out if the samba server should have access to each file one the non-native file system. Not sure if I like the idea of that - if any of the files should be "no access" it should be the "/etc/passwd" file. The alternative is to completely ignore the non-native file system's security.

Solar wrote:

Unless, of course, that other OS implements drivers for your FS...

I wouldn't consider this likely as I'll be doing everything possible to prevent it (encrypted file system structures, no public documentation, no public source code, etc).

Ah... so even if it would be technically possible to have a secure shared data partition, you placed a bolt in front of it in best Microsoft tradition... nah, don't like it. But it's your OS.

Step 1, find features people want that other OSs don't have. Step 2, implement those features in my OS. Step 3, force people to use my OS in order to use those features. After this I'll do the reverse - find features people want to use on other OSs that can't be done on my OS and get them to work on my OS (either by implementing them or emulating the other OS).

Have you ever heard someone say "I really want to run Linux, but I've got a lot of files for MS Office that I can't use on Linux, so I'm forced to stick with Windows"? It doesn't have to be MS office files - it could be games, a CAD package, some commercial accountancy software, a video driver, etc.

It's one of the reasons open source software won't ever kill Microsoft - everything good gets ported to Windows anyway, so there's never much reason for people to leave Windows.


Cheers,

Brendan

Well those are stupid arguments since linux does have support for many of those things. AFAIK OO.o supports most of the M$ Office file formats.

BTW, calling everything other than your filesystem "legacy" sounds stupid- and it stinks like Windows.

Although you do have a fairly good idea, not telling everyone else how it works means your OS will never rise, even to the height of linux. Your OS is too much like Windows, except "more secure." But since nobody has any idea how it works, it's impossible to make any other OSes compatible with yours (unless they reverse-engineer your source code, in which case it must not be as secure as you thought, although it would be difficult to do so ;D )

I'll probably make my own filesystem eventually, though I plan always to be compatible with as many other filesystems as possible. I will support all the others... plus my own. However, that's a long time in the future, and, for me, FAT12 is the easiest and fastest (all I have to do is right-click and hover over "Send To >" and click "3 1/2" Floppy (A:)" and the kernel is on the disk!

Thanks bubach, by the way, I got your bootloader working fine.

Hi,

NotTheCHEAT wrote: BTW, calling everything other than your filesystem "legacy" sounds stupid- and it stinks like Windows.

I don't like Microsoft too much, but my OS will (hopefully) become a commercial thing (shareware, donations or something). This leads to adopting some of the tactics commercial OS developers use, like calling anything that's not native "legacy".

Unlike almost everyone, my goal is to eventually make enough money from my OS to be able to work on the OS full time. I'm not talking millions of dollars, just enough to pay the rent and buy a new computer every few years, if you see what I mean.

I assume you (and probably most OS developers here) have no intention of ever attempting to recoup any of the development costs, and haven't considered ways that could improve the chances of recouping costs in such a hostile market.

NotTheCHEAT wrote:Although you do have a fairly good idea, not telling everyone else how it works means your OS will never rise, even to the height of linux. Your OS is too much like Windows, except "more secure." But since nobody has any idea how it works, it's impossible to make any other OSes compatible with yours (unless they reverse-engineer your source code, in which case it must not be as secure as you thought, although it would be difficult to do so ;D )

I don't know how Windows, OS X or even SkyOS "works", but this wouldn't prevent me from buying any of them if they do what I need them to do. Even though I'm running Gentoo (Linux) on some of my computers I've never really looked at any of the source code and have no idea how it works either.

You say "it's impossible to make any other OSes compatible with yours" as if this is a bad thing, which is where I mostly disagree. Compatibility can be provided through standard/public interfaces (samba, nfs, ftp, http, fat, iso9660, etc), while preventing people from stealing my code or design and building their own version.

For example, imagine you've come up with an idea that could make your OS much better than Windows for one specific purpose. How would you prevent Microsoft (or some third party Windows developer) from implementing this idea? Patents are one possibility, but I have moral objections. On the other hand, calling it a "trade secret" and never providing enough information about it to anyone I consider both fair and reasonable.


Cheers,

Brendan

Although I disagree with it (my OS will be Open Source, and completely free as in price as well as in freedom) I respect your choice as it's your OS

Not that it's bad to keep people from stealing your ideas, but you could give documentation on some things which, if stolen by M$ for example, would not hurt you.

I really have a hard time seeing any hobby-OS-dever making any money off their OS, with M$ and Mac and linux dominating the "market" (or non-market, in linuxes case). But good luck all the same!

Sorry for quoting from this old post....

Solar wrote:

Solar wrote:Unless, of course, that other OS implements drivers for your FS...

I wouldn't consider this likely as I'll be doing everything possible to prevent it (encrypted file system structures, no public documentation, no public source code, etc).

Ah... so even if it would be technically possible to have a secure shared data partition, you placed a bolt in front of it in best Microsoft tradition... nah, don't like it. But it's your OS.

I'm one of those who believe that the _only_ way of really securing data on storage is encryption. I respect when you don't want to share source code, or don't want to expose public documentation, but these can (IMHO) not be considered as means to protect your data. But public documentation can also help you on the market, I think, because it shows that you believe in your products secureness and that you don't have to hide weaknesses and/or bugs. You can't really prevent people from reverse engineering your data structures anyway, once there is enough interest in doing so.

cheers Joe

Hi,

NotTheCHEAT wrote:Not that it's bad to keep people from stealing your ideas, but you could give documentation on some things which, if stolen by M$ for example, would not hurt you.

I will be providing lots of documentation, including the kernel API, almost every user-level messaging interface, example/template application source code, etc. Currently I'm re-writing my utility to convert source code into html so that I can provide much more of the OSs source code on the internet (so specific/sensitive source code files remain unavailable instead of entire modules that contain these files).

The only documentation I intend to supress will be some parts of the scheduler's code, some parts of the VFS code, the entire networking protocol used between computers within a cluster, the entire native file system code and "on-disk" data structures, all encryption/decryption code, all keys used for encryption/decryption, and all code that works on user's passwords. This still leaves a huge amount of documentation and online code.

NotTheCHEAT wrote:I really have a hard time seeing any hobby-OS-dever making any money off their OS, with M$ and Mac and linux dominating the "market" (or non-market, in linuxes case). But good luck all the same!

Here's a few:

http://www.skyos.org/
http://www.yellowtab.com/

AFAIK neither of these OSs have any "special" features (ie. neither do anything that Windows/Linux/Mac users can't do). Despite this they're still at least recouping some of the costs of development.

IMHO making money isn't impossible (it's just very difficult) .


Cheers,

Brendan

Hi,

JoeKayzA wrote:I'm one of those who believe that the _only_ way of really securing data on storage is encryption. I respect when you don't want to share source code, or don't want to expose public documentation, but these can (IMHO) not be considered as means to protect your data. But public documentation can also help you on the market, I think, because it shows that you believe in your products secureness and that you don't have to hide weaknesses and/or bugs. You can't really prevent people from reverse engineering your data structures anyway, once there is enough interest in doing so.

I guess I should point out that I want to protect data stored on disk, and protect ideas used within my native file system from being implemented by other file system creators. The former requires encrypting the data, while the later requires that source code and implementation details aren't made public.

In general no form of security is ever 100% secure, but by making things 99.99% secure you can make it too hard for people to bother with.

Specifically, I intend to support encrypted partitions where every single sector on the partition/disk (regardless of whether it's used or unused) is encrypted. This will be transparent to the file system driver, and should even allow things like encrypted FAT and encrypted ISO9660. My own native file system will refuse to work on non-encrypted partitions. This will make it very difficult to reverse engineer from the data on the disk.

This makes disassembling the native file system driver the weak point. I'm still looking into this area, however I do intend combining this code and the VFS code into a single binary to make relevant pieces harder to find. IMHO trying to figure out what a large amount of code written in assembly actual does is quite difficult without any documentation or comments (not impossible, but not so easy that everyone will try it).

One of my concerns here (in regard to protecting my source code) is to prevent people from downloading my source code, making a few minor changes (remove all security/permission checks), recompiling the module and using it in a working OS. Consider Linux - with a little prior knowledge it'd be possible for someone to create a completely insecure Linux kernel, then use a boot-CD to replace the kernel used on a secure server. I can imagine a disgruntled employee doing this without anyone noticing until several weeks have passed (long enough for all data to be copied elsewhere).

For Linux this isn't so much of a problem because you need physical access to the server itself, which can be locked inside a secure room with fingerprint scanners on the doors, movement sensors, etc. For a distributed OS the problem is worse because you only need physical access to one computer of many. For my distributed OS, if the secretary's laptop is compromised the entire cluster of computers is compromised. Therefore, for marketting purposes I must be able to convince potential users that this (and other security problems) can't happen, otherwise no-one will be able to trust my OS (and no-one will use it, regardless of any other benefits).


Cheers,

Brendan

well, i'm glad for you you still hope to make money with your OS. There's a long time i dropped that and went for "okay, let's just bring those ideas to the real world" and then "well, if i wish it to work one day, i'd rather have to appeal more developers with catchy ideas that i wish to implement, so there's no point keeping idea secret until code is released" .

Encrypted partitions could indeed be a solution (i'm thinking at KaZaa protocol that is encrypted too and hasn't been broken so far - to my best knowledge), but i doubt you could achieve anything worth it about execution speed ...

Those examples may be making some money, but are they making real money? It's not that it's impossible to make money with a hobby OS, it's just that it's highly unlikely, and extremely difficult, because the market is already dominated by two or three giants (literally; even linux makes money for some people). You never even hear about those other OSes except online. Besides, if they're making money, is it really a hobby OS (I guess I was stupid to call it that myself)?

In short, it's not impossible but IMO it's highly unlikely. But good luck anyways, and may you prove us wrong!