OSDev.org

Hi,

Candy wrote:Mine should also be a life's work, but I'm more of the decent basis stuff... I'm working on (simultaneously) a few compilers (c, c++, assembler to go with it, interface languages for own idea and possibly some more), compiler writing tools (to make beforesaid quicker and easier), my kernel, drivers, userland libraries, an slight adaptment of the stl, a filesystem for removable media, a filesystem for fixed media, a user interface (only in my mind atm, just started to try out an idea on graphical font rendering without bitmaps) and a few programs to run in it (text editor, internet browser). My one idiocy is that I'm refusing any and all existing code and designs unless I fully agree with them, or for compatibility. ELF has passed my judgement (as first standard I actually endorse), FAT/NTFS/HPFS/BeFS etc. have all failed, so I'm making a new one, including linux driver etc....

I take "breaks" - they're experiments that normally last about a month. One was a 3D renderer (matrices, polygon drawing techniques), one was an emulator (researching the performance effects of messaging on a multi-process emulation of SMP), and one was a language called "VIAL" (Versatile Intermediate Assembly Language). This was a machine independant low level language, intended for the back end of compilers (compiled or interpretted byte-code).

I never use existing code. Partly because most of the time I can't (my OS design has it's peculiarities) and partly because I like doing things myself. I don't want compatibility for similar reasons (except where it's an "operational requirement" - iso9660, FAT, SFS, UDP/TCP/IP, FTP, HTTP, NNTP, SMTP, etc).

Candy wrote:Add to that a little touch of perfectionism (if I find a design flaw all code is chucked and rewritten to conform to the new design) and it's taking forever already .

This is partly why I'm making the new version of my OS very modular (so I can chunk a small module instead of chucking the entire kernel).

Candy wrote:Positive points: I've gone from two studies to only a single full-time-plus-overwork-job so I've got loads of time on my hands, so I can catch up with 2 years of OS work. Have got some 20-30 hours a week now, so I'm starting to pick up pace again (with some non-direct-kernel projects first).

I was running an electrical contracting business, but dumped it and became a student again to free up more time. Now I'm behind in my study...

Candy wrote:Brendan, it sounds like we'll be colleagues for a lifetime to come

Don't worry - you're on my "4 years time" list..


Cheers,

Brendan

Brendan wrote: I take "breaks" - they're experiments that normally last about a month. One was a 3D renderer (matrices, polygon drawing techniques), one was an emulator (researching the performance effects of messaging on a multi-process emulation of SMP), and one was a language called "VIAL" (Versatile Intermediate Assembly Language). This was a machine independant low level language, intended for the back end of compilers (compiled or interpretted byte-code).

I'm busy with mostly directly applicable stuff for my OS, that still needs porting. I'm hoping to craft this font renderer in such a way that it's using the basic libraries, but I'm not sure it's entirely possible yet...

I never use existing code. Partly because most of the time I can't (my OS design has it's peculiarities) and partly because I like doing things myself.

Dito... I'm having trouble with mp3 and flate compression though, mp3 stuff has some options but I have no idea what I'm doing wrong with flate... been a while since I tried so it's not very important, but still...

This is partly why I'm making the new version of my OS very modular (so I can chunk a small module instead of chucking the entire kernel).

And that's why the very next feature my kernel will have is module support, so that the core is plain done (no chucking) and that the rest is modular (module chucking).

I was running an electrical contracting business, but dumped it and became a student again to free up more time. Now I'm behind in my study...

How was it to step back from a full monthly wage to a minimum-ish wage (or no wage) ?

Don't worry - you're on my "4 years time" list..

What's your "4 years time" list ?

Hi,

Candy wrote:

This is partly why I'm making the new version of my OS very modular (so I can chunk a small module instead of chucking the entire kernel).

And that's why the very next feature my kernel will have is module support, so that the core is plain done (no chucking) and that the rest is modular (module chucking).

So far I'm up to 11 modules used during boot. Add to this about 16 different "kernel modules" and at least 3 "system modules". That gives me around 32 modules just for the "core" (no apps, no drivers, no GUI, etc).

Candy wrote:How was it to step back from a full monthly wage to a minimum-ish wage (or no wage) ?

I wasn't making a huge amount of cash, and for self-employment payment is erratic (for e.g. you could do a large job, spend a pile of cash on materials and wages, get paid nothing for 2 months and then get a large wad of cash). Now I'm getting less cash, but my expenses are much less and it's regular/predictable.

The biggest difference is "hassles per day"...

Candy wrote:

Don't worry - you're on my "4 years time" list..

What's your "4 years time" list ?

"probably 4 years before it'll run basic applications on "generic" hardware... After that I'm hoping others will help - GUIs, device drivers, applications, etc."

The "4 years time" list is a mental list of names, beginning with Solar (fixing up the mostly broken C libraries I'll probably have) and including yourself, Pype, Colonel Kernel, and many many others. I thought that while I was being optimistic... ;D


Cheers,

Brendan

I guess this may be kind of off topic but my reason for writing an OS is to test some ideas I have, not unlike most of y'all. They mostly relate to weird ways of supporting some strange hardware configurations but I am fascinated by you people's ideas. Keep up the good work!

One day I figured that since PS/2 keyboards and mice use the same physical connector and communication protocol it should be possible to swap them and still be able to run them. This led me to write a DOS TSR to see if I could run two keyboards on one computer and make them appear as one, since really the BIOS supports only one. I did get it to work successfully and even wrote some code that wasn't integrated into the TSR that allowed hot-swapping and would alert you via IRQs (well insertion anyway, you'd have to poll periodically to check if anything was unplugged). A good friend once joked about what would happen if you held down the "L" key on one and "O" on the other. ;D Anyway I also wrote some code to test the swapping of mouse and keyboard and found that many modern keyboard controllers like to automatically translate scancodes from the keyboard port so data from the mouse in the keyboard port would be corrupted. I think there is a way around this, some bit to turn off translation, but I have yet to try it. I read about this prior to these tests but also I found a page that described how some keyboard controllers don't forward any commands meant for the keyboard to it, except for setting LEDs. This KBC would handle everything else like scancode repetitions itself. All in the name of appearing compatible, I guess. ::)

I have also experimented with what I call VESA/VGA hybrid modes. I accidentally created something similar while writing code to set 800x600x16 on a plain VGA. In this example program I forgot to unlock the clocking registers that were configured for a text mode so I got a 70-something Hz refresh rate instead of 48. I used the mode data from TWEAKVGA btw. I have also read of people making unchained VESA modes to avoid bank switching but this would limit resolution since it's only really useful for modes that fit their framebuffer within 256 KB. Of course using all of this stuff would be up to the user and the responsibility of any consequences would be theirs since I really can't guarantee any operation like this will work. A related idea would be having many PCI video cards on one system and running them all using VESA but doing dynamic reconfigurations to switch between them (but being able to render to only one at a time). I don't know if this is possible though since I have yet to read the official PCI docs. You may have noticed with multiple user input and output devices I'm thinking of allowing many users per computer; I feel computers these days are a little too powerful to have one per user especially given multi-core CPUs.

Yet another thing I thought up was bus and device re-enumeration after resuming from suspension. I have some old laptops with only APM support and you can swap in and out things like floppy and CD-ROM drives while suspended, and in order to be able to use them you would normally have to reboot. I of course plan on supporting legacy hardware like ISA since reviving old machines has always been a dream of mine and I probably wouldn't be able to catch up to current standards anyway.

With all my far-fetched ideas you may be thinking I have no code to back it up. I do however have some and I've been experimenting with pmode and hardware interfacing but I want to sort out most of how these ideas would be implemented and their consequences before I actually do code them. Happy coding!

Hi,

blip wrote:With all my far-fetched ideas you may be thinking I have no code to back it up. I do however have some and I've been experimenting with pmode and hardware interfacing but I want to sort out most of how these ideas would be implemented and their consequences before I actually do code them. Happy coding!

Not at all! If I had code to back-up all my ideas I'd be a very happy man, but this isn't going to happen without a lot of work.

BTW, you might be interested in this newsgroup post... The last paragraph may sound familiar..


Cheers,

Brendan

blip: video cards with VBE 2 (VESA) support allow you to have a full frame buffer. I've written a test that switches to 800 x 600 in 24 or 32 bits color. So that's not really a problem except you don't have any hardware acceleration support or vertical retrace sync etc.

See this thread for the example:

http://www.mega-tokyo.com/forum/index.php?board=1;action=display;threadid=9169

I'm wondering if the whole multiple keyboard / mouse thing isn't a lot easier todo with USB devices?

p.s. that VBE / VESA method only supports one (the main) graphics card. Unfortunately no dual monitor output. Of course you could setup other video graphics to "standard" VGA resolutions.

Haven't I read in several places that due to the PS/2 mice and keyboards not needing to support hot-plugging (in the standard at least), the controllers in some motherboards (the cheaper ones usually IIRC, though also some more expensive ones that stick to the letter of the standard) can be fried if you try plugging/unplugging the devices too much?

I read the whole thing, Brendan. Major deja vu. When reading your post I was like "whaaaaaat" since you are the first person I know of that has also programmed this idea. I have other crazy ideas dealing with non-standard floppy formats including breaking the 2 MB "barrier" by using perpendicular recording on outer tracks of HD disks given a proper drive and controller. The idea is not totally mine, some guy wrote a utility over a decade ago that could make almost 2 MB disks from 3.5" HD floppies and almost 4 MB with ED ones. It also included routines to write to 5.25" 360 KB floppies in a 1.2 MB drive that would write at a higher density on outer tracks. Funny thing, I also considered support for user-modified hardware. This one project I have yet to do with a friend deals with modifying a floppy drive so that you can control motor speed, slowing it on outer tracks in order to fit more data. I'm interested in good floppy support since some of my old computers don't have working hard drives, including two ancient laptops.

I have additionally thought of integrating USB devices into this scheme, Kemp, but that is a long way off, that is if I ever do get to it anyway. I have read that some KBCs and keyboards/mice can fry if hotplugged, and I do not doubt it, but I have never had a problem myself and really all responsibility lies with the user. If you guys want to know, when you insert a PS/2 device it performs the Basic Assurance Test and sends the completion status byte, and this is what interrupts you upon plugging it in. I do not recall off the top of my head whether it sends its ID also or if you have to query it but keyboards seem to default to scancode set 1 though I have heard of some that only support set 2. Actually my multi-user idea stemmed from a system I had setup where me and my brother could both use the same computer in Windows XP (two mice and two monitors) but we did have to share a keyboard and be aware of who's program had focus.

Rob, I have read the VBE 3.0 document several times and I feel comfortable with my understanding. In one of the versions (I think 3.0 though) you have flags that tell you whether or not the modes are VGA compatible so you can implement vertical retrace synching in the appropriate cases and stay away from VGA registers otherwise. I gave the example of setting the 800x600x16 mode on VGA since it is not well known and just plain neat for anyone running a vanilla VGA. Also since it is an unchained mode you can optimize some drawing routines especially horizontal lines and area fills. This is another advantage of VESA/VGA hybrids, if you can get them to work that is. I have yet to read the PCI docs to find out if it is possible to dynamically reassign PCI resources to be able to use VESA on multiple video cards without blanking the display. Like I previously said, if you could you would be limited to rendering to one at a time but that's better than being limited to using VESA only on the primary card.

P.S. Sorry for the long post again. :-\

blip: personally I prefer the higher resolutions & colors, no 16 color modes for me.

Actually VBE 2.0 (or 1.2, forgot) already supports the VGA compatability flag. The problem is that on the machines I've tested it with it isn't available for any of the higher resolutions (modern cards). So that doesn't help me much. Guess I'll have to live without vsync for a while.

Hi,

blip wrote:I read the whole thing, Brendan. Major deja vu. When reading your post I was like "whaaaaaat" since you are the first person I know of that has also programmed this idea.

I didn't do hot swap support (it didn't occur to me at the time ), but it did handle the same PS/2 device being disconnected and reconnected. This is mainly because I'm use some older mechanical KVM (Keyboard, Mouse, Video) switches here (which are just electrical switches that allow the same devices to be switched to different computers). For most OSs, if you use a switch like this the keyboard and mouse lose their state (LEDs, typmatic rate, etc). When my OS received the BAT it'd resend the device state so this didn't happen. This means you should be able to unplug a device after it's been detected and then plug it back in, but if the device wasn't plugged in during boot or if you swapped it with a different kind of device you'd be in trouble. BTW on reconnections you get a BAT, but you don't get the device ID unless you request it..

For floppy disks, the same version of the OS handled 1680 KB disks (similar to a 1440 KB disk but formatted with 21 sectors per track instead of 18). IIRC Microsoft themselves used this format - I think it was Win95 or Win98 that came as a set of 12 1680 KB disks...

The Commodore 64 did pack more data onto outer tracks (with intelligent disk drives that had their own CPU, RAM, etc and operated more like a primitive file server than a disk device). I'm not sure if they used variable speed drive motors or if they read/wrote the data at variable frequencies (I'd assume the latter). PC hardware isn't capable of doing any of this (it's cheaper to manufacture), but does support 4 different data frequencies (the jump from 512 Kbps to 1 Mbps is too much though - you'd want 512 Kbps, 520 Kbps, 528 Kbps, 530 Kbps, etc).

I've never even seen a "2 MB disk drive" - I think they were available for 3 minutes and got made obsolete by CD-ROM soon after.

All OSs support user-modified and user-invented hardware (it's called "write your own driver" hardware)..

USB devices would also work for "many user computers", but USB is much harder to write drivers for. In this case the maximum limit would depend on how many video cards you could put in the computer. I've heard of computers with 9 video cards - can you imagine a computer with 9 users all doing their own thing?

For a distributed OS (where a user isn't limited to the resources of one computer as much) this sort of thing is important. Consider the hardware costs - an office with 5 "dual-user" computers running my OS compared to the same office with 10 "single-user" computers and an expensive server running Windows or Linux (where most of the computers spend most of the time waiting for a keypress anyway).


Cheers,

Brendan

I read a paper once by guy that R&Ded an idea in which he migrated the entire OS from one computer to another. Now that was pretty sweet.

Edit: link: http://www.diku.dk/~jacobg/nomadxen.pdf

Self-migration looks impossibly difficult for me but it is nonetheless very neat. Hats off to anyone that tries it!

Folks, I admit my KBC knowledge is rusty. I found PS/2 hot-swapping support a little tricky (only implemented in test programs in DEBUG) since most of the troubles occur when you try to send a command to the device and nothing is there. For a while all I knew is that it would freeze your system, and this was a problem with my TwoKeybs TSR if you unplugged one and pressed a lock key that would make it try to update both sets of LEDs. Some time later I wrote a test program to find the value of the status register when this was attempted. I found there was a timeout error bit set and apparently modifying the KBC command byte fixes this (IIRC my code cleared some inhibit flag). By doing this you can periodically check for an unplugged device without inadvertently locking out KBC interrupts. BTW Windows 98 is able to load with two keyboards but it takes about 10 seconds to timeout trying to communicate with the secondary keyboard as if it were a mouse.

One time I tried to write my own serial and PS/2 mouse driver TSR that included all these capabilities but it proved difficult to implement everything I wanted. For one it would probably be smaller than Microsoft's. Redoing BIOS keyboard support to work on a mouse in primary port and keyboard in auxillary was difficult and would break any USB keyboard support. Also pretty much every program that talks to the hardware directly expects the keyboard on the primary port, something a V86 monitor would be well-suited to emulating but wasn't in the plans and by doing so you would prevent things like EMM386 from loading or it would keep the TSR from loading, depending on which was initiated first. Along the way I didn't perform enough testing as I coded, so something messed up the stack but I never figured out what. Really this whole task should be in an OS and not hacked on to DOS so I quit that.

I use DMF disks all the time especially with my older machines. However I didn't know the C64 could roughly maintain data density like that. I have one (been sitting on a shelf for months) but the extent of what I can do is limited to what I can PEEK and POKE in since I don't have any assemblers, debuggers, or interfaces with a PC. I admit 2 MB disks aren't spectacularly larger than DMF ones but they are around the physical limit.

I wonder if anyone has seriously considered using video memory as swap space. For example, my sister's computer has 512 MB of RAM but also an AGP 4x card with 256 MB. I haven't the slightest clue how this would be implemented but at least in theory you could speed up swapping pages out by sending in parallel one fraction to the hard drive and the rest to the video card. Perhaps one could use VBE mode 81FFh or set up many offscreen buffers in VRAM? Just guessing.

Self-migration looks impossibly difficult for me but it is nonetheless very neat. Hats off to anyone that tries it!

It's identical to teleportation and the same difficulty, except that programming is defining how to do something in a language for the purpose of defining how. So, instead of making all your atoms stop moving instantly, you tell the atoms to freeze themselves. Then, you transport the freeze-dried "corpse" to the correct place and you tell it to relive. Some parts are slightly harder and need to be fixed up (if you were breathing you need to be reconnected to the generic air at the new place instead of having a loose link to the previous place... if you were on the phone with a friend the link will have to be restored to the new cell or to a copy/teleportation of your friend.

In OS terms: You tell the process to freeze (which is kind of trivial, although it would probably prefer to know before you freeze it, due to some weird algorithms not liking freezing). You then transport the process image, reconnect it (open files, socket connections etc.) and restart it. It's probably easiest to kill the source process after it's restarted at the other point. For sockets, I imagine a new protocol is defined at the OS level where it is told by a given system that a certain protocol connection on a given port needs to reconnect transparently to another system. It kind of needs to be protected and secure because if you could sniff traffic you could redirect any traffic on your computer. On the other hand, just making user accounts work properly would also help with that .

Anybody feel like making a proof-of-concept? I think I'm not going to add this to my list . Also, you seem to like the idea more.

Thanks for the informative explanation. With that comment I gave I was trying to imply that I wasn't planning to implement that feature. I like most if not all the ideas presented in this thread but it's just a matter of devotion, time, and compromise (sounds a lot like how people describe marriage ).

I am going to be implementing migration, but my intent is to use it more to the effect of transparent load distribution over a network (meaning the processes that are migrated are more just back end things instead of entire sessions of a program's execution as the user sees it), but with the right design it may be possible to also migrate the front end independently....

I really should avoid getting interested in more lofty goals, but please keep us up to date on your progress here. This is always interesting stuff.