512B (Selfer)

[shikhin]

Hi,

I've updated the Gist, and here's the hexdump:

Code: Select all

EA 10 7C 00 00 B0 24 31 DB B4 0E CD 10 F4 EB FD 31 DB 8E D3 BC 00 7C 8E DB 8E C3 9C 58 80 F4 30 50 9D 9C 59 30 EC 84 E4 75 DB FC 88 16 00 05 B0 03 CD 10 B5 26 B4 01 CD 10 31 F6 BD 00 FE 31 FF BB 41 00 E8 54 01 81 ED 00 02 4B 75 F6 80 3E 00 7E 90 75 09 89 EB 81 C3 00 02 E9 AB 00 66 60 06 B8 00 B8 8E C0 31 FF B9 00 02 66 B8 30 0B 30 0F F3 66 AB 89 EB 01 F5 95 C1 E6 02 26 66 C7 04 30 04 30 04 BE 06 0F B9 02 00 E8 4C 01 26 88 14 26 88 74 FE 83 EE 04 C1 E8 08 E2 EE 31 F6 8A 07 E8 36 01 26 88 34 26 88 54 02 43 83 C6 04 81 FE 00 08 72 EA 07 66 61 89 EB 01 F3 30 E4 CD 16 80 FC 4B 75 04 4E E9 9C 00 80 FC 4D 75 04 46 E9 93 00 80 FC 40 75 07 89 1E 01 05 E9 87 00 80 FC 42 75 02 EB 05 80 FC 41 75 1B 8B 3E 01 05 8A 0D 88 0F 80 FC 42 74 07 46 FF 06 01 05 EB 67 4E FF 0E 01 05 EB 60 80 FC 3B 75 08 55 FF D3 5D 31 F6 EB 53 80 FC 3E 75 10 31 FF 47 89 EB C1 EB 09 83 EB 3E E8 77 00 EB 3E 80 FC 3F 75 17 55 BD 00 FE 31 FF 47 BB 41 00 E8 63 00 81 ED 00 02 4B 75 F6 5D EB 22 80 FC 50 75 0C 81 FD 00 FE 74 17 81 C5 00 02 EB 11 80 FC 48 75 13 81 FD 00 7C 74 06 81 ED 00 02 EB 00 81 E6 FF 01 E9 F3 FE 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 3C 1B 74 E9 B9 02 00 2C 30 3C 09 76 08 2C 07 3C 0F 76 02 2C 20 86 C4 E2 EE C0 E0 04 C1 E8 04 88 07 46 EB C9 66 60 93 31 F6 89 EB BE 03 00 31 D2 B9 12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 32 FE 66 61 C3 0F B6 D0 C1 E2 04 C0 EA 04 B0 02 80 FA 09 7F 05 80 C2 30 EB 03 80 C2 37 86 F2 FE C8 75 ED C3 00 00 00 00 00 00 00 55 AA

The changelog is as follows:

• The code now loads the first few sectors from the floppy at 0x7C00 to 0xFFFF. F5 writes these all back to disk. F4 writes the current sector to disk. There is no inter-segment handling, anymore.
• F1 calls the current index. It does a simple call, so you can now return with 0xC3 (ret). Also, you don't need to save/restore BP/SI anymore.
• If the byte at 0x7E00 is 0x90 (nop), then it automatically calls it. Note that the procedure followed here is the same described in the above point.
• F6 behaves like the previous retain (copy). F8 behaves like the previous paste, which I now call move. F7 is paste, since it INCREMENTS both the pointers, as opposed to F8.

I still have 7 bytes to spare, and maybe I can remove back any dependence on function keys, if you've got some other cool idea. I hope I've done everything you described in your last but one post. A manual, describing the location of everything important, would be written, as soon as we figure out which one should be the final version.

There's also the issue of how to make backups. Ideally you should be able to do so without using another operating system, so if you're using floppy disks you should simply be able to switch disks and press w to create a perfect copy. BwtSecOS also fails at the moment in that regard as it would save everything except the bootsector, so new code would need to be written by the user to save the bootsector (though that would be a simple matter of creating a block of seven bytes to define what's to be saved and where it's to be saved to before calling the existing save routine in the bootsector.)

For floppies, I suppose you could simply switch them, yes.

If working from flash drives things become more problematic as some BIOSes may only support one flash drive such that it's the only place you can save the OS from within the OS. This is why being able to save through LBA to a partition is important: you would be able to edit the LBA start location before pressing w to save a copy to a new location in the partition, so if the main version got into an unrecoverable mess, the bootsector could still edit itself and change the start LBA, and then it could jump to its own first byte to load in an undamaged copy from another location in the partition. This would allow you to store thousands of copies on the same flash drive as you develop your OS and to backtrack to a stable version whenever a problem turns up.

I don't think anyone has a partition on a typical flash drive. You could, however, save it in different LBAs inside the flash drive via changing one value in the EventLoop.Write/EvenLoop.Save functions. I'll add that to the manual, obviously. Also, note that you can probably create a more "terse" backup system in the second sector yourself!

Regards,
Shikhin

[DavidCooper]

Hi,

Wow - that's a radical rewrite done at some speed, and it seems to work perfectly! I'm definitely going to try to build something on top of this, and it'll force me to work in hex which will be a challenge.

I like F5 for "save" as "5" looks a bit like "S". Having F4 to save a single modified sector is a good extra feature which will reduce floppy disk grinding, but it also retains a significant flavour of the original design.

The call works beautifully, including the automatic call on loading if a nop's been saved in the right place, so now Selfer can hide itself and boot up the OS it's been used to build without a user knowing anything about it. This nop would only be put in at a late stage: during the development phase the code can be checked by running it with F1 while still allowing Selfer to modify it if it crashes, but once it's reached the point where it's safe to run it automatically every time, that's when the nop can be written in, while a ret will be all it takes to return to Selfer to enable the OS to be saved.

The copy & paste or move functions are much improved now that it can copy in both directions - that'll be useful for closing gaps when code is simplified instead of just filling the recovered space with nops. I didn't think there'd be enough space to do that, so well done.

• I still have 7 bytes to spare, and maybe I can remove back any dependence on function keys, if you've got some other cool idea. I hope I've done everything you described in your last but one post. A manual, describing the location of everything important, would be written, as soon as we figure out which one should be the final version.

It may be complete now, but don't be too quick to spend those 7 bytes - save them up for now. I want to take my time to read the source carefully, and working on building something with Selfer may reveal something that might need those bytes applied to it.

There's also the issue of how to make backups. Ideally you should be able to do so without using another operating system, so if you're using floppy disks you should simply be able to switch disks and press w to create a perfect copy. BwtSecOS also fails at the moment in that regard as it would save everything except the bootsector, so new code would need to be written by the user to save the bootsector (though that would be a simple matter of creating a block of seven bytes to define what's to be saved and where it's to be saved to before calling the existing save routine in the bootsector.)

For floppies, I suppose you could simply switch them, yes.

I made a mistake in what I said there - I can't use the code in BwtSecOS to save the bootsector itself as it would save it in a state that would prevent the copy from loading, so I'd need to write code in another sector specifically to save the bootsector. I haven't looked to see how you've done things with Selfer, but hopefully you've found a way to avoid this problem.

I don't think anyone has a partition on a typical flash drive. You could, however, save it in different LBAs inside the flash drive via changing one value in the EventLoop.Write/EvenLoop.Save functions. I'll add that to the manual, obviously. Also, note that you can probably create a more "terse" backup system in the second sector yourself!

Yes - anything that's missing can be added by the user straight away. The important thing is to provide the most important functionality in the bootsector to minimise the amount of work the user has to do, but it's hard to say which functionality is most vital. As long as there's enough there to do the job, then it's passed the only crucial test, so anything else is a bonus.

All the flash drives I've worked with do have a partition on them, the location of the partition being set out in a partition table in the MBR. What I do with them is create a second partition by changing the top of the first partition in the partition table, for example turning a FAT32 partition from 15GiB to 14Gib and then making a partition for my OS at the 14GiB point. I then format the drive in Windows so that the FAT32 partition is adjusted to fit the reduced space. I think Windows simply goes by the partition table in the MBR and the byte that labels it as a FAT32 partition, but I'm going to have to check the BPB values (such as sector count) to make sure it's adjusting everything properly - Windows certainly reports a reduced capacity for the FAT32 partition after the change, but I can't remember now if I checked to see if the BPB values were also changed when I set up my flash drives, so it may be that when the FAT32 partition is full it might overrun into my partition. If that happens, the cure would be to change the BPB values before formatting the drive. I'll check the numbers later and edit the answer to that question here.

Okay I'm going to go and study the source code, and then I'm going to try try to build a little tool on top of Selfer.

[DavidCooper]

Okay - I've studied the source now.

Code: Select all

    ; At si * 4 (since every si represents two bytes, i.e. two characters, 
    ; and there are two bytes for each character in display memory) get red color.
    shl si, 2
    mov dword [es:si], 0x04300430

I think the 04 parts of 04300430 are the red colour, but it's dark red and hard to read. Try 0C300C30 for the brighter red.

More importantly though, if I'm reading things correctly, you're loading and saving a single sector for each BIOS call, and you're also doing it backwards. This means that a real floppy disk would have to spin 65 times instead of 4 to load or save everything, and a flash drive would have to save half a megabyte of flash memory 65 times instead of 4, which means 16 times as much wear as is technically necessary. [In order to save a sector, a flash drive has to read half a megabyte, modify 512 bytes of it and write the half megabyte back again.] That could be avoided though by just saving single sectors when they're modified, so it's actually workable just so long as the user understands the risks and avoids using F5 wherever possible. There are no problems with it at all when using a disk image in an emulator, of course, which is the way most people would use it.

Because it uses CHS, it would be impossible to use it in a high-altitude partition on a flash drive, so the FAT32 partition would need to be moved to a higher LBA starting point and a second partition (second entry in the partition table) would then be created underneath it. I don't know how big that partition could be, but if the emulated floppy allows more than 80 tracks it may provide plenty of space. As you say though, the user can write their own code to write to disk via LBA, so they can concentrate on developing that code first and liberate themselves from using Selfer's save routines... only it wouldn't be able to load back in from high altitude as it always has to load in on booting via CHS, so they'd still be stuck with using low altitude LBAs.

The best way forward may be to have two versions of Selfer: one loading and saving using CHS (for floppy disks) and the other loading and saving using LBA for flash drives. In BwtSecOS I provided code capable of doing both, but it cost a lot of space and resulted in very limited byte-editing routines, although I also used up a fair bit of space with the decimal/hex mode switch: there's probably no way to fit everything we'd like into a single sector, so every solution is probably going to be a heavy compromise.

Anyway, it looks as if I can build stuff in the second sector without needing to change it to work with future versions of Selfer, so I can get on with building my tool now.

[DavidCooper]

And here's the tool, adapted from Machine Editor in my own OS. Put this in the second sector and boot Selfer:-

Code: Select all

90 B8 00 00 8E C0 8E D8 66 B8 56 42 45 32 66 A3 00 80 B8 00 4F BF 00 80 06 1E CD 10 1F 07 66 A1 00 80 66 3D 56 45 53 41 74 01 C3 FC BE 0E 80 AD 50 AD 8E D8 5E BF 00 82 B9 00 02 B8 00 00 8E C0 F3 A4 B8 00 00 8E D8 90 BE 00 82 BF 00 84 AD 3D FF FF 75 01 C3 91 B8 01 4F 56 57 06 1E CD 10 1F 07 5F 5E B8 00 02 03 F8 EB E4

What it does is collect VESA tables to let you explore them before you try to write code to handle their content. Use the down-cursor key to scroll through them. At 8000h we have the table returned by function 4F00h int 10h. At 8200 we have a copy of the mode list, just in case the BIOS put it somewhere where Selfer can't view it. [The location where the mode list was actually put by the BIOS is found at the address and segment stated at byte 0E of the table loaded to 8000h, and on my one of my old 486s (neither work any more so I can't check) I think it was put somewhere in the screen memory area which Selfer can't reach.] From 8400 onwards we have the tables returned by function 4F01h int 10h, one for each mode in the list.

[shikhin]

Hi,

Wow - that's a radical rewrite done at some speed, and it seems to work perfectly! I'm definitely going to try to build something on top of this, and it'll force me to work in hex which will be a challenge.

Hehe, thanks.

I made a mistake in what I said there - I can't use the code in BwtSecOS to save the bootsector itself as it would save it in a state that would prevent the copy from loading, so I'd need to write code in another sector specifically to save the bootsector. I haven't looked to see how you've done things with Selfer, but hopefully you've found a way to avoid this problem.

I save everything important at 0x500 onwards; thus, nothing should be put on disk (and it should be safe to save the bootsector).

All the flash drives I've worked with do have a partition on them, the location of the partition being set out in a partition table in the MBR. What I do with them is create a second partition by changing the top of the first partition in the partition table, for example turning a FAT32 partition from 15GiB to 14Gib and then making a partition for my OS at the 14GiB point. I then format the drive in Windows so that the FAT32 partition is adjusted to fit the reduced space. I think Windows simply goes by the partition table in the MBR and the byte that labels it as a FAT32 partition, but I'm going to have to check the BPB values (such as sector count) to make sure it's adjusting everything properly - Windows certainly reports a reduced capacity for the FAT32 partition after the change, but I can't remember now if I checked to see if the BPB values were also changed when I set up my flash drives, so it may be that when the FAT32 partition is full it might overrun into my partition. If that happens, the cure would be to change the BPB values before formatting the drive. I'll check the numbers later and edit the answer to that question here.

...

Because it uses CHS, it would be impossible to use it in a high-altitude partition on a flash drive, so the FAT32 partition would need to be moved to a higher LBA starting point and a second partition (second entry in the partition table) would then be created underneath it. I don't know how big that partition could be, but if the emulated floppy allows more than 80 tracks it may provide plenty of space. As you say though, the user can write their own code to write to disk via LBA, so they can concentrate on developing that code first and liberate themselves from using Selfer's save routines... only it wouldn't be able to load back in from high altitude as it always has to load in on booting via CHS, so they'd still be stuck with using low altitude LBAs.

The best way forward may be to have two versions of Selfer: one loading and saving using CHS (for floppy disks) and the other loading and saving using LBA for flash drives. In BwtSecOS I provided code capable of doing both, but it cost a lot of space and resulted in very limited byte-editing routines, although I also used up a fair bit of space with the decimal/hex mode switch: there's probably no way to fit everything we'd like into a single sector, so every solution is probably going to be a heavy compromise.

Hrm. For that, I'd also need to be aware about the partition table (and which entry I am), and then have some code for int 0x13 extended. I could create another version of Selfer for that, but I suppose everyone would be willing to simply wipe their flash drive clean for Selfer.

Code: Select all

    ; At si * 4 (since every si represents two bytes, i.e. two characters, 
    ; and there are two bytes for each character in display memory) get red color.
    shl si, 2
    mov dword [es:si], 0x04300430

I think the 04 parts of 04300430 are the red colour, but it's dark red and hard to read. Try 0C300C30 for the brighter red.

I've changed that. I'll probably add it to the Gist with the manual.

More importantly though, if I'm reading things correctly, you're loading and saving a single sector for each BIOS call, and you're also doing it backwards. This means that a real floppy disk would have to spin 65 times instead of 4 to load or save everything, and a flash drive would have to save half a megabyte of flash memory 65 times instead of 4, which means 16 times as much wear as is technically necessary. [In order to save a sector, a flash drive has to read half a megabyte, modify 512 bytes of it and write the half megabyte back again.] That could be avoided though by just saving single sectors when they're modified, so it's actually workable just so long as the user understands the risks and avoids using F5 wherever possible. There are no problems with it at all when using a disk image in an emulator, of course, which is the way most people would use it.

Yes. For reading multiple sectors, there are a lot of hassles involved (e.g., only read sectors in one track; try to read single sector if contiguous fails). Mine would be a bit slower, but it's guaranteed to work. Obviously, F4 has been provided so that an actual floppy user can avoid writing the entire floppy every time.

And here's the tool, adapted from Machine Editor in my own OS. Put this in the second sector and boot Selfer:-

Code: Select all

90 B8 00 00 8E C0 8E D8 66 B8 56 42 45 32 66 A3 00 80 B8 00 4F BF 00 80 06 1E CD 10 1F 07 66 A1 00 80 66 3D 56 45 53 41 74 01 C3 FC BE 0E 80 AD 50 AD 8E D8 5E BF 00 82 B9 00 02 B8 00 00 8E C0 F3 A4 B8 00 00 8E D8 90 BE 00 82 BF 00 84 AD 3D FF FF 75 01 C3 91 B8 01 4F 56 57 06 1E CD 10 1F 07 5F 5E B8 00 02 03 F8 EB E4

What it does is collect VESA tables to let you explore them before you try to write code to handle their content. Use the down-cursor key to scroll through them. At 8000h we have the table returned by function 4F00h int 10h. At 8200 we have a copy of the mode list, just in case the BIOS put it somewhere where Selfer can't view it. [The location where the mode list was actually put by the BIOS is found at the address and segment stated at byte 0E of the table loaded to 8000h, and on my one of my old 486s (neither work any more so I can't check) I think it was put somewhere in the screen memory area which Selfer can't reach.] From 8400 onwards we have the tables returned by function 4F01h int 10h, one for each mode in the list.

Really neat. I think I'd use something similar with Selfer, to grab off the EDID data from my older monitor (since I wanted to do that for some experimentation).

Anyway, it's nice to see one of my project reaching most of it's goals. I'm particularly happy with what all I've been able to achieve with Selfer, given the fact that I doubted I'd even fit half of it in 512 bytes in the first place.

Regards,
Shikhin

[DavidCooper]

Hi Shikhin,

I meant to say last night: I found a couple of 04s in the bootsector and changed them to 0C - it turned the red of the cursor bright, and F4 saved the change successfully, so it's always bright now.

Having worked with it for a while I can tell you a few of the things that could be better but are probably impossible to improve given the limitations of space. Not having the ability to move the cursor up and down the screen means it takes a long time to get it where you want it. To fix that would probably need too much extra code, so I doubt it can be done without losing other functionality.

Something that might be possible to fix is the cursor position after using F1 - it jumps back to the start of the sector instead of staying where it was when F1 was pressed.

There is something that's very easy to fix though, and that's the crash that happens if the code you run with F1 changes DS - a push and pop around the call would fix that. And I've just tested ES - i see that it causes a freeze on ret if it's been changed, so that's two pushes and two pops required.

Realistically though, while we've both shown that one sector is enough for a self-contained development system to continue to build itself and an OS, it would actually be better to use two sectors and make something that's good enough for the user to be happy to go on using its display & edit routines for some time rather than immediately feeling the need to write something better to replace them. Rather than going on polishing something that will always be an unfortunate compromise, it might be better to build a 1KB system, although that could still be done by starting with a single-sector system which can be used to modify itself to turn it into a more polished 1KB development system. It might be possible to write everything in such a way that extra functionality can be patched into the first sector from the second to remove all the major deficiencies while still depending on the original routines, and the code in the third sector can then be run automatically if there's a nop in its first byte, just as happens with the first byte of the second sector already.

With BwtSecOS I was able to patch in new code to allow vertical cursor movement, or rather the equivalent of it as the cursor actually stayed where it was while everything else moved, but the display & edit code works in such an uncomfortable way (for experimental artistic reasons) that it would need complete replacement. The trick would be to write code that can still go on being used while the functionality is enhanced, and the challenge would be to minimise the amount of dead code in the bootsector once the enhancements have been added - ideally all of it should continue to be useful.

[shikhin]

Hi,

I've updated the Gist, as well as added your extension as a comment (on the Gist page, not in Selfer.asm). Here's the hexdump:

Code: Select all

EA 0F 7C 00 00 B0 24 31 DB B4 0E CD 10 EB FE 31 DB 8E D3 BC 00 7C 8E DB 8E C3 9C 58 80 F4 30 50 9D 9C 59 31 C8 84 E4 75 DC FC 88 16 00 05 B0 03 CD 10 B5 26 B4 01 CD 10 31 F6 BD 00 FE 31 FF BB 41 00 E8 5B 01 81 ED 00 02 4B 75 F6 89 EB 80 C7 02 80 3E 00 7E 90 0F 84 BC 00 66 60 06 B8 00 B8 8E C0 31 FF B9 00 02 66 B8 30 0B 30 0F F3 66 AB 89 EB 01 F5 95 C1 E6 02 26 66 C7 04 30 0C 30 0C BE 06 0F B9 02 00 E8 55 01 26 88 14 26 88 74 FE 83 EE 04 C1 E8 08 E2 EE 31 F6 8A 07 E8 3F 01 26 88 34 26 88 54 02 43 83 C6 04 81 FE 00 08 72 EA 07 66 61 30 E4 CD 16 89 EB 01 F3 80 FC 4B 75 04 4E E9 AB 00 80 FC 4D 75 04 46 E9 A2 00 80 FC 50 75 06 83 C6 28 E9 97 00 80 FC 48 75 06 83 EE 28 E9 8C 00 3C 6B 75 07 89 1E 01 05 E9 81 00 3C 6D 75 02 EB 04 3C 70 75 1A 8B 3E 01 05 8A 0D 88 0F 3C 6D 74 07 46 FF 06 01 05 EB 64 4E FF 0E 01 05 EB 5D 3C 72 75 0C 55 56 1E 06 FF D3 07 1F 5E 5D EB 4D 3C 73 75 0E 31 FF 47 C1 EB 09 83 EB 3E E8 6E 00 EB 3B 3C 77 75 17 55 BD 00 FE 31 FF 47 BB 41 00 E8 5B 00 81 ED 00 02 4B 75 F6 5D EB 20 3C 78 75 0C 81 FD 00 FE 74 16 81 C5 00 02 EB 10 3C 7A 75 13 90 90 81 FD 00 7C 74 04 81 ED 00 02 81 E6 FF 01 E9 E4 FE 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 3C 1B 74 E9 B9 02 00 2C 30 3C 09 76 02 2C 07 86 C4 E2 F4 C0 E0 04 C1 E8 04 88 07 46 EB CF 66 60 93 31 F6 89 EB BE 03 00 31 D2 B9 12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 2C FE 66 61 C3 0F B6 D0 C1 E2 04 C0 EA 04 B0 02 80 FA 09 7F 05 80 C2 30 EB 03 80 C2 37 86 F2 FE C8 75 ED C3 00 55 AA

The changelog is as follows:

• Previous sector is now via 'z', while next sector is accessed via 'x'. Run is 'r', retain is 'k', move is 'm' and paste is 'p'. 's' saves the current sector; 'w' writes all of them.
• I simplified some code in the Input hex thingy. The way I see it, smaller case should've stopped working. However, it still works (which is mildy funny). I suppose it's some side-effect of the way I convert ASCII to hex. I'll go through it via the Bochs debugger anyway, just to ensure I don't introduce any bugs.
• The down key moves 40 bytes ahead. The up key 40 bytes before. The problem is that this fails at the last line, since it's not exactly 40 bytes, and it moves a little more ahead. I hope everyone can bear with that.

I also added two consecutive NOPs in the code, which are preceded by a (short) jmp. The purpose of this is that you can replace these bytes with a jump to your own code, which can add additional input parsing. For example, you can test for a key like 'i', which can enable or disable interrupts. Of course, you got to jump back from your code to where the short jmp was supposed to jump. I'll detail this in the manual, if this is the last version.

Having worked with it for a while I can tell you a few of the things that could be better but are probably impossible to improve given the limitations of space. Not having the ability to move the cursor up and down the screen means it takes a long time to get it where you want it. To fix that would probably need too much extra code, so I doubt it can be done without losing other functionality.

Done, via only removing function key support and a little bit of Input meddling. I still need to completely debug the input thingy -- maybe there's some hidden bug lurking in there? You can help too!

Something that might be possible to fix is the cursor position after using F1 - it jumps back to the start of the sector instead of staying where it was when F1 was pressed.

There is something that's very easy to fix though, and that's the crash that happens if the code you run with F1 changes DS - a push and pop around the call would fix that. And I've just tested ES - i see that it causes a freeze on ret if it's been changed, so that's two pushes and two pops required.

Fixed.

Realistically though, while we've both shown that one sector is enough for a self-contained development system to continue to build itself and an OS, it would actually be better to use two sectors and make something that's good enough for the user to be happy to go on using its display & edit routines for some time rather than immediately feeling the need to write something better to replace them. Rather than going on polishing something that will always be an unfortunate compromise, it might be better to build a 1KB system, although that could still be done by starting with a single-sector system which can be used to modify itself to turn it into a more polished 1KB development system. It might be possible to write everything in such a way that extra functionality can be patched into the first sector from the second to remove all the major deficiencies while still depending on the original routines, and the code in the third sector can then be run automatically if there's a nop in its first byte, just as happens with the first byte of the second sector already.

The aim of my project was to see how far possible it is to stretch a self-building system in 512 bytes. I think we both managed to do pretty well with Selfer (and BwtSecOS). Of course, we can probably extend this to 1KiB and do a lot of more interesting things. 32KiB, and I think we can even manage some DOS-compatibility. 128KiB, and maybe we achieve some POSIX-compliance too? With 512 bytes, we've achieved something usable. I, myself, if I were to write something in hex, could live with Selfer for a long time before I needed any additional capability. I'd perhaps add "sector" copy & paste, but the user can do that himself. Let's just celebrate the fact that Selfer has achieved most of what we planned, anyway.

Oh, and if you want to send in some donations, I'd be willing to accept! </shameless-plug>

With BwtSecOS I was able to patch in new code to allow vertical cursor movement, or rather the equivalent of it as the cursor actually stayed where it was while everything else moved, but the display & edit code works in such an uncomfortable way (for experimental artistic reasons) that it would need complete replacement. The trick would be to write code that can still go on being used while the functionality is enhanced, and the challenge would be to minimise the amount of dead code in the bootsector once the enhancements have been added - ideally all of it should continue to be useful.

You can patch in for additional capability in Selfer too. I just made it a bit easier, so...

Regards,
Shikhin

EDIT: It appears as if the Hexadecimal Input code accepts both upper and lower case as a side effect of the way it converts it. Woot!

[DavidCooper]

Hi Shikhin,

I don't know how you managed it, but you've found yet room for functionality that I thought wouldn't fit. We both missed something though, and this is more good news - the BIOS can do some more of work for you. When you had < and > functions, I noticed that they only worked if you held the shift key down. I now notice that the functions tied to various letter keys don't work if you hold the shift key down, and in the source code I can see that you used lower case. It looks as if the protection against accidental presses of these keys can be obtained simply by changing them to capital letters in the source so that a shift key has to be used with them, so this is far better than putting them on the F1-F12 keys. No protection needed on z and x, of course, and k probably doesn't need it either.

I've ... added your extension as a comment (on the Gist page, not in Selfer.asm).

I'd like to improve it a bit as I was being very lazy when I wrote it, using hex numbers I already knew rather than working out others which would have been more direct. Also, because you've removed the freeze/crash on ret if DS/ES modified problem, I can cut my code down further.

[*]I simplified some code in the Input hex thingy. The way I see it, smaller case should've stopped working. However, it still works (which is mildy funny). I suppose it's some side-effect of the way I convert ASCII to hex. I'll go through it via the Bochs debugger anyway, just to ensure I don't introduce any bugs.

"a": 97-55=42
"A": 65-55=10
The difference between them is still 32, so when you throw away bit-5 (using shl al,4 and shr ax,4) you lose that difference.

[*]The down key moves 40 bytes ahead. The up key 40 bytes before. The problem is that this fails at the last line, since it's not exactly 40 bytes, and it moves a little more ahead. I hope everyone can bear with that. [/list]

That's not a problem at all - it's removed the real problem and made working with Selfer enormously more efficient.

I also added two consecutive NOPs in the code, which are preceded by a (short) jmp. The purpose of this is that you can replace these bytes with a jump to your own code, which can add additional input parsing. For example, you can test for a key like 'i', which can enable or disable interrupts. Of course, you got to jump back from your code to where the short jmp was supposed to jump. I'll detail this in the manual, if this is the last version.

That's good, but I'm wondering if it might be better to put the nops at the start of .Input - I think that would allow the user to add code not only to test for other key presses, but also to eliminate all the unused keys that might lead to accidental modifications of bytes.

Let's just celebrate the fact that Selfer has achieved most of what we planned, anyway.

Indeed - it's looking very good now. I'd still like it to be able to boot from a high partition and to write everything in four BIOS calls so that it can be used elegantly on flash drives, but that is asking too much as a lot of functionality would need to be moved out of the bootsector and then rebuilt in the second sector. Maybe an alternative version of Selfer which doesn't work on floppy disks could solve that problem though while keeping everything else in the bootsector - having two versions would be a reasonable compromise, and floppies are sufficiently obsolete that any user working from a flash drive (if they want to work directly on real hardware rather than within an emulator) probably wouldn't care if their code can't be run from a floppy disk any more than they'd like it to be run from paper tape with holes punched in it. Optical disks aren't too attractive either, so the most important place to be is on something that behaves like a hard drive.

Oh, and if you want to send in some donations, I'd be willing to accept! </shameless-plug>

Perhaps I should be asking for 1% of any donations - I hope that doesn't sound too greedy, but I've been spending a lot of time helping a rival to overtake BwtSecOS instead of developing BwtSecOS.

EDIT: It appears as if the Hexadecimal Input code accepts both upper and lower case as a side effect of the way it converts it. Woot!

You can thank whoever it was that designed the ASCII layout.

[shikhin]

Hi,

I don't know how you managed it, but you've found yet room for functionality that I thought wouldn't fit. We both missed something though, and this is more good news - the BIOS can do some more of work for you. When you had < and > functions, I noticed that they only worked if you held the shift key down. I now notice that the functions tied to various letter keys don't work if you hold the shift key down, and in the source code I can see that you used lower case. It looks as if the protection against accidental presses of these keys can be obtained simply by changing them to capital letters in the source so that a shift key has to be used with them, so this is far better than putting them on the F1-F12 keys. No protection needed on z and x, of course, and k probably doesn't need it either.

I find it a little limiting and slow to press shift while working inside Selfer. Moreover, only 's' and 'w' really clash with hexadecimal characters -- those two should cause the minimum damage anyway (unless you save something screwed). Obviously, the final manual would tell you the address of those bytes, so then you can change that yourself.

I've ... added your extension as a comment (on the Gist page, not in Selfer.asm).

I'd like to improve it a bit as I was being very lazy when I wrote it, using hex numbers I already knew rather than working out others which would have been more direct. Also, because you've removed the freeze/crash on ret if DS/ES modified problem, I can cut my code down further.

I'd love better extensions.

I also added two consecutive NOPs in the code, which are preceded by a (short) jmp. The purpose of this is that you can replace these bytes with a jump to your own code, which can add additional input parsing. For example, you can test for a key like 'i', which can enable or disable interrupts. Of course, you got to jump back from your code to where the short jmp was supposed to jump. I'll detail this in the manual, if this is the last version.

That's good, but I'm wondering if it might be better to put the nops at the start of .Input - I think that would allow the user to add code not only to test for other key presses, but also to eliminate all the unused keys that might lead to accidental modifications of bytes.

D'oh. I miswrote that. That should've been: "Of course, you got to jump back from your code to a specific address which I'll provide in the manual." From there, you can probably just check if input key is not hexadecimal characters at all and if not, you can jump the address I provide, or if it is a valid character, you can jump to "parse input" address. I'll provide the minor details in the manual.

Let's just celebrate the fact that Selfer has achieved most of what we planned, anyway.

Indeed - it's looking very good now. I'd still like it to be able to boot from a high partition and to write everything in four BIOS calls so that it can be used elegantly on flash drives, but that is asking too much as a lot of functionality would need to be moved out of the bootsector and then rebuilt in the second sector. Maybe an alternative version of Selfer which doesn't work on floppy disks could solve that problem though while keeping everything else in the bootsector - having two versions would be a reasonable compromise, and floppies are sufficiently obsolete that any user working from a flash drive (if they want to work directly on real hardware rather than within an emulator) probably wouldn't care if their code can't be run from a floppy disk any more than they'd like it to be run from paper tape with holes punched in it. Optical disks aren't too attractive either, so the most important place to be is on something that behaves like a hard drive.

Oh, I forgot to write this.

Here's a new Gist, and the hexdump is as follows:

Code: Select all

EA 0F 7C 00 00 B0 24 31 DB B4 0E CD 10 EB FE B4 41 BB AA 55 CD 13 72 ED 31 C0 8E D0 BC 00 7C FC 66 8B 4C 08 8E D8 88 16 00 05 66 89 0E 02 05 B0 03 CD 10 B5 26 B4 01 CD 10 31 F6 BD 00 FE 31 FF BB 41 00 E8 59 01 81 ED 00 02 4B 75 F6 89 EB 80 C7 02 80 3E 00 7E 90 0F 84 BA 00 66 60 B8 00 B8 8E C0 31 FF B9 00 02 66 B8 30 0B 30 0F F3 66 AB 89 EB 01 F5 95 C1 E6 02 26 66 C7 04 30 0C 30 0C BE 06 0F B9 02 00 E8 4F 01 26 88 14 26 88 74 FE 83 EE 04 C1 E8 08 E2 EE 31 F6 8A 07 E8 39 01 26 88 34 26 88 54 02 43 83 C6 04 81 FE 00 08 72 EA 66 61 30 E4 CD 16 89 EB 01 F3 80 FC 4B 75 04 4E E9 AB 00 80 FC 4D 75 04 46 E9 A2 00 80 FC 50 75 06 83 C6 28 E9 97 00 80 FC 48 75 06 83 EE 28 E9 8C 00 3C 6B 75 07 89 1E 01 05 E9 81 00 3C 6D 75 02 EB 04 3C 70 75 1A 8B 3E 01 05 8A 0D 88 0F 3C 6D 74 07 46 FF 06 01 05 EB 64 4E FF 0E 01 05 EB 5D 3C 72 75 0C 55 56 1E 06 FF D3 07 1F 5E 5D EB 4D 3C 73 75 0E 31 FF 47 C1 EB 09 83 EB 3E E8 6E 00 EB 3B 3C 77 75 17 55 BD 00 FE 31 FF 47 BB 41 00 E8 5B 00 81 ED 00 02 4B 75 F6 5D EB 20 3C 78 75 0C 81 FD 00 FE 74 16 81 C5 00 02 EB 10 3C 7A 75 13 90 90 81 FD 00 7C 74 04 81 ED 00 02 81 E6 FF 01 E9 E6 FE 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 3C 1B 74 E9 B9 02 00 2C 30 3C 09 76 02 2C 07 86 C4 E2 F4 C0 E0 04 C1 E8 04 88 07 46 EB CF 66 60 BE 06 05 66 31 C9 66 89 4C 04 66 89 4C 0C 66 C7 04 10 00 01 00 66 03 5C FC 89 6C 04 66 89 5C 08 C1 E7 08 B4 42 01 F8 8A 16 00 05 CD 13 72 03 66 61 C3 B0 40 E9 2F FE 0F B6 D0 C1 E2 04 C0 EA 04 B0 02 80 FA 09 7F 05 80 C2 30 EB 03 80 C2 37 86 F2 FE C8 75 ED C3 00 00 00 00 00 00 00 55 AA 

The changelog is as follows:

• It expects to be passed control from a valid and generic MBR, with proper stuff set up.
• It doesn't check for 386 or not, since those bytes were used to instead check if the Int 0x13 extensions are available or not.
• I have 7 bytes left with, so maybe some interesting use can come up with them.

Oh, and if you want to send in some donations, I'd be willing to accept! </shameless-plug>

Perhaps I should be asking for 1% of any donations - I hope that doesn't sound too greedy, but I've been spending a lot of time helping a rival to overtake BwtSecOS instead of developing BwtSecOS.

I'm willing to give 49%, since you've put in almost equal effort in Selfer. Of course, 100% of nil is still... nil.

Regards,
Shikhin

[dozniak]

I'm willing to give 49%, since you've put in almost equal effort in Selfer. Of course, 100% of nil is still... nil.

Proper first step would be to convert it from a gist into a proper repository with history.

[shikhin]

Hi,

I'm willing to give 49%, since you've put in almost equal effort in Selfer. Of course, 100% of nil is still... nil.

Proper first step would be to convert it from a gist into a proper repository with history.

At first, I was thinking of creating a new repository named 512B or something, in which I'd store all pretty little bootsectors. Then I realized that mostly all of the bootsectors would be only 1 file, anyway. It didn't make much sense to me to have a repository for 1 file (which might expand to more, later). I poked around a bit, and I remembered of Gist. Gists are pretty cool -- they're nearly like normal git repositories. You can see all my gists from here. You can can view the history for it too ("Revisions" on the left for each gist), can fork them, comment on them, and even star them.

I don't see what's the problem with them remaining as Gists. If you can cite any reason to switch to full fledged git repositories hosted on GitHub, I'd be sure to consider.

NOTE: As for the manual, I think I'd be putting up a polished version of that as a comment on both the files, once we (me and DavidCooper) agree on which is the final version.

Regards,
Shikhin

[DavidCooper]

Hi Shikhin,

I find it a little limiting and slow to press shift while working inside Selfer. Moreover, only 's' and 'w' really clash with hexadecimal characters -- those two should cause the minimum damage anyway (unless you save something screwed). Obviously, the final manual would tell you the address of those bytes, so then you can change that yourself.

It seriously isn't going to slow you down, and if you press m or p by mistake you could change an important byte without noticing. If you press r by mistake it'll typically cause a crash. If you press w by mistake you'll make 65 unnecessary rewrites of the same half megabyte block on a flash drive. These things do need to be guarded against. Even shift+s isn't going to slow you down, so I'd definitely put it on that as well.

I also added two consecutive NOPs in the code, which are preceded by a (short) jmp. The purpose of this is that you can replace these bytes with a jump to your own code, which can add additional input parsing. For example, you can test for a key like 'i', which can enable or disable interrupts. Of course, you got to jump back from your code to where the short jmp was supposed to jump. I'll detail this in the manual, if this is the last version.

That's good, but I'm wondering if it might be better to put the nops at the start of .Input - I think that would allow the user to add code not only to test for other key presses, but also to eliminate all the unused keys that might lead to accidental modifications of bytes.

D'oh. I miswrote that. That should've been: "Of course, you got to jump back from your code to a specific address which I'll provide in the manual." From there, you can probably just check if input key is not hexadecimal characters at all and if not, you can jump the address I provide, or if it is a valid character, you can jump to "parse input" address. I'll provide the minor details in the manual.

I still think the nops are in the wrong place:-

Code: Select all

cmp al, 'z'
jne .Input
nop
nop

It looks to me as if they'll only ever be run when you press z.

Oh, I forgot to write this. ...

That'll do nicely! I've just put it on a flash drive to test it, but I'm going to post this before I run it and I probably won't be back online till late tomorrow to tell you how it went. I plan to boot it from within my OS, booting it up from a subpartition of the multifloppy partition in which my OS resides (hundreds of different versions sit in different subpartitions, all of them bootable), but I will need to write a little bit of new code to line up ES:DI on a dummy partition table entry before it will work the same way with Selfer. My plan then will be to disable the w function and write new code to save everything in a manner that causes 1/16th as much wear, as well as adding code to modify the LBA stored at 502h so that Selfer too can store multiple versions of itself in different subpartitions.

I'm willing to give 49%, since you've put in almost equal effort in Selfer. Of course, 100% of nil is still... nil.

Even if it's nil, the percentage of nothing is still crucial. I won't accept more than 10%.

[shikhin]

Hi,

It seriously isn't going to slow you down, and if you press m or p by mistake you could change an important byte without noticing. If you press r by mistake it'll typically cause a crash. If you press w by mistake you'll make 65 unnecessary rewrites of the same half megabyte block on a flash drive. These things do need to be guarded against. Even shift+s isn't going to slow you down, so I'd definitely put it on that as well.

I suppose you're right -- I've updated both the Gists, with everything now in capital.

I still think the nops are in the wrong place:-

Code: Select all

cmp al, 'z'
jne .Input
nop
nop

It looks to me as if they'll only ever be run when you press z.

Yes, they only ever run when you press 'z'. However, the purpose of this is that you can rewrite that piece (from Selfer) to be something like:

Code: Select all

cmp al, 'z'
jne 0x7E00

At 0x7E00, you can then have something like:

Code: Select all

cmp al, 'i'                      ; Any character here. 
jne EventLoop.Input      ; If not 'i', must be hexadecimal input.

; DO MAGIC HERE.

; Go back to the event loop in Selfer.
jmp EventLoop.Next

Note that "jne .Input" in Selfer was a relative jump, and thus I needed to add two nops so that you can replace it with something like "jne 0x7E00". I hope this makes sense, now.

That'll do nicely! I've just put it on a flash drive to test it, but I'm going to post this before I run it and I probably won't be back online till late tomorrow to tell you how it went. I plan to boot it from within my OS, booting it up from a subpartition of the multifloppy partition in which my OS resides (hundreds of different versions sit in different subpartitions, all of them bootable), but I will need to write a little bit of new code to line up ES:DI on a dummy partition table entry before it will work the same way with Selfer. My plan then will be to disable the w function and write new code to save everything in a manner that causes 1/16th as much wear, as well as adding code to modify the LBA stored at 502h so that Selfer too can store multiple versions of itself in different subpartitions.

Heh, yeah -- that's a nice side effect, so you can change the dword at 0x502 to store multiple versions. As for writing new code to save better, I'd love to have that in Selfer, but making it "fool (BIOS) proof" in the limited amount of bytes would be tough...

Even if it's nil, the percentage of nothing is still crucial. I won't accept more than 10%.

10% it is (though I still support 49%).

Regards,
Shikhin

[dozniak]

I don't see what's the problem with them remaining as Gists. If you can cite any reason to switch to full fledged git repositories hosted on GitHub, I'd be sure to consider.

Many of the gists have 1 revision, even though they are clearly a modifications of a previous one. To see what changed, i have to chase them down, either from the posts here or from a chronological list and run manual diffs, not very handy. E.g. the selfer (extended) could just grow out of normal selfer, either as a branch or a natural development as master, not separate gist.

It's also more visible on the main github site and in your profile. Gists are much more hidden.

[shikhin]

Hi,

I don't see what's the problem with them remaining as Gists. If you can cite any reason to switch to full fledged git repositories hosted on GitHub, I'd be sure to consider.

Many of the gists have 1 revision, even though they are clearly a modifications of a previous one. To see what changed, i have to chase them down, either from the posts here or from a chronological list and run manual diffs, not very handy. E.g. the selfer (extended) could just grow out of normal selfer, either as a branch or a natural development as master, not separate gist.

Erm, no. I've got only two Gists in total -- one is Selfer, which is the normal version and it has 9 revisions. The extended version could be a fork of the original one, yes. I was stupid and forgot to do that, but, Gist still allows you to do it (even if I didn't).

It's also more visible on the main github site and in your profile. Gists are much more hidden.

That depends. If more and more people use Gists for this purpose, maybe GitHub includes it in your main profile, and more people also make sure to check people's Gists out?

Regards,
Shikhin