OSDev.org

Posted: **Sat Mar 16, 2013 10:00 am**

Hi,

Inspired by this thread, and especially DavidCooper's BwtSecOS, I took out some time to write Selfer. Selfer has got similar goals to BwtSecOS, and even though I cheated a bit by writing it in assembly, I think it does the job pretty well...

So, what all does it do?

First of all, Selfer assumes 0x7C00 to represent LBA 0, 0x7E00 to represent LBA 1, and so on. It then displays a byte stream from 0x7C00 to 0x7E00, makes the first byte "green" in color, displays the address of the first byte down below, and enters into an input loop, which, for the following characters, behaves as mentioned:

Left, or right arrow key. You can move between offsets into the sector, like this. Left key for previous byte; right key for next byte. I display a nice green color to indicate which is the "current" byte, but don't really change the cursor position. It also wraps around, and displays the address of the current byte down below.
Down, or up arrow key. You can move between sectors, like this. Up key for previous sector, and down key for next sector. Moving between sectors automatically writes the current one to disk. Also, note that the lower bound is 0x7C00; the upper bound is 0x80000. If you look at the hexdump, it's pretty easy to change these values yourself.
'j'. This jumps to whatever the current address 0xYXXXX is, like 0xY000:0xXXXX. It also pushes a return address, so you can do a 'ret', and get back into the loop.
'r' & 'p'. 'r' copies the current value of the byte we're at, while 'p' pastes the value you copied last. It's supposed to paste 0x00, if you copied nothing.
'w'. That writes the current sector. Just in case you don't want to go the next/previous sector for writing.
'>' & '<'. '>' increments the current byte (greater than), while '<' decrements it (smaller than).
Input. Other than that, you can directly input hexadecimal values (both upper and lowercase accepted). It takes in the input, and moves to the next byte. It's not exactly safe, so try only to enter hexadecimal values.

I'm really hoping for someone like Brendan dissecting it, and probably helping me figure out how to reduce it's size, make it neater, add more features, etc. It was fun working on this, and it took me a day or two, actually.

Thanks to Antti, for the idea; DavidCooper, for BwtSecOS; nortti for attempting to figure out what it's about from the hexdump.

Bugs

I've tried to fit a lot in there, so there are some bugs which I'm trying to fix without cutting down on functionality. These are (and the list would be extended as/when bugs are found):

In 'j' handling, while I do jump between segments, I don't push the segment for Selfer, which means that if the code jumped to returns without resetting CS, plausible bugs can ensue! But then, again, I don't reset 'ebp' either. I don't fix a lot of stuff. Returning from code you jumped to, isn't... advisable. EDIT: FIXED. You now need a far ret to properly return back to the event loop. It then resets ebp, and everything should work fine (as long as you don't touch the first 6 bytes of 0x500, and the stack ain't messed up).

Regards,
Shikhin

EDIT: Things have changed for Selfer. You'd need to scroll down a bit, to see what all's different, now.

Posted: **Sat Mar 16, 2013 5:06 pm**

I've just run it in Bochs (after going through the disassembled version and deleting everything except the hex in order to be able to paste it into a hex editor to create a disk image). I like it a lot - I see that it can save the modified bootsector too (I think the 3rd last byte is safe to change), but it looks as if it it could kill a cheap flash drive pretty quickly with all that saving, assuming that it can run from one (I haven't tried). If you hold down the down-cursor key it will write a half megabyte block of flash memory on each repeat just to modify a single sector. It's absolutely fine to use as an image in an emulator though.

I think you should drop the > and < functions as it's almost always going to be much quicker typing hex values in, so the space used for the > and < code can be recovered to enhance the hex input. I only provided functions like > and < in BwtSecOS because that was the only way to modify bytes there due to the lack of space left after providing both hex and decimal display modes.

I also doubt that the copy byte and paste byte functions are useful in their current form, although it would certainly be useful if it could copy strings of bytes from one location to another, with each press of p copying another byte across - that would make a huge difference whenever you need to add code into the middle of a routine to fix a bug as it would make it easy to open up enough space for the necessary modifications, though clearly the copy and paste would have to start at the far end and work backwards towards the point where you need to make a space. (There isn't the same need to close gaps as you can just fill them with nops or jump past them.)

Here's the hex on its own just in case anyone else needs to create an image via a hex editor (to save them doing all the same work again):-

Code: Select all

EA 3A 7C 00 00 66 01 F5 66 89 EB 66 C1 EB 04 81 
E3 00 F0 8E EB 89 EB 66 29 F5 C3 B0 24 31 DB B4 
0E CD 10 EB FE 66 89 EB 66 81 EB 00 7C 00 00 66 
C1 EB 09 31 FF 47 E8 05 01 C3 31 DB 8E D3 BC 00 
7C 8E DB 9C 58 F6 C4 80 75 D1 80 F4 30 50 9D 9C 
59 31 C8 84 E4 75 C4 FC 88 16 00 05 88 1E 01 05 
B0 03 CD 10 66 0F B7 EC 66 31 F6 31 FF E8 CE 00 
E8 2A 01 E8 8F FF 30 E4 CD 16 80 FC 4B 75 04 4E 
E9 B4 00 80 FC 4D 75 04 46 E9 AB 00 3C 6A 75 07 
68 89 7C 0F A8 53 CB 3C 72 75 09 65 8A 07 A2 01 
05 E9 93 00 3C 70 75 09 A0 01 05 65 88 07 E9 86 
00 3C 77 75 05 E8 6D FF EB 7D 80 FC 50 75 0B 66 
81 FD 00 FE 07 00 74 6F EB 0E 80 FC 48 75 29 66 
81 FD 00 7C 00 00 74 5F E8 4A FF 80 FC 50 75 0A 
66 81 C5 00 02 00 00 43 EB 08 66 81 ED 00 02 00 
00 4B 4F E8 48 00 EB 3F 3C 3E 75 05 65 FE 07 EB 
36 3C 3C 75 05 65 FE 0F EB 2D 30 E4 66 C1 E0 10 
CD 16 86 E0 66 C1 E8 08 B9 02 00 2C 30 3C 09 76 
08 2C 07 3C 0F 76 02 2C 20 86 C4 E2 EE C0 E0 04 
C1 E8 04 65 88 07 46 81 E6 FF 01 E9 32 FF 66 60 
93 31 F6 E8 BF FE 8C E9 8E C1 BE 03 00 31 D2 B9 
12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 
C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 
73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 9F FE 66 61 
C3 0F B6 C2 C1 E0 04 C0 E8 04 B2 02 3C 09 7F 04 
04 30 EB 02 04 37 86 E0 FE CA 75 F0 C3 66 60 B8 
00 B8 8E C0 31 FF B9 00 04 B8 30 0F F3 AB 66 89 
E9 01 F1 C1 E6 02 26 C6 44 01 02 26 C6 44 03 02 
31 F6 E8 40 FE 65 8A 17 E8 B6 FF 26 88 24 26 88 
44 02 83 C6 04 43 81 FE 00 08 72 E9 66 89 CA BE 
0E 0F B9 04 00 E8 99 FF 26 88 04 26 88 64 FE 83 
EE 04 66 C1 EA 08 E2 ED 66 61 C3 00 00 00 55 AA

[By the way, I took Brendan's advice and replaced my keyboard reading code in BwtSecOS with the Int16h BIOS function, thereby allowing the interrupts to stay enabled at all times so that the floppy disk motor will switch off automatically - I'd forgotten that would be an issue on older machines because I only use USB floppy drives now which always stop their own motors themselves without waiting for the BIOS to tell them to do so.]

Edit:-

"00000058 88160005 mov [0x500],dl ; save registers for later use"

I read somewhere that some BIOSes use the byte there as part of the BDA. Can anyone confirm that?

Edit: Have also run it in QEMU where it worked fine, though it took a few attempts to get QEMU to agree to work with it (the difficulty being in the way I tried to select the image I wanted QEMU to run).

Posted: **Sat Mar 16, 2013 11:04 pm**

Hi,

DavidCooper wrote:I've just run it in Bochs (after going through the disassembled version and deleting everything except the hex in order to be able to paste it into a hex editor to create a disk image). I like it a lot - I see that it can save the modified bootsector too (I think the 3rd last byte is safe to change)

Hrm. The version nortti disassembled was a bit older one, as compared to the Gist paste. I recommend that you use the Gist version, since it's now got 0 free bytes. It's pretty easy to make the Gist paste into a bootable floppy. Add something like "times 1474560 - ($ - $$) db 0" in the absolute end of the file. Then, assuming you've got nasm installed, just run "nasm Selfer.asm -o Selfer.flp." Selfer.flp should then be bootable!

DavidCooper wrote:It looks as if it it could kill a cheap flash drive pretty quickly with all that saving, assuming that it can run from one (I haven't tried). If you hold down the down-cursor key it will write a half megabyte block of flash memory on each repeat just to modify a single sector. It's absolutely fine to use as an image in an emulator though.

That's a valid issue. I'm thinking of tracking if the last input was next/previous sector, and if so, it skips the writing part? Obviously, that can be tricked (increase byte, then decrease byte, so nothing's been changed, as such), but we're not really here to trick Selfer.

DavidCooper wrote:I think you should drop the > and < functions as it's almost always going to be much quicker typing hex values in, so the space used for the > and < code can be recovered to enhance the hex input. I only provided functions like > and < in BwtSecOS because that was the only way to modify bytes there due to the lack of space left after providing both hex and decimal display modes.

I'm not too sure how to enhance the hex input more (it works, as of now); however, I can delete that and use the space reclaimed to implement some of the features you've suggested.

DavidCooper wrote:I also doubt that the copy byte and paste byte functions are useful in their current form, although it would certainly be useful if it could copy strings of bytes from one location to another, with each press of p copying another byte across - that would make a huge difference whenever you need to add code into the middle of a routine to fix a bug as it would make it easy to open up enough space for the necessary modifications, though clearly the copy and paste would have to start at the far end and work backwards towards the point where you need to make a space. (There isn't the same need to close gaps as you can just fill them with nops or jump past them.)

I like the idea; I'm going to try to implement it, though I'd need to cut down some bytes, across.

DavidCooper wrote:I read somewhere that some BIOSes use the byte there as part of the BDA. Can anyone confirm that?

I'm not too sure; I usually just refer to the wiki's article on the memory map, where it suggests that 0x500 is free for use. Also, AFAIR, MS-DOS uses that space to store some sectors too, so it should be fine.

Thanks for trying Selfer out! I'm going to update this post as soon as I go around to doing some of the things you suggested.

Regards,
Shikhin

EDIT: 200th post!

Posted: **Sun Mar 17, 2013 3:52 am**

Hi,

I updated the Gist, with the following features added/modified/removed:

'>' & '<' was removed, as rightly suggested by DavidCooper.
'r' and 'p' functionality was modified, a bit. How they function now is: at any address, you press 'r' and it "retains" the current address. Then, wherever you press 'p', it copies the value, and moves one cell behind. You press 'p' again, and it copies the value from behind the address you retained, and then moves one cell behind too. It's a bit like memcpy in case you want to move some buffer *ahead*, when the two buffers are overlapping. Someone might want to copy normally (i.e., cells move forward); in that case, you can simply look into the source and change the two decrements in the paste handling to increments.
Now, unless you write some hexadecimal value or paste something, sectors won't be written to disk.

Do note that the 'r' and 'p' handling for inter-segments is poor. If you're at address 0x10000 for example, it's going to fail horribly. Try to keep things simple, alright?

I'm open to suggestions, and everything. Please keep 'em coming. Here's the hexdump, by the way, if you're like DavidCooper:

Code: Select all

EA 33 7C 00 00 66 01 F5 66 89 EB 66 C1 EB 04 81 E3 00 F0 8E EB 89 EB 66 29 F5 C3 B0 24 31 DB B4 0E CD 10 EB FE 66 89 EB 66 C1 EB 09 83 EB 3E 31 FF 47 C3 31 DB 8E D3 BC 00 7C 8E DB 9C 58 80 F4 30 50 9D 9C 59 30 EC 84 E4 75 D0 FC 88 16 00 05 B0 03 CD 10 66 0F B7 EC 66 31 F6 E8 43 01 E8 A4 FF 30 E4 CD 16 80 FC 4B 75 04 4E E9 CD 00 80 FC 4D 75 04 46 E9 C4 00 3C 72 75 0B 8C 2E 02 05 89 1E 04 05 E9 B5 00 3C 70 75 19 66 8B 3E 02 05 8E C7 66 C1 EF 10 26 8B 0D 65 89 0F 4E FF 0E 04 05 E9 93 00 3C 6A 75 09 6A 00 68 54 7C 0F A8 53 CB 3C 77 75 08 E8 6E FF E8 88 00 EB 7F 80 FC 50 75 0B 66 81 FD 00 FE 07 00 74 71 EB 0E 80 FC 48 75 38 66 81 FD 00 7C 00 00 74 61 E8 48 FF 80 3E 01 05 01 74 03 E8 5B 00 80 FC 50 75 0A 66 81 C5 00 02 00 00 43 EB 08 66 81 ED 00 02 00 00 4B 4F E8 40 00 C6 06 01 05 01 EB 32 30 E4 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 B9 02 00 2C 30 3C 09 76 08 2C 07 3C 0F 76 02 2C 20 86 C4 E2 EE C0 E0 04 C1 E8 04 65 88 07 46 30 C0 A2 01 05 81 E6 FF 01 E9 19 FF 66 60 93 31 F6 E8 BB FE 8C E9 8E C1 BE 03 00 31 D2 B9 12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 9B FE 66 61 C3 0F B6 C2 C1 E0 04 C0 E8 04 B2 02 3C 09 7F 04 04 30 EB 02 04 37 86 E0 FE CA 75 F0 C3 66 60 B8 00 B8 8E C0 31 FF B9 00 04 B8 30 0F F3 AB 66 89 E9 01 F1 C1 E6 02 26 66 81 24 00 02 00 02 31 F6 E8 3E FE 65 8A 17 E8 B8 FF 26 88 24 26 88 44 02 83 C6 04 43 81 FE 00 08 72 E9 66 89 CA BE 0E 0F B9 04 00 E8 9B FF 26 88 04 26 88 64 FE 83 EE 04 66 C1 EA 08 E2 ED 66 61 C3 00 55 AA

Regards,
Shikhin

Posted: **Sun Mar 17, 2013 3:38 pm**

Running it now - it works fine. I see that when copying, when you hit the start of the sector it starts to read from the previous sector and writes to the end of the current sector, but it hasn't crashed. If you were actually to build an OS with this you'd probably put one or two routines in each sector and space them out well so that you can use the copy and paste function to open up spaces within them to add modifications without any of your other routines being affected, so there would rarely be a need to run the copy and paste function through sector boundaries.

You've certainly created something up to the task - any additional capability can be written using Selfer itself. The main problem with it would be reading your code as you'd always need to drag the cursor through the point you're trying to read in order to tell where one byte ends and the next starts. I don't know if you can reclaim enough space to put gaps between them or to colour alternate bytes differently, though it isn't hugely important as anyone who actually used a program of this kind would immediately want to develop their own display & edit routines and only really depend on the bootsector code after that for loading and saving.

I certainly prefer the way you've done things with a moving cursor and the bytes staying in place - I wrote BwtSecOS partly for artistic reasons, inspired by the idea of a Turing Machine with a tape (memory) running through a stationary processor, and with that image in my mind I simply had to see how it would actually look for real. Having seen it, I think it would drive you to drink if you actually used it for any length of time!

Posted: **Sun Mar 17, 2013 3:58 pm**

Having thought about it a bit more, it would be a lot less costly just to colour the tens and units differently rather than making alternate bytes different colours. I reckon if you can do that, it can be considered to be a fully competent single-sector development system.

Edit: I also forgot to say that by enhancing the hex input, I was thinking in terms of displaying the first digit when it's typed in, but that isn't vital and shouldn't be a priority.

What actually matters most is getting the j function sorted, because I've just tried to run a ret instruction with it and it immetiately jumps back to address 7BFE, then the down-cursor key takes the display two bytes on from there to 7C00 but calls it 7DFE. I can go to and fro between these two display positions by using the cursor keys up, down, up, down, etc., but if I try to go back further than 7BFE it then freezes up and I have to reboot.

Edit2: I didn't try the j function before today because I don't know machine code in hex form and I just assumed it would work, but I've just written code to add AL and CL together and to try to save the result somewhere, but it isn't working so far. I've just accidentally saved an accidentally modified version of the boot sector and it won't boot any more, so I'll have to start from scratch...

Edit3: Recovered - only had to change first byte back to EA. Right, I've tried to run the following code, typed in at 7E01:-

Code: Select all

B0 0A (mov AL,10),
B1 07 (mov CL,7),
02 C1 (add AL,CL),

1E (push DS),
33 C9 (xor CX,CX),
8E D9 (mov DS,CX),

A2 00 7E (mov [7E00],AL) [not sure how to write that in assembly language, but it's supposed to write the content of AL to the start of the second sector.]

1F (pop DS)

C3 (ret)

It should write the value 11h to the start of the second sector, but it doesn't. I've checked my hex values by converting them back to decimals and it all looks correct.

Edit4: Just tried the same code on the original version of Selfer and it works fine - a bug must have got into Selfer during later modifications.

Posted: **Sun Mar 17, 2013 8:06 pm**

Hi,

DavidCooper wrote:Running it now - it works fine. I see that when copying, when you hit the start of the sector it starts to read from the previous sector and writes to the end of the current sector, but it hasn't crashed. If you were actually to build an OS with this you'd probably put one or two routines in each sector and space them out well so that you can use the copy and paste function to open up spaces within them to add modifications without any of your other routines being affected, so there would rarely be a need to run the copy and paste function through sector boundaries.

It works okay on sector boundaries. Fails a bit when you're just on the start of a segment (0xYXXXX; if XXXX are 0x0000, then it suffers a bit of a problem).

DavidCooper wrote:You've certainly created something up to the task - any additional capability can be written using Selfer itself. The main problem with it would be reading your code as you'd always need to drag the cursor through the point you're trying to read in order to tell where one byte ends and the next starts. I don't know if you can reclaim enough space to put gaps between them or to colour alternate bytes differently, though it isn't hugely important as anyone who actually used a program of this kind would immediately want to develop their own display & edit routines and only really depend on the bootsector code after that for loading and saving.

I could reclaim enough bytes to display every alternate 4 bits in a different color, if I tried hard; would that do?

DavidCooper wrote:I certainly prefer the way you've done things with a moving cursor and the bytes staying in place.

Hehe, thanks!

DavidCooper wrote:Edit: I also forgot to say that by enhancing the hex input, I was thinking in terms of displaying the first digit when it's typed in, but that isn't vital and shouldn't be a priority.

That'd be tough, right now. I'll look into it, though, if I get around to getting few more empty bytes.

DavidCooper wrote:What actually matters most is getting the j function sorted, because I've just tried to run a ret instruction with it and it immetiately jumps back to address 7BFE, then the down-cursor key takes the display two bytes on from there to 7C00 but calls it 7DFE. I can go to and fro between these two display positions by using the cursor keys up, down, up, down, etc., but if I try to go back further than 7BFE it then freezes up and I have to reboot.

Heh, I'm sorry. I changed another thing, and forgot to mention it in the latest post (though I edited the original one to include it). Earlier, I used to jump between segments properly with 'j', but the code I jumped to would have had to reset CS to 0, to properly get back into the event loop. I got around to saving a few more extra bytes, so now I "push 0x0000" too. The proper way to return back would be a retf opcode (0xCB). Also note that I use the value of ESP to reset the value of EBP (which points at the current sector). This is so that I don't have to place any restrictions on the code being run itself; however, if the value of ESP is trashed, things can go bad.

DavidCooper wrote:Edit2: I didn't try the j function before today because I don't know machine code in hex form and I just assumed it would work, but I've just written code to add AL and CL together and to try to save the result somewhere, but it isn't working so far. I've just accidentally saved an accidentally modified version of the boot sector and it won't boot any more, so I'll have to start from scratch...

Edit3: Recovered - only had to change first byte back to EA. Right, I've tried to run the following code, typed in at 7E01:-
Code: Select all
B0 0A (mov AL,10),
B1 07 (mov CL,7),
02 C1 (add AL,CL),

1E (push DS),
33 C9 (xor CX,CX),
8E D9 (mov DS,CX),

A2 00 7E (mov [7E00],AL) [not sure how to write that in assembly language, but it's supposed to write the content of AL to the start of the second sector.]

1F (pop DS)

C3 (ret)
It should write the value 11h to the start of the second sector, but it doesn't. I've checked my hex values by converting them back to decimals and it all looks correct.

I modified 0xC3 in the end to 0xCB, and it works.

Also... I did something stupid. Right now, how it works is: as soon as you press 'j', it pushes a return segment and return address, as well as the segment and address to jump to. It then "retf"'s, which reaches into the code you want to jump to. Once you return from the code via another far ret (0xCB, to remind you), it then jumps back to the starting of the EventLoop. Here, it clears out EBP (i.e., moves the value of ESP into it, so don't mess with the stack!), as well as ESI. It then reads the first sector, and displays the values of it again...

The problem is, if you store anything at say 0x7E00, once you try to press the down key to go to the next sector, it'd read it from floppy and overwrite anything you wrote at 0x7E00. I suppose you'd have to deal with this, somehow. Maybe store values at somewhere beyond 0x80000, and write some sort of another output function to display and check the values. I checked the values via Bochs' debugger, and it looks fine to me.

Edit4: Just tried the same code on the original version of Selfer and it works fine - a bug must have got into Selfer during later modifications.

Hehe, the original version only demanded a ret. Fixed that, though it appears to be causing a bit of a pain (I just wanted to handle segments properly!).

Regards,
Shikhin

Posted: **Sun Mar 17, 2013 8:51 pm**

Hi,

I modified the Gist a bit. I hope you (DavidCooper) now know how to properly assemble it (I'd be willing to provide the hexdump, otherwise, but it just makes these posts a bit unreadable). Bytes start from the color red.

I've got another idea or two on how to cut size, so maybe I can fit in something to enhance hex input? However, I do hope it can now considered to be a "fully competent single-sector development system."

Regards,
Shikhin

P.S. The color scheme can be changed. Please suggest better ones?

Posted: **Mon Mar 18, 2013 2:32 pm**

Okay - I'd like to learn how to do it the proper way. I added the "times 1474560 - ($ - $$) db 0" part to the end of the file, as here:-

Code: Select all

...
BIOSSignature:
    dw 0xAA55

times 1474560 - ($ - $$) db 0

Then I got this when using "nasm Selfer.asm -o Selfer.flp" in Nasm (without the quotes):-

Selfer.asm:465: error: TIMES value -7 is negative

So, for the moment I can't run it unless I make a list file, edit out everything except the hex, etc. (or I could spend a few hours reading the manual to try to find out what's wrong). The result is that I haven't seen how much you've fixed since the last version.

I could reclaim enough bytes to display every alternate 4 bits in a different color, if I tried hard; would that do?

If it simply goes from white to cyan to white to cyan etc. that would do the job fine.

On the hex input issue, there's really no need to display half the byte while it's being typed in, so forget that I ever mentioned that. There is a real problem with the input though - if you start typing in a number by accident, you're then commited to changing it no matter what, which is likely to cause a crash if you're viewing the boot sector at the time. If you make this mistake elsewhere, you might at least get a chance to read the byte you're accidentally modifying before it's changed, unless you press another key before realising your error, by which time you've lost the original value. This is the main hazard with the system as you can make such an error by accidentally typing almost any key. If you do it in the boot sector and don't notice, you might then save the modified bootsector by moving to look at the next sector, so a bug may creep in without being noticed and wipe out lots of your work later on. Even a careful user would be bound to trip up on this at some point, so if any room can be found to fix this, it's worth doing. I won't suggest a solution as it's your program and I don't want to impose my idea of a fix on it, but it's likely to be hard to find room for it whatever way you try to do it.

The copy function has been used for real now, by the way - I needed it and it was there. I think the k key would sound more like a copy function than r though. If you changed that, it would also free up the r key to operate the "run" function that you have linked to the j key - the jump which is actually a call. Having said that though, it's also the case that both the jump and the paste functions could cause a crash if they're pressed by mistake, and adding protection to them would be costly, so moving these functions to the F1-F12 key row might be the best compromise.

With the jump function, I did try a far ret but thought it didn't work either - I now realise though that it would have been because the sector wasn't saved and my code simply disappeared when I ran it. This business of results disappearing is actually a serious problem as it will make debugging extremely difficult - you need to be able to post results somewhere and be able to view them.

It works okay on sector boundaries. Fails a bit when you're just on the start of a segment (0xYXXXX; if XXXX are 0x0000, then it suffers a bit of a problem).

Yes, I realised I'd misunderstood how it worked and meant to say so earlier. I don't think it's a problem though because by the time a programmer has reached the first segment boundary they should have switched to using their own edit & display code long ago: 32K is a huge amount of space to work with.

So, there are three issues that would be worth trying to fix: readability (being able to see byte boundaries clearly), accidental modifications (need a way to confirm that new input is wanted) and the ability to view results (so that you can see what your code actually does, thereby enabling you to look at the contents of registers at various points along the way). If you can fix all that, you'll then have created an educational tool of some value: people could use it to help them learn to read machine code in hex, perhaps by being challenged to create as much additional functionality as they can within the second sector.

Posted: **Mon Mar 18, 2013 5:10 pm**

DavidCooper wrote:Okay - I'd like to learn how to do it the proper way. I added the "times 1474560 - ($ - $$) db 0" part to the end of the file, as here:-
Code: Select all
...
BIOSSignature:
    dw 0xAA55

times 1474560 - ($ - $$) db 0
Then I got this when using "nasm Selfer.asm -o Selfer.flp" in Nasm (without the quotes)

The problem is in the code itself and not what you're doing

Code: Select all

1E8 0FB6C2                      movzx ax, dl
1EB C1E004                      shl ax, 4
1EE C0E804                      shr al, 4
1F1 B202                        mov dl, 2
                                  
                        .Loop:
1F3 3C09                        cmp al, 9
1F5 7F05                        jg .Char
                               
1F7 0430                        add al, 48
1F9 E90200                      jmp .Next
                                  
                       .Char:
1FC 0437                        add al, 55
                                  
                       .Next

1FE 86E0                        xchg ah, al
                 Boot signature just got overwritten

200 FECA                        dec dl
202 75EF                        jnz .Loop
                                
204 C3                          ret
                                  
            ; Padding.
   512                                  ;times 510 - ($ - $$) db 0
                         Evaluates to a negative number
   513                                  
   514                                  BIOSSignature:
   515 00000205 55AA                        dw 0xAA55

Posted: **Mon Mar 18, 2013 9:32 pm**

DavidCooper wrote:So, for the moment I can't run it unless I make a list file, edit out everything except the hex, etc. (or I could spend a few hours reading the manual to try to find out what's wrong). The result is that I haven't seen how much you've fixed since the last version.

nasm -v please.

Posted: **Mon Mar 18, 2013 11:03 pm**

Hi,

DavidCooper wrote:Okay - I'd like to learn how to do it the proper way. I added the "times 1474560 - ($ - $$) db 0" part to the end of the file, as here:-
Code: Select all
...
BIOSSignature:
    dw 0xAA55

times 1474560 - ($ - $$) db 0
Then I got this when using "nasm Selfer.asm -o Selfer.flp" in Nasm (without the quotes):-

Selfer.asm:465: error: TIMES value -7 is negative
So, for the moment I can't run it unless I make a list file, edit out everything except the hex, etc. (or I could spend a few hours reading the manual to try to find out what's wrong). The result is that I haven't seen how much you've fixed since the last version.

It appears to be a case of "no optimization" by default in versions prior to NASM 2.09. This leads to a bigger binary size. Either update NASM, or add a "-Ox" somewhere in the command line. Here's the hexdump, anyway (though I recommend you trying to assemble it):

Code: Select all

EA 33 7C 00 00 66 01 F5 66 89 EB 66 C1 EB 04 81 E3 00 F0 8E EB 89 EB 66 29 F5 C3 B0 24 31 DB B4 0E CD 10 EB FE 66 89 EB 66 C1 EB 09 83 EB 3E 31 FF 47 C3 31 DB 8E D3 BC 00 7C 8E DB 9C 58 80 F4 30 50 9D 9C 59 30 EC 84 E4 75 D0 FC 88 16 00 05 B0 03 CD 10 66 0F B7 EC 66 31 F6 66 60 B8 00 B8 8E C0 31 FF B9 00 02 66 B8 30 0C 30 0F F3 66 AB 66 89 E9 01 F1 C1 E6 02 26 66 C7 04 30 02 30 02 31 F6 E8 80 FF 65 8A 17 E8 55 01 26 88 24 26 88 44 02 83 C6 04 43 81 FE 00 08 72 E9 66 89 CA BE 0E 0F B9 04 00 E8 38 01 26 88 04 26 88 64 FE 83 EE 04 66 C1 EA 08 E2 ED 66 61 E8 48 FF 30 E4 CD 16 80 FC 4B 75 04 4E E9 CC 00 80 FC 4D 75 04 46 E9 C3 00 3C 72 75 0B 8C 2E 02 05 89 1E 04 05 E9 B4 00 3C 70 75 19 66 8B 3E 02 05 8E C7 66 C1 EF 10 26 8B 0D 65 89 0F 4E FF 0E 04 05 E9 92 00 3C 6A 75 08 0E 68 54 7C 0F A8 53 CB 3C 77 75 08 E8 13 FF E8 88 00 EB 7F 80 FC 50 75 0B 66 81 FD 00 FE 07 00 7D 71 EB 0E 80 FC 48 75 38 66 81 FD 00 7C 00 00 76 61 E8 ED FE 80 3E 01 05 01 74 03 E8 5B 00 80 FC 50 75 0A 66 81 C5 00 02 00 00 43 EB 08 66 81 ED 00 02 00 00 4B 4F E8 40 00 C6 06 01 05 01 EB 32 30 E4 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 B9 02 00 2C 30 3C 09 76 08 2C 07 3C 0F 76 02 2C 20 86 C4 E2 EE C0 E0 04 C1 E8 04 65 88 07 46 30 C0 A2 01 05 81 E6 FF 01 E9 BE FE 66 60 93 31 F6 E8 60 FE 8C E9 8E C1 BE 03 00 31 D2 B9 12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 40 FE 66 61 C3 0F B6 C2 C1 E0 04 C0 E8 04 B2 02 3C 09 7F 04 04 30 EB 02 04 37 86 E0 FE CA 75 F0 C3 00 00 55 AA

DavidCooper wrote:
I could reclaim enough bytes to display every alternate 4 bits in a different color, if I tried hard; would that do?
If it simply goes from white to cyan to white to cyan etc. that would do the job fine.

It goes from red to white; but I can change the color scheme. Try it out.

DavidCooper wrote:On the hex input issue, there's really no need to display half the byte while it's being typed in, so forget that I ever mentioned that. There is a real problem with the input though - if you start typing in a number by accident, you're then commited to changing it no matter what, which is likely to cause a crash if you're viewing the boot sector at the time. If you make this mistake elsewhere, you might at least get a chance to read the byte you're accidentally modifying before it's changed, unless you press another key before realising your error, by which time you've lost the original value. This is the main hazard with the system as you can make such an error by accidentally typing almost any key. If you do it in the boot sector and don't notice, you might then save the modified bootsector by moving to look at the next sector, so a bug may creep in without being noticed and wipe out lots of your work later on. Even a careful user would be bound to trip up on this at some point, so if any room can be found to fix this, it's worth doing. I won't suggest a solution as it's your program and I don't want to impose my idea of a fix on it, but it's likely to be hard to find room for it whatever way you try to do it.

The fix I see is that, after entering one keypress (which isn't any special function like copy/paste), if the user enters "ESC", the first keypress is ignored. I already have two bytes to spare, so I guess I only need two or three more bytes to implement that. Freeing two or three bytes, on the other hand, would be tough...

DavidCooper wrote:The copy function has been used for real now, by the way - I needed it and it was there. I think the k key would sound more like a copy function than r though. If you changed that, it would also free up the r key to operate the "run" function that you have linked to the j key - the jump which is actually a call. Having said that though, it's also the case that both the jump and the paste functions could cause a crash if they're pressed by mistake, and adding protection to them would be costly, so moving these functions to the F1-F12 key row might be the best compromise.

Yes, moving them to the function keys seems more sensible. I'll do that.

DavidCooper wrote:With the jump function, I did try a far ret but thought it didn't work either - I now realise though that it would have been because the sector wasn't saved and my code simply disappeared when I ran it. This business of results disappearing is actually a serious problem as it will make debugging extremely difficult - you need to be able to post results somewhere and be able to view them.

I can fix that, though it appears that I'd need to impose some restrictions on the user code, though. As always, I'll try.

DavidCooper wrote:So, there are three issues that would be worth trying to fix: readability (being able to see byte boundaries clearly), accidental modifications (need a way to confirm that new input is wanted) and the ability to view results (so that you can see what your code actually does, thereby enabling you to look at the contents of registers at various points along the way). If you can fix all that, you'll then have created an educational tool of some value: people could use it to help them learn to read machine code in hex, perhaps by being challenged to create as much additional functionality as they can within the second sector.

Readability has already been fixed. Ability to view results would soon be added. Accidental modifications... could require some thought. I'll look into it, but creating an educational tool of some value surely has it's incentive.

@TightCoderEx -- update NASM, or add "-Ox" in the command line?

Regards,
Shikhin

Posted: **Tue Mar 19, 2013 12:08 am**

Hi,

I've updated the Gist, as always. Here's the hexdump:

Code: Select all

EA 33 7C 00 00 66 01 F5 66 89 EB 66 C1 EB 04 81 E3 00 F0 8E EB 89 EB 66 29 F5 C3 B0 24 31 DB B4 0E CD 10 EB FE 66 89 EB 66 C1 EB 09 83 EB 3E 31 FF 47 C3 31 DB 8E D3 BC 00 7C 8E DB 9C 58 80 F4 30 50 9D 9C 59 30 EC 84 E4 75 D0 FC 88 16 00 05 B0 03 CD 10 66 0F B7 EC 66 31 F6 66 60 B8 00 B8 8E C0 31 FF B9 00 02 66 B8 30 0C 30 0F F3 66 AB 66 89 E9 01 F1 C1 E6 02 26 66 C7 04 30 02 30 02 31 F6 E8 80 FF 65 8A 17 E8 57 01 26 88 24 26 88 44 02 83 C6 04 43 81 FE 00 08 72 E9 66 89 CA BE 0E 0F B9 04 00 E8 3A 01 26 88 04 26 88 64 FE 83 EE 04 66 C1 EA 08 E2 ED 66 61 E8 48 FF 30 E4 CD 16 80 FC 4B 75 04 4E E9 98 00 80 FC 4D 75 04 46 E9 8F 00 3C 6B 75 0B 8C 2E 02 05 89 1E 04 05 E9 80 00 3C 70 75 19 66 8B 3E 02 05 8E C7 66 C1 EF 10 26 8B 0D 65 89 0F 4E FF 0E 04 05 E9 99 00 3C 72 75 08 0E 68 62 7D 0F A8 53 CB 3C 77 75 08 E8 13 FF E8 8A 00 EB 4B 80 FC 50 75 0B 66 81 FD 00 FE 07 00 7D 3D EB 0E 80 FC 48 75 3D 66 81 FD 00 7C 00 00 76 2D E8 ED FE 80 3E 01 05 01 74 03 E8 5D 00 80 FC 50 75 0A 66 81 C5 00 02 00 00 43 EB 08 66 81 ED 00 02 00 00 4B 4F E8 42 00 C6 06 01 05 01 81 E6 FF 01 E9 F2 FE 66 C1 E0 10 CD 16 86 E0 66 C1 E8 08 3C 1B 74 E9 B9 02 00 2C 30 3C 09 76 08 2C 07 3C 0F 76 02 2C 20 86 C4 E2 EE C0 E0 04 C1 E8 04 65 88 07 46 30 C0 A2 01 05 EB C3 66 60 93 31 F6 E8 5E FE 8C E9 8E C1 BE 03 00 31 D2 B9 12 00 F7 F1 88 D1 FE C1 88 C6 80 E6 01 D1 E8 88 C5 8A 16 00 05 C1 E7 08 F8 B8 01 02 01 F8 CD 13 73 0C 30 E4 CD 13 4E 75 EF B0 40 E9 3E FE 66 61 C3 0F B6 C2 C1 E0 04 C0 E8 04 B2 02 3C 09 7F 04 04 30 EB 02 04 37 86 E0 FE CA 75 F0 C3 55 AA

The changelog is as follows:

You get colors to indicate bytes!
'k' and 'p' is copy&paste. 'r' is run. 'w' is write. I hope that works with all.
It doesn't clear the value of EBP and ESI, which means that the if you "far ret" back into the loop, you land into the same sector. Thus, you can now observe changes and such that you write in the same sector. Obviously, this would mean placing a restriction on the user code -- while, now, you can mess with the stack as much as you want, you'd need to save and restore EBP and ESI. You shouldn't have much of an urge to modify EBP in the first place. If you want to modify ESI, just push and pop it back? This, though, means that you can debug and the like.
If you enter one keypress (NOT a special function), and then press ESC, the first keypress is ignored. I couldn't add any more safety, unfortunately.

So, can we call it a day?

Regards,
Shikhin

Posted: **Tue Mar 19, 2013 6:15 pm**

Hi Shikhin,

"Nasm Selfer.asm -o Selfer.flp -Ox" worked, though Bochs couldn't find it until I changed the file into .img, but I'm there now. (I used the hexdump first though as it's much quicker.)

Shikhin wrote:It goes from red to white; but I can change the color scheme. Try it out.

A bit too much of a contrast there for me, but the best solution would simply be to provide in the manual the address(es) of the byte(s) which control the red colour and let users change it to a colour value they prefer. My guess is that a brighter colour closer to white would look less stark, such as cyan, yellow or magenta (which are twice as bright as blue green and red).

The fix I see is that, after entering one keypress (which isn't any special function like copy/paste), if the user enters "ESC", the first keypress is ignored. I already have two bytes to spare, so I guess I only need two or three more bytes to implement that. Freeing two or three bytes, on the other hand, would be tough...

Esc after one accidental key press should catch most mistakes, so it's a huge improvement.

Yes, moving them to the function keys seems more sensible. I'll do that.

Or let the user change them, though I don't know what values the Function keys return in AL and it may be to expensive to switch to using the scancode values in AH, so I'm guessing that you haven't moved them there yet for that reason. Even so, accidental key presses are more likely to be keys round the edge or next to keys you're meant to be pressing. K, L and ; are probably the least likely keys to be pressed by mistake, so the user could modify the program to use those for copy, paste and run if they know the addresses to write the new values to (which they would be doing from within Selfer and not with the source code).

Shikhin wrote: [*]It doesn't clear the value of EBP and ESI, which means that the if you "far ret" back into the loop, you land into the same sector. Thus, you can now observe changes and such that you write in the same sector. Obviously, this would mean placing a restriction on the user code -- while, now, you can mess with the stack as much as you want, you'd need to save and restore EBP and ESI. You shouldn't have much of an urge to modify EBP in the first place. If you want to modify ESI, just push and pop it back? This, though, means that you can debug and the like.

So long as the user knows to save and restore them it should be fine. If they are only using BP and SI rather than EBP and ESI, pushing and popping BP an SI would be sufficient. I used to use EBP extensively as a general purpose register, but I now use it primarily to point directly at blocks of local variables so that as many routines as possible can be re-entrant (and without having to move values onto the stack and back off it again), but there's no barrier to this as BP can simply be saved on the stack at the start and restored before the ret.

There's still an awkward limitation in that results can only be seen if they're posted into the sector you're viewing when you press r, but there's a workaround for that as you can use one sector as a debugging space and run the code you're testing with a far call from there with all the results being posted back to the debug sector. Another option would be to write code to display results to a free area of the screen, but it's much easier just to post results into memory and to examine them via the existing display routine.

So, can we call it a day?

It's getting into the area of diminishing returns - more and more work would be needed for less and less gain. The system you've designed for loading and modifying sectors was an interesting experiment, but I think it causes most of the problems, and it would still wear a flash drive excessively. I'd prefer to press w once or twice per session to save all my work in one go and not have things being saved and loaded automatically every time I make a small modification and take a quick look at another sector. Changing that might also win back some space, and limiting the active working zone to the first segment of memory could also simplify things while still leaving plenty of room to develop an operating system. However, the way you've done things is interesting precisely because it's so different form the way I built mine, and the most important thing is that it also works. I think this may well be the right time to stop, so you could maybe round it off by writing a final version of the manual.

The manual's already been written of course, but there are a few things missing from it. The user shouldn't need to understand the code in Selfer at all, but (s)he might like to know which byte(s) need to be changed to modify the display colour and which bytes need to be changed to move the w, k, p and r functions to different keys of their own choosing (so that they can do it within Selfer rather than modifying the source code). The most important missing information is details of where it would be best to patch in a jump/call to make it bypass Selfer's display code and boot up the user's operating system directly (once the user has built enough of an OS for it to support itself), at which point Selfer would serve as a loader.

Oh, and once you've settled on the final version, make sure you comment all your code properly - I still haven't managed to find my way through a lot of it and can't work out how it loads everything in in the first place. I think I'm going to have to look at the machine code in decimal form before I can follow it all...

Oh - that's a big problem! I haven't looked at it in decimal form, but I've done a new experiment in Bochs just to check something and I now realise why I couldn't see where it loads everything at the start - it's because it doesn't load anything at the start at all. The only way to run any OS which you build with Selfer is going to be to use Selfer to scroll down through it first (or at least into the second sector) before you can jump into its code. I thought from my earlier experiments in Bochs that it loaded everything up front, but I realise now that values were simply being retained in memory in Bochs and weren't being loaded in again by Selfer (reset in Bochs is not like pressing the reset button on a real machine as it preserves memory content).

That's it settled then - I think you are going to need to redesign the system for loading & saving code. It needs to load a large chunk of memory on booting (e.g. from 7E00 to FFFF of segment 0) and then to save that whole chunk (plus the bootsector) back to disk when the user presses w. The bootsector will also need to be saved so that it can be modified from within Selfer and for those modifications to remain in place. (I avoided the need for any modifications to the bootsector in BwtSecOS to be saved by making it run the code at the start of the second sector automatically if the second sector starts with a nop, but Selfer in its current form would need to have a jump/call patched into it and for the change to be saved.)

I don't know if you're still going to want to finish this project now as you doubtless have other fish that need frying, but changing the way it saves would make it a lot more flash-drive friendly. What you've already built is already a really good demo though, so the option is there just to declare the job as good enough and leave it as is. It would be hard, but I think there is enough there for someone to build an OS and to patch the bootsector with new load code at some point so that Selfer would effectively be replaced with a loader designed specifically for the new OS - they'd need to write their own display & edit code, then hack the bootsector with it and save the result using their own code too, and then they'd have to hope it all works first go.

Posted: **Wed Mar 20, 2013 1:01 pm**

There's also the issue of how to make backups. Ideally you should be able to do so without using another operating system, so if you're using floppy disks you should simply be able to switch disks and press w to create a perfect copy. BwtSecOS also fails at the moment in that regard as it would save everything except the bootsector, so new code would need to be written by the user to save the bootsector (though that would be a simple matter of creating a block of seven bytes to define what's to be saved and where it's to be saved to before calling the existing save routine in the bootsector.)

[Edit: Correction - the bootsector in BwtSecOS can't be saved using the bootsector's own save code as it would save it in a form which would fail to boot properly, so separate code to call the BIOS from another sector would have to be written to do that.]

If working from flash drives things become more problematic as some BIOSes may only support one flash drive such that it's the only place you can save the OS from within the OS. This is why being able to save through LBA to a partition is important: you would be able to edit the LBA start location before pressing w to save a copy to a new location in the partition, so if the main version got into an unrecoverable mess, the bootsector could still edit itself and change the start LBA, and then it could jump to its own first byte to load in an undamaged copy from another location in the partition. This would allow you to store thousands of copies on the same flash drive as you develop your OS and to backtrack to a stable version whenever a problem turns up.

Edit: I also meant to add that just loading and saving a single side of a single track would provide plenty of working space for a new operating system to be built, so that should save a fair bit of space in the bootsector as it can be done with a single call using very simple code to set up for it. It would be up to the user to make sure they don't write any code beyond the last sector that gets saved.

OSDev.org

512B (Selfer)

512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)

Re: 512B (Selfer)