Page 1 of 2
Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 4:15 am
by Waszka
Hi all,
I have recently finished my write to FAT12 procedure which looks like this:
Code: Select all
1. Find free cluster in FAT directory (if entry == 0 it's free) (I know that clusters 0 and 1 are restricted and cannot be used)
2. Finde free entry in Root directory (if entry.name == "\0\0\0\0..." it's free)
3. Fill root directory entry with: name, size, first cluster. Fill FAT cluster entry with 0xff8
4. Write content of the file to [free_cluster + 31] to floppy disk (31 is because Data Directory starts from there)
5. Save changes to root and FAT
Procedure assumes there is free space for file, file is no longer that cluster size.
After saving I can read the file using procedure from BrokenThornEntertainment or my own one, but the file is not visible under Linux
and the file is lost after copying any file from Linux to floppy disk.
What do I forget about during this procedure?
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 4:49 am
by Gigasoft
This is completely wrong because you forgot to read the FAT specification. If the first byte of a directory entry is 0, the directory ends right there. On the other hand, if the first byte is 0xe5, you have a free directory entry but the directory does not end. By the way, there is no cluster 0 or 1. Cluster numbering starts with 2, so you have to subtract 2 when calculating the offset from the data start.
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 5:49 am
by Waszka
Gigasoft wrote:This is completely wrong because you forgot to read the FAT specification. If the first byte of a directory entry is 0, the directory ends right there. On the other hand, if the first byte is 0xe5, you have a free directory entry but the directory does not end. By the way, there is no cluster 0 or 1. Cluster numbering starts with 2, so you have to subtract 2 when calculating the offset from the data start.
I read FAT specification on Wikipedia and here
http://www.forensicswiki.org/wiki/FAT so that's probably why I didn't get the whole thing. I promise to read official documentation this evening
But there are some things I couldn't understand from your post: 0xe5 is a sigma, and it is used when deleting files (as far as I'm aware it's used ONLY in this situation).
So when I have first byte = 0 it should mean that entry is free for use.
The last thing is the one about clusters. I understand that
Cluster numbering starts with 2
means that FAT entry number 0 refers to cluster 2 in Data Directory, right? So I must add 2 instead of substracting?
EDIT: Ok, I think I got it - in FAT entry number 0 refers to cluster 0 in Data Directory but this cluster's address is 2. Like FAT[0] = 0x2 so it means that file has part of it in Data[0]. Is it true?
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 6:07 am
by zhiayang
Waszka wrote:
I read FAT specification on Wikipedia and here
http://www.forensicswiki.org/wiki/FAT so that's probably why I didn't get the whole thing. I promise to read official documentation this evening
But there are some things I couldn't understand from your post: 0xe5 is a sigma, and it is used when deleting files (as far as I'm aware it's used ONLY in this situation).
So when I have first byte = 0 it should mean that entry is free for use.
Indeed, 0x00 means "Entry is available and no subsequent entry is in use. Also serves as an end marker when DOS scans a directory table". Gigasoft is also correct here; when you encounter a '0x00' entry, that means you've reached the last entry in the directory. It's like a NULL on the end of a string.
Except that entries past that are also valid, up to the cluster boundary. Simple analogy: you malloc(32). You strcpy() "Hello, World!" to the buffer. You copied 14 bytes (including the nullptr). The nullptr means you've reached the end of the string, but you know you still have 18 bytes to work with. Similarly, the cluster size and number of clusters for the directory dictates how many directory entries there can be inside it, the 0x0 only serves to tell you where the last valid entry is.
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 6:12 am
by Gigasoft
So when I have first byte = 0 it should mean that entry is free for use.
Right now you are skipping over any entry which has 0 in the first byte but not in the second to eleventh bytes, which means that any other OS will stop looking once it gets to the entry you skipped, and your file will not be found.
means that FAT entry number 0 refers to cluster 2 in Data Directory, right? So I must add 2 instead of substracting?
EDIT: Ok, I think I got it - in FAT entry number 0 refers to cluster 0 in Data Directory but this cluster's address is 2. Like FAT[0] = 0x2 so it means that file has part of it in Data[0]. Is it true?
No, you didn't get it. The first cluster, the one at the beginning of the data area, is numbered with a "2" and corresponds to FAT entry 2. FAT entries 0 and 1 contain something else, unrelated to files and clusters.
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 6:22 am
by Waszka
Gigasoft wrote:Right now you are skipping over any entry which has 0 in the first byte but not in the second to eleventh bytes, which means that any other OS will stop looking once it gets to the entry you skipped, and your file will not be found.
Ok, I get it - it was my fault to write this pseudo-code so wrong... I meant that I just look at the first byte in root directory and when it's 0 or sigma I put file there.
Sorry for my mistake.
No, you didn't get it. The first cluster, the one at the beginning of the data area, is numbered with a "2" and corresponds to FAT entry 2. FAT entries 0 and 1 contain something else, unrelated to files and clusters.
So I should never look at FAT entry 0 and 1 (like FAT[0] or FAT[1] is bad
) and I can only do FAT[ sektor ] = DATA[ sektor-2].
Thanks for that.
And what about formatting?
Here
http://c-jump.com/CIS24/Slides/FAT/lect ... ctor_bytes in chapter Formatting Floppy it writes that during format Root Directory entries will be overwritten with 0xF6. 0xF6 = 11110110. There is no byte 0 in the beginning...
Re: Files written to FAT12 not visible in real OS.
Posted: Tue Jul 15, 2014 7:10 am
by Gigasoft
Don't trust everything you read on the Internet. You can't just go around reading every random page you find written by some stranger. The root directory is initialized with 0's. The data area is optionally initialized with 0xf6's on a floppy.
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 12:21 am
by Waszka
Ok, so I've tried to correct those errors - I start looking for free space in FAT from third entry (assuming FAT[0] and FAT[1] are first two).
When I find FAT = 0 I take this sector as free and start streaming data to floppy_disk_sector[33 + (i - 2)]. I also put in FAT entry 0xff8 (no more cluster for this file).
With root directory I scan only for entry->name[0] == ( '\0' || `sigma`) and then I put entry.
Unfortunately I have still error somewhere and I'll investigate this when I find some time.
But my question is: "what should entry in root directory contain"? Is it ok just to fill just NAME (with extension), FIRST_CLUSTER, and SIZE?
I can't find the answer in the papers - it's like they show the fields, how to fill them, but no info about leaving 'em empty.
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 12:41 am
by Combuster
Finding the solution to this problem is fairly easy.
1: You make an image of the original floppy - if you don't have one already
2: You use Linux to write the file to disk
3: You make another image of the floppy.
4: You put back the original image.
5: You reboot into your OS and try to write the file
6: Back in linux, make another image of the floppy
7: Find the bytes that differ between 3 and 6.
8: ...
9: Profit.
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 3:23 pm
by Waszka
How can I detect if my floppy image is really formated FAT12?
I have all required fields in the boot sector, and my df -T shows:
Code: Select all
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/loop0 vfat 1424 34 1390 3% /mnt/disk
Judging from this it's FAT12. What's more the read function works as expected (I think that because I can view files, files in folders, and I can load kernel in stage2 bootloader)
But there's one thing that seems not to be detected: I'm talking about entries in FAT. While parsing them I get the following:
Code: Select all
FAT_ENTRY_No VALUE
0 -> 0x0
1 -> 0x0
2 -> 0x0
3 -> 0x4
4 -> 0xFFFFFF00
5 ->0xFF0
6 -> 0xFFFFF800
...
First of all these values are integer (?!) not short. Next thing is second entry is "empty" which is quite odd.
But again -> when reading those specific sectors on floppy (floppy_read(33)) I get empty 512 bytes. So indeed it's free.
Quite the opposite is happening with entry number 14 -> it also shows 0x0 as a value but reading floppy sector (33 + (14-2)) = 45 I get some data there. Also writing there results in system not booting and bochs showing error about trying to read sector outside floppy boundaries.
Here is the code for getting FAT entries:
Code: Select all
int fat12_find_if_free_cluster(int free_cluster)
{
unsigned char* sector;
/* Calculate FAT entry offset, etc. */
unsigned int FAT_Offset = free_cluster + (free_cluster / 2); //multiply by 1.5 (o3mujemy adres info o nast. klastrze)
unsigned int FAT_Sector = 1 + (FAT_Offset / ROZMIAR_SEKTORA);
unsigned int entryOffset = FAT_Offset % ROZMIAR_SEKTORA;
/* Read fat sectors */
sector = (unsigned char*) floppy_read_sector( FAT_Sector );
memcpy((unsigned char*)FAT, sector, 512);
sector = (unsigned char*) floppy_read_sector ( FAT_Sector + 1 );
memcpy((unsigned char*)FAT + ROZMIAR_SEKTORA, sector, 512);
/* Read where is next cluster of file (if any) */
short nextCluster = *(unsigned short*)&FAT[entryOffset];
short actual_cluster;
if( nextCluster & 0x0001 )
actual_cluster = nextCluster >> 4; //grab high 12 bits
else
actual_cluster = nextCluster & 0x0FFF; //grab low 12 bits
if(actual_cluster == 0) {
actual_cluster = 0xff8;
/* Zapisujemy w FAT */
if(nextCluster & 0x0001) {
nextCluster &= 0x000F;
nextCluster |= (actual_cluster << 4);
} else {
nextCluster &= 0xF000;
nextCluster |= actual_cluster;
}
short* tmp = (short*)&FAT[entryOffset];
*tmp = nextCluster;
floppy_write_sector(FAT_Sector, FAT);
floppy_write_sector(FAT_Sector+1, FAT+ROZMIAR_SEKTORA);
return free_cluster;
}
return -1;
}
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 3:36 pm
by Waszka
Whoooa!
I got it! It was me not umounting image of floppy after uploading new kernel. Files were there, but system flushed it's "temporary" image to disk and overrode my files!
But I still have one problem:
When I start writing from the second FAT entry I get to the point where after second write I put data to 14 sector of data. And then bochs get the error:
Code: Select all
[FDD ]
Message: io: norm r/w parms out of range: sec#01h cyl#6fh eot#01h head#01
EDIT: It turned out that probably something was corrupted within FAT directory. Created clean floppy disk and everything works. I'll let you know how it works after some longer work with it.
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 5:37 pm
by zhiayang
Waszka wrote:Whoooa!
I got it! It was me not umounting image of floppy after uploading new kernel. Files were there, but system flushed it's "temporary" image to disk and overrode my files!
Bah, happens all the time -- I had to get my makefile to mount the disk to copy the file, then unmount before starting QEMU, then remount after QEMU exits...
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 6:02 pm
by Nable
requimrar wrote:mount the disk to copy the file
Did you think about using `mtools' package? Or something
similar.
Re: Files written to FAT12 not visible in real OS.
Posted: Wed Jul 16, 2014 7:36 pm
by zhiayang
Nable wrote:requimrar wrote:mount the disk to copy the file
Did you think about using `mtools' package? Or something
similar.
1. DITO doesn't work for FAT32 I don't think.
2. mtools' interface just seems poor and inconvenient. I'd rather wait the few seconds for hdiutil to attach my image.
Re: Files written to FAT12 not visible in real OS.
Posted: Thu Jul 17, 2014 12:05 am
by Combuster
1. DITO doesn't work for FAT32 I don't think.
2. mtools' interface just seems poor and inconvenient. I'd rather wait the few seconds for hdiutil to attach my image.
Ignorance is not an excuse. It does support all FAT sizes, and one of it's key advantages is that it's command line and doesn't need to mount anything, making it one of the rare tools that can be automated and actually work identically on all three platforms alike (though I haven't tested OSX personally).