Files written to FAT12 not visible in real OS.

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Files written to FAT12 not visible in real OS.

Post by Waszka »

Hi all,
I have recently finished my write to FAT12 procedure which looks like this:

Code: Select all

1. Find free cluster in FAT directory (if entry == 0 it's free) (I know that clusters 0 and 1 are restricted and cannot be used)
2. Finde free entry in Root directory (if entry.name == "\0\0\0\0..." it's free)
3. Fill root directory entry with: name, size, first cluster. Fill FAT cluster entry with 0xff8
4. Write content of the file to [free_cluster + 31] to floppy disk (31 is because Data Directory starts from there)
5. Save changes to root and FAT
Procedure assumes there is free space for file, file is no longer that cluster size.
After saving I can read the file using procedure from BrokenThornEntertainment or my own one, but the file is not visible under Linux
and the file is lost after copying any file from Linux to floppy disk.

What do I forget about during this procedure?
Gigasoft
Member
Member
Posts: 856
Joined: Sat Nov 21, 2009 5:11 pm

Re: Files written to FAT12 not visible in real OS.

Post by Gigasoft »

This is completely wrong because you forgot to read the FAT specification. If the first byte of a directory entry is 0, the directory ends right there. On the other hand, if the first byte is 0xe5, you have a free directory entry but the directory does not end. By the way, there is no cluster 0 or 1. Cluster numbering starts with 2, so you have to subtract 2 when calculating the offset from the data start.
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Re: Files written to FAT12 not visible in real OS.

Post by Waszka »

Gigasoft wrote:This is completely wrong because you forgot to read the FAT specification. If the first byte of a directory entry is 0, the directory ends right there. On the other hand, if the first byte is 0xe5, you have a free directory entry but the directory does not end. By the way, there is no cluster 0 or 1. Cluster numbering starts with 2, so you have to subtract 2 when calculating the offset from the data start.
I read FAT specification on Wikipedia and here http://www.forensicswiki.org/wiki/FAT so that's probably why I didn't get the whole thing. I promise to read official documentation this evening :)
But there are some things I couldn't understand from your post: 0xe5 is a sigma, and it is used when deleting files (as far as I'm aware it's used ONLY in this situation).
So when I have first byte = 0 it should mean that entry is free for use.

The last thing is the one about clusters. I understand that
Cluster numbering starts with 2
means that FAT entry number 0 refers to cluster 2 in Data Directory, right? So I must add 2 instead of substracting?
EDIT: Ok, I think I got it - in FAT entry number 0 refers to cluster 0 in Data Directory but this cluster's address is 2. Like FAT[0] = 0x2 so it means that file has part of it in Data[0]. Is it true?
User avatar
zhiayang
Member
Member
Posts: 368
Joined: Tue Dec 27, 2011 7:57 am
Libera.chat IRC: zhiayang

Re: Files written to FAT12 not visible in real OS.

Post by zhiayang »

Waszka wrote: I read FAT specification on Wikipedia and here http://www.forensicswiki.org/wiki/FAT so that's probably why I didn't get the whole thing. I promise to read official documentation this evening :)
But there are some things I couldn't understand from your post: 0xe5 is a sigma, and it is used when deleting files (as far as I'm aware it's used ONLY in this situation).
So when I have first byte = 0 it should mean that entry is free for use.
Indeed, 0x00 means "Entry is available and no subsequent entry is in use. Also serves as an end marker when DOS scans a directory table". Gigasoft is also correct here; when you encounter a '0x00' entry, that means you've reached the last entry in the directory. It's like a NULL on the end of a string.

Except that entries past that are also valid, up to the cluster boundary. Simple analogy: you malloc(32). You strcpy() "Hello, World!" to the buffer. You copied 14 bytes (including the nullptr). The nullptr means you've reached the end of the string, but you know you still have 18 bytes to work with. Similarly, the cluster size and number of clusters for the directory dictates how many directory entries there can be inside it, the 0x0 only serves to tell you where the last valid entry is.
Gigasoft
Member
Member
Posts: 856
Joined: Sat Nov 21, 2009 5:11 pm

Re: Files written to FAT12 not visible in real OS.

Post by Gigasoft »

So when I have first byte = 0 it should mean that entry is free for use.
Right now you are skipping over any entry which has 0 in the first byte but not in the second to eleventh bytes, which means that any other OS will stop looking once it gets to the entry you skipped, and your file will not be found.
means that FAT entry number 0 refers to cluster 2 in Data Directory, right? So I must add 2 instead of substracting?
EDIT: Ok, I think I got it - in FAT entry number 0 refers to cluster 0 in Data Directory but this cluster's address is 2. Like FAT[0] = 0x2 so it means that file has part of it in Data[0]. Is it true?
No, you didn't get it. The first cluster, the one at the beginning of the data area, is numbered with a "2" and corresponds to FAT entry 2. FAT entries 0 and 1 contain something else, unrelated to files and clusters.
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Re: Files written to FAT12 not visible in real OS.

Post by Waszka »

Gigasoft wrote:Right now you are skipping over any entry which has 0 in the first byte but not in the second to eleventh bytes, which means that any other OS will stop looking once it gets to the entry you skipped, and your file will not be found.
Ok, I get it - it was my fault to write this pseudo-code so wrong... I meant that I just look at the first byte in root directory and when it's 0 or sigma I put file there.
Sorry for my mistake.

No, you didn't get it. The first cluster, the one at the beginning of the data area, is numbered with a "2" and corresponds to FAT entry 2. FAT entries 0 and 1 contain something else, unrelated to files and clusters.
So I should never look at FAT entry 0 and 1 (like FAT[0] or FAT[1] is bad :P) and I can only do FAT[ sektor ] = DATA[ sektor-2].
Thanks for that.

And what about formatting?
Here http://c-jump.com/CIS24/Slides/FAT/lect ... ctor_bytes in chapter Formatting Floppy it writes that during format Root Directory entries will be overwritten with 0xF6. 0xF6 = 11110110. There is no byte 0 in the beginning...
Gigasoft
Member
Member
Posts: 856
Joined: Sat Nov 21, 2009 5:11 pm

Re: Files written to FAT12 not visible in real OS.

Post by Gigasoft »

Don't trust everything you read on the Internet. You can't just go around reading every random page you find written by some stranger. The root directory is initialized with 0's. The data area is optionally initialized with 0xf6's on a floppy.
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Re: Files written to FAT12 not visible in real OS.

Post by Waszka »

Ok, so I've tried to correct those errors - I start looking for free space in FAT from third entry (assuming FAT[0] and FAT[1] are first two).
When I find FAT = 0 I take this sector as free and start streaming data to floppy_disk_sector[33 + (i - 2)]. I also put in FAT entry 0xff8 (no more cluster for this file).
With root directory I scan only for entry->name[0] == ( '\0' || `sigma`) and then I put entry.
Unfortunately I have still error somewhere and I'll investigate this when I find some time.
But my question is: "what should entry in root directory contain"? Is it ok just to fill just NAME (with extension), FIRST_CLUSTER, and SIZE?
I can't find the answer in the papers - it's like they show the fields, how to fill them, but no info about leaving 'em empty.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Files written to FAT12 not visible in real OS.

Post by Combuster »

Finding the solution to this problem is fairly easy.

1: You make an image of the original floppy - if you don't have one already
2: You use Linux to write the file to disk
3: You make another image of the floppy.
4: You put back the original image.
5: You reboot into your OS and try to write the file
6: Back in linux, make another image of the floppy
7: Find the bytes that differ between 3 and 6.
8: ...
9: Profit.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Re: Files written to FAT12 not visible in real OS.

Post by Waszka »

How can I detect if my floppy image is really formated FAT12?
I have all required fields in the boot sector, and my df -T shows:

Code: Select all

Filesystem     Type     1K-blocks      Used Available Use% Mounted on
/dev/loop0     vfat          1424        34      1390   3% /mnt/disk
Judging from this it's FAT12. What's more the read function works as expected (I think that because I can view files, files in folders, and I can load kernel in stage2 bootloader)
But there's one thing that seems not to be detected: I'm talking about entries in FAT. While parsing them I get the following:

Code: Select all

 FAT_ENTRY_No   VALUE
0 -> 0x0
1 -> 0x0
2 -> 0x0
3 -> 0x4
4 -> 0xFFFFFF00
5 ->0xFF0
6 -> 0xFFFFF800
...
First of all these values are integer (?!) not short. Next thing is second entry is "empty" which is quite odd.
But again -> when reading those specific sectors on floppy (floppy_read(33)) I get empty 512 bytes. So indeed it's free.
Quite the opposite is happening with entry number 14 -> it also shows 0x0 as a value but reading floppy sector (33 + (14-2)) = 45 I get some data there. Also writing there results in system not booting and bochs showing error about trying to read sector outside floppy boundaries.

Here is the code for getting FAT entries:

Code: Select all

int fat12_find_if_free_cluster(int free_cluster)
{
  unsigned char* sector;

  /* Calculate FAT entry offset, etc. */
  unsigned int FAT_Offset = free_cluster + (free_cluster / 2); //multiply by 1.5 (o3mujemy adres info o nast. klastrze)
  unsigned int FAT_Sector = 1 + (FAT_Offset / ROZMIAR_SEKTORA);
  unsigned int entryOffset = FAT_Offset % ROZMIAR_SEKTORA;

  /* Read fat sectors */
  sector = (unsigned char*) floppy_read_sector( FAT_Sector );
  memcpy((unsigned char*)FAT, sector, 512);
  sector = (unsigned char*) floppy_read_sector ( FAT_Sector + 1 );
  memcpy((unsigned char*)FAT + ROZMIAR_SEKTORA, sector, 512);

  /* Read where is next cluster of file (if any) */
  short nextCluster = *(unsigned short*)&FAT[entryOffset];
  short actual_cluster;

  if( nextCluster & 0x0001 )
    actual_cluster = nextCluster >> 4;        //grab high 12 bits
  else
    actual_cluster = nextCluster & 0x0FFF;    //grab low 12 bits

  if(actual_cluster == 0) {
      actual_cluster = 0xff8;

      /* Zapisujemy w FAT */
      if(nextCluster & 0x0001) {
          nextCluster &= 0x000F;
          nextCluster |= (actual_cluster << 4);
      } else {
          nextCluster &= 0xF000;
          nextCluster |= actual_cluster;
      }
      short* tmp = (short*)&FAT[entryOffset];
      *tmp = nextCluster;
      floppy_write_sector(FAT_Sector, FAT);
      floppy_write_sector(FAT_Sector+1, FAT+ROZMIAR_SEKTORA);

      return free_cluster;
  }

  return -1;
}
User avatar
Waszka
Member
Member
Posts: 38
Joined: Tue Apr 22, 2014 11:30 am
Location: Poland

Re: Files written to FAT12 not visible in real OS.

Post by Waszka »

Whoooa!
I got it! It was me not umounting image of floppy after uploading new kernel. Files were there, but system flushed it's "temporary" image to disk and overrode my files!
But I still have one problem:
When I start writing from the second FAT entry I get to the point where after second write I put data to 14 sector of data. And then bochs get the error:

Code: Select all

[FDD ]
Message: io: norm r/w parms out of range: sec#01h cyl#6fh eot#01h head#01
EDIT: It turned out that probably something was corrupted within FAT directory. Created clean floppy disk and everything works. I'll let you know how it works after some longer work with it.
User avatar
zhiayang
Member
Member
Posts: 368
Joined: Tue Dec 27, 2011 7:57 am
Libera.chat IRC: zhiayang

Re: Files written to FAT12 not visible in real OS.

Post by zhiayang »

Waszka wrote:Whoooa!
I got it! It was me not umounting image of floppy after uploading new kernel. Files were there, but system flushed it's "temporary" image to disk and overrode my files!
Bah, happens all the time -- I had to get my makefile to mount the disk to copy the file, then unmount before starting QEMU, then remount after QEMU exits...
Nable
Member
Member
Posts: 453
Joined: Tue Nov 08, 2011 11:35 am

Re: Files written to FAT12 not visible in real OS.

Post by Nable »

requimrar wrote:mount the disk to copy the file
Did you think about using `mtools' package? Or something similar.
User avatar
zhiayang
Member
Member
Posts: 368
Joined: Tue Dec 27, 2011 7:57 am
Libera.chat IRC: zhiayang

Re: Files written to FAT12 not visible in real OS.

Post by zhiayang »

Nable wrote:
requimrar wrote:mount the disk to copy the file
Did you think about using `mtools' package? Or something similar.
1. DITO doesn't work for FAT32 I don't think.
2. mtools' interface just seems poor and inconvenient. I'd rather wait the few seconds for hdiutil to attach my image.
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Files written to FAT12 not visible in real OS.

Post by Combuster »

1. DITO doesn't work for FAT32 I don't think.
2. mtools' interface just seems poor and inconvenient. I'd rather wait the few seconds for hdiutil to attach my image.
Ignorance is not an excuse. It does support all FAT sizes, and one of it's key advantages is that it's command line and doesn't need to mount anything, making it one of the rare tools that can be automated and actually work identically on all three platforms alike (though I haven't tested OSX personally).
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
Post Reply