OSDev.org

Hi

It's been a while since I've been on OSDev type forums (I used to be on the Mega-Tokyo.com OSDev forums).

I was reading through this article on bit-tech.net and saw on this page:
http://www.bit-tech.net/hardware/storag ... rd-disks/2

Windows XP and other older OSes work by accessing specific disk sectors, not by using 'atomic writes' that work on byte-level data placements that we see in Windows Vista, 7, Server 2008, MacOS 10.4, 10.5 and 10.6 (Tiger, Leopard and Snow Leopard) and recent Linux kernels from 2.6.31 onwards.

I realise this probably isn't the best explanation of the subject in the world. but I'm a bit confused. I always thought that an OS accesses a hard disk a sector at a time, but has this changed since Linux 2.6.31, Win Vista and OSX Tiger?

What does this mean for us OSDev people. Is this different method in the current or future ATA spec, or is there anywhere else we can find out about this. Is this going to be required for these "new fangled" hard disks with 4KB sectors (as opposed to the normal 512KB sectors that have been around since the stone age)

Thanks
srg4096

This is a 'popularized' explanation of the concept of a 'journaling' filesystem. Probably best to start reading in wikipedia and then move on to google. http://en.wikipedia.org/wiki/Journaling_file_system

No, it does not replace ATA in any way. It is a higher-level protocol that must adapt itself to the underlying hardware, somehow. When a process tries to modify information in a file, the filesystem needs to modify 512byte blocks on the disk. However, the whole point of journaling is that there is an algorithm for 'perfectly undoing' any series of writes to the disk in case of an error, and there is also an algorithm for redoing a series of writes, from a previously stored state.

I personally think that journaling filesystems are a waste of time and resources. They do not increase the underlying reliability of the hardware enough to be worth the incredible overhead. If you do code up an OS, then yes, it is almost certain that you will need to support them at some point, eventually.

The newfangled 4K-sector disks are still accessed in 512byte mode. The underlying 4K structure is completely hidden from everything external to the drive.

Is this related to the subject of Journaling file systems though? Windows XP used a journaling file system, but it seems here that something fundamentally changed for Vista etc?

So the 4KB sectors are only internally and not shown on the outside?

Thanks
srg4096

bewing wrote:I personally think that journaling filesystems are a waste of time and resources. They do not increase the underlying reliability of the hardware enough to be worth the incredible overhead. If you do code up an OS, then yes, it is almost certain that you will need to support them at some point, eventually.

You've never had any data trashed because of a power failure (or kernel panic, or...) before then? I have, but never with a journalled file system

I've had some work lost because it was still only in memory and I hadn't hit "save" yet. A journaling FS doesn't help that at all. I've seen work lost because of a total disk failure. Journaling doesn't help that, either. I've had work lost because a buggy M$ program interacted with a buggy M$ OS to overwrite most of a disk that it shouldn't have even been accessing. Journaling doesn't help that either. Lost data that was already stored on a disk, because of a power failure etc.? Nope, I've never seen that even once.

@srg4096 -- yes, the 4K sectors are internal only.
And AFAIK, the journaling in XP was not complete.

I've had work lost because the power went out while the file system was still updating some internal structures. Complete FS trashed.

Journalled file systems journal their own structures, and update them atomically. Never have I had a journalled FS trashed - not even a ReiserFS partition, which is well known for being completely unrecoverable if things go wrong (If you need to reiserfsck, prey, because your file system is getting mangled).

Journalling protects against the loss of data due to an supposedly atomic action not being atomic. It is good for much the same reason a journalled database is good.

I really have my doubts that what the original article was refering to was anything to do with Journaling, I think this is at a lower level than that. It sounds like modern OS's work bytes at a time rather than sectors at a time. Maybe I'm getting something wrong though.

Thanks
srg4096

The point is that operating on a disk byte by byte is simply an abstraction. There may be some layers of the OS that see the disk as a series of bytes, and some that see it as a series of blocks. However, because the interface to ATA drives requires operating on blocks, not bytes, there must be at least some layer that uses blocks. Everything that is not the ATA driver shouldn't have to worry about the block size for anything but optimization, so there must also be some later that uses bytes. Saying that some system uses one method exclusively is wrong.

@srg4096: You said yourself that you thought the article was wrong about some things. What NickJohnson said is exactly right.
Despite any implications in the article, there are no current hardware devices anywhere that read and write random-length bytestreams. All hardware storage devices always store blocks of various sizes, still -- at their lowest hardware level. Anything else is just an abstraction.