Creating NVMe wiki article

[codepilot]

I would like to create an NVMe article with working code. This would be the first article that I am creating. Is there anything that I should keep in mind?

[iocoder]

I would like to create an NVMe article with working code. This would be the first article that I am creating. Is there anything that I should keep in mind?

Would be great if you can explain the differences between AHCI and NVMe, and how the same SSD device implements backward compatibility? For example, if an SSD device supports both SATA Express and NVM Express, do I need to do something in order to detect this setup and only use NVMe?

Do we have to consider different form factors? For example, if an SSD is attached to the system through an M.2 port, would the logical interface of the NVMe controller be different than an SSD attached through PCIe slot?

Thanks!

[codepilot]

Totally explaining differences...

Big one is up to 64 thousand requests can be made instead of just 32(sata), and (nvme)bunch of queues, instead of 1(sata) my samsung drive supports 8 queues(nvme), and I know a lot of the more advanced ones support a bunch more.

Sata has rather complicated setup for drivers to send and receive, while nvme has a simple single posted write that can submit up to 64 thousand requests.

I will put few open source examples on github.

Also, I have been making windows kernel a while, and I am working on translating to linux kernel modules also, so the code will work a lot of environments for easy inclusion in any custom OS.

I will look into the sata express setup for using nvme, because i would bet that it needs something extra to put it ready for nvme.

I have done u.2 and m.2 and not seen any difference. There is nothing in the spec that says they would have any difference. Maybe only advanced pcie specific features such as controling speed would matter, idk yet, will read more.

My OS is around 20 years old now, and I feel ancient, and I can say from a driver standpoint nvme is beautiful, and i cannot stand any form of storage tech that came before by comparison.

Anyone making a new OS should just make it for nvme and forget anything else ever existed, that is why i think an article needs to be made. I volunteer

[Korona]

An NVMe article sounds great! (It's a piece of hardware that I wanted to support for some time; but so far, I didn't have time to write a driver.)

I'd resist the temptation to put a lot of example code into the article, as any code will be highly OS-specific. In the article on Intel graphics, I tried to find a middle ground by explaining the high-level control flow of the driver while linking to the C source code in Linux (but any other OS would also be equally good as long as the code is somewhat clean). I'm not claiming that this is the best way; however, IMO code-heavy wiki articles are not very useful. As an example, look at the ATA article. It has assembly level code that is somewhat hard to read (certainly harder to read than code in a high-level language). It's fixed to 32-bit registers and uses a non-standard calling convention. It does not use interrupts. IMO, the article would be better off if that code was removed. While it's of course also possible to improve the code's quality, I fear that any code in the wiki will bitrot over time (let's be realistic here: not many people will have the NVMe expertise + knowledge of your code to maintain it in the future).

[iocoder]

Big one is up to 64 thousand requests can be made instead of just 32(sata), and (nvme)bunch of queues, instead of 1(sata) my samsung drive supports 8 queues(nvme), and I know a lot of the more advanced ones support a bunch more.

This is really impressive.

I will look into the sata express setup for using nvme, because i would bet that it needs something extra to put it ready for nvme.

Yeah that was one of my concerns:



Photo is taken from Wikipedia. Can you see the grey box that is labeled 'Configurable SATA port or PCI lane'? I think this is a crucial part of the game. By playing with this component, the SSD can either appear to the OS as a legacy SATA device or as a PCI device.

Maybe this configuration is done through BIOS Setup? In that case the OS shouldn't worry about it, it would be a matter of how the end user configures their firmware.

I have done u.2 and m.2 and not seen any difference. There is nothing in the spec that says they would have any difference. Maybe only advanced pcie specific features such as controling speed would matter, idk yet, will read more.

That makes sense, thanks for confirming.

Anyone making a new OS should just make it for nvme and forget anything else ever existed

Couldn't agree more. We are in an era where things started to become more unified and standard specifications are becoming more mature. We will be in 2020 very soon, I think it is more useful to learn NVMe (which sounds like it is well designed; so your mind will learn good and structured concepts) rather than learning legacy stuff (like AHCI, ATA PIO, etc.). It is the same argument why we don't need real and protected modes in modern Intel CPUs... It is not worth it!

[codepilot]

An NVMe article sounds great! (It's a piece of hardware that I wanted to support for some time; but so far, I didn't have time to write a driver.)

I'd resist the temptation to put a lot of example code into the article, as any code will be highly OS-specific. In the article on Intel graphics, I tried to find a middle ground by explaining the high-level control flow of the driver while linking to the C source code in Linux (but any other OS would also be equally good as long as the code is somewhat clean). I'm not claiming that this is the best way; however, IMO code-heavy wiki articles are not very useful. As an example, look at the ATA article. It has assembly level code that is somewhat hard to read (certainly harder to read than code in a high-level language). It's fixed to 32-bit registers and uses a non-standard calling convention. It does not use interrupts. IMO, the article would be better off if that code was removed. While it's of course also possible to improve the code's quality, I fear that any code in the wiki will bitrot over time (let's be realistic here: not many people will have the NVMe expertise + knowledge of your code to maintain it in the future).

I am planning on putting link to a GitHub set of examples, and over time hopefully as many languages as people would like. Also, being an open source repository should help people keep the bitrot at bay. The article is based on words tho, not code.

[codepilot]

Maybe this configuration is done through BIOS Setup? In that case the OS shouldn't worry about it, it would be a matter of how the end user configures their firmware.

I don't know really, I don't have a motherboard like that anymore, and I never had a drive like that. I kinda remember it being a bios thing tho.

[codepilot]

I made a page https://wiki.osdev.org/NVMe and https://github.com/codepilot/NVMe-how-to

I am going to start off with the headers and start to fill them in

Could I get some help making some headers or FAQ entries so I can get some direction of what people need?

I am aiming to make examples in C++ to get started, but I am open to other languages by request

[rdos]

I'm not overly impressed by this new interface. It contains a huge amount of bloat & conditional things that might or might not be supported. Extended error codes are more or less useless. A good disc controller specification should NEVER return any type of error code unless the drive is beyond repair. I like the health report and error logs though, as those are preventive and not fatal realtime errors. And what is the point of the administrative interface? Just returning basic drive parameters (like number of sectors) is quite enough, and partitions should be part of the operating system not the drive.

If I could decide, I'd use AHCI over NVMe. AHCI is also severely bloated, but at least it contains large enough LBAs by default unlike the legacy IDE specification.

[codepilot]

I'm not overly impressed by this new interface. It contains a huge amount of bloat & conditional things that might or might not be supported. Extended error codes are more or less useless. A good disc controller specification should NEVER return any type of error code unless the drive is beyond repair. I like the health report and error logs though, as those are preventive and not fatal realtime errors. And what is the point of the administrative interface? Just returning basic drive parameters (like number of sectors) is quite enough, and partitions should be part of the operating system not the drive.

If I could decide, I'd use AHCI over NVMe. AHCI is also severely bloated, but at least it contains large enough LBAs by default unlike the legacy IDE specification.

I super duper agree, I wish for an interface that reads and writes data, and write-zero(deallocate) is nice. All I need to know is how many sectors/how big. Everything else, especially optional stuff and complicated ways to break are not super helpful.

If you want to interrupt the normal flow of operations, it had better be fatal.

NVMe does one thing that is better than AHCI(6Gbit/s), it may be way faster, only limited by the interface being PCIe, is a big deal. PCIe 3.0 4x lanes is 32Gbit/s, standard M.2 drives, doubles to 64Gbit/s with PCIe 4.0

Programming in general seams to be an exorcise of handling innumerable errors and exceptions for a plethora of specifications that are improperly implemented and full of errata.

Just for fun sometimes, when I really want to get depressed, I read some open-source driver for something or another that should be short and simple. It is mostly filled up with comments and code to identify and handle various errata.