64-bit addressing really necessary?

[rdos]

IMO 64 bit addressing is necessary now. My OS maps the complete physical memory to 0xFFFF800000000000. It would take to much address space doing that with 32 bit addressing, even with things like PAE.

I see no reason why you need to map all physical memory. For allocation purposes, one bit per 4k page is enough. This supports 1TB of physical memory using only 32MB of virtual memory. Today you cannot buy a regular motherboard that supports more than 128GB, and so support for 1TB clearly is enough.

Also, you could use even less virtual memory by using 2M pages for some physical memory areas.

IOW, the physical memory manager clearly doesn't need a 64-bit address space.

I wouldn't call programmers who take advantage of it lazy, as I like to work smarter, not harder . With x86_64 in particular, you get way more registers, and for me that is the biggest advantage. Back when I maintained a 32 bit version of my OS, the 32 bit version was a lot slower then the 64 bit version. Why? Because in C with 32 bit x86, parameters are passed on the stack, while on x86_64, they are passed in registers.

That's compiler dependent. You can pass parameters in registers in C when using 32-bit.

[nexos]

I see no reason why you need to map all physical memory. For allocation purposes, one bit per 4k page is enough. This supports 1TB of physical memory using only 32MB of virtual memory. Today you cannot buy a regular motherboard that supports more than 128GB, and so support for 1TB clearly is enough.

Also, you could use even less virtual memory by using 2M pages for some physical memory areas.

IOW, the physical memory manager clearly doesn't need a 64-bit address space.

I see what you're saying, but if you map the complete physical memory and make your physical memory allocator use a free list, then your allocator uses 0 bytes of RAM. Not only that, but it is O(1) as well.

That's compiler dependent. You can pass parameters in registers in C when using 32-bit.

System V ABI compilers like gcc or clang use the behavior as described above. In order to have faster programs and better optimizations, I stick with GCC.

[Octocontrabass]

System V ABI compilers like gcc or clang use the behavior as described above.

You may be interested in the -mregparm option. The Linux kernel uses -mregparm=3 by default. Since it changes the ABI, you may also need a specially compiled libgcc.

[rdos]

I see what you're saying, but if you map the complete physical memory and make your physical memory allocator use a free list, then your allocator uses 0 bytes of RAM. Not only that, but it is O(1) as well.

Linked list algorithms don't work well with a lock-free operation, and I think physical memory allocation must be lock-free since it can happen at any time if you use lazy allocation of physical pages. This is particularly the case in multicore environments. By using a bitmap you can use locked instructions to clear a bit and get a result back if it succeeded (lock btr), which is atomic, and thus ISR and multicore safe and provides a good basis for creating lock-free physical memory allocation.

That's compiler dependent. You can pass parameters in registers in C when using 32-bit.

System V ABI compilers like gcc or clang use the behavior as described above. In order to have faster programs and better optimizations, I stick with GCC.

OpenWatcom can use either a register based interface or a stack based. I always use the register based, and it doesn't require anything in the code, just a compiler setting which I have as default for RDOS applications.

[Korona]

Today you cannot buy a regular motherboard that supports more than 128GB, and so support for 1TB clearly is enough.

What's a regular motherboard?

...

I've seen this "there are no machines with more than a couple of GiB of RAM" claim so many times on this forum. It's still not true. Screenshot attached. (The machine is idle. The 11 GiB in use are mostly 'struct page' + a few daemons. There are several 100s of GiB of page cache due to previous jobs.)

[Schol-R-LEA]

Today you cannot buy a regular motherboard that supports more than 128GB, and so support for 1TB clearly is enough.

What's a regular motherboard?

In this context, I'd expect that rdos means one for a consumer or 'prosumer' CPU, such as an i[3579] or Ryzen, as opposed to a workstation system (WS Xeon or Threadripper) or server (HPC Xeon or Epyc). At this particular point, most of the CPUs in question are limited to 64GiB or 128GiB in physical addressing.

However, it has to be understood that this doesn't mean they cannot address more than that; it only means that they cannot address more physical RAM than that. It is a limitation in how many address pins the CPU has.

Perhaps more importantly, WS and HPC motherboards - or rather, CPUs, since that is where the limitation is actually occurring - have significantly more address lines, with some being capable of addressing over 1 TB.

Furthermore, it is likely that the next generation of CPU sockets will add one or more addressing pins - keeping in mind that each additional addressing pin doubles the limit, and it is likely that any new socket will add four or more new address pins as well as other pins for purposes aside from memory addressing.

Now, more than a few motherboards have their own limit based on the amount of RAM which can be fitted to them, especially SFF mobos which often have only two DIMM slots. But this is separate from the matter of CPU addressing capacity. Most ATX mobos will work with at least one type of DIMM which can, when loaded with a full complement, bring it to the addressing limit.

I've seen this "there are no machines with more than a couple of GiB of RAM" claim so many times on this forum. It's still not true.

Well, no it isn't, and hasn't been for some time. The average new system today has 8GiB (which has been roughly standard for about five years) or 16GiB (which has become increasingly common over the past 2 years, following the drop in RAM prices in late 2018 - though this bump is mainly seen in gaming systems, as opposed to the much more numerous business and 'home productivity' systems). Even inexpensive systems generally have 8GiB at present, and 32GiB is not especially uncommon - 64GiB is still a bit of an outlier for consumer/prosumer systems, but given the increasing use of video editing software it is not unknown.

And anyone who is going to spend the money for, say, a Threadripper 3995 is likely to load at least 64GiB RAM and likely 128GiB or even 256GiB - and it is much more likely that they will pop for ECC memory (even if it isn't a requirement, as it is on Xeons), as stability is likely to be a bigger factor than raw speed on a system of that caliber.

That having been said, it is also true that a lot of older systems still run with less than 8GiB of memory.

[Schol-R-LEA]

I don't think a 64-bit address space is needed unless we are talking about really complex stuff.

This is moot, since as I have said before, it is likely that both Intel and AMD will drop support for 32-bit protected mode in the near future, just as they are already planning to do with with real mode (and 16-bit p-mode) in the next iteration of x86 systems. Note that MS have already announced that 32-bit Windows 10 will be EOL soon, and some Linux distros have done so already. Without the need to support multiple CPU modes for backwards compatibility, I cannot see the chip manufacturers continuing to add features no one is using.

Regardless of whether it is in any way necessary, it is going to be mandatory sooner than you expect.

[Korona]

Yes, today's consumer PCs have significantly less than 1 TiB of RAM.

If the question is: are 64 bits necessary to run vim and a terminal, then the answer is certainly: no. If the question is: are there algorithms and inputs that require 64 bits to work today, the answer is a clear: yes.

Also, limiting new operating systems to the current average is certainly not a good idea. After all, 64K turned out *not* to be enough for everybody.

To further add to that: the amount of memory where switching to 64 bits makes sense is not >4 GiB but rather around 756 MiB or so since systems that cannot address all memory at the same time are at a significant disadvantage. For example, Linux devs experienced significant speedups of x86_64 over i386 for ~765 MiB of memory due the reduced TLB contention when all physical memory is mapped at the same time.

[rdos]

I don't think a 64-bit address space is needed unless we are talking about really complex stuff.

This is moot, since as I have said before, it is likely that both Intel and AMD will drop support for 32-bit protected mode in the near future, just as they are already planning to do with with real mode (and 16-bit p-mode) in the next iteration of x86 systems. Note that MS have already announced that 32-bit Windows 10 will be EOL soon, and some Linux distros have done so already. Without the need to support multiple CPU modes for backwards compatibility, I cannot see the chip manufacturers continuing to add features no one is using.

Regardless of whether it is in any way necessary, it is going to be mandatory sooner than you expect.

I suspect you might be right, mostly because of EFI. Basically every new computer sold today boot the OS with 64-bit EFI, and so the processor would not be required to boot up in real mode anymore, or even support any of the 32-bit or 16-bit stuff. Still, there are a lot of old stuff in the embedded area, and so I don't see it coming soon that every new machine sold will lack 32-bit protected mode. But some certainly might, and it might disappear in the near future.

At least the current generation of Ryzen threadripper still has protected mode.

[rdos]

In this context, I'd expect that rdos means one for a consumer or 'prosumer' CPU, such as an i[3579] or Ryzen, as opposed to a workstation system (WS Xeon or Threadripper) or server (HPC Xeon or Epyc). At this particular point, most of the CPUs in question are limited to 64GiB or 128GiB in physical addressing.

Well, I initially looked at Epyc and multi-CPU boards, but decided that Threadripper offered more power with the same investment. At that time (about a year ago), there were no motherboards for Threadripper that supported more than 128GB.

[rdos]

I don't think a 64-bit address space is needed unless we are talking about really complex stuff.

This is moot, since as I have said before, it is likely that both Intel and AMD will drop support for 32-bit protected mode in the near future, just as they are already planning to do with with real mode (and 16-bit p-mode) in the next iteration of x86 systems. Note that MS have already announced that 32-bit Windows 10 will be EOL soon, and some Linux distros have done so already. Without the need to support multiple CPU modes for backwards compatibility, I cannot see the chip manufacturers continuing to add features no one is using.

Regardless of whether it is in any way necessary, it is going to be mandatory sooner than you expect.

I suspect you might be right, mostly because of EFI. Basically every new computer sold today boot the OS with 64-bit EFI, and so the processor would not be required to boot up in real mode anymore, or even support any of the 32-bit or 16-bit stuff. Still, there are a lot of old stuff in the embedded area, and so I don't see it coming soon that every new machine sold will lack 32-bit protected mode. But some certainly might, and it might disappear in the near future.

At least the current generation of Ryzen threadripper still has protected mode.

There is another dimension to this too. In long mode there is compatibility mode that many older 32-bit applications still will use in 64-bit Windows. I don't think they can remove this anytime soon. The issue here is that compatibility mode is essentially the same thing as protected mode, and so they won't save much by removing protected mode. Also, compatibility mode today includes 16-bit protected mode support too, but they might be able to remove that.

Given that, I can still run my 32-bit OS in compatibility mode, and so the removal of protected mode will not be a "show stopper". but the removal of compatibility mode would be. I still use some 16-bit drivers, but those would certainly be possible to move to 32-bit.