OSDev.org

Posted: **Sun Jul 30, 2017 5:36 pm**

Making the hardware of a video card to have its very own RAM is part of the hardware acceleration at the hardware level. After all, if it's hardware acceleration it has to start from the hardware, absolutely nothing to do with creating software but the device.

It's because of the card. When I used it once for rendering a SketchUp scene under XP, that machine performed faster. Machines performed faster with that card.

You can test the performance of an onboard video card and a dedicated video one by running your favorite applications, recording TV, playing, rendering. Even Flash feels really great with this card instead of using the default onboard shared video, and that's using only a generic VBE 2.0 driver, not even a native one.

Even with just 32 or 64 MB of video memory enabled, you will find that a dedicated card will always make the system perform the best. For real-time video there cannot really be intensive swapping out of memory pages. It would become almost unusable. There is bandwidth consumption involved when the graphics chip keeps the video active, it needs to read memory all the time to hold the screen, when the cards have their own RAM, that frees the rest of the system from the load of refreshing the screen and holding it according to what video RAM contains, hardware cursors, fonts, palettes, making the system's memory hardware controllers busy waiting while the current frame finishes to be rendered to the final screen hardware.

That could probably explain why I was able to watch TV with great frame rates in a Pentium I at 100 MHz CPU when using Trident video cards with their own RAM (soldered on the PCI card).

Posted: **Mon Jul 31, 2017 5:41 am**

~ wrote:Making the hardware of a video card to have its very own RAM is part of the hardware acceleration at the hardware level. After all, if it's hardware acceleration it has to start from the hardware, absolutely nothing to do with creating software but the device.

It's because of the card. When I used it once for rendering a SketchUp scene under XP, that machine performed faster. Machines performed faster with that card.

You can test the performance of an onboard video card and a dedicated video one by running your favorite applications, recording TV, playing, rendering. Even Flash feels really great with this card instead of using the default onboard shared video, and that's using only a generic VBE 2.0 driver, not even a native one.

Even with just 32 or 64 MB of video memory enabled, you will find that a dedicated card will always make the system perform the best. For real-time video there cannot really be intensive swapping out of memory pages. It would become almost unusable. There is bandwidth consumption involved when the graphics chip keeps the video active, it needs to read memory all the time to hold the screen, when the cards have their own RAM, that frees the rest of the system from the load of refreshing the screen and holding it according to what video RAM contains, hardware cursors, fonts, palettes, making the system's memory hardware controllers busy waiting while the current frame finishes to be rendered to the final screen hardware.

That could probably explain why I was able to watch TV with great frame rates in a Pentium I at 100 MHz CPU when using Trident video cards with their own RAM (soldered on the PCI card).

I'm not really sure what to say to all that...

Are you suggesting that a 32/64MB discrete video card will _always_ make the system perform _the best_? Compared to for example Intel's Iris Plus 650? Or even a slightly older HD 4400? You can check the benchmark score for those two integrated solutions vs your 512MB AGP card, which got better score? Of course your 512MB AGP is older, so it's not a fair comparison if age is considered.

AFAIK, the main issue is not with discrete VRAM, or RAM bandwidth. For 1080p you have approx 2Mpix, each taking 4B that's 8MiB for each frame, even with 100 fps that's "only" 0.800GiB/s, current bandwidths are around 20-40GiB/s.

Posted: **Mon Jul 31, 2017 6:24 am**

LtG wrote:
Octacone wrote:I managed to solve it. Thanks for letting me know that memory_upper and memory_lower fields are not reliable.
So basically I tried to sum up all the memory map lengths, and it matched perfectly with what I was expecting.
What a stupid little misinterpretation. I didn't know it was that easy.
Now emulators return ~4 GB as expected, except Bochs. (which is always special, it also has a shorter memory map emulation so this behavior is expected)
There is one more thing doe - when I run this thing on my PC it only returns 4.3 GB instead of 8 GB. Is this supposed to happen?
With regards to the lower/upper, if I were to tell you the system has 8 GiB RAM, but I don't tell you at what _physical_ address it is, how would you ever be able to use it? By assuming it starts at zero and goes up to 8GiB? That's a relatively fair assumption, but in reality there are holes in it that are used for other purposes and are not RAM. As a consequence even if the lower/upper reported everything you still wouldn't be able to reliable use it.

So you need to look at the memory map which tells you exactly where is available RAM, that you can use as physical memory. You should look the detecting memory wiki page as well as the other related memory pages and maybe the threads where Brendan has in lengths explained how to do this stuff properly. For instance the memory map is not "sanitized" for you, which means you need to sort it, remove overlaps and in cases of overlap decide what's safe (err on the side of caution), combine adjacent areas if they are of same type, etc.. Most stuff I've seen give a nice clean memory map that is sorted, but not all devices.

As for the 4,3GiB vs 8GiB, as far as I know that shouldn't happen. So I'd still look at your code to find the reason, are you fully printing the entire memory map? Have you used some other OS (Windows, Linux, etc) on that same PC? If so, do they report all of the RAM? If you think they do, check and verify. Can you copy that memory map here? You didn't mention where you got the previous memory map from, so I don't know what that is..

I already have a working memory map implementation. The picture I showed you is actually mine, screenshot of my OS printing that memory map.
What do you mean by overlaps? Like 0x0 to 0x9FC00 and then 0x0 to 0x40000000, is that an overlap you are talking about?
That is my main PC, it has a dual boot setup, every other OS detects it perfectly.
As I already mentioned that is a legit memory map my OS outputs.

Posted: **Mon Jul 31, 2017 6:43 am**

I presumed it was your OS printing the memory map, what you haven't mentioned if it's from real hardware and whether it's the 8GiB machine and only reporting 4,XGiB, and whether other OS's report all the RAM correctly on that same machine. It's useful to be more specific, so we (I) don't have to guess. When I have to guess the number of possibilities usually grows exponentially, and writing all of those in a post takes too much time.

You may want to check your code, the mmap now that I looked at it more closely seems to mismatch the length and (end - start). For instance chunk 6 starts at 0x0 and ends at 0x100000000, which would mean 4GiB, yet the size says 1GiB.

The overlap you mention is the type of overlap I was referring to, though in this case that might be caused by your code and not really part of the mmap given by BIOS/GRUB. IIRC most modern systems give nice sane mmaps, sorted and without overlaps, but it is not guaranteed and so you have to prepare for the worst and sanitize the mmap yourself to be safe.

Posted: **Mon Jul 31, 2017 11:35 am**

LtG wrote:I presumed it was your OS printing the memory map, what you haven't mentioned if it's from real hardware and whether it's the 8GiB machine and only reporting 4,XGiB, and whether other OS's report all the RAM correctly on that same machine. It's useful to be more specific, so we (I) don't have to guess. When I have to guess the number of possibilities usually grows exponentially, and writing all of those in a post takes too much time.

You may want to check your code, the mmap now that I looked at it more closely seems to mismatch the length and (end - start). For instance chunk 6 starts at 0x0 and ends at 0x100000000, which would mean 4GiB, yet the size says 1GiB.

The overlap you mention is the type of overlap I was referring to, though in this case that might be caused by your code and not really part of the mmap given by BIOS/GRUB. IIRC most modern systems give nice sane mmaps, sorted and without overlaps, but it is not guaranteed and so you have to prepare for the worst and sanitize the mmap yourself to be safe.

That picture is displaying 4 GB memory map. It is from a virtual machine (QEMU).
Most modern system do give nice and sane maps okay, what about emulators?
Could that 0x0 be something else perhaps? I think that should be like 0xFFC00000 - 0xFFFFFFFF?

Posted: **Mon Jul 31, 2017 12:11 pm**

Octacone wrote: Most modern system do give nice and sane maps okay, what about emulators?
Could that 0x0 be something else perhaps? I think that should be like 0xFFC00000 - 0xFFFFFFFF?

emulators almost always give nice, simple, well formatted list

to me, that last entry looks to me like it should be 0x1_0000_0000 (that is, 4GB) but you are ignoring the upper 32bits of the address, so it becomes 0

Posted: **Mon Jul 31, 2017 1:31 pm**

LtG wrote:Are you suggesting that a 32/64MB discrete video card will _always_ make the system perform _the best_? Compared to for example Intel's Iris Plus 650? Or even a slightly older HD 4400? You can check the benchmark score for those two integrated solutions vs your 512MB AGP card, which got better score? Of course your 512MB AGP is older, so it's not a fair comparison if age is considered.

AFAIK, the main issue is not with discrete VRAM, or RAM bandwidth. For 1080p you have approx 2Mpix, each taking 4B that's 8MiB for each frame, even with 100 fps that's "only" 0.800GiB/s, current bandwidths are around 20-40GiB/s.

If you implement the best video chipset as video that takes/shares memory from main system RAM, you will destroy its performance.

A video chipset will always be faster if it's implemented as dedicated video with its very own RAM when it needs to access RAM to refresh the whole display features at 30 FPS or more, and several Megabytes at that speed. At least RAM access between hardware devices is much faster than doing the same in software, but it still affects.

Integrated video cards that share main system RAM are not normal.

Most old PCI video cards had their own memory so they didn't need to make the main RAM controller chips busier. They were like dedicated video accelerated through implementing them with their own RAM.

Maybe that's why PCI cards without their own onboard RAM didn't work in some Pentium I motherboards. Probably they were made for systems that were able to share main RAM with the video. At the time those cards had less than 32 or 16MB, so it was common even for Trident VGAs.

So in those cases the system was as balanced and ran as fast as it already was capable of.

It is not normal that current dedicated video cards make look onboard RAM as a rarity and shared video as the norm when initially it wasn't so and when it's possible and more efficient to separate main RAM and video RAM buses for obviously more speed.

If you have a 2 GHz single-core CPU and a 400 MHz video card, in separate buses, obviously it would be as if you had added a Pentium I to your system just for faster video, but the moment you share main RAM for several things, the performance is cut down.

The RAM itself is slower than any CPU, so loading it with more accesses than just code execution and data access will affect the system noticeably by a human.

Don't forget that accessing video contents and displaying them at a speed that is perceivable by the human eye can block the execution a bit, but that's why people ends up looking for faster video and systems as soon as they run intensive multimedia applications that become laggy.

Even setting up CPU MTRRs to use write combine for video (write combine if I remember correctly, as in MenuetOS) results in noticeably faster video. So we can see that something as simple as 30 FPS video, or 60 Hz, etc., for the actual screen hardware, can affect everything when seen as an electronics system.

Remember that if your video looks slow, that huge slowing down will slow down all of your programs as well, and even more if they use the screen at all. One way to escape that would simply be to avoid using video at all, disable the display in hardware so that it shuts off, and execute most critical programs as if it was a headless machine... maybe that's why servers prefer to be headless, to cut down an unnecessary slow down for a display chip that it won't be running anyway.

Posted: **Mon Jul 31, 2017 3:22 pm**

Looks like there is no way for me to print 64 bit values. Shifting does not work either. 32 bit protected mode refuses to work with 64 bit values.
Anybody has an idea on how to print 64 bit values on a 32 bit system??

Posted: **Mon Jul 31, 2017 3:56 pm**

I made a tutorial about printing FPU numbers in February, which can handle 80-bit floats and 64-bit integers:
http://f.osdev.org/viewtopic.php?t=31357

Code:
http://devel.archefire.org/tmp/FPU_float_print_demo.zip

You can build 64-bit variables adjusting with carry and the like.

Or you could just use the FPU, load values as 80-bit decimals and retrieve as 64-bit integer.

You can make an FPU-only printing routine that uses the FPU to generate each character in the numeric string.

In this case you can just load a 64-bit integer number which will be converted to 80-bit float internally, but since you will only use integers, you can simply print the part that is greater than zero, and in this case divide by 16 instead of 10 and print hexadecimal digits instead of decimal ones.

You will be using solely the FPU even to convert each binary value to its corresponding ASCII digit. That's what the demo function does, for decimals. The hexadecimal version would need a little bit of more work.

Posted: **Mon Jul 31, 2017 4:01 pm**

Octacone wrote:Looks like there is no way for me to print 64 bit values. Shifting does not work either. 32 bit protected mode refuses to work with 64 bit values.
Anybody has an idea on how to print 64 bit values on a 32 bit system??

if your just printing them as hex, its incredibly easy:

first print the upper 32 bits
then print the lower 32 bits immediately following it (without the "0x" prefix -- though you might want to put a space between them to make it easier to read)

Posted: **Mon Jul 31, 2017 5:10 pm**

JAAman wrote:
Octacone wrote: Most modern system do give nice and sane maps okay, what about emulators?
Could that 0x0 be something else perhaps? I think that should be like 0xFFC00000 - 0xFFFFFFFF?
emulators almost always give nice, simple, well formatted list

to me, that last entry looks to me like it should be 0x1_0000_0000 (that is, 4GB) but you are ignoring the upper 32bits of the address, so it becomes 0

I don't know what happened but that last 0x0 used to be 0xFFC0000. (old picture of mine confirmed this)

JAAman wrote:
Octacone wrote:Looks like there is no way for me to print 64 bit values. Shifting does not work either. 32 bit protected mode refuses to work with 64 bit values.
Anybody has an idea on how to print 64 bit values on a 32 bit system??
if your just printing them as hex, its incredibly easy:

first print the upper 32 bits
then print the lower 32 bits immediately following it (without the "0x" prefix -- though you might want to put a space between them to make it easier to read)

Upper then lower, OH!
I guess that is why it didn't work. I somehow flipped those two around and since my hex function is very dynamic it got me confused.
Because it has 4 output types: 0xBEEF, 0x0000BEEF, BEEF and 0000BEEF.
Now it works wonderfully, +1. But only with manually entered values and not variables. Could be that my uint64_t is not long enough.
Basically this works:

Code: Select all

TUI.Put_Hex((8589934592 >> 32), 0x0E, 1);
TUI.Put_Hex((8589934592), 0x0E, 4);

but this doesn't:

Code: Select all

uint64_t bytes = 8589934592;
TUI.Put_Hex((bytes >> 32), 0x0E, 1);
TUI.Put_Hex((bytes), 0x0E, 4);

Both uint64_t (unsigned long int) and sizeof(8589934592) are 4 bytes long. Why would that cause a problem?
Edit: it works with unsigned long long, which is 8 bytes in size.
Edit 2: conclusion: wrong types defined after all.
Edit 3: after some slight variable length mods my real PC reports 8.2 GB of total RAM.

Posted: **Mon Jul 31, 2017 10:11 pm**

Octacone wrote:Edit 2: conclusion: wrong types defined after all.

Are you defining uint64_t yourself instead of using the definition provided by your cross-compiler? It should be provided by the <cstdint> header.

Posted: **Tue Aug 01, 2017 4:30 am**

Octocontrabass wrote:
Octacone wrote:Edit 2: conclusion: wrong types defined after all.
Are you defining uint64_t yourself instead of using the definition provided by your cross-compiler? It should be provided by the <cstdint> header.

I don't like to include anything I haven't reviewed or written myself. It just breaks the flow.

After some slight printing mods, this is how it looks like:
Real PC: //hand written...took ages to write

Emulator:

: newmmap.png (7.06 KiB) Viewed 4511 times

Another very important question:
If there are lets say 64 MB of RAM how can 0xFFFFFFFF exist? It is not virtual memory, how is this possible to maintain that?

Posted: **Tue Aug 01, 2017 5:34 am**

Octacone wrote:I don't like to include anything I haven't reviewed or written myself. It just breaks the flow.

I think accidentally defining uint64_t as a 32-bit integer is breaking your flow even more...

Octacone wrote:After some slight printing mods, this is how it looks like:

The end address looks reasonable, but the start address and length both seem to be wrong.

Octacone wrote:Another very important question:
If there are lets say 64 MB of RAM how can 0xFFFFFFFF exist? It is not virtual memory, how is this possible to maintain that?

The chipset can leave empty holes in the memory map if it wants to. There is no requirement to have RAM in one continuous block. There are also memory-mapped peripherals, which can also be mapped just about anywhere. For example, 0xFFFFFFFF is the BIOS ROM.

Posted: **Tue Aug 01, 2017 6:18 am**

Octocontrabass wrote:
Octacone wrote:I don't like to include anything I haven't reviewed or written myself. It just breaks the flow.
I think accidentally defining uint64_t as a 32-bit integer is breaking your flow even more...

It breaks even more than that.

Octocontrabass wrote:
Octacone wrote:After some slight printing mods, this is how it looks like:
The end address looks reasonable, but the start address and length both seem to be wrong.

Roger that.

Octocontrabass wrote:
Octacone wrote:Another very important question:
If there are lets say 64 MB of RAM how can 0xFFFFFFFF exist? It is not virtual memory, how is this possible to maintain that?
The chipset can leave empty holes in the memory map if it wants to. There is no requirement to have RAM in one continuous block. There are also memory-mapped peripherals, which can also be mapped just about anywhere. For example, 0xFFFFFFFF is the BIOS ROM.

++ combined reply ++

pitfall wrote:
Another very important question:
If there are lets say 64 MB of RAM how can 0xFFFFFFFF exist? It is not virtual memory, how is this possible to maintain that?
As said before, chipset maps RAM and IO memory wherever it "wants" to.

On my laptop (with UEFI and BIOS emulation layer), I did some tests and ended up with a somewhat strange but logical (as it makes sense) memory map, as only 2GiB are mapped before 4GiB mark...

So it mapped :

0-640KiB as conventional memory
640KiB-1MiB for legacy BIOS data and firmware areas (as conventional memory AND memory mapped ROM)
1MiB-2GiB as conventional memory
3GiB-4GiB as various IO and mapped ROM address ranges, including local APIC and IO/APICs
4GiB-6GiB as conventional memory
7GiB-9GiB as conventional memory
10GiB-12GiB as conventional memory

I still don't know exactly why it created 1GB holes between memory slots...

So please, do not assume RAM slots are mapped contiguously, it may seem a fair guess, but it can be totally wrong and you can hit a non-mapped / out-of-RAM address.

Please note another thing : on (very) old systems, a hole (called ISA-hole) may exist between 15MiB and 16MiB. That's why BIOS int15h, ah=88h returned available memory BEFORE 16MiB... I still have a 80486 with this memory hole.

Here were my 2 cents,

pit'

That is horrible. Why isn't that standardized and made contiguous.
It makes my life so much more complicated.
How am I supposed to use it with my bitmap based PMM? I used to do it like bitmap_area = from 0x0 to total memory (e.g if you have 2 GB it would be 0x80000000). Aka I assumed it was contiguous.
Here is my current plan:
1.Map every "reserved type" area as used. (PMM)
2.Make bitmap start form 0x0 and end at the last memory map address given. Mark every free area as usable. (PMM)
3.Profit?

Before doing so (if you guys agree it is a good thing) I need to fix the actual memory map itself.

OSDev.org

GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory

Re: GRUB Returning Less Memory