Developing hobby OS on Raspberry Pi 3

[zaval]

it's a very decent rpi in fact. far better, than anything they have done before - 4 core Cortex-A72, 4 GB of LPDDR4 memory, USB3, dual display support, Gigabit Ethernet and that's for 55 dollars - very good. also, they made goodies - cute keyboard/hub and mouse, the case and cables as well. ^_^ but yeah, the documentation, where it is? for the OS hobbyists, the documentation is the most needed feature, and it's still missing. also weird, that they still haven't added eMMC, why? In this light, I am wondering (again and again) - have they finally implemented UHS-I support for SD cards. or still 20 MB/s? one can easily guess, that 20 MB/s on the boot medium, where the OS and programs live, dumbs the board down hardly.

[Octacone]

it's a very decent rpi in fact. far better, than anything they have done before - 4 core Cortex-A72, 4 GB of LPDDR4 memory, USB3, dual display support, Gigabit Ethernet and that's for 55 dollars - very good. also, they made goodies - cute keyboard/hub and mouse, the case and cables as well. ^_^ but yeah, the documentation, where it is? for the OS hobbyists, the documentation is the most needed feature, and it's still missing. also weird, that they still haven't added eMMC, why? In this light, I am wondering (again and again) - have they finally implemented UHS-I support for SD cards. or still 20 MB/s? one can easily guess, that 20 MB/s on the boot medium, where the OS and programs live, dumbs the board down hardly.

It seriously pisses me off. They talk about it like it's some kind of a miracle, when in reality you can only run stinking Linux on it. What about us hobbyists?
It isn't friendly at all, can't find the documentation, closed source GPU chips...
Also about the storage, just give us a damn SATA (hell even PATA is ok) controller, not an SD card - which was never supposed to be used for that purpose in the first place.

[iansjack]

Not just Linux. Also RiscOS and FreeBSD (or even Windows 10 IOT). Hobby OSs are possible, but it's a bit more work than on an x86 PC - not surprising considering that they have been developed for nearly 40 years now. Hence there is a wealth of knowledge and documentation. But it's the challenge that makes it fun.

If you don't like SD cards just use USB storage. It's plenty fast enough for the class of machine, and you can boot from USB.

Don't get pissed off about the fact that decent computers are available nowadays for $30. Just be grateful that you are not living in the times when a very basic x86 PC set you back $1500 (about $4500 in today's money). So buy yourself 170 of them and make a decent little cluster.

[Octacone]

Not just Linux. Also RiscOS and FreeBSD (or even Windows 10 IOT). Hobby OSs are possible, but it's a bit more work than on an x86 PC - not surprising considering that they have been developed for nearly 40 years now. Hence there is a wealth of knowledge and documentation. But it's the challenge that makes it fun.

If you don't like SD cards just use USB storage. It's plenty fast enough for the class of machine, and you can boot from USB.

Don't get pissed off about the fact that decent computers are available nowadays for $30. Just be grateful that you are not living in the times when a very basic x86 PC set you back $1500 (about $4500 in today's money). So buy yourself 170 of them and make a decent little cluster.

They (so called development boards and operating system that they support) all fall under the same category. Commercialism and capitalism. I've never said anything about hating SD cards, I was actually referring to zaval, it's the speed that matters. They are OK for data logging, but not for installing an OS onto them since their write cycles are limited IIRC and they are also slow for that purpose. Also USB is unpractical, not rigid (speaking of the connector) and impractical for embedded/proprietary projects. True, Raspberry PI is a great little machine in a compact form factor but still lagging far behind (in some aspects). It would be nice for them to offer (not all boards get them) some x86 features and make them customizable at an increased price point ofc.

[bzt]

Raspberry Pi 4 is out with some interesting new features. USB3, dual-HDMI, giga-ethernet and fast RAM up to 4G just to name a few.
From programmer's point of view, backward compatible, but there's an integrated GIC too, and the Cortex-A72 supports ARMv8a. GPU is now VC VI with Mesa drivers.

Cheers,
bzt

[iansjack]

They (so called development boards and operating system that they support) all fall under the same category. Commercialism and capitalism.

You are surprised that the manufacture of computers - even those costing only $30 - falls under the category of commercialism and capitalism? Do you suppose the $1500 ($4500 in today's money) IBM PC wasn't predicated on such capitalist principles? Do you suppose that Linux is based more on capitalist values than Microsoft or Apple's offerings? 40 years ago you'd have given your back teeth for a $30 dollar computer running a free operating system.

The fact is that the Raspberry Pi has opened up computing to millions of people in the third world who can't afford the hundreds or thousands of dollars required by conventional PCs, not to mention educational uses in the developed world. It may not suit your particular needs, but don't knock the power for good that it has been.

[eekee]

Advice for RPi haters: Buy secondhand laptops, relax, have fun. I do! I mean look for laptops "sold as seen" "for parts", but pick brands and ranges known to last well. Thinkpad T series is mostly good, X series not too bad. Thinkpad R series were supposed to be cheap, but both the R400 Thinkpads I bought are great. R400 is like X61 but with a screen that lasts longer, and all fitted into a big solid T61 case.

Advice for RPi lovers: Unless you specifically want ARM or lots of GPIO, see above!

Anyone remember the ZX81? It was under £50 when PCs were over £1000, and if you think the Raspberry Pi sucks... ohhhh boy, oh boy oh boy, it SUCKED! Didn't stop my brother in law soldering up a 16KB RAM pack, or me from soldering up a bus extender and a parallel IO card; GPIO in modern terms. It didn't stop Maplin from offering a whole line-following robot kit based around it, either.

[zaval]

eekee, what you say is totally out of context. RPi "sucks" not because it is weak compared to x86 machines, it's absolutely irrelevant, it does suck because:
1) its hardware is weak compared to competitors' ARM boards for the same price (RPi4 excluded, since it's not an underdog anymore)
but mostly because:
2) hobby OS guy won't be able to program it due to lack of documentation. Just comparison - everything you could get from the official site about RPi4 SoC is a brochure 13 pages worth. At the same time ARM SoC vendors like NXP and TI release technical reference manuals for their SoC several thousand pages worth. People learn how to program other vendors' SoC reading these (because same or similar IPs).
And this is to iansjack - ARM is not new, it's 30, SBCs aren't new, the first appeared more than decade ago I guess, RPi is not fun to write OS for because you cannot get information on its hardware. And it's not RPi foundation to put blame on, it's Broadcom. Why RPi should care about tens of weirdos, tryna write their OSes? Most people over there use that raspbian thing. But that makes RPi de facto OS development unfriendly thing.
ah, yet, personally I have yet one:
3) VideoCore GPU playing the role of system controller, a very weird organization, where CPU is not "central". But eh, it's nothing compared to the 2.

PS. And to answer my question made in the previous post here, - no, they haven't included UHS-I support.

[eekee]

ARM SoC vendors like NXP and TI release technical reference manuals for their SoC several thousand pages worth. People learn how to program other vendors' SoC reading these (because same or similar IPs).

This is a point worth highlighting and repeating. Go where the docs are, guys!

... Old laptops are better documented than RPi. @ zaval! (I know some PC chipsets aren't documented well enough, I'm just teasing.)

More seriously, I don't think it's bad to have the graphics core bring up the "central" processor. Modern CPUs are ridiculously complicated things, the busses around them also non-trivial, and even the RAM is crazy. I can well imagine they need help. AFAIK graphics... what's the word? used to be "chipsets"... have their own CPUs to organize their own wierdness, why shouldn't that processor help with the rest of the system too?

[bzt]

hobby OS guy won't be able to program it due to lack of documentation

You keep saying that, but guess what, these guys haven't read your post!

- https://github.com/s-matyukevich/raspberry-pi-os
- https://github.com/moizumi99/RPiHaribote
- https://github.com/jsandler18/raspi-kernel
- https://github.com/qwqw330/raspiOS
- https://github.com/weidongguo/RaspiOS
- https://github.com/rsta2/circle
- https://github.com/ultibohub/Core
- https://github.com/dwelch67/raspberrypi
- https://github.com/LdB-ECM/Raspberry-Pi
- ...and the list just goes on and on...

Just because you can't find the documentation, doesn't mean there's none. Interestingly everybody else except you could find them.
And many from the above list have linked those "non-existstent" docs in their README's, that must be some kind of magic!

Cheers,
bzt

[zaval]

Interestingly everybody else except you could find them.
And many from the above list have linked those "non-existstent" docs in their README's, that must be some kind of magic!

It's not magic, it's a pretty common thing for now - you again playing a smart aleck.
Ok, I am a stupid dumb4ss, have been looking at the official site and didn't find TRMs. Neither I did by following the links you provided. Please note, - TRM on Cortex-Axx is NOT TRM on SoC. Could you help me and give a link for the BCM2711 TRM or BCM2836*, from where I can get at least the info on how to program SD/MMC, USB, Ethernet etc. let alone GPU. So?

I actually want TRM for BCM2711, but I would be happy to see you showing me TRM on ANY Broadcom SoC.

[iansjack]

From bzt's first link:

https://github.com/raspberrypi/document ... -.V2-1.pdf

[Candy]

I found nearly everything on the Pi 1-3 to be well documented, except for the USB controller. Now that the Pi 4 is out, I'm starting back up and have looked into what it has, and I'm working on the wiki page for the Pi 4 to have links to all docs & Linux drivers for relevant bits.

I need somebody to take a Pi 4 with 4GB of ram to check how the memory layout there is done; on a Pi 4 with 1GB it's similar/same-ish to the Pi 3+ with 1GB. I have bootlogs and all other related stuff for the 1GB one that I have to fill out that page more.

Come and help! https://wiki.osdev.org/Raspberry_Pi_4

[zaval]

From bzt's first link:

https://github.com/raspberrypi/document ... -.V2-1.pdf

204 pages... kinda thing after 13 pages of BCM2711. let's see.

1 Introduction
1.1 Overview
BCM2835 contains the following peripherals which may safely be accessed by the ARM:
• Timers
• Interrupt controller
• GPIO
• USB
• PCM / I2S
• DMA controller
• I2C master
• I2C / SPI slave
• SPI0, SPI1, SPI2
• PWM
• UART0, UART1
The purpose of this datasheet is to provide documentation for these peripherals in sufficient
detail to allow a developer to port an operating system to BCM2835.
There are a number of peripherals which are intended to be controlled by the GPU. These are
omitted from this datasheet. Accessing these peripherals from the ARM is not recommended.

LOL. that's all? UARTs and SPIs?

... Well, this is basically what I tried to say - Broadcom SoCs are NOT ARM processors. You can't even program SD card controller! I2C, SPI and UART? seriously? accessing other peripherals from the ARM "is not recommended"?

Just read the quoted lines:

There are a number of peripherals which are intended to be controlled by the GPU. These are
omitted from this datasheet. Accessing these peripherals from the ARM is not recommended.

Were they high making this? So ethernet, wifi, blutooth, sd cards are "intended to be controlled by the GPU"? entirely? not even a chance to program them? amazing. and of course that CPU/GPU mutant isn't nearly known how to program it. except those moronic mailboxes. and of course, this all lets one very "good" example of how to create an OS on ARM and in general. That "OS" would be able to only make blinking LEDs by itself, because GPIO is accessible! tada! Honestly, I'm shocked. Basically all RPi "system" programming boils down to knocking on that VC thing and asking it to do everything - hey, read SD card please, oh, send this on ethernet... It's even cringiest, than I thought. Because I thought peripherals are accessible and controllable from ARM, just aren't documented. Nope, they aren't even accessible.
And this:

1.2 Address map
1.2.1 Diagrammatic overview
In addition to the ARM’s MMU, BCM2835 includes a second coarse-grained MMU for
mapping ARM physical addresses onto system bus addresses. This diagram shows the main
address spaces of interest:

coARSE-grained - good term for this, by the way! Oh, boy. ARM physical addresses are not enough. there are additional "system bus addresses".


But it's expected - since ARM is not CPU there, it doesn't own system address space.

in sum - we still don't have info on how to program the SD controller. One of the first crucial peripherals on an SBC. No, we do have the info. And it says - we cannot program it! Yay. Good luck with that to everybody wishing to mess around with this abysmal design. Fortunately, there is plenty of real ARM computers, with the normal architectural organization. By the way, next year Rockchip will release rk3588 - octacore dual-clustered design with x4 Cortex-A76 and x4 Cortex-A55! both are the newest ARM thingies. And IIRC, it will be made with something like 7nm or 12nm node size! awesome.

[iansjack]

I would be happy to see you showing me TRM on ANY Broadcom SoC.

Oh dear. Happiness is so ephemeral.

I can't do all your research for you, but you did read the chapter on the EMMC, didn't you?