OSDev.org

I am going to take a shot at ARM development. However looking at hundred of choices I am swamped.
I am looking to get compelte development kit that can power up and start programming in minutes, including desktop programming software etc, What is available?

I am literally looking to develop bootloader + some aspect of kernel development.
For that I am looking for faily powerful CPU 8-16 core, minimum 512M-1GB and around similar storage space at 512M.

Your only real option for desktop level of features, at the lowest possible cost (under £10) is the RaspberryPi Zero. This obviously doesn’t have required the level of CPU power you want... but then that’s what the RapsberryPi 4 is for.

bzt has written some RaspberryPi tutorials which is also very helpful.

-edit- I found this serial kernel loader really helpful (for RaspberryPi 3) development work: https://github.com/robey/c3r3s

bloodline wrote:bzt has written some RaspberryPi tutorials which is also very helpful.

Thanks! Here's the link.

ggodw000 wrote:I am looking to get compelte development kit that can power up and start programming in minutes,

I've also written a loader for the Pi, which makes this possible, see BOOTBOOT. But is for 64 bit kernels only (meaning you'll need Raspberry Pi 3 with Cortex-A52 at least), and it does not contain a "complete development kit", rather it allows you to use the standard GNU toolchain without tricks and Assembly magic. An example Hello World kernel in C is included which compiles to x86 and ARM just as-is (there's also a Rust kernel example, but I don't know if that works for ARM). You'll realize pretty soon that physically removing / reattaching the SD card and mounting / umounting on your PC to copy your latest code on it is cumbersome, so BOOTBOOT is also capable of loading your OS image over serial line.

If you're looking for a Multiboot solution, I'd recommend jncronin's rpi-boot. It is too bloated to my taste, and also note there's no official Multiboot spec for ARM, so it's not guaranteed to be compatible with other implementations. Advantage that is can also boot your kernel over serial.

Another option would be berryboot, but again, that's also too bloated to my taste, so I don't use it. It is a very complex GUI installer solution which happens to have a somewhat-Multiboot-compatible loader component.

ggodw000 wrote:including desktop programming software etc, What is available?

...however I don't think you'll find such a thing. For desktop programming, I'd recommend RaspiOS (formerly Rasbian), which is a debian-based Linux distro. The whole point of programming bare metal is that you don't have a system nor a desktop environment.

ggodw000 wrote:I am literally looking to develop bootloader + some aspect of kernel development.

For that, you'll have to take a deep dive in the ARM and BCM specs. Sadly they are not as easy to find as their x86 counterparts, and they are full of typos and errors. Always look for errata. My loader is MIT licensed, you're free to study its source.

ggodw000 wrote:For that I am looking for faily powerful CPU 8-16 core, minimum 512M-1GB and around similar storage space at 512M.

I don't think such a devboard exists. Qualcomm has just recently announced 8 cores Snapdragon (but that's not SMP, meaning cores aren't symmetric). Apple's M1 has 8 cores, but only 4 is symmetric. Raspberry Pi 3 has 4 cores, 1GB RAM, and the storage is up to you. Raspberry Pi 4 has also 4 cores, but can be purchased with more than 1G RAM too.

Cheers,
bzt

bzt wrote:

bloodline wrote:bzt has written some RaspberryPi tutorials which is also very helpful.

Thanks! Here's the link.

ggodw000 wrote:I am looking to get compelte development kit that can power up and start programming in minutes,

I've also written a loader for the Pi, which makes this possible, see BOOTBOOT. But is for 64 bit kernels only (meaning you'll need Raspberry Pi 3 with Cortex-A52 at least), and it does not contain a "complete development kit", rather it allows you to use the standard GNU toolchain without tricks and Assembly magic. An example Hello World kernel in C is included which compiles to x86 and ARM just as-is (there's also a Rust kernel example, but I don't know if that works for ARM). You'll realize pretty soon that physically removing / reattaching the SD card and mounting / umounting on your PC to copy your latest code on it is cumbersome, so BOOTBOOT is also capable of loading your OS image over serial line.

If you're looking for a Multiboot solution, I'd recommend jncronin's rpi-boot. It is too bloated to my taste, and also note there's no official Multiboot spec for ARM, so it's not guaranteed to be compatible with other implementations. Advantage that is can also boot your kernel over serial.

Another option would be berryboot, but again, that's also too bloated to my taste, so I don't use it. It is a very complex GUI installer solution which happens to have a somewhat-Multiboot-compatible loader component.

ggodw000 wrote:including desktop programming software etc, What is available?

...however I don't think you'll find such a thing. For desktop programming, I'd recommend RaspiOS (formerly Rasbian), which is a debian-based Linux distro. The whole point of programming bare metal is that you don't have a system nor a desktop environment.

ggodw000 wrote:I am literally looking to develop bootloader + some aspect of kernel development.

For that, you'll have to take a deep dive in the ARM and BCM specs. Sadly they are not as easy to find as their x86 counterparts, and they are full of typos and errors. Always look for errata. My loader is MIT licensed, you're free to study its source.

ggodw000 wrote:For that I am looking for faily powerful CPU 8-16 core, minimum 512M-1GB and around similar storage space at 512M.

I don't think such a devboard exists. Qualcomm has just recently announced 8 cores Snapdragon (but that's not SMP, meaning cores aren't symmetric). Apple's M1 has 8 cores, but only 4 is symmetric. Raspberry Pi 3 has 4 cores, 1GB RAM, and the storage is up to you. Raspberry Pi 4 has also 4 cores, but can be purchased with more than 1G RAM too.

Cheers,
bzt

Ok, raspberry PI3/4 has its own O/S, does that mean I can bypass O/S and directly boot from my own bootloader? For that, I am assuming there is going to be some way to flash the whatever drive with the bootloader code. I will need to look at spec.

ggodw000 wrote:
Ok, raspberry PI3/4 has its own O/S, does that mean I can bypass O/S and directly boot from my own bootloader? For that, I am assuming there is going to be some way to flash the whatever drive with the bootloader code. I will need to look at spec.

You can Google, or check the RaspberryPi forums for this information, but the RaspberryPi doesn’t use onboard flash. It loads the kernel from a FAT32 formatted SD card (partition) upon boot. You can put whatever OS you like on that SD card, including your own (kernel or bootloader).

You can also boot from a USB device on newer models.

ggodw000 wrote:Ok, raspberry PI3/4 has its own O/S

The Pi only has it's own OS in the sense that any computer does. There are a number of alternatives, and you can boot your own kernel.

ggodw000 wrote:Ok, raspberry PI3/4 has its own O/S, does that mean I can bypass O/S and directly boot from my own bootloader?

No and yes

ggodw000 wrote:For that, I am assuming there is going to be some way to flash the whatever drive with the bootloader code. I will need to look at spec.

Yes. The following files are loaded from the SD card in order:
bootcode.bin - raw binary for the GPU. Only RPi3 needs this, RPi4 has it in a flash
startX.elf - another binary for the GPU. Replacing this can make the board boot from USB, CD-ROM, etc.
kernelX.img - raw ARM binary, loaded by startX.elf (simple startX.elf loads it from the same SD card). There are different files for AArch32 and Aarch64

With RaspiOS, the Linux kernel is in the kernelX.img file compiled for ARM. If you're aiming at an ARM OS, that's the file you can replace. But there are experimental OSes written for the VC GPU, those replace startX.elf.

Cheers,
bzt

bllodline wrote: Your only real option for desktop level of features, at the lowest possible cost (under £10) is the RaspberryPi Zero. This obviously doesn’t have required the level of CPU power you want... but then that’s what the RapsberryPi 4 is for.

that's not true. to name a few, that have boards, beating the dust out of RPi.
Pine - Pine64, Rock64 and Co. look at "devices" section and then on wiki
Radxa - Rock Pi and Co
Hardkernel - Odroids
FriendlyARM - NanoPi's
Sinovoip - Bananas
Xunlong - Oranges
Khadas - Khadas VimNs
besides, the author didn't say, he looks for something under £10, but none of the listed boards producers make "overpriced" things, they all are pretty humble in pricing. The link for Xunlong is for their Aliexpress store, but it's legit, and they respond there, you may ask things, I don't know/remember if they have their own site.

IMHO, RPi is the worst choice for the "developing bootloader + some aspect of kernel development", that is for "bare metal programming" - it's not even ARM, it's VC plus ARM as a toy controller. all the mentioned above, on the other hand, are real ARM computers, and you can bypass linux and directly handle the board. heck, you can even bypass uboot and take the whole control over everything, including with SDRAM initialization and other fun stuff. if you want something most powerful of this sea, than the best, still very affordable in price are Odroid-N2+ or any RK3399 based board, Pine even has a laptop with this chip for 200$. but the boards are from 60-85$ depending on amount of RAM mostly. and important enough - rk3399 is (relatively) well documented, at least if you can google (you'll need the leaked v1.3 TRM instead of halfassed "opensauce" v1.4 TRM. in short, Odroid N2+ (Amlogic S922X rev C) is hexa core (4 Cortex-A73 + 2 Cortex-A53), slightly more powerful, than RK3399, but very slightly, the latter, on the other hand (also hexa core 2 Cortex-A72 + 4 Cortex-A53) - has more interesting peripherals, and better documented.

PS. ah, and if you aren't satisfied with performance offering, the mentioned boards make, look at this "monsta", still not whoppingly overpriced, here, this one and Co. but the SoCs they have, are from Huawei, with all the consequences. it's not "apple silicone b00b5", but it's a really cool board, it has even UFS3 storage! for the osdev, that would be fantastic. just as NVMe. linux is frying it god awully though - I watched the video on youtube. running linux, this board looks like its capabilities are dumbed down by 10. but you probably are gonna create something better, than linux? there is also Asus Tinkerboards, Nvdia boards, but the latter are priced higher, than all aforementioned.

zaval wrote:IMHO, RPi is the worst choice for the "developing bootloader + some aspect of kernel development"

I disagree. Neither of the boards you have listed has as many tutorials, as big community and as many documentation as the Raspberry Pi. The chance to get any help is the highest with a Raspberry Pi.

zaval wrote:you can bypass linux and directly handle the board. heck, you can even bypass uboot and take the whole control over everything

You can do all of these with an RPi as well.

Cheers,
bzt

bzt wrote:

zaval wrote: IMHO, RPi is the worst choice for the "developing bootloader + some aspect of kernel development"

I disagree. Neither of the boards you have listed has as many tutorials, as big community and as many documentation as the Raspberry Pi. The chance to get any help is the highest with a Raspberry Pi.

tutorials and community support on such a rare hobby as osdev? technical reference manuals are the best friends for this and Broadcom isn't even close to being "friendly" with this respect. for many peripheral controllers, their "documentation" sais straight: you'd better off to not program them directly via ARM. meaning you need to use VC <-> ARM interfacing. and that was exactly why I said, it's the worst choice for the ARM bare metal experience - you need to implement a lot of RPi only things, that will have no use for any other ARM board. summarizing, your ARM code becomes full of Broadcom specific hacks and cruft. even in places, where it's unnecessary! their SoCs are the "2 hostesses in the kitchen" case, and that doesn't make things more clear, easy, portable etc. opposite.

bzt wrote:

zaval wrote: you can bypass linux and directly handle the board. heck, you can even bypass uboot and take the whole control over everything

You can do all of these with an RPi as well.

yes, maybe, I wasn't objecting this, but still, for normal ARM boards, you would need to learn about only interface between ROM code, which is very minimalistic (basically, you would have to know, what a signature/structure to put and where for the ROM code to recognize you and load into SRAM) and then your code takes control over the whole platform - you can even implmemnt your Secure Monitor. with Broadcom, for such a level of "taking control", you would need to replace VC firmware. is it possible? And I heard, Broadcom SoC have screwed up EL3, meaning, it's made absolutely not in accordance with the ARM ARM (Architecure Reference Manual). this is, of course, in no way interferes with the OS development, it does so for FW development, but as I mentioned in the above paragraph, there are nasty things about RPi SoCs, that make OS development not as fun as well.

zaval wrote:tutorials and community support on such a rare hobby as osdev?

Yes. There are plenty of those for RPi:
https://wiki.osdev.org/ARM_RaspberryPi_Tutorial_C
https://www.cl.cam.ac.uk/projects/raspb ... orials/os/
https://github.com/LdB-ECM/Raspberry-Pi (including FreeRTOS port)
https://github.com/bztsrc/raspi3-tutorial
https://github.com/s-matyukevich/raspberry-pi-os
https://github.com/moizumi99/RPiHaribote
...just to name a few, but there's a lot more. And for the community support, take a look at RPi forums, C/C++ topic, Bare metal topic etc. plus you can also ask here.

zaval wrote:technical reference manuals are the best friends for this and Broadcom isn't even close to being "friendly" with this respect.

Documentation could be improved, agreed. However RPi is one of the best documented among all the boards. But there are lots of other resources than reference manuals, see Ultibo's wiki for example with a very detailed documentation on all of the RPi's peripherals. Our own wiki has a fair amount of info on Raspberry Pi (and has absolutely nothing on Pine nor on Orange for that matter).

zaval wrote:you need to implement a lot of RPi only things

I don't know about that. Actually the RPi specific part is pretty small, and with RPi4 you'll have exactly the same GIC as all the other ARM boards for example.

zaval wrote:for normal ARM boards, you would need to learn about only interface between ROM code, which is very minimalistic

Same with the RPi. The firmware is executed on the GPU, but you don't have to care or know about that. You just fill up some structures in memory to communicate with the firmware, that's all. It is called Mailboxes, suprisingly simple.

zaval wrote:with Broadcom, for such a level of "taking control", you would need to replace VC firmware. is it possible?

First, you don't need to replace firmware to get EL3, a single line in config.txt will do. Second, yes, replacing the VC firmware is as simple as replacing startX.elf on the SD card. Little Kernel is ported to the VC for example.

Cheers,
bzt

ggodw000 wrote:I am going to take a shot at ARM development. However looking at hundred of choices I am swamped.
I am looking to get compelte development kit that can power up and start programming in minutes, including desktop programming software etc, What is available?

I am literally looking to develop bootloader + some aspect of kernel development.
For that I am looking for faily powerful CPU 8-16 core, minimum 512M-1GB and around similar storage space at 512M.

I was going to write a funny reply, but you're not really asking about the target system, are you? It looks to me like you're asking about the development system. I assume you want to write code for the dev system too, otherwise its architecture wouldn't matter. It's still quite easy: there are lots of ARM server boards in ITX format - the same as PC motherboards. They run Linux and there will be desktop Linux distributions for them. (They do have some graphics output.)

I'm guessing they won't be more than 4 core, (I'm not up to date,) but honestly, to start programming in "minutes" is a very very far from a difficult requirement. I remember when some guy got Gentoo Linux booting to a shell in under 1.3 seconds! I am not exaggerating. You won't do that with modern hardware because initializing some peripheral hardware takes stupid amounts of time, but if you're prepared to cut down your desktop Linux a bit, under 1 minute is still reasonable. Drop all desktop environments and use the command line for file management; you'll be surprised. Vim needs nothing of the disgusting desktop megabloatware. Gcc and Clang certainly don't.

As for compilation speed, honestly, before everyone started using the disgusting megabloatware features of C++, I was very happy recompiling my entire Linux system on a Core 2 Quad with 4GB RAM. Back in 2008 I sometimes even compiled the system while playing Second Life which demanded 3 cores and had to download tons of textures into RAM. It was fine; I just had it run 2 parallel compiles instead of 6. It was a lightweight system: no desktop environment, just a shell and a traditional window manager, but it did have Firefox.

Finding the right distro can be a bit of a pain, to be honest. Last time I checked -- a few years ago now -- Arch was horribly flakey, requiring far more system administration work than Debian. Debian is a bit of a cesspit for bloatware. Alpine was good, but might have changed. To get GVim and a web browser which don't need megabloatware desktop libraries, you may have to compile them yourself, but you shouldn't be scared of that if you're in OS dev.

If you install FreeBSD instead, you'll get a fairly small core system which is easier to manage than Linux. All the "ports" (packages) are compiled by default. The ports files take options you can use to exclude some bloat. OpenBSD has the same advantages but slows down the boot process by relinking everything for security reasons. I don't know anything about other BSDs.

tl;dr: Get an ARM server board in ITX format. Put FreeBSD on it. Have fun!