Low-level programming on a smartphone

[Bherzet]

Hello!

I have an old smartphone lying around (Sony Xperia E2303, a.k.a. M4 Aqua) and I'd love to try and create some sort of a bootdemo or perhaps even a really basic operating system. Unfortunately, I hadn't been able to find any resources regarding low-level programming for modern smartphones. I found only a handful of relevant topics on this board [1, 2], but none of them were particularly helpful.

On IBM PC-compatible, things are simple. When the machine is started, on-board firmware gets loaded into memory and executed. It performs some initialization and once everything's ready, it starts looking for a bootable medium. It either looks for 0x55aa boot signature at the end of the first sector or directly loads a certain file from the filesystem if it's UEFI. Than it's a matter of a single JMP and you're free to do whatever you want. BIOS is our best friend and writing a simple demo is actually a no-brainer because of that.

I have no idea how things work on a modern smartphone. I've found some explanations of the boot process [3, 4], but it's difficult to make sense of it. Let me explain what I mean.

This is Pavel Dubrova's effort to bring Android 11 with a mainline Linux kernel [5] to Xperia M4 Aqua. In order for me to flash it to my device, I have to:

1. Power off the device, press Volume Up and while holding it, plugin the USB cable to the computer. LED now turns blue and device is in a so-called fastboot mode. I would assume anything related to fastboot mode is stored in a different part of the internal flash memory, so that hard-bricking the device is less of a possibility, but that's actually in contrary to [3] (see the comments).

2. Once the fastboot is on, I can now flash contents to certain partitions. Here, Pavel uses just boot, system, cache and userdata. This is interesting, because the boot.img file, which Pavel provided and which goes into the boot partition, actually doesn't seem to contain low-level code (directly "chewable" by the CPU). I tried examining it and it showed this:

Code: Select all

$ file boot.img 
boot.img: Android bootimg, kernel, ramdisk, page size: 2048, cmdline (androidboot.hardware=tulip androidboot.boot_devices=soc/7824900.sdhci androidboot.memcg=1 cgrou)

$ hexdump -C -n 100 boot.img 
00000000  41 4e 44 52 4f 49 44 21  f7 01 90 00 00 80 00 80  |ANDROID!........|
00000010  f0 c8 0c 00 00 00 00 81  00 00 00 00 00 00 00 00  |................|
00000020  00 01 00 80 00 08 00 00  00 00 00 00 49 01 00 16  |............I...|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  61 6e 64 72 6f 69 64 62  6f 6f 74 2e 68 61 72 64  |androidboot.hard|
00000050  77 61 72 65 3d 74 75 6c  69 70 20 61 6e 64 72 6f  |ware=tulip andro|
00000060  69 64 62 6f                                       |idbo|
00000064

So this is basically something like a /boot partition of a standard Linux system. There has to be an actual bootloader somewhere, and indeed there is. When I write "fastboot flash" in the shell and press TAB twice (I don't know if there's a better way to list all available partitions), it shows a couple more partitions, namely: bootloader, radio and recovery.

So, the bootloader partition it is (hopefully, probably). Unfortunately, I couldn't find a way to read the partition via fastboot, so I can't really be sure. What I'm really concerned about is that:

1. I would very much want to avoid hard-bricking the device. Although out-dated, it's still a very nice piece of hardware and I want to eventually reflash it with a stock ROM and use it as a secondary smartphone. Since I'm not sure where the fastboot mode actually comes from and what would happen if I just flashed a random garbage to the bootloader partition, I have to stay void of such experiments for now.

2. Even if I didn't risk hard-bricking the device, I still eventually want to recover (preferably) the stock bootloader – or whatever I have in there currently, because I vaguely remember unlocking the bootloader and rooting the device several years ago. This wouldn't be a problem if I could simply perform a backup first.

For a moment, I thought I could perhaps avoid touching the bootloader whatsoever and just play inside the recovery partition. Perhaps I could take a look at some project like TWRP for inspiration, right? Well, no. Again, it seems to be just a kernel that gets loaded using the bootloader…

Any advices on how to flash my own assembly code into the bootloader partition without risking hard-bricking the device would be greatly appreciated. And sorry for the lengthy post, but I wanted to provide enough context.

[austanss]

If you backup the stock bootloader, you can easily reflash it later. As for low-level programming though, you'll have a hard time finding resources on programming for Qualcomm (presumably) chips.

[eekee]

Have the PostMarketOS and LegacyOS communities been any help? They're both concerned with porting newer Linux kernels to older smartphones if I understand right. I thought the PostMarketOS wiki pretty good but I wasn't looking for actual developer details at the time.

[zaval]

with ARM devices, the start sequence is basically the same as with x86, it's just not documented. it's a chain of "loaders", started with the ROM code. if you had documentation, you could back up what is there and then wipe out it and put your substitutions. the earlier stage it would be, the more information you would need. and it's a Qualcomm chip. forget it.
you can pretend you are android/linux, and see what happens. can you find the kernel there? substitute it with your own. even if you figure out the format expected and all what's needed to be recognized and run, what your code would do? do you have UART? no. are you able to draw on the panel? no. you have zero information on the hardware, you are presumably running on. it's a hopeless effort.

Don't you want to try something very similar, but way more friendly, than this? if you do, then try Single Board Computers. any of them would be much more feasible and enjoyable to work with - you would have UART, and thus possibility to break into its "FW" and ask kindly to load you thing. attaching a monitor is possible as well. finally, the lately versions of the most commonly used "FW" there - uboot, do support UEFI, so you could run some code, line by line the same for x86 and ARM. I did so, until it goes to the OS Loader backend, where architecture specific things kick in, the code is the same.

Or, take a look at developer friendly smartphones, like Pinephone.

[austanss]

with ARM devices, the start sequence is basically the same as with x86, it's just not documented. it's a chain of "loaders", started with the ROM code. if you had documentation, you could back up what is there and then wipe out it and put your substitutions. the earlier stage it would be, the more information you would need. and it's a Qualcomm chip. forget it.
you can pretend you are android/linux, and see what happens. can you find the kernel there? substitute it with your own. even if you figure out the format expected and all what's needed to be recognized and run, what your code would do? do you have UART? no. are you able to draw on the panel? no. you have zero information on the hardware, you are presumably running on. it's a hopeless effort.

Don't you want to try something very similar, but way more friendly, than this? if you do, then try Single Board Computers. any of them would be much more feasible and enjoyable to work with - you would have UART, and thus possibility to break into its "FW" and ask kindly to load you thing. attaching a monitor is possible as well. finally, the lately versions of the most commonly used "FW" there - uboot, do support UEFI, so you could run some code, line by line the same for x86 and ARM. I did so, until it goes to the OS Loader backend, where architecture specific things kick in, the code is the same.

Or, take a look at developer friendly smartphones, like Pinephone.

I find it extremely toxic to come onto this forum and post on threads you have insufficient knowledge on, and tell them it's too difficult. OS development isn't supposed to be easy. I don't have much knowledge in the way of ARM development/Qualcomm-specifics. But I'm not going to destroy their hopes and discourage them from what they are trying to do. Exploring frontiers is how we make progress. This person is willing to take on this challenge. The benefits can apply to all of us. The information they gain can be redistributed among the other OS developers on this forum/wiki. And yet, you want to discourage them.

[iansjack]

I find it extremely toxic to come onto this forum and post on threads you have insufficient knowledge on

I agree.

I don't have much knowledge in the way of ARM development/Qualcomm-specifics.

I agree.

[eekee]

On topic, as far as I know there is always a serial port in some form or other. JTAG is one likely possibility. Sometimes you have to solder 3 wires or so, but the electronics will be there because the hardware had to be debugged somehow. I think JTAG is sometimes integreated into a USB port (which saves on soldering), but I don't know any details.

On argument ( ):

I find it extremely toxic to come onto this forum and post on threads you have insufficient knowledge on, and tell them it's too difficult.

I have to agree. This wasn't the worst case I've seen; there was this one time when a young lad was pushed into depression (hopefully temporary!) and deleted his cross-compiler due to people telling him it was too difficult. Why did they tell him that? Due to a misunderstanding of terminology only!

OS development isn't supposed to be easy. I don't have much knowledge in the way of ARM development/Qualcomm-specifics. But I'm not going to destroy their hopes and discourage them from what they are trying to do. Exploring frontiers is how we make progress. This person is willing to take on this challenge. The benefits can apply to all of us. The information they gain can be redistributed among the other OS developers on this forum/wiki.

Sound reasoning as far as I can see. I don't know if I'll ever stop offering alternatives as I did in the thread where someone wants to use USB ACM as an alternative to RS-232, but I'll try to be more careful about it.

[xeyes]

I find it extremely toxic to come onto this forum and post on threads you have insufficient knowledge on

I agree.

I don't have much knowledge in the way of ARM development/Qualcomm-specifics.

I agree.

Well said.

To share something with @Bherzet or anyone else who might be interested in this

I've worked (as in, part of job) on similar projects. There were certainly no shortage of challenges. Despite all the 'internal' documents and knowledge; 'proprietary' hardware and software tools; official support from SOC and other hardware vendors; and of course we don't locked down the early devices to hinder ourselves.

My 2 cents: Nothing is impossible. I have a lot of admiration for people who can pull this off without leaked/pirated info and tools. Not only in their technical competency but more in their years or decades of life that they can afford to dedicate to fruitless trial and errors.

For us normal people, if reading the freely available x86 manuals and developing for a still very open platform like the PC is too easy or uninteresting to you, perhaps consider something like this https://shop.intrinsyc.com/products/snapdragon-855-hdk

[austanss]

I find it extremely toxic to come onto this forum and post on threads you have insufficient knowledge on

I agree.

I don't have much knowledge in the way of ARM development/Qualcomm-specifics.

I agree.

I originally thought I agreed with you, until I read it again and noticed you severely took my post out of context.