More on Backups

So in my last post, I detailed some vague, abstract plans for performing a backup of the OEM state of my device. The last thing I want to do is brick a device whose manufacturer offers zero support in my country. (Trust me, I tried to call them, and the representative had no idea what I was talking about.) I was rambling something about partitions and boots and whatever. This time, I'd like to go from the abstract to the concrete: I'm going to be screwing around with my phone.

Android's partitions typically reside on non-volatile NAND memory on a special device or set of devices on the phone. I will be doing a raw byte-level copy of these partitions, which are exposed through special files in /dev/mtd. The mtd bit stands for "Memory Technology Device," and was added to the Linux kernel as an abstraction layer between the Linux block and character device layer and flash hardware.

Why an abstraction layer? Well, NAND flash memory has somewhat idiosyncratic characteristics on the low level, most notably that ones cannot be written to a block. Strange, I know, but bear with me. When you write data to a regular disk, such as a hard drive, you can tell a byte to become set to whatever value you like. Flash doesn't support writes so much as it supports "programming." When you erase a block of flash, its bits all get set to one, and when you write you just set some of those bits to zero. Setting the back to one, however, requires wiping an entire block.

Needless to say, this strange usage pattern requires some trickery to make it transparent to the boring old block and character device abstraction in the kernel. The MTD interface accomplishes exactly that. (The more adventurous of you might want to take a look at the kernel source for the msm_nand platform driver, which is what powers Android's flash support.) This interface is exposed through /dev/mtd, and can be accessed and read as though they were regular files.

So then, when I look at this directory, here is what I see:

# ls /dev/mtd
mtd4ro
mtd4
mtd3ro
mtd3
mtd2ro
mtd2
mtd1ro
mtd1
mtd0ro
mtd0


Not the most descriptive bit of output, but it's a start. A little more information can be gained by looking at  /proc/mtd :

# cat /proc/mtd
dev:    size   erasesize name
mtd0: 01fa0000 00020000 "boot"
mtd1: 01fa0000 00020000 "cache" 
mtd2: 01fa0000 00020000 "recovery" 
mtd3: 15600000 00020000 "system" 
mtd4: 21c20000 00020000 "userdata"


Much better. This tells me the exact layout and size of the partitions on this device. The sizes are 32MB for the boot, cache, and recovery partitions, a comfy 358MB for the system partition, plus 566MB for the user data. 


So then, how to back these up? That depends on how we want to restore things. Upon asking for help from the Cyanogenmod development IRC channel, I was advised to be careful to copy out-of-band data, or the flash metadata about each block, such as bad blocks and so on. I found that if I were to be using various low-level tools, this would have been a concern. However, I will be using fastboot to do the flashing, and so can rely on the bootloader to handle bad blocks and such.

So then, actually performing the backup is easy. The data will emerge from the NAND partition and be presented to by the kernel as a simple device:

cat /dev/mtd/mtd0 > /sdcard/boot-mtd0.img

Fun fact: my hunch from my last post about this phone not using a recovery partition was confirmed when I opened up my recovery backup to find nothing but ones. After copying each partition, and subsequently pulling the data off, I now have a working backup of every single relevant byte on my phone.

Or at least in theory I do. I still need to test this. There is only one way to test this, and that is the scary one: wipe a partition, see that something is wrong, and restore it. I chose boot as my victim:

fastboot erase boot
# Tense reboot. LG logo lights up. Fails to boot. 
fastboot flash boot boot-mtd0.img
# *Really* tense reboot. LG logo lights up...and...

To my glee, the thing comes back to life! I've tempted fate and walked away unscathed! As of right now, I have a complete backup of the working state of my phone, and should anything go awry, I can fall back on these images, flashing them onto the device via fastboot.

So then, where do we go from here? In the background, I've downloaded and compiled my starting point: Cyanogenmod 7.1 for the Nexus One. For my next episode, I will naively attempt to plop the Nexus One image onto the phone and see what happens.

Stay tuned!

The Golden Rule

The Golden Rule of computing is "data you don't have copies of is data you don't care about." This is a rule I live by in my digital life, faithfully making copies of all the things I know I'd have a hard time getting back in the event of a theft, fire or, far more likely, my own blundering.

One of these things is the current state of my phone. I am very tempted to rush headlong into Android development and nuke my current installation because hell, getting away from this setup is why I'm porting Cyanogenmod in the first place! However, my data-safety tendencies kick in before my curiosity gets the better of me.

So then, how do you back up a device like an Android phone? The naive answer is "just run adb pull /". For the purposes of grabbing the relevant files, this would work fine (nevermind the fact that adb won't accept that particular command). However, the Android root filesystem contains not just files, but also mount points of various important partitions. In order to jury rig a working backup ROM, I'd have to faithfully preserve this layout.

Consider first a Linux machine. A Linux machine's hard drive is laid out in partitions. The bulk of the spaces belongs to data partitions, but some partitions have special purposes, with which you might be familiar. The swap partition serves as a place to store swapped-out pages in the virtual file system. The boot partition stores the kernel and all files required to start the operating system at boot time. Extended partitions operate as a hack to allow for more than four partitions on master boot record-organized disks.

The Android disk is laid out in a similar fashion, with various partitions serving various purposes. An excellent and concise description of each partition can be found here, so I won't go into what each partition does. Rather, I'll give a description of the relevant concerns you must keep in mind when handling each partition during a port.

One of the crucial points with my situation is that this phone has very little support and community based around it. A successful port will involve combining the widely available portions of the android platform, i.e. Cyanogenmod, with LG's own modifications and closed source libraries, like the camera, radio, etc. For this reason, it's important that I handle each partition extremely carefully to avoid destroying data.

/boot

The boot partition contains the bare minimum required to bootstrap the system, namely the bootloader, the kernel, and the init binary. I will end up writing and compiling a custom kernel, probably backporting LG's changes, but I would be very very hesitant to do any harm to the bootloader. If I screw up the bootloader, the phone will be bricked, and I will have to throw it out and give up on my little project, because LG USA doesn't support this model.

As an aside, please, please, please correct me if I'm wrong. That last point is a source of some anxiety on my part.

There is an very nice wiki page describing the byte-level layout of the boot image, and I will most likely be using the tools described there to unpack and repack the boot image as necessary. This allows me to modify the kernel, but still keep LG's own init scripts in place until I'm ready to remove them.

/recovery

So what if you do screw up the boot partition? You can always fall back on the recovery partition. This partition is a free standing (not so sure about this point), bootable partition containing some code to fix your screwups. I believe this partition is typically used to perform carrier over-the-air updates, although I suspect LG doesn't use it. LG's update process (at least for this phone) is an oldschool "connect your phone to your windows box and run the update application" type deal, so this partition might not contain anything useful, if it exists at all. 

That being said, I would be very surprised if this partition wasn't there, so I'll assume I have some space to work with. Writing my own recovery image would be very useful, since it would grant the ability to screw around with the various partitions without worrying too much about how to bring the phone back to life. 

/system

This bit contains the proper Android stuff. For what it's worth, it's going to be completely blown away and replaced with Cyanogenmod. The only thing it's really good for is the closed source OEM libraries, including the graphics and camera libraries, so it will have to be preserved at least in filesystem form. 

There are other partitions, but I'm not really concerned with them at this stage. A backup would consist of grabbing each of these partitions, byte by byte, and a recovery would consist of booting into some minimally useable state, be it recovery or bootloader, and flashing each partition with those backed up images. 

So then, this has been a fine bit of theory, but my next post will describe the practical concerns with backing up these partitions, and subsequent posts will describe what goodies I find on them once I unpack them. 

Hello World!

So a little while ago, I came into possession of a rooted LG Optimus Z. The my LG-using readers might point out that they have never heard of such a phone, and I respond "for good reason." The Optimus Z, also known as the LG SU950/KU9500, is a high end Korea-only phone made by LG and sold under the CYON brand, on South Korea Telecom.

It came out came out in 2010, was trumpeted and heralded by western tech media, and, as is the fate even of American tech products, was promptly forgotten. It came out running Android 2.1 Eclair, and was promised an upgrade to 2.2 Froyo in the fall of the same year.

So great! I now own and use a Korea-only GSM phone! I've always wanted an Android phone to play with, and I now that I've got one promise I'll feed it and pet it and clean up after it and can we keep it, can we keep it please please? I started reading my email on it, I installed Google Voice to enjoy the benefits of free text messaging, didn't install Angry Birds, and was generally pleased.

Fast forward a few weeks. Once the giddiness that every geek knows all too well faded away, I had a few gripes. The translation wasn't perfect, which is surprising given that the thing had to be localized from English. The Google Voice app took a painfully long time to open. Three quarters of the applications are in Korean, and some of them work with services that don't even exist in the States.

Beyond that, I had some real serious problems. The thing has the annoying tendency to crash at the most inconvenient time. There is nothing worse than having an alarm clock that goes off once, crashes while you snooze, and lets you sleep an hour and half longer than you meant to. What's more, the text messaging application limits the length of my messages to 80 characters. I'll leave the exact reason for this as a puzzle for my more programming-savvy readers.

So what choice do I have? You might think the promise of the firmware update was foreshadowing for some mean trick on LG's part, but it's not. The update to 2.2 came out last year. So why not apply it and hopefully get these bugs fixed?

Because this thing is rooted. I have no backup of this rooted image, I have no idea of how it was rooted, and if I were to install LG's Android 2.2, I'd be stuck with just a boring old phone. I got it rooted, and I don't think I could possibly go about rooting it myself. If there's already a hack out there, I can't speak Korean, so any and all documentation would be useless. So I had to fall back on plan B: Port Cyanogenmod.

Over the next few months, assuming my classes allow me the time, I will be working on porting Cyanogenmod to this phone. If you've ever been curious about what goes into creating a custom ROM or Android port, then come along and find out with me. I will be covering things that I know and am familiar with, as well as reporting wonderful new things I discover.

This is going to be fun.