These are just some notes I made when creating my own mini-distro after wanting something more custom than just using buildroot or making the official firmware more slim. For people other than me, I suggest looking through (C)LFS or running postmarketOS instead once this reader’s pull request gets integrated into upstream.
Two things that’ll greatly help with this is having serial terminal access with the four uart pins near the top right in the back of the reader, near the uSD card slot (I don’t connect the 5V pin as my reader doesn’t really turn on anything other than the power LED). I suggest maybe soldering female pin headers to there to make your life easier (you can later cut out a hole in the back cover or desolder the headers once you’re done). Other than that, I suggest installing QEMU with ARM userspace to test programs that you have built or running them on a separate ARM device like a Raspberry Pi.
Prelude
Ever since I learnt that the official firmware for the Clara was just using a modified Linux kernel with busybox as coreutils and many other libraries, I just knew that I had to minimize it. I also saw that it was using glibc for it’s libc, which I really dislike as statically linking C programs against it was a pain in my experience, compared to something like musl and uclibc. It’s also much larger than them and I don’t use any of glibc extensions so it seemed like a waste of space to me.
Initially when I replaced Nickel with Plato, I was able to shave about
100 MiB after I removed /usr/local (which contains Nickel, Qt and a few
other things), from 189 MiB to 74 MiB, but I still wanted to make it
smaller.
Using buildroot, I was able to get it under 2 MiB (!!) which was a
little less than half the size of an uncompressed armhf Alpine Linux
minirootfs (4.9M for 3.14). With Busybox, it was pretty much working
out of the box, with serial terminal access! But waiting around 15
minutes for the toolchain to build each time I wanted to change
something in the rootfs took way too long, although it could’ve been
minimized if I used ccache with a fairly large cache size. I still
found that it compiled and installed a lot of things I wouldn’t be
using (particularly in /usr) even after disabling almost all of the
third-party packages.
I’ve uploaded the config file and the resulting rootfs for buildroot 2021.05. The root password by default is changeme. EDIT 2022-10-21: gone, build it yourself
Of course the rootfs I got from buildroot nor me making the official firmware smaller is the point of this article, and the actual point is making one yourself! (or rather what I did to make my own)
Cross-toolchain
For now as of July 22, 2021, I’m using my distro (Void Linux)’s packaged cross toolchain for armhf musl, but eventually I would be using my own.
I’m not compiling off of the device itself as it would be somewhat slow for bigger programs, which is currently primarily the Linux kernel, U-Boot, and the toolchain itself, considering that the ereader’s CPU (Freescale i.MX 6SLL) is a single core running up to 1 GHz. Including the development tools and headers would also take up more space on the device itself, and since the terminal can currently only be accessed through it’s serial/uart pins, I don’t think it’s ideal.
TODO: include steps to create own toolchain (probably based off of gcc
4.7.3 as that doesn’t require c++)
Building the rootfs
Assuming you made a new filesystem on your rootfs’s partition, it’ll likely be empty with no directories you’d expect to find on a regular distro. So you’ll just have to make them.
cd /path/to/rootfs
mkdir bin dev etc proc sbin
Your binaries would usually go in /bin, the uSD card, ttymxc0, and
other devices would go in /dev, felker init’s default program/script to
execute is usually in /etc/rc, /proc is optional but I have it mounted
to see what is currently mounted through /proc/mounts (or mount(1)
without any arguments) as well as to see my disk usage through df(1).
/sbin is there to place the init in as /sbin/init is the default init
path the kernel looks at.
toybox
Now on to the main part of the distro, the userspace. I intend to keep
it fairly minimal so I’ve chosen to use toybox along with a slightly
modified version of felker (musl dev)’s init, as well as dash as
the main shell since toybox doesn’t include one as of 0.8.5 (though
it’ll probably be there by 1.0). I’ll also be statically linking all
the programs that’ll be used so I wouldn’t have to worry about shared
libraries not being included/copied over, and also including LTO for
slightly faster binaries. Originally, I tried going with sinit, sbase,
and ubase but I was having trouble getting serial terminal access with
getty to /dev/ttymxc0 (the default serial tty, at least with the
vendor kernel). I didn’t have this problem with busybox’s and toybox’s
getty however. My config for toybox was also about 81K smaller than my
trimmed sbase-box and ubase-box (352K compared to 267K+166K) where I
removed programs that I won’t use from ${BIN} in their respective
Makefiles.
EDIT 2022-10-21: also gone
First I suggest exporting some environment variables to set the toolchain used as well as enabling static linking and LTO.
export CROSS_COMPILE="arm-linux-musleabihf-" # change to your cross-tc
export CC="${CROSS_COMPILE}gcc"
export LDFLAGS="--static"
export CFLAGS="-flto -static"
export ARCH=arm # for compiling the linux kernel
To compile toybox, get the source from
https://landley.net/toybox/downloads/ (or clone the upstream repo).
Then run make menuconfig (optionally with make defconfig before it) and
change it as you see fit. Personally, I disabled most of the programs I
wouldn’t use and kept only the ones that’ll help with fixing a problem.
Finally, make sure to run make.
make defconfig
make menuconfig
make
To move it to your rootfs and set it’s symlinks, you could probably run
make install after setting PREFIX to your rootfs’s /bin directory, but
I did it manually.
# automatic (didn't test, check README)
make PREFIX=/path/to/rootfs/bin/ install
# (semi?) manual
cp toybox /path/to/rootfs/bin
# add symlinks if doing manual and you want them
cd /path/to/rootfs/bin
for prog in $(qemu-arm ./toybox); do ln -s toybox "$prog"; done
dash
Also as of toybox 0.8.5, a shell still isn’t included (probably would
be included by 1.0 according to scripts/install.sh as well as a few
other programs like gzip), so a separate shell would need to be built.
Any can be used but dash would be shown as an example as I was able to
get a static binary without too much trouble.
First obtain the source and cd into its
untarred directory. Assuming your CC and CFLAGS are set, you can run
these steps:
autoreconf -fiv
./configure --host=$CROSS_COMPILE --with-libedit
make
${CROSS_COMPILE}strip src/dash
As this is going to be used as the main shell, I’ve decided to just
copy it to /bin/sh in the rootfs directory, though copying it there but
as /bin/dash and /bin/sh being symlinked to dash is also an option.
cp src/dash /path/to/rootfs/bin/sh
# or
cp src/dash /path/to/rootfs/bin
cd /path/to/rootfs/bin
ln -s dash sh
felker’s init
The init is just a single file that you can get
from felker’s site or the github
gist.
I haven’t had a good experience with the default startup program
(/etc/rc) as a shell script with execve() run on it so I’d change it
to execvp() and remove the third argument (specifies environment). To
compile and install the init, all you need to do is run:
$CC $CFLAGS -o init init.c
cp init /path/to/rootfs/sbin
Instead of /etc/rc being a shell script, you can also make a C program
that does whatever you think is needed for a proper startup. I’ll still
use a shell script though which is linked here.
EDIT 2022-10-21: you get the idea, it’s gone.
/etc/passwd
Copying the rootfs’s contents to your device’s/uSD card’s root
partition and then turning the device on should now work with a login
prompt shown in the serial terminal. However, you probably wouldn’t be
able to login to any user. So you’ll have to create a file at
/path/to/rootfs/etc/passwd. For an empty password to root, you can use
this, though I suggest setting a password as soon as you login:
# in rootfs's /etc/passwd
root::0:0:root:/root:/bin/sh
With the passwd file created/updated, you should now be able to login
to root after the rootfs is copied to your uSD card. Your rootfs so far
should now be around 550-560K, which is much much smaller than the
original firmware’s, though it’ll likely be much larger to maybe a few
megabytes once a proper reader software is added.
Custom Linux Kernel
WARNING: I haven’t actually gotten the kernel to load in u-boot yet. It
just hangs in the “Starting kernel ...” step and the init doesn’t get
loaded, so I’m assuming the kernel itself isn’t either. If anyone out
there has gotten a custom kernel working in the Kobo Clara HD, please
send me an email or message on xmpp.
UPDATE Jul 28, 2021: Gave up on it as I just couldn’t get any kernels I
built (both vendor and akemnade’s mainline) to boot. But neither did
postmarketOS boot beyond the initial initramfs messages without the log
file being created. So I’ll revisit this for later.
EDIT 2022-10-21: I have gotten this working, but have been unable to
get Plato build for musl, so I will have to either continue fighting
with the crab or create my own with fbink, as that still works.
Separate article on this later.
My next big step is compiling my own kernel for the Clara HD. With the
default configuration built for the vendor kernel, it appears to be
about 3M, so my goal is to build a kernel that is smaller than that
while retaining only the functionality that I need. I’m also not going
to include networking support as that is unneeded for my purposes, but
I suggest just keeping it if you’re unsure. The wifi driver for the
Kobo Clara HD is available as an out-of-tree driver.
You should first obtain the kernel source, with two main options, the vendor kernel and the mainline kernel (with akemnade’s patches). For the latter, you need to clone the repo and switch to the latest kobo/drm-merged branch (kobo/merged-5.13 as of July 25, 2021).
After you’ve got them and assuming the CROSS_COMPILE and ARCH
environment variables are set, you’d want to configure the kernel.
I had a hard time compiling the vendor kernel with many things disabled, so I’ve kept my config somewhat similar to the default config. The config I used is available here (EDIT: dead).
make menuconfig
make zImage
Assuming it compiles properly and arch/arm/boot/zImage exists, all
that’s needed to is to write it to your uSD card at the 1M offset.
dd if=/path/to/kernel/zImage of=/path/to/uSDdev bs=512 seek=2048
Custom U-Boot
I have not done this yet, nor really plan to, but if you do manage to compile the Kobo’s vendored u-boot source, then all you’d have to do to install it is:
dd if=u-boot-file of=/dev/mmcblk0 bs=128k count=1 seek=6
If I remember correctly, this command was included in an older
firmware’s startup script/rcS for updating udev, and it should still
work.