Vineet's site

*nix workflow after nearly a decade (presentation)

2026-04-13T17:30:00Z

note: there's some artifacts like not knowing how to do org-mode's en-dashes in markdown when I converted this for my site. this was originally shown in org-mode via org-present.el.

who am I and why am I here

former e-board officer, been using *nix as my daily driver for nearly a decade
started at 12 as a 4chan /g/ kid -- any cringe can be blamed on that
- I haven't gone on 4chan since like 2020, I have no idea what kind of cesspool even /g/ has become (back in the day, there were nice threads on there everywhere except /pol/ based on how the poltards were described)
guiding principles throughout: unix purist, minimalist, anti-bloat
this is roughly how that went

how it started: luke smith

his early video on why should one use terminal programs was my gateway drug into everything
- since I used youtube in like 2011, this was the sole good thing that ever came out of the algorithm thing that people talk about for me specifically
from there: different distros (he started with parabola on his librebooted x200), tiling WMs (i3, dwm), suckless and minimalism, LaTeX
parabola's wiki in 2017 still had the beginner's guide arch had removed -- effectively the same distro anyway
- parabola is just a more libre and gnu version of arch
this was 2017-2018, before he quit being a linguistics professor in georgia, moved to a cabin in florida, and started complaining about les youths
didn't follow other linux channels until much later -- pretty much just david wilson's system crafters for emacs and guix (more on this at the end)

distro hopping

started on parabola -> arch shortly after for wifi drivers (was interesting finding out about non-free firmware as soon I start using linux)
set up gentoo through the handbook to actually learn how linux works underneath
did a full LFS+BLFS. twice.
- if I didn't get into Guix, I would've done it again after getting frustrated with regular distros for the nth time.
- side-tangent: I already disliked that it made it seem like creating a distro was more complicated than it really can be because of how many dependencies stuff like gcc and glibc need, but now it's even adding systemd on top of that? I don't recommend anymore. too many points of failure where it can go wrong.

distro hopping 2

then: CRUX, Sabotage, KISS, Alpine -- hopping between them over whatever small thing annoyed me at the time
found CRUX through z3bra on nixers.net discussing differences between it and gentoo
- felt like home: just enough packages for a minimal base, liked being a package maintainer for a while
- power user burnout got to me every single time after a few months of usage though where I then quickly reinstall arch
sabotage was unique and interesting but only lasted one hop
- it had a similar idea to nixos and guix where there's a singular store directory but

distro hopping 3

KISS was what I wished CRUX was, but it didn't have the drunk tux mascot so it only got a couple hops
sabotage and KISS found through people on IRC and XMPP
learned a lot about sysadmin and writing packages through all of this
choices eventually converged to: arch (easy), CRUX (fuzzy inside), gentoo (USE flags deepened the purism path, less package maintenance burden than CRUX)

os hopping -- the BSDs

found them in the continued pursuit of minimalism and purism
direct descendants of bell labs research unix
freebsd: what I wished linux was -- better documentation, first-class integrated ZFS
openbsd: high quality documentation in form of man pages (rather than freebsd's handbook in addition to man pages), security-above-all-else focus (though didn't impact game emulation much), tight-knit mature community (unlike most of linux), crux's packaging was heavily inspired by openbsd's, the mascot, and it's canadian

os hopping -- tangent with plan9 / 9front

grade 12 high school phase
the creators of UNIX at the end of Research UNIX v7 and v8 pretty much thought "how can we do this better?" and made a separate OS that is like a reimagining of UNIX
- "everything is a file" taken further into even the networking
- e.g. in BSD and its derivatives for networking via sockets are all based around the C sockets API while plan9 put even that into abstracted files and made 9p (plan9 protocol; even windows used it in wsl1 before they switched to virtualization)
loved the simplicity and elegance of how its programmed
- it uses a different flavour of C
rio/8.5 and acme showed me that mice aren't inherently bad for computer use when designed /properly/
- i.e. not context menu galore
couldn't daily it: i219-v and r8168 ethernet drivers weren't working correctly even after trying to patch them with my then-meager C skills like changing IRQ interrupt numbers and stuff

distro hopping fin

OSes I used the longest without hopping: crux, gentoo, and openbsd
left crux and gentoo mainly due to power user burnout from package maintenance
initially left openbsd for windows virtualization and gaming (though I ended up doing more FOSS engines and retro emulation under it than expected)
later left openbsd again and freebsd as well because of the amdkfd driver being patched out, meaning I can't use ROCm on my GPUs for GPGPU compute
- it only leaves ggml's vulkan kernels which is quite different from doing CUDA/HIP

window managers

started with i3 -- keyboard-only workflow, most popular on r/unixporn at the time
- back when every post was either i3-gaps or bspwm, not sway and hyprland like today
switched to bspwm: more minimal, more unix philosophy -- keybind handling split into a separate program (sxhkd)
used herbstluftwm for a bit -- liked the manual tiling
dwm became my main WM for a long time: minimalism + functionality, suckless, elegant
brief switch to spectrwm -- visually similar to dwm but easier to configure
- based on my screenshots that lasted only a few months before switching back to dwm
briefly tried sway to see if wayland was worth it -- switched back to dwm
- only really used sway when too lazy to set up xorg on a new system

some screenshots from other people

reflects what I see as a *nix desktop is

window managers -- i3 (from xaetlas on r/unixporn in 2021)

I couldn't find the picture I saw from luke smith's website, maybe mandela effect

window managers -- bspwm (from gamehelp16 on r/unixporn in 2017)

window managers -- herbsluftwm (from nerdypepper/abhinav on r/unixporn in 2017)

window managers -- cwm (from z3bra in 2018)

and now my rices

I didn't keep screenshots archived from before 2020. some good ones were lost. :(

2020 -- spectrwm

2023 -- cwm

2024 -- dwm

2025 -- sway

2025 -- dwm

2026 -- (not mine, joey's) hyprland since it looks cool

editors: before emacs

before linux: eclipse for java, notepad++ for everything else, brief atom stint
switched to vim when I started linux
- terminal programs like it were part of what got me to switch in the first place
- learning your tools actually matters -- this comes up a lot
- avoided pre-made configs: they were bloated and changed defaults in ways that were hard to reason about as a new user
brief emacs attempt, but not knowing lisp made it impossible to properly configure
- also noticeably sluggish on windows on the same hardware that ran arch and gentoo fine
after learning vim enough (motions, basic ex commands) -- wanted more minimalism
- used vis and nvi for a while
plan9 phase: used acme briefly, learned sam through its manual
- sam taught me ed, ed taught me basic regex
- regex + vi motions: cannot overstate how much this improved editing

nixos

peer pressured into it by functional programming friends in junior year of college
main selling point: entire system configured through a single file, dotfiles managed in one language
- if I hadn't used nixos, I likely would not have gotten as deep into guixsd later
- guix itself was originally based on nix
frustrating at first because I didn't know nixlang
- same situation as when I first tried emacs in 2020
- once I learned more nixlang, nixos felt much nicer to use
but: still had to deal with systemd, and its administration felt very bureaucratic and corporate

nixos

the broader nixpkgs community was being overrun by new users blindly copy-pasting configs without understanding them -- everyone ended up with the same setup
- felt like it was becoming less for power users
the DSL is a special snowflake -- they should have used something else as a base like haskell or anything they actually took inspiration from
ROCm support issue had a light-hearted tone that implied it wouldn't be seriously maintained by the team
- that thread was marked stale three months ago and has had no activity for the past eight months -- right after I stopped using nixos

first attempt at guix

briefly tried guix after nixos
not knowing guile scheme made it hard to use -- same pattern as nixos and emacs before it
also trying to fold my existing suckless + vi workflow into it was difficult at the time
switched back to rotating between arch, CRUX, gentoo, and nixos despite their respective shortcomings

editors: almost emacs

the completion package situation had changed significantly since I last looked at emacs
it previously felt like you /needed/ something like ivy or helm -- packages that fundamentally changed how the editor felt, not improved it
what changed my mind:
- orderless -- fuzzy searching without the bloat
- vertico -- the vertical completion menu ivy used to give
- consult -- better autocomplete layered over existing emacs functions
these packages improve what's already there rather than overriding it
finding them was part of my gateway drug into giving emacs a proper second chance

the lispy gopher show

heard about it while briefly on the fediverse through prahou (author/artist of unix_surrealism from analog_nowhere)
it and system crafters were what actually made me learn lisp and scheme properly rather than just look things up
had learned some functional programming through OCaml and nix the prior couple summers -- learning lisp and scheme was much easier with that background
with vi-like editors I never needed to write scripts in them or touch their config much
- regex + vi motions + a few bridge packages covered most gaps
- with emacs, not knowing lisp meant always looking things up instead of just writing what I needed
the parentheses were daunting before. now they make me feel fuzzy inside, the same way learning linux did in middle school

why emacs now and not before

the main reason I resisted emacs was it seeming like a monolithic kitchen sink
- "an operating system that lacks a good editor" is not an entirely inaccurate description
but the way I was applying the unix philosophy had a cohesion problem
- mutt looks different from newsboat looks different from lynx -- all configured in different syntaxes
- vi keybinds and ncurses in common is about as far as the cohesion went
with emacs: yes it's one big program, yes I'm not using POSIX shell to glue things together
- lisp is a more powerful language than shell, which depends on other programs written in other languages to do even simple things
- everything goes through text-based buffers -- integration is genuinely easier
pre-made configs like doom emacs or spacemacs were also a turnoff
- same problem as pre-made bloated vim configs: you end up learning the pre-made config, not the editor
- neovim has a similar situation now

guix + emacs

guix got me to learn guile scheme properly
now write most of my programs in guile scheme instead of defaulting to C99 (suckless style) or python
guix also finally gave me what nixos pointed toward: system + home config + dotfiles all managed in one language, without systemd, without the corporate feel, without the special snowflake DSL problem

current stack

editor, scheme via geiser, elisp via eval-* functions: emacs, obviously
org-mode: document editing, task planning, habit tracking
- worth calling out separately -- it's much more than a markup language
- some people use emacs entirely for org-mode
email: notmuch + mbsync + msmtp via notmuch.el directly (same setup as before, just dropped mutt/aerc)
RSS: elfeed
browser: eww primarily, qutebrowser for js-heavy sites (canvas, etc.)
document viewing: pdftools + docview instead of mupdf
- mupdf still for large PDFs like textbooks where it's just faster
background programs: dtach + its emacs management package instead of tmux
local LLMs: gptel talking to vllm and llama.cpp running on my desktop
window manager: EXWM instead of dwm or anything else
- worked fine for games -- elite dangerous and others ran through it without emacs getting in the way

*nix workflow after nearly a decade (raw braindump)

2026-04-13T12:00:00Z

I was asked a few times from members of my university's cybersecurity club (I was a former e-board member and it's my main club) over the past months of how I use a computer since apparently how I use it is different from how others do it. They also found it fascinating that I used Linux/*BSD for as long as I did, and I'm probably among very few others if any at my university that used it for like 9.5 years. It's organized in what got me into this in the first place, my OS-hopping, my editor-hopping, what my current workflow is, and my (maybe lack of meaningful) thought process behind each switch. The TL;DR of that is I'm like a purist (primarily with the UNIX philosophy) and minimalist (reducing "bloat" as much as possible) and that guided a lot of my decisions up till now at the end of college. Any mistakes I made or any "cringe" I did like being a 4chan /g/ kid can be blamed on me being 12/13 at the time. The following is the raw braindump before I condensed it into point form.

I was introduced to Linux via Luke Smith's early video on why use terminal programs. Then by watching his other videos, I got introduced to stuff like different kinds of distros (he used Parabola at the time and Parabola's wiki in 2017 hadn't removed the beginners guide page unlike Arch wiki, though in hindsight they're basically the same), tiling window managers (i3, dwm), suckless movement and minimalism (was the start of me obsessing over purism even to my detriment sometimes as in spending too much time that I miss deadlines), using LaTeX for documents and presentations, etc. This was during his early days back in 2017 and 2018, before he quit being a linguist professor in Georgia and moved to a cabin in Florida and now complains about les youths. I didn't watch other Linux-related channels until more recently with David Wilson's System Crafters for Emacs and Guix (more on this at the very end, it's a recent change that fundamentally changed how I do stuff, this will come up a lot).

This put me on the path of both distro-hopping and WM-hopping. Reminder that I started this journey when I was 12 (now 21) so I had lots of time on my hands. I started with Parabola and then switched to Arch shortly after for wifi drivers, then setup Gentoo through reading its handbook to learn about Linux more, and then later did a full LFS+BLFS twice, then CRUX, Sabotage, KISS, Alpine, and then switched between them depending on what little thing annoyed me at the time. Of these, CRUX really made me feel at "home" with giving me just enough packages for a minimal base and I liked being a package maintainer for a short while. I found CRUX through z3bra on the nixers.net forum talking about the differences between it and Gentoo. Sabotage (the way it did stuff was unique and interesting to me at the time but only used for one hop) and KISS (what I wished CRUX was but because it didn't have the drunk tux mascot I didn't use it much beyond a couple hops) from people on IRC and XMPP. I learned a lot about system administration and writing my own packages through distro-hopping. Eventually, the choices converged to Arch (if I wanted something easy), CRUX (because it made me feel fuzzy inside), Gentoo (USE flags put me further on the path of purism, not as much need to maintain so many packages in my overlay unlike CRUX). I later also found out about the BSDs in my goal of more minimalism and purism. It being direct descendants of the venerable Bell Labs's Research UNIX. FreeBSD was what I wished Linux was and I liked its better documentation and integrated first-class ZFS support. OpenBSD I loved for its documentation, focus on security above all else, tight-knit and knowledgeable mature community (unlike with most of Linux), package management feeling more similar to CRUX, mascot, and being Canadian (patriotism I guess). Among the OSes I used the longest without hopping, CRUX, Gentoo, and OpenBSD are the ones I really used. I usually switched away from CRUX due to power user burnout from maintaining packages for adding and making them more minimal, same from Gentoo but to lesser extent due to many other overlays, and from OpenBSD for Windows virtualization at the time and gaming (though I did find out more about FOSS engines and did more retro emulation under OpenBSD). I also had a phase in high school grade 12 with plan9/9front. Really loved the simplicity and elegance of it. Its windowing manager rio/8.5 and acme editor also showed me that mice aren't an inherently bad thing for computer use when designed properly. I couldn't go further with dailying it because my i219-v and r8168 driver wasn't working properly even after I tried patching it with my then meager C skills. Later in college around junior year, I was peer pressured into NixOS from my functional programming friends. I liked the idea behind it and it made systemd somewhat usable, but I disliked the special snowflake DSL (they should have used something else as a base like Haskell or anything else they took inspiration from) and it liked pulling in all sorts of transient dependencies, the exact opposite of what I wanted in Linux since I started using it years ago. What really put me off more than transient deps was poor and inconsistent documentation (like pretty much nothing about flakes: there was a disconnect in documentation of what the broader community used and what upstream deemed stable). But, it had advantages like helping organize my system+home configs and dotfiles all in one place which was very nice. So I briefly tried Guix, but once again without knowing its language was hard to use and also trying to mould my existing suckless+vi non-emacs worklow into it was hard at the time. So I once again switched back and forth between Arch, CRUX, Gentoo, and NixOS despite their shortcomings I detailed earlier: my power user burnout or getting bored. Continuing later for recent switch to Guix+Emacs.

So now about window manager choice. I started with i3 since that's what Luke Smith used and I was curious about how a keyboard-only workflow would look like. It's also what the most popular WM on r/unixporn was. Back in the day when pretty much every post there was either a close-to-default i3-gaps setup or bspwm instead of sway and hyprland now. I then switched to bspwm because it was more minimal and UNIX philosophy like where it split keybind handling into separate program sxhkd. I also used herbstluftwm, the manual tiling was nice. Then comes dwm, my main WM of choice for a long time since it blended minimalism with functionality and also by suckless so was elegant. However, I still did end up switching to spectrwm since it looked similar visually but I found it easier to config. According to my screenshots though, that didn't last for long. I think I only stayed on it for a few months before switching back to dwm. I did briefly try out sway to see if Wayland was really all that great, but ultimately switched back to dwm again, and only really used it when I felt too lazy to setup Xorg on a new system and not using my pre-existing configs. Again, continuing later for my recent switch to Guix+Emacs.

Missing what editor I used is criminal for this kind of topic. Going back to before Linux, I used Eclipse for Java and Notepad++ for regular stuff, and then I think I used Atom for a brief period of time (VS Code wasn't out yet or something in 2016/17). I didn't do much programming back then, partly because getting dependencies and compiling anything is a pain if my experience with compiling aseprite (sprite editor with cmake buildsystem) was anything to go off of. When I started using Linux, I also started with Vim. Programs like it were what got me to switch in the first place. Turns out learning your tools is important (and that comes up a lot). Even back then I didn't like using much of the pre-made configs, they were bloated and harder to reason about as a new user since defaults were changed. Also had a brief stint with Emacs, but not knowing basic Lisp made it hard to "know" it and it was bloated (common joke is it's an operating system that lacks a good editor) and also not optimized on Windows (same config from Linux) where it was just noticeably more sluggish on muh gamin' laptop when running Win10 instead of like Arch or Gentoo. After learning Vim enough like how to move easily and basic ex commands, I of course wanted more minimalism. Had a nice time with vis and nvi, used them for a while. In my plan9 phase, I used acme briefly and learned a lot about sam through its manual, which also taught me how to use ed the standard text editor, which also taught me (basic) regex which I can't overstate how much better it made editing files in addition to vi motions.

Finally, my current workflow and choices. It's now all based around Lisp and Scheme because they make me feel all fuzzy inside when learning it, just like I did when learning to use Linux in middle school and how to program in C. The propaganda that got me into using this now was when I heard about the Lispy Gopher Show when I was briefly on the Fediverse through Prahou (the author/artist of unix_surrealism from analog_nowhere). While I did briefly use Emacs in the past, I did not learn it properly. As in, not knowing Lisp meant I needed to look a lot of things up instead of just writing them myself. Technically with the vi-like editors, I also didn't write scripts in them nor needed to touch their config due to how minimal they already were by default and that regex+vi-motions going a long way and there being packages to bridge the remaining gap for integrations. But with Emacs, the parentheses I guess made it more daunting or something. This sort of mirrors what I did when I first got into Linux (well Arch) as I read a lot through its wiki. But compared to just regular system administration where you run a bunch of commands or even making your own distro via LFS, I guess properly learning an editor and its programming language was too much for me at the time. For new Emacs users, it was also recommended a lot that they use a pre-made opinionated config like Doom-Emacs or Spacemacs (neovim has a similar situation) and that really put me off since that felt like I would be learning instead how those work instead of the editor itself. Now that I know some functional programming through OCaml and to some extent Nix from a couple summers ago, learning Lisp and Scheme was actually much easier. Compared to using a vi-like editor, using Emacs seemed nicer for working with these kinds of languages since I can selectively execute parts of a program, simliar to something like Jupyter notebook without it being so web- and Python-focused and being an inefficient use of system resources. The main reason why I did not want to use Emacs previously was that it seemed like a monolithic kitchen sink of everything. I think it being described as an operating system within an operating system isn't too much of an inaccurate description. However, compared to Linux and Xorg proper where I tried following minimalism and the UNIX philosophy as much as I could (i.e. small, minimal, self-contained programs for a specific task), there was not much cohesion between them. At most, they'll have vi-keybinds and happen to use ncurses but the layout of everything looks different and configured with different syntaxes (e.g. mutt the mail client looks different from newsboat the RSS reader and different from lynx and links2). Meanwhile with Emacs, yes there are multiple packages and yes I'm not necessarily using POSIX shell scripts for connecting things together, but Lisp is a more powerful language than something like shell that depends on other programs (written in other languages) to do even simple stuff. With Emacs, I now have everything integrated into a single program, everything properly goes through text-based buffers, and as a result is even easier to integrate around because everything is primarily text. Also previously, some popular packages that people use also seemed like it may have contributed to Emacs's bloat in my point of view at that time like /needing/ something like ivy or helm. There's better packages available that do the same thing now, like orderless (for fuzzy searching) combined with vertico (for the vertical complete menu that ivy gave), and consult for better autocomplete for various existing Emacs functions, among other stuff. Now it's my main editor since like last summerish, and I configured it to be my editor, Scheme programming via Geiser, Lisp via just Emacs (since for now I'm mainly using Emacs Lisp and not Common Lisp+SLIME) and using the eval-* functions a lot (for both Lisp and Scheme), document editing and task planning and habit tracking through org-mode (some people use Emacs just for org-mode since it's much more than just a markup language), email through notmuch+mbsync+msmtp (stayed pretty much the same except I'm using notmuch.el directly instead of through like mutt or aerc), RSS through elfeed, browsing primarily through eww (though I have to use qutebrowser for js-heavy sites like Canvas), document viewing through pdftools and docview instead of through mupdf (though for some larger PDFs like textbooks, mupdf is just faster but not needed for most of my PDFs), backgrounded programs through dtach and its associated emacs management package instead of tmux, gptel for interacting with my local LLMs running on my desktop through vllm, and finally as my X window manager via EXWM instead of dwm or any other WM I used to use in the past. Works well when playing games too like Elite Dangerous, Emacs doesn't get in the way.

I know this was very ramble-y. I just braindumped since there's so much to cover and a lot of stuff to remember over the past 9 years that I couldn't think in a more organized way without having all of it written down. I probably missed some stuff, so any questions?

In the actual presentation, I missed mentioning my peripherals (a Ferris Sweep with Kailh Choc Ambient Nocturnal switches for laptop, Corne with Zealios Zilent v2 switches for my desktop, and a Ploopy Adept as my mouse for both) as well since that's also different from how most people use it, as well as my current project "X380", a modded stripped-down X280 (I'll make a post when it's more finalized after fixing my 3D printer).

CrewCTF2024 misc/Sniff Writeup

2024-08-04T12:00:00Z

Challenge

Description

I came across this mysterious device. So I hooked up my logic analyzer and recorded somebody using it. (capture.sol) This challenge has two flags in the flag{} format
The first (easier) is the password that was typed on the keyboard.
The second (significantly harder) is what was display on the screen after the password was entered.

Images

Intro

Instead of sleeping, I made the mistake^Wwise decision of looking at my Discord notification that said there was a hardware challenge in this CTF. It just so happened that there it was using an ATmega-powered keyboard and an e-paper screen, and it almost seemed like a coincidence since I was designing my own keyboard and wanted to interface with an e-paper screen in the near future. This seemed like a great learning opportunity so I started working on the two-part challenge.

There were a few files inside the dist.zip, with the most interesting one being capture.sal which was technically a zip but actually the analyzer file for Salae. Sadly it seemed to need their proprietary program to open.

The first thing I did was figuring out what each channel was connected to and what it meant. It seemed that the keyboard and display were controlled by the Raspberry Pi which then seemed to go to the logic analyzer. So I looked at what each channel was connected to and based on its connected pin on the Pi, I found its function via pinout.xyz. I ended up with this:

Channel 0: P03  I2C SDA
Channel 1: P05  I2C SCL
Channel 2: P11  GPIO 17 (busy)
Channel 3: P13  GPIO 27 (reset)
Channel 4: P15  GPIO 22 (data/command)
Channel 5: P21  MOSI
Channel 6: P23  SPI0 SCLK
Channel 7: P24  SPI0 CE0

Part 1

In Logic 2, I opened the I2C analyzer and outputted the dump in the terminal tab into a file

It seemed there was a lot of NUL bytes being sent, probably indicating that there wasn’t anything during that cycle, and a few seconds later there were also some other different bytes with NUL and some 0x01 bytes in between, and these seemed to be printable ASCII.

read to 0x5F ack data: 0x01
read to 0x5F ack data: 0x01
read to 0x5F ack data: 0x66
read to 0x5F ack data: 0x6c
read to 0x5F ack data: 0x61
read to 0x5F ack data: 0x67
read to 0x5F ack data: 0x7b
read to 0x5F ack data: 0x37
read to 0x5F ack data: 0x01
read to 0x5F ack data: 0x31
read to 0x5F ack data: 0x37
read to 0x5F ack data: 0x66
read to 0x5F ack data: 0x37
read to 0x5F ack data: 0x35
read to 0x5F ack data: 0x01
read to 0x5F ack data: 0x33
read to 0x5F ack data: 0x32
read to 0x5F ack data: 0x7d
read to 0x5F ack data: 0x01
read to 0x5F ack data: 0x0d

Filtering out the 0x00 and 0x01 data bytes and converting to ASCII results in flag{717f7532}.

Part 2

I first outputted the SPI dump from the analyzer into a file and kept only the MOSI and MISO columns.

Time [s],Packet ID,MOSI,MISO
4.108880200000000,0,0x12,0x00
5.109988000000000,0,0x01,0x00
5.110044320000000,0,0xF9,0xFF
5.110062800000000,0,0x00,0xFF
5.110081280000000,0,0x00,0xFF
5.110125600000000,0,0x3A,0x00
5.110167440000000,0,0x1B,0xFF
5.110210120000000,0,0x3B,0x00
5.110251760000000,0,0x0B,0xFF

To actually understand what’s going on, I couldn’t find any proper documentation initially. There wasn’t even a proper datasheet on DigiKey; the “datasheet” was just a summary of the product.

Then I found the Python library source from Pimoroni of their epaper screens, which is probably what was used to make this challenge.

def setup(self):
  """Set up Inky GPIO and reset display."""
  if not self._gpio_setup:
      if self._gpio is None:
          try:
              import RPi.GPIO as GPIO
              self._gpio = GPIO
          except ImportError:
              raise ImportError('This library requires the RPi.GPIO module\nInstall with: sudo apt install python-rpi.gpio')
      self._gpio.setmode(self._gpio.BCM)
      self._gpio.setwarnings(False)
      self._gpio.setup(self.dc_pin, self._gpio.OUT, initial=self._gpio.LOW, pull_up_down=self._gpio.PUD_OFF)
      self._gpio.setup(self.reset_pin, self._gpio.OUT, initial=self._gpio.HIGH, pull_up_down=self._gpio.PUD_OFF)
      self._gpio.setup(self.busy_pin, self._gpio.IN, pull_up_down=self._gpio.PUD_OFF)

      if self._spi_bus is None:
          import spidev
          self._spi_bus = spidev.SpiDev()

      self._spi_bus.open(0, self.cs_pin)
      self._spi_bus.max_speed_hz = 488000

      self._gpio_setup = True

  self._gpio.output(self.reset_pin, self._gpio.LOW)
  time.sleep(0.5)
  self._gpio.output(self.reset_pin, self._gpio.HIGH)
  time.sleep(0.5)

  self._send_command(0x12)  # Soft Reset
  time.sleep(1.0)
  self._busy_wait()

def _update(self, buf_a, buf_b, busy_wait=True):
  """Update display.

  Dispatches display update to correct driver.

  :param buf_a: Black/White pixels
  :param buf_b: Yellow/Red pixels

  """
  self.setup()

  self._send_command(ssd1608.DRIVER_CONTROL, [self.rows - 1, (self.rows - 1) >> 8, 0x00])
  # Set dummy line period
  self._send_command(ssd1608.WRITE_DUMMY, [0x1B])
  # Set Line Width
  self._send_command(ssd1608.WRITE_GATELINE, [0x0B])
  # Data entry squence (scan direction leftward and downward)
  self._send_command(ssd1608.DATA_MODE, [0x03])
  # Set ram X start and end position
  xposBuf = [0x00, self.cols // 8 - 1]
  self._send_command(ssd1608.SET_RAMXPOS, xposBuf)
  # Set ram Y start and end position
  yposBuf = [0x00, 0x00, (self.rows - 1) & 0xFF, (self.rows - 1) >> 8]
  self._send_command(ssd1608.SET_RAMYPOS, yposBuf)
  # VCOM Voltage
  self._send_command(ssd1608.WRITE_VCOM, [0x70])
  # Write LUT DATA
  self._send_command(ssd1608.WRITE_LUT, self._luts[self.lut])

  if self.border_colour == self.BLACK:
      self._send_command(ssd1608.WRITE_BORDER, 0b00000000)
      # GS Transition + Waveform 00 + GSA 0 + GSB 0
  elif self.border_colour == self.RED and self.colour == 'red':
      self._send_command(ssd1608.WRITE_BORDER, 0b00000110)
      # GS Transition + Waveform 01 + GSA 1 + GSB 0
  elif self.border_colour == self.YELLOW and self.colour == 'yellow':
      self._send_command(ssd1608.WRITE_BORDER, 0b00001111)
      # GS Transition + Waveform 11 + GSA 1 + GSB 1
  elif self.border_colour == self.WHITE:
      self._send_command(ssd1608.WRITE_BORDER, 0b00000001)
      # GS Transition + Waveform 00 + GSA 0 + GSB 1

  # Set RAM address to 0, 0
  self._send_command(ssd1608.SET_RAMXCOUNT, [0x00])
  self._send_command(ssd1608.SET_RAMYCOUNT, [0x00, 0x00])

  for data in ((ssd1608.WRITE_RAM, buf_a), (ssd1608.WRITE_ALTRAM, buf_b)):
      cmd, buf = data
      self._send_command(cmd, buf)

  self._busy_wait()
  self._send_command(ssd1608.MASTER_ACTIVATE)

It was also communicating over SPI which seemed to indicate that this was the proper library. Then I looked at the setup() and _update() functions in library/inky/inky_ssd1608.py, and the SPI commands that were sent in the analyzed dump log matched exactly, including each byte of the LUT table.

All the SPI commands used in the library are used with named constants that are defined library/inky/ssd1608.py:

"""Constants for SSD1608 driver IC."""
DRIVER_CONTROL = 0x01
GATE_VOLTAGE = 0x03
SOURCE_VOLTAGE = 0x04
DISPLAY_CONTROL = 0x07
NON_OVERLAP = 0x0B
BOOSTER_SOFT_START = 0x0C
GATE_SCAN_START = 0x0F
DEEP_SLEEP = 0x10
DATA_MODE = 0x11
SW_RESET = 0x12
TEMP_WRITE = 0x1A
TEMP_READ = 0x1B
TEMP_CONTROL = 0x1C
TEMP_LOAD = 0x1D
MASTER_ACTIVATE = 0x20
DISP_CTRL1 = 0x21
DISP_CTRL2 = 0x22
WRITE_RAM = 0x24
WRITE_ALTRAM = 0x26
READ_RAM = 0x25
VCOM_SENSE = 0x28
VCOM_DURATION = 0x29
WRITE_VCOM = 0x2C
READ_OTP = 0x2D
WRITE_LUT = 0x32
WRITE_DUMMY = 0x3A
WRITE_GATELINE = 0x3B
WRITE_BORDER = 0x3C
SET_RAMXPOS = 0x44
SET_RAMYPOS = 0x45
SET_RAMXCOUNT = 0x4E
SET_RAMYCOUNT = 0x4F
NOP = 0xFF

I then noticed that there was a long string of bytes being sent after a 0x24 which in the library indicated that it was the memory buffer for the black/white channel ending with a 0x00 MISO, with the yellow/red channel afterward with a 0x26 MOSI and also ended with 0x00 MISO

...
0x19,0xFF
0x01,0xFF
0x00,0xFF
0x3C,0x00
0x01,0xFF
0x4E,0x00
0x00,0xFF
0x4F,0x00
0x00,0xFF
0x00,0xFF
0x24,0x00
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
0xFF,0xFF
...

There also seemed to be two different updates at around 5 seconds and 70 seconds.

However, the number of bytes written was 4250, which wasn’t the 3812.5 or 2756 bytes I was expecting. This wasn’t divisible by 250 nor 122 and so I was stuck for a long time. Looking through the library source for more than an hour with my tired self didn’t help much either. As a last ditch attempt, I tried converting the raw bytes into an image via Pillow, I used the L mode (8bpp) and just got an uninteresting garbled image.

Part 2 Part 2: Electric Boogaloo

After waking up and working on the CTF after it ended, my partner asked on the Discord and found some interesting very helpful information. It turned out the image was a packed 1bpp image. This meant that each byte in the memory framebuffer contained 8 pixels (8 bits / 1 bits per pixel = 8 pixels).

# under show()
buf_a = numpy.packbits(numpy.where(region == BLACK, 0, 1)).tolist()
buf_b = numpy.packbits(numpy.where(region == RED, 1, 0)).tolist()

In the Python source, this was shown by buf_a and buf_b being packed bits of 1bpp via NumPy. I don’t know much about NumPy, so this was a skill issue as I initially assumed it was a complicated way of saving all the black and red pixels into lists. This is a good reminder that the documentation should be checked for all unfamiliar functions instead of naively assuming what they seem to do.

Also in addition to the screen being rotated by 90 degrees, the vertical resolution is actually 136 pixels and not 120 according to the driver.

Knowing all this solved all my problems as 4250 * 8 was indeed divisible by 250 and the actual vertical resolution 136.

All I had to do was change the Pillow mode when converting the bytes to an image from L (8bpp) to 1 (1bpp) and the (rotated) resolution from (250, 16) to (136, 250) and got an actual image.

I used the first updated bytes which was the screen shown in the challenge’s screenshots. Using the second update’s bytes gave half of the flag in the black/white channel and the other half in the yellow/red channel.

Each character index in both channels seemed to alternate, so the actual flag was flag{ec9cf2b7}. After I finished writing this writeup and seeing the two images side-by-side, they probably could’ve been overlayed after one’s colours are inverted, and is probably what was meant by “stitching” the channels together.

Conclusion

This was my most favourite CTF challenge by far and I learned a lot, especially about stuff I wanted to learn like how SPI e-paper screens work and not be lost with I2C. I am personally now curious whether the SPI screens can be interfaced directly with the MCU instead of going through an intermediate daughterboard/HAT and how different the protocol for parallel screens are since they’re much faster and use more pins.

Fixing bitmap font fallbacks in the st terminal

2023-11-24T12:00:00Z

tldr, change FC_SCALABLE in x.c from 1 to 0. (comes from the font2 patch)

For some context, I have been using xterm for a long while when I'm on OpenBSD since it is included by default in Xenocara with Terminus as my default font, and the main reason why I did not use st again was that my bitmap fallback font for CJK was not loading. Instead, I get an ugly sans-serif scaled font that looked very out of place in my otherwise clean and crisp bitmap terminal.

Yes, I did make sure that the font2 patch for st was applied correctly.

The X11 font string for reference is Fixed: -misc-fixed-medium-r-normal-ja-18-120-100-100-c-180-iso10646-1

It also didn't help that fontconfig was unable to find the font either no matter how much I looked for it with fc-list and fc-match. The weirder thing is that when I installed GNU Unifont to my fonts directory, fontconfig was able to find it and st loaded it (I put a printf in the xloadfonts() function in x.c), but the same old ugly scaled font was still being shown for CJK. The weirderer thing was that Unifont was rendering just fine when being used as the main font instead of in font2.

I thought to myself why this was happening and wasn't able to find out, until I reread the font loading portion in x.c's xloadsparefonts() function that came part of the font2 patch.

It had set the FC_SCALABLE boolean to 1 (true). That explained why the fallback font rendered fine as the main font and not fallback. Setting that boolean to 0 (false) fixed my fallback font not matching issue, and now I have clean and crisp looking text that I can read more easily.

I already disliked fontconfig, freetype, xft, and friends (don't get me started on pango and harfbuzz), but this incident made me dislike it further.

DEADFACE CTF 2023 Shattered Dreams Writeup

2023-10-26T12:00:00Z

DEADFACE is on the brink of selling a patient's credit card details from the Aurora database to a dark web buyer. Investigate Ghost Town for potential leads on the victim's identity.

A huge hint was dropped immediately, so I went to Ghost Town to find a thread titled "We got a potential buyer".

The flag's format is flag{Firstname Lastname}.

lilith, the original poster of the thread, said the victim's SHA1 hash we need to look for is "911d1fc5930fa5025dbc2d3953c94de9e4773584" and showed how she calculated that, including the (lack of) delimeter.

So, we can easily bruteforce getting this SHA1 hash by repeating what lilith did.

The first three fields (card number, expiration, CCV) are values from the billing table and the rest of the fields is all the fields in the patient table.

CREATE TABLE `billing` (
  `billing_id` int(11) NOT NULL AUTO_INCREMENT,
  `patient_id` int(11) NOT NULL,
  `credit_type_id` int(11) NOT NULL,
  `card_num` varchar(24) NOT NULL,
  `exp` varchar(8) NOT NULL,
  `ccv` varchar(4) NOT NULL,
  PRIMARY KEY (`billing_id`),
  UNIQUE KEY `card_num` (`card_num`),
  KEY `fk_billing_patient_id` (`patient_id`),
  KEY `fk_billing_credit_type_id` (`credit_type_id`),
  CONSTRAINT `fk_billing_credit_type_id` FOREIGN KEY (`credit_type_id`) REFERENCES `credit_types` (`credit_type_id`) ON DELETE CASCADE,
  CONSTRAINT `fk_billing_patient_id` FOREIGN KEY (`patient_id`) REFERENCES `patients` (`patient_id`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=14443 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
...
CREATE TABLE `patients` (
  `patient_id` int(11) NOT NULL AUTO_INCREMENT,
  `first_name` varchar(32) NOT NULL,
  `last_name` varchar(64) NOT NULL,
  `middle` varchar(8) DEFAULT NULL,
  `sex` varchar(8) NOT NULL,
  `email` varchar(128) NOT NULL,
  `street` varchar(64) NOT NULL,
  `city` varchar(64) NOT NULL,
  `state` varchar(8) NOT NULL,
  `zip` varchar(12) NOT NULL,
  `dob` date NOT NULL,
  PRIMARY KEY (`patient_id`),
  UNIQUE KEY `email` (`email`)
) ENGINE=InnoDB AUTO_INCREMENT=18542 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

Since there are no delimeters, they can just be concatenated with each other and then piped to sha1. The difficult part I had was properly concatenating those values because I was not able to read the MySQL dump with sqlite3 nor mariadb.

I noticed that each of the rows that were inserted into the tables were delimited by a comma, similar to CSV.

INSERT INTO `patients` VALUES (8151,'Lorrayne','Covey','E','Female','lcovey0@wunderground.com','40411 Old Shore Street','Houston','TX','77201','1985-02-08'),(8152,'Eddy','Omand','S','Female','eomand1@mysql.com','454 Pine View Alley','Columbus','OH','43226','1959-09-29'),(8153,'Renard','Berre','O','Male','rberre2@friendfeed.com','3496 Merrick Center','Pittsburgh','PA','15235','1978-12-05'),(8154,'Galven','Nardrup','M','Male','gnardrup3@mac.com','0 American Road','Denver','CO','80241','1984-04-09'), ...

So I was able to easily convert it into a CSV with the following command:

grep 'INSERT INTO `patients`' aurora.sql \
    | sed 's/^INSERT[^(]*//' \
    | awk '{gsub(/,\(/, "\n"); gsub(/\)*/, ""); print}'

Part of the output is now:

...18532,'Becka','Hurlin','T','Female','bhurlin80d@yolasite.com','58 Amoth Way','Ventura','CA','93005','1953-11-14'
18533,'Aluin','Horwell','O','Male','ahorwell80e@cbc.ca','3 Oriole Terrace','Miami','FL','33190','1984-06-24'
18534,'Glennis','Walder','R','Female','gwalder80f@cnet.com','966 Packers Hill','Topeka','KS','66617','1950-01-21'
...

One problem I had when I used tr to replace ( with \n was that one of the names had () in their name for some reason, which messed up the concatenating of the two files to get the hash. I originally just manually edited it, but the above with awk is cleaner. The same goes with the 'INSERT INTO ...' part messing things up for the same reason. I didn't save my ~/.ksh_history file with the commands I ran, so this is a non-ugly version I remade with more awk.

To properly concatenate the data fields, the comma and single quotes should also be removed, which tr can be used for unlike before.

grep 'INSERT INTO `patients`' aurora.sql \
    | sed 's/^INSERT[^(]*//' \
    | awk '{gsub(/,\(/, "\n"); gsub(/\)*/, ""); print}' \
    | tr -d "'," >patients_.txt

Part of the output prior to being written to a file is now:

18527TannerMasselinAMaletmasselin808@google.es10080 Reindahl CourtBoca RatonFL334871957-08-15
18528MerrelYeudeDMalemyeude809@ca.gov7934 Katie PassSaint PaulMN551881951-04-07
18529JeffVan BaarenMMalejvanbaaren80a@sphinn.com58097 Autumn Leaf DriveNew OrleansLA701421984-01-13

It is written to a file so that it can be easy to concatenate both the billing and patient data by using the paste command.

The next part is doing the same with the billing data. Unlike with the patient data, only three fields from the billing table is used instead of all, so cut or awk can be used with the delimeter set to a comma.

Before that, we need to know what index (base 1) the three fields are at. The credit card number, expiry date, and ccv are fields 4, 5, and 6.

grep 'INSERT INTO `billing`' aurora.sql \
    | sed 's/^INSERT[^(]*//' \
    | awk '{gsub(/,\(/, "\n"); gsub(/\)*/, ""); print}' \
    | cut -d, -f4,5,6 \
    | tr -d "'," >billing_.txt

Some of the output looks like

51087507745678202025-01403
50483770976260922023-07242
50483739134688352023-12501

Both tables with the fields have been parsed and saved to two different files, but simply using cat on both the files would simply print the contents of the second file after the first file has finished printing, but we need each line of both the files to be joined together! That's where the paste command comes in. It combines each line of its input files, which is exactly what is required.

These merged lines are then fed into sha1, and finally grep can be used to look for the hash we need and print the victim's name if it shows up.

target_hash="911d1fc5930fa5025dbc2d3953c94de9e4773584"

paste billing_.txt patients_.txt | tr -d '\t' \
    | while read line; do
    echo -n "$line" | sha1 \
        | grep "$target_hash" && echo "$line" && break
    done

The outputted line of the victim who had the same hash is:

50483743238485412026-0498316314BertonLuchettiXMalebluchetti6ar@taobao.com39 Meadow Ridge TerraceClevelandOH441251964-10-29

So, the victim is Berton X. Luchetti, and the flag is flag{Berton Luchetti}.

This challenge would probably have been easier if I was able to use proper SQL commands, but I couldn't do that and standard UNIX tools saved the day. I did the exact same process of parsing the tables for all the other SQL challenges and I found it funny that I solved all of them without needing to run a single SQL command (partly because they didn't load the file).

BCTF23 crypto/Electronical (medium) Writeup

2023-10-26T12:00:00Z

I do all my ciphering electronically. https://electronical.chall.pwnoh.io/

When going to the linked site, you get told to encrypt any message or view the site's source code. After submitting a message to encrypt, it returns some hex string.

The source is:

from Crypto.Cipher import AES
from flask import Flask, request, abort, send_file
import math
import os

app = Flask(__name__)

key = os.urandom(32)
flag = os.environ.get('FLAG', 'bctf{fake_flag_fake_flag_fake_flag_fake_flag}')

cipher = AES.new(key, AES.MODE_ECB)

def encrypt(message: str) -> bytes:
    length = math.ceil(len(message) / 16) * 16
    padded = message.encode().ljust(length, b'\0')
    return cipher.encrypt(padded)

def decrypt(msg: str) -> bytes:
    return cipher.decrypt(msg)

@app.get('/encrypt')
def handle_encrypt():
    param = request.args.get('message')

    if not param:
    	return abort(400, "Bad")
    if not isinstance(param, str):
    	return abort(400, "Bad")

    print(encrypt(param + flag))

    return encrypt(param + flag).hex()

@app.get('/source')
def handle_source():
    return send_file(__file__, "text/plain")

@app.get('/')
def handle_home():
    return """
    	<style>
    		form {
    			display: flex;
    			flex-direction: column;
    			max-width: 20em;
    			gap: .5em;
    		}

    		input {
    			padding: .4em;
    		}
    	</style>
    	<form action="/encrypt">
    		<h2><i>ELECTRONICAL</i></h2>
    		<label for="message">Message to encrypt:</label>
    		<input id="message" name="message"></label>
    		<input type="submit" value="Submit">
    		<a href="/source">Source code</a>
    	</form>
    """

if __name__ == "__main__":
    app.run()

It seems that the flag is appended to the user's message and then encrypted with AES-ECB. The total message is also padded to be a multiple of 16 bytes.

According to Wikipedia, ECB (electronic codebook) works by dividing a message into blocks of a certain size (like 16 bytes). The problem however is that ECB doesn't attempt to make any encrypted block unique like by adding a salt or nonce, so any blocks of data that are identical would also be identical when encrypted. Wikipedia also has an interesting example of encrypting an image of Tux and a mountain (on French Wikipedia) with AES.

Through some more searching online, it seems a way to exploit this is with something called a Chosen Plaintext Attack. Since the message before the flag is controlled by us the user (attacker?) and the flag is appended to the end, the provided message can be made in a way that only one byte of the flag needs to be bruteforced at a time.

Let's say that this is our message: thischallengesucksFLAG{5om3_!mp0r74nt_$3cr37}

This string is 45 characters, so the server would pad this with 3 \0 characters to make it evenly divisible by 16 characters.

b'thischallengesucksFLAG{5om3_!mp0r74nt_$3cr37}\x00\x00\x00'

We know what "thischallengesucks" is, but FLAG and anything else after is appended by the server and is what we're trying to find.

"thischallengesucks" is 18 characters, but if we send a 15 character string, then the first block to be encrypted would be "thischallengesu?", where ? is the mystery character.

For readability purposes, I'm going to use repeated "0" characters needed instead of "thischallengesucks".

When passing "000000000000000?" to the server, a certain hex string would be returned (newlines every 32 characters not included in original):

b57189530dacbb9c5707c1cb0b044a34
5377049685bb9553a73e4408565505dd
0c614f69c4749b10f8cbc9c735fd7314
5a9ae527825603a8eb0dba6a0347a4e5

Replacing the ? with any other character would result in only the first row being changed, like with "000000000000000A":

b2457a857e82a1d5ad919a4bdaf9133a
7835c84bc75d836fad8ca5fbcec086ff
937cf83a682fa26162a65f2295b2b119
6b398dd6f75e212b1633c5189bdb5689

Since the other three blocks remained the same, the last character in the message being sent simply needs to bruteforced with every printable character until it results in the same block from ?. In this case, that character would be F:

b57189530dacbb9c5707c1cb0b044a34
5377049685bb9553a73e4408565505dd
0c614f69c4749b10f8cbc9c735fd7314
5a9ae527825603a8eb0dba6a0347a4e5

(0000000000000, 14 characters long)
5ec61f1209adfeff202edbba28339f83
4f7cc7a4c0c553380874383e93408678
ca91d95b091956edb162da583b51051b
33b91ab14b8807348fc98bf223b4b3a5

(0000000000000FL, 16 characters long)
5ec61f1209adfeff202edbba28339f83
4f7cc7a4c0c553380874383e93408678
ca91d95b091956edb162da583b51051b
33b91ab14b8807348fc98bf223b4b3a5

Then the 0 left pad would be decreased by one character and the process repeats until the whole block is done. However, a flag usually won't be just 16 characters long. I had some difficulty trying to bruteforce the 17th character and above because I was prepending and appending the zeros within a single block (between 0 and 15 padded 0s), but that was among a few other issues I had that were the result of the message I was sending being in the format of "pad + known_flag + brute_single_char + pad" where this only worked for the first block. This did not work later because those messages would have the pad bytes in the middle of the message, which did not go well.

In the end, I realized that I can check the target block hexstring by sending only the padded 0 bytes (or anything else of that length) without other characters and then append my known bytes of the flag and a single other character to fill that block to brute force that last character.

A visual representation is this:

Block size: 8 characters

7 pad, 0 known
XXXXXXX?
XXXXXXXF

6 pad, 1 known
XXXXXX??
XXXXXXFL

5 pad, 2 known
XXXXX???
XXXXXFLA

...

0 pad, 7 known
FLAG{5o?

8 known
FLAG{5om

This only decrypts the first block, so how I decrypted each additional block was by prepending another block of pad characters (blocksize - 1) and repeating the process.

7 pad, 7 known
XXXXXXXFLAG{5om?
XXXXXXXFLAG{5om3

6 pad, 8 known
XXXXXXFLAG{5om3?
XXXXXXFLAG{5om3_

...

5 pad, 26 known 
XXXXXFLAG{5om3_!mp0r74nt_$3cr37?
XXXXXFLAG{5om3_!mp0r74nt_$3cr37}

...
0 pad, 31 known
FLAG{5om3_!mp0r74nt_$3cr37}\0\0?
FLAG{5om3_!mp0r74nt_$3cr37}\0\0\0

After some automating help with python, I was able to finally get the flag.

Flag: bctf{1_c4n7_b3l13v3_u_f0und_my_c0d3b00k}

My python file to solve this was:

from requests import get
from requests.utils import quote

# list of characters that will be bruteforced, these are the printable chars
chars = [chr(i) for i in range(ord(' '), ord('~') + 1)]
# nul character is also checked because that's the pad character
chars += '\0'

def encrypt(msg):
    #url = "https://electronical.chall.pwnoh.io/encrypt?message="
    url = "http://localhost:5000/encrypt?message="
    return get(url + quote(msg)).content;

def calc_padding_for_known():
    # divided by 2 because each hex byte is 2 characters long
    cur = len(encrypt("0")) // 2

    # 16 chosen because that's the padding chosen in app.py on the server
    for i in range(2, 16):
    	tmp = len(encrypt("0" * i)) // 2
    	if tmp > cur:
    		return tmp, tmp - cur, i - 1
    # shouldn't come here
    return 0,0,0

totalblocks, bs, pad = calc_padding_for_known()

print(f"Block size of padding: {bs}, {pad}")

# first block are known to not be the flag (is all 0 being encrypted)
# second block is what's being bruteforced
# third block's last character is unknown and being compared with second block
#msg = "0" * (bs + bs - 1) + "a" * 1 + "0" * (bs - 1)
#cur = encrypt(msg)
#print(cur)
flag = ""
curflag = ""

tbs = bs * 2

for j in range(totalblocks // bs):
    for i in range(1, bs + 1):
    	known = "0" * (bs * (1 + j) - len(flag) - 1)
    	msg = known
    	target = encrypt(msg).decode("utf-8")

    	print(f"\nNew target message: {msg}")
    	print("New target message return:")
    	print('\n'.join([target[A:A + tbs] for A in range(0, len(target), tbs)]))

    	target = target[tbs * (0 + j):tbs * (1 + j)]
    	print(f"New target block: {target}")

    	for c in chars:
    		msg = known + flag + c
    		print(f"Current character: {c}")
    		print(f"Current message: {msg}")
    		print(f"Target block: {target}")
    		cur = encrypt(msg).decode("utf-8")
    		print('\n'.join([cur[A:A + tbs] for A in range(0, len(cur), tbs)]))

    		print(f"Current block: {cur[:tbs]}")

    		if (cur[tbs * (0 + j):tbs * (1 + j)] == target):
    			flag += c
    			print(flag)
    			break

    		print("\n")
    curflag += flag
print(flag)

DEADFACE CTF 2023 Host Busters 3 Writeup

2023-10-26T12:00:00Z

Continue characterizing the machine. Is there any way you can escalate to a user that has permissions the vim user does not have? Find the flag associated with this user. Submit the flag as flag{flag_here}.

vim@ghost404.deadface.io letmevim

You first login to vim, which has vim open. Then you escape from it like you did in the OverTheWire Bandit challenges with :set shell=bash and :shell. Now you have a proper shell over SSH.

The first thing I looked at after mistaking Host Busters 1's key in the home directory as 3 was look at what other user home directories there were by running ls /home. I saw there were a few users, notably gh0st404 and spookyboi.

gh0st404's user home directory had his OpenSSH private key as world-readable and in plain sight not in his .ssh hidden directory. It being world-readable would have had OpenSSH scream at you, but them being stupid was good for us.

So, once you use that SSH private key to login as gh0st404, you can check the contents of hostbusters3.txt and you got the flag.

cat hostbusters3.txt

"This is why you should have come to the Monday meetings for OverTheWire." ~Joey, FPUSEC President

Here's an asciinema of the entire thing in action.

CSAW23 rev/Rebug1 Writeup

2023-09-28T12:00:00Z

Can't seem to print out the flag :( Can you figure how to get the flag with this binary?

An innocent looking binary is given that asks for a string:

./test.out
Enter the String: rptuainadui
that isn't correct, im sorry!

This is part of the rev category (which I think is for reverse engineering). You could bruteforce this yes, but I found it easier to put this into a decompiler like the ones on DogBolt (Decompiler Explorer) to see what it's doing.

Decompiled main function (via angr):

int main()
{
    char v0;  // [bp-0x448] 
    unsigned int v1;  // [bp-0x41c]
    char v2;  // [bp-0x418]
    char v3;  // [bp-0x408]
    unsigned long long v4;  // [bp-0x18]
    unsigned int v5;  // [bp-0x10]
    unsigned int v6;  // [bp-0xc]
    unsigned long long v8;  // rax

    printf("Enter the String: ");
    __isoc99_scanf("%s", (unsigned int)&v3);
    for (v6 = 0; (&v3)[v6]; v6 += 1);
    if (v6 == 12)
    {
    puts("that's correct!");
    v4 = EVP_MD_CTX_new();
    (unsigned int)v8 = EVP_md5();
    EVP_DigestInit_ex(v4, v8, 0x0, v8);
    EVP_DigestUpdate(v4, "12", 0x2, "12");
    v1 = 16;
    EVP_DigestFinal_ex(v4, &v2, &v1, &v2);
    EVP_MD_CTX_free(v4);
    for (v5 = 0; v5 <= 15; v5 += 1)
    {
    	sprintf(&(&v0)[2 * v5], "%02x", (&v2)[v5]);
    }
    printf("csawctf{%s}\n", (unsigned int)&v0);
    return 0;
    }
    printf("that isn't correct, im sorry!");
    return 0;
}

This along with the rest of the decompiled binary can't be simply compiled again as-is because there are a few issues, like the OpenSSL functions being called having an extra argument added to the end.

When looking at the functions being called, it seems that the flag is just an md5 of the number 12. The program also seems to give the flag itself if you give it the character with the ASCII value of 12 (form feed).

The line that that has the data being checksummed is this:

EVP_DigestUpdate(v4, "12", 0x2, "12");

I did try piping the form feed character via printf to the binary, but it did not like that, so it seems that the only way to get the flag is through another way.

While you could just create a very simplified version of the decompiled source with OpenSSL's crypto library (which is what I did originally), it's much easier to just pass the number 12 to a pre-installed md5 command (md5 on OpenBSD, md5sum on Linux).

$ echo -n 12 | md5
c20ad4d76fe97759aa27a0c99bff6710

This CTF's flags were in the format of csawctf{somethinghere}, as also seen in the decompiled source, so the actual flag was this:

csawctf{c20ad4d76fe97759aa27a0c99bff6710}

This was my first time doing any decompilation of a program, and I think this was a good start to learn reverse engineering.

Creating a Temporary File Hoster

2022-04-27T12:00:00Z

For the past couple years, whenever I wanted to upload a file, I would curl the file to lainsafe, i/u.kalli.st, and recently ttm.sh.

Since I want to selfhost, I thought i can just use either of what those three used. Earlier today though, I realized I could just copy the file(s) I want to upload via rsync/scp to a public directory that gets served by an httpd or gopherd.

From what I understand, the previous file hosters had a program running that read the file that the user uploads to them, does some renaming, and writes that to a directory that is served. After some time, that file is deleted. The first part can be handled via rsync/scp like mentioned previously. For automatic deletion, I recently saw in find's man page that it can list that haven't been modified via the -mtime flag, so that can be used with a cron job.

But while thinking of this idea, I got stumped by how to print back the url to this file that is uploaded since printing the filename as is appended to its baseurl, there could be spaces and other invalid unescaped characters which programs trying to download it may not like.

I thought I could just create a separate program for this. However, doing this seemed more complicated than just copying the file to the server. So, with the help of awk and some StackExchanging, I've been able to do it.

upfile.sh:

#!/bin/sh
urlencode() {
     awk '
BEGIN { for (i = 1; i < 256; i++) hex[sprintf("%c", i)] = sprintf("%%%02X", i) }
{
     for (i = 1; i <= length($0); i++) {
             c = substr($0, i, 1)
             printf("%s", c ~ /^[-._~0-9a-zA-Z]$/ ? c : hex[c])
     }
     printf "\n"
}
'
}

FILE="$1"
SERVER="REPLACEME"
BASEURL="https://u.$SERVER"

[ -z "$1" ] && exit 1

scp "$FILE" "$SERVER":files/ || exit 1
printf "%s/" "$BASEURL"
basename "$FILE" | urlencode

Then to purge these files after they become too old (e.g. 3 days), you can put something like this in a cron job to run daily (replace file directory):

0 0 * * * find /path/to/dir/ -mtime +3 -exec rm {} \;

You can also put this command in /etc/daily.local or /etc/cron/daily, or whatever file your root crontab's @daily runs (if there is one).

And that's it! The only difficult part that I experienced was encoding the name of the file and originally did that in C. However, having a mixed C and shell program just for file uploading didn't sit right with me. It seems like whenever you're in doubt, you can rely on awk huh.

Kobo Clara HD Custom Linux Distro/RootFS

2021-07-22T12:00:00Z

These are just some notes I made when creating my own mini-distro after wanting something more custom than just using buildroot or making the official firmware more slim. For people other than me, I suggest looking through (C)LFS or running postmarketOS instead once this reader's pull request1 gets integrated into upstream.

Two things that'll greatly help with this is having serial terminal access with the four uart pins near the top right in the back of the reader, near the uSD card slot (I don't connect the 5V pin as my reader doesn't really turn on anything other than the power LED). I suggest maybe soldering female pin headers to there to make your life easier (you can later cut out a hole in the back cover or desolder the headers once you're done). Other than that, I suggest installing QEMU with ARM userspace to test programs that you have built or running them on a separate ARM device like a Raspberry Pi.

Prelude

Ever since I learnt that the official firmware for the Clara was just using a modified Linux kernel with busybox as coreutils and many other libraries, I just knew that I had to minimize it. I also saw that it was using glibc for it's libc, which I really dislike as statically linking C programs against it was a pain in my experience, compared to something like musl and uclibc. It's also much larger than them and I don't use any of glibc extensions so it seemed like a waste of space to me.

Initially when I replaced Nickel with Plato, I was able to shave about 100 MiB after I removed /usr/local (which contains Nickel, Qt and a few other things), from 189 MiB to 74 MiB, but I still wanted to make it smaller.

Using buildroot, I was able to get it under 2 MiB (!!) which was a little less than half the size of an uncompressed armhf Alpine Linux minirootfs (4.9M for 3.14). With Busybox, it was pretty much working out of the box, with serial terminal access! But waiting around 15 minutes for the toolchain to build each time I wanted to change something in the rootfs took way too long, although it could've been minimized if I used ccache with a fairly large cache size. I still found that it compiled and installed a lot of things I wouldn't be using (particularly in /usr) even after disabling almost all of the third-party packages.

I've uploaded the config file and the resulting rootfs for buildroot 2021.05. The root password by default is changeme. EDIT 2022-10-21: gone, build it yourself

Of course the rootfs I got from buildroot nor me making the official firmware smaller is the point of this article, and the actual point is making one yourself! (or rather what I did to make my own)

Cross-toolchain

For now as of July 22, 2021, I'm using my distro (Void Linux)'s packaged cross toolchain for armhf musl, but eventually I would be using my own.

I'm not compiling off of the device itself as it would be somewhat slow for bigger programs, which is currently primarily the Linux kernel, U-Boot, and the toolchain itself, considering that the ereader's CPU (Freescale i.MX 6SLL) is a single core running up to 1 GHz. Including the development tools and headers would also take up more space on the device itself, and since the terminal can currently only be accessed through it's serial/uart pins, I don't think it's ideal.

TODO: include steps to create own toolchain (probably based off of gcc 4.7.3 as that doesn't require c++)

Building the rootfs

Assuming you made a new filesystem on your rootfs's partition, it'll likely be empty with no directories you'd expect to find on a regular distro. So you'll just have to make them. cd /path/to/rootfs mkdir bin dev etc proc sbin

Your binaries would usually go in /bin, the uSD card, ttymxc0, and other devices would go in /dev, felker init's default program/script to execute is usually in /etc/rc, /proc is optional but I have it mounted to see what is currently mounted through /proc/mounts (or mount(1) without any arguments) as well as to see my disk usage through df(1). /sbin is there to place the init in as /sbin/init is the default init path the kernel looks at.

toybox

Now on to the main part of the distro, the userspace. I intend to keep it fairly minimal so I've chosen to use toybox along with a slightly modified version of felker (musl dev)'s init2, as well as dash3 as the main shell since toybox doesn't include one as of 0.8.5 (though it'll probably be there by 1.0). I'll also be statically linking all the programs that'll be used so I wouldn't have to worry about shared libraries not being included/copied over, and also including LTO for slightly faster binaries. Originally, I tried going with sinit, sbase, and ubase but I was having trouble getting serial terminal access with getty to /dev/ttymxc0 (the default serial tty, at least with the vendor kernel). I didn't have this problem with busybox's and toybox's getty however. My config for toybox was also about 81K smaller than my trimmed sbase-box and ubase-box (352K compared to 267K+166K) where I removed programs that I won't use from ${BIN} in their respective Makefiles. EDIT 2022-10-21: also gone

First I suggest exporting some environment variables to set the toolchain used as well as enabling static linking and LTO.

export CROSS_COMPILE="arm-linux-musleabihf-" # change to your cross-tc
export CC="${CROSS_COMPILE}gcc"
export LDFLAGS="--static"
export CFLAGS="-flto -static"
export ARCH=arm # for compiling the linux kernel

To compile toybox, get the source from https://landley.net/toybox/downloads/ (or clone the upstream repo). Then run make menuconfig (optionally with make defconfig before it) and change it as you see fit. Personally, I disabled most of the programs I wouldn't use and kept only the ones that'll help with fixing a problem. Finally, make sure to run make.

make defconfig
make menuconfig
make

To move it to your rootfs and set it's symlinks, you could probably run make install after setting PREFIX to your rootfs's /bin directory, but I did it manually.

# automatic (didn't test, check README)
make PREFIX=/path/to/rootfs/bin/ install

# (semi?) manual
cp toybox /path/to/rootfs/bin

# add symlinks if doing manual and you want them
cd /path/to/rootfs/bin
for prog in $(qemu-arm ./toybox); do ln -s toybox "$prog"; done

dash

Also as of toybox 0.8.5, a shell still isn't included (probably would be included by 1.0 according to scripts/install.sh as well as a few other programs like gzip), so a separate shell would need to be built. Any can be used but dash would be shown as an example as I was able to get a static binary without too much trouble.

First obtain the source3 and cd into its untarred directory. Assuming your CC and CFLAGS are set, you can run these steps:

autoreconf -fiv
./configure --host=$CROSS_COMPILE --with-libedit
make
${CROSS_COMPILE}strip src/dash

As this is going to be used as the main shell, I've decided to just copy it to /bin/sh in the rootfs directory, though copying it there but as /bin/dash and /bin/sh being symlinked to dash is also an option.

cp src/dash /path/to/rootfs/bin/sh
# or
cp src/dash /path/to/rootfs/bin
cd /path/to/rootfs/bin
ln -s dash sh

felker's init

The init is just a single file that you can get from felker's site2 or the gist on github7. I haven't had a good experience with the default startup program (/etc/rc) as a shell script with execve() run on it so I'd change it to execvp() and remove the third (specifies environment). To compile and install the init, all you need to do is run:

$CC $CFLAGS -o init init.c
cp init /path/to/rootfs/sbin

Instead of /etc/rc being a shell script, you can also make a C program that does whatever you think is needed for a proper startup. I'll still use a shell script though which is linked here. EDIT 2022-10-21: you get the idea, it's gone.

/etc/passwd

Copying the rootfs's contents to your device's/uSD card's root partition and then turning the device on should now work with a login prompt shown in the serial terminal. However, you probably wouldn't be able to login to any user. So you'll have to create a file at /path/to/rootfs/etc/passwd. For an empty password to root, you can use this, though I suggest setting a password as soon as you login:

# in rootfs's /etc/passwd
root::0:0:root:/root:/bin/sh

With the passwd file created/updated, you should now be able to login to root after the rootfs is copied to your uSD card. Your rootfs so far should now be around 550-560K, which is much much smaller than the original firmware's, though it'll likely be much larger to maybe a few megabytes once a proper reader software is added.

Custom Linux Kernel

WARNING: I haven't actually gotten the kernel to load in u-boot yet. It just hangs in the "Starting kernel ..." step and the init doesn't get loaded, so I'm assuming the kernel itself isn't either. If anyone out there has gotten a custom kernel working in the Kobo Clara HD, please send me an email or message on xmpp.

UPDATE Jul 28, 2021: Gave up on it as I just couldn't get any kernels I built (both vendor and akemnade's mainline) to boot. But neither did postmarketOS boot beyond the initial initramfs messages without the log file being created. So I'll revisit this for later.

EDIT 2022-10-21: I have gotten this working, but have been unable to get Plato build for musl, so I will have to either continue fighting with the crab or create my own with fbink, as that still works. Separate article on this later.

My next big step is compiling my own kernel for the Clara HD. With the default configuration built for the vendor kernel, it appears to be about 3M, so my goal is to build a kernel that is smaller than that while retaining only the functionality that I need. I'm also not going to include networking support as that is unneeded for my purposes, but I suggest just keeping it if you're unsure. The wifi driver for the Kobo Clara HD is available as an out-of-tree driver8.

You should first obtain the kernel source, with two main options, the vendor kernel9 and the mainline kernel (with akemnade's patches)10. For the latter, you need to clone the repo and switch to the latest kobo/drm-merged branch (kobo/merged-5.13 as of July 25, 2021).

After you've got them and assuming the CROSS_COMPILE and ARCH environment variables are set, you'd want to configure the kernel.

I had a hard time compiling the vendor kernel with many things disabled, so I've kept my config somewhat similar to the default config. The config I used is available here (EDIT: dead).

make menuconfig
make zImage

Assuming it compiles properly and arch/arm/boot/zImage exists, all that's needed to is to write it to your uSD card at the 1M offset. dd if=/path/to/kernel/zImage of=/path/to/uSDdev bs=512 seek=2048

Custom U-Boot

I have not done this yet, nor really plan to, but if you do manage to compile the Kobo's vendored u-boot source, then all you'd have to do to install it is:

dd if=u-boot-file of=/dev/mmcblk0 bs=128k count=1 seek=6

If I remember correctly, this command was included in an older firmware's startup script/rcS for updating udev, and it should still work.

Kobo Clara HD Notes for Plato (and KSM)

2021-03-27T12:00:00Z

These are my notes for getting Plato on the Kobo Clara HD from scratch as well as some notes for getting KSM to work, but I now boot directly into Plato instead of through KSM.

Previously, I didn't really like using KOReader because it was kind of slow and was written in Lua. At the time of using Plato, it seemed nice but it didn't cover thumbnails for books, which while it is a minor detail, I find books easier to be recognized with a cover thumbnail in addition to their title. This was added in release 0.9.10 but as an optional feature which I didn't somehow see until recently when I retried it. HOWEVER again, I didn't like using k/fmon as I had to still use Nickel to get back into KOReader/Plato/whatever alternate reader when I wanted to go away from using Nickel.

<ignore> That was when I found out about KSM and how there was a working version for the Clara HD. KSM is like an alternate bootloader for the Kobo readers and it apparently doesn't work very well with newer models like the Clara HD and up, but someone got it to work with those devices. KSM 09 is apparently not maintained anymore and I'm not sure if KSM 10 is being developed or not since I'm pretty sure it's closed source.

Development and support for KSM stopped some time ago. Therefore, do not use it!

"That sign can't stop me because I can't read!" - Me imitating D.W. from the PBS Kids cartoon Arthur on Mar. 26, 2021 when I saw that it can be used on my Kobo

The latest firmware version that KSM sort of supports is v4.25.15875 but it can probably work with a newer version like v4.26+ that would likely only need a couple changes to /etc/init.d/rcS, if any changes were needed at all. I'll be using v4.26 for the rest of this article/guide. </ignore>

Recently, after seeing how my Kobo boots into KSM and Nickel through the rcS file, I realized that I could've instead just booted directly into Plato, and plato.sh (the script that runs Plato) has a standalone option that supports just that! The KSM notes are still going to be here in case someone still wants to use KSM.

Installing Plato (or probably any other reader like KOReader)

This part probably applies to any other reader other than Plato like KOReader but I haven't personally tested them. All you have to do is get the latest release at Plato's repo and unzip it's contents into a folder called plato in /path/to/kobo/mount/.adds, the latter folder of which should have already been created by KSM if you are using that. If you are using KSM, there should be a new option below "start nickel" called "start plato" when you have rebooted the device. Read below if you aren't using KSM.

Loading Plato on Boot

Since I don't want to load Nickel only to load into another reader like the recommended options in Plato's forum thread (kfmon, fmon, and NickelMenu) suggest, I noticed that I could have booted into Plato directly. The only requirements for doing this having access to the rootfs, so either through a telnet/ssh session, or having the sd card's root/first partition mounted to your computer, or just ftp/rsyncing the files to your Kobo.

First I suggest making a copy of rcS if you haven't already in case an update overwrites it. My copy is named custominit.sh. Next you'll want the Kobo's /etc/inittab to boot with custominit.sh instead of rcS: /etc/inittab:

#::sysinit:/etc/init.d/rcS
::sysinit:/etc/custominit.sh

The rest of the lines don't need to change. Then you should open custominit.sh in your favourite editor to add the lines at the bottom but before hindenburg is executed:

cd /mnt/onboard/.adds/plato     # or whereever Plato is
PLATO_STANDALONE=1 ./plato.sh

You would probably also want to remove the lines where Nickel-specific programs/scripts are running like nickel, hindenburg, pickel, sickel, etc.

Now on subsequent boots, Plato should automatically have been loaded. Boot times may also be slightly faster! :D

Installing KSM 09 (not doing anymore)

First you would want to download the Clara HD version of KSM 09 and the fix for v4.25. Then, you would want to unzip the KoboRoot.tgz with separate filenames so they don't replace each other and we would untar those into the same directory. After that, we would cd into the directory and tar it's contents into a new KoboRoot.tgz and place it in /path/to/kobo/mount/.kobo/.

An example of what I did after downloading and unzipping the files are below:

mkdir koboroot
tar -xvf KoboRoot-main.tgz -C koboroot
tar -xvf KoboRoot-v4.25-darkmodefix.tgz -C koboroot
cd koboroot
tar -czvf ../KoboRoot.tgz .
cd ..
rm -r koboroot

After your Kobo untars it and you wait a while, you should be presented with KSM's main screen :D ksm09's main screen running on the kobo clara hd

Auto-Boot into Plato instead of Nickel via KSM (not doing anymore)

First make sure USB support is enabled in KSM and then mount your Kobo to your computer. Once mounted, go to /path/to/kobo/mount/.adds/kbmenu_user/confoptions and edit ksm_ini_options.txt in your favourite editor. You should see many options that are listed but the one that we're interested in is ksmAutoselectoption which may have start_nickel and start_koreader already and what we want to do is add ksmAutoselectoption=start_plato. After a quick restart to reload the options file, you should be able to see the new option in KSM's settings under [general] and add item if it wasn't already added. Now Plato should auto-boot on subsequent powerons.

Kobo Clara HD Notes for Nickel

2021-01-13T12:00:00Z

My ereader of choice is the Kobo Clara HD and I particularly like it because my eyes hurt less when reading for long periods of time compared to when I read on my phone or when I still had my iPad. It also had much longer battery life and only need to charge it about once every two weeks when I read for about 4 hours on average daily.

However, the two notable things I don't like about it is it's included telemetry, like using Google Analytics by default and keeping a unique salt

Spyware/Anti-Features:

Google Analytics (a lot of actions, if not everything, is sent to Google)
Auto-update by default
- I prefer being able to review what the new update provides and choose not to apply it
- I don't like the new redesign in firmware v4.23.15505

I'm also assuming your Kobo reader and it's SD card's device file would be would located at /dev/sdf and be mounted at /mnt/kobo.

If you're going to not be using Nickel and instead be using something like Plato, there's a newer version of this article available here, but the notes are for ~KSM~ loading Plato directly and not though k/fmon because I don't want to load Nickel if I'm already using a different reader.

Upgrade/Backup Included SD Card

While the included 8GB microSD card is decent for storing your ebook library that may not have a lot of images, that would likely not be enough if you were aiming to read some comics on your ereader as they can be pretty big (quite a few of mine are over a gigabyte, with some over. Luckily, you can replace the microSD card with another one.

Before upgrading, you should backup the SD card to into an image file so the filesystem would be preserved when putting the contents of the image on the new SD card. I'm using the command dd but there might be another program doing the same thing. Even if you're not going to upgrade, I still suggest to backup the SD card in case something goes wrong.

dd if=/dev/sdf of=kobo_sd.img conv=sync

After this is done, you can plug in your new SD card and reimage kobo_sd.img onto it. With dd, you can do something like:

dd if=kobo_sd.img of=/dev/sdf conv=sync

Checking it's partition table via lsblk or fdisk -l should show three partitions. If you replaced the SD card with something bigger, than you should resize the third partition.

Bypassing Registration On Setup

When setting up your Kobo, you will be asked to sign into a Kobo account. There are other options like logging in via Google, Walmart, and other stores, but I don't like having to login to a device that would likely not be connected to the public internet. Fortunately, you can bypass this by choosing that you cannot connect to a Wi-Fi network and mount your Kobo to your computer. In, .kobo/KoboReader.sqlite, you can run:

echo "INSERT INTO user(UserID,UserKey) VALUES('1','');" \
     | sqlite3 KoboReader.sqlite

This way you don't have to install their application just to be able to use your device.

Note: Do not try doing this when you still have your SD card mounted before you setup your device. The device's screen would likely not update, at least on an early firmware version like v4.7.10733.

Blocking Google Analytics and other Telemetry

Just adding 0.0.0.0 analytics.google.com to /etc/hosts may be enough to block most of the telemetry from being sent. However, you can try intercepting what connections your Kobo is making via mitmproxy set to transparent mode or using a hosts file that blocks all connections to Google (but not necessarily to Kobo's servers) like Baobab's hosts file (raw). EDIT 2022-10-21: Baobab has deleted his account from Codeberg for quite a while, so these two links are dead. Instead, I now recommend Steven Black's instead (raw).

To put the hosts file without root (which will be detailed in another section), you can make a directory called etc, put the hosts file in there, and tar it into a file called KoboRoot.tgz.

mkdir etc
wget -O etc/hosts https://codeberg.org/baobab/hosts/raw/branch/master/hosts
tar czvf KoboRoot.tgz etc
cp KoboRoot.tgz /mnt/kobo/.kobo/

When you move a tar file with that name into your Kobo's .kobo folder, it's contents gets untarred into it's root at / when the device is turned on again, which is usually done for their updates but can be used for custom files like this and gaining root access.

Gaining Root Access via Telnet

To gain root access, we first need to get the /etc/inittab and /etc/inetd.conf which you can get from mounting the SD card's first partition into your computer (the second partition seems to be like a backup). You should copy those two files into a folder called etc somewhere (probably not on the SD card).

In the etc/inittab file, you should add these two lines:

::sysinit:/etc/custominit.sh
::respawn:/usr/sbin/inetd -f /etc/inetd2.conf

You would want to rename the etc/inetd.conf file you copied into etc/inetd2.conf (or whatever the custom inetd.conf's filename is) and when editing that, you should add:

23 stream tcp nowait root /bin/busybox telnetd -i

However, if there is already a commented line for root telnet in the inetd2.conf, you should probably still add the above line and ignore the commented line as that may or may not work (didn't for me).

To actually start inetd, you should add these lines somewhere in /etc/custominit.sh:

mkdir -p /dev/pts
mount -t devpts devpts /dev/pts
/usr/sbin/inetd /etc/inetd2.conf

After that, you just have to tar the etc/ folder again and copy it to your Kobo's onboard/third partition's .kobo folder.

tar czvf KoboRoot.tgz etc
cp KoboRoot.tgz /mnt/kobo/.kobo/

Now you could put your SD card back into your Kobo provided that they are already unmounted and turn your Kobo back on.

After connecting to the WiFi, simplying telnetting (?) into your Kobo and logging in as root should give you a root shell. :D

telnet $KOBO_IP

By default, root has no password so you should change it with passwd.

Getting SSH and SFTP access via Dropbear

I'm using Dropbear instead of OpenSSH because it's better suited for embedded hardware like the Kobo Clara HD. Obviously we can't copy a binary compiled for amd64 or whatever architecture your compiling computer is running so we would have to cross-compile for our ereader.

Fortunately, we are not required to cross-compile gcc/clang and friends as we can simply download the linaro arm toolchain which has the binaries for gcc and others included. You could get the toolchain here and you should get the release that matches your host's

architecture. After untarring the file, you should also set your PATH variable to the toolchain's bin/ folder so you don't have to manually set the CC and CXX variables when building Dropbear.

wget https://releases.linaro.org/components/toolchain/binaries/latest-7/arm-linux-gnueabihf/gcc-linaro-7.5.0-2019.12-x86_64_arm-linux-gnueabihf.tar.xz
tar xvf gcc-linaro-7.5.0-2019.12-x86_64_arm-linux-gnueabihf.tar.xz
export PATH=$(pwd)/gcc-linaro-7.5.0-2019.12-x86_64_arm-linux-gnueabihf/bin:$PATH

Now you could get the source for Dropbear and cross-compile it. The source can be found on their homepage or github repo.

wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2020.81.tar.bz2
tar xvf dropbear-2020.81.tar.bz2
cd dropbear-2020.81
./configure --enable-static --host=arm-linux-gnueabihf
# MULTI=1 combines the binaries like busybox does and is also smaller in size
make MULTI=1 PROGRAMS="dropbear dropbearkey"

Now you only need to copy the dropbearmulti binary over to your Kobo. What I've done is running python3 -m http.server and downloading the file onto my Kobo but you could also just copy it onto the microSD card.

wget your.computer.ip:8000/dropbearmulti
chmod +x dropbearmulti
mv dropbearmulti /usr/bin
cd /usr/bin
# below are optional but dropbear(key) would be an argument for dropbearmulti
ln -s dropbearmulti dropbear
ln -s dropbearmulti dropbearkey

Now you only need to generate the host keys. My client key is ed25519 so I'm not going to generate the others.

mkdir /etc/dropbear
dropbearkey -t ed25519 -f /etc/dropbear/dropbear_ed25519_host_key
dropbear -F -r /etc/dropbear/dropbear_ed25519_key

Now you could ssh into your Kobo and login as root. Remember to change root's password beforehand though if you haven't already! I suggest copying your public key to your Kobo via ssh-copy-id so you don't have to enter root's password all the time and so password-based logins can be disabled in dropbear.

To start it on boot, you could add the following line to /etc/inetd2.conf:

22 stream tcp nowait root /usr/bin/dropbearmulti dropbear -i -r /etc/dropbear/dropbear_ed25519_key

For some reason, the symlink wasn't resolving for me inetd so I had to call the multi-binary directly. You could also add the command/args into /etc/custominit.sh.

FTP Access

If you don't or can't use sftp or scp for some reason, there's always ftp :D There's a ftp daemon included in busybox so all we have to do is enable it in /etc/inetd2.conf:

21 stream tcp nowait root /bin/busybox ftpd -w -S /

This would share the entire filesystem so you may or may not want to restrict the shared directory to maybe just your ebook directory (/mnt/onboard) and move the files out via telnet or ssh. EDIT 2022-10-21: A chroot would also work.

References and Other Links

VFIO Install Notes

2020-10-17T12:00:00Z

You should first go look at the Arch Wiki on it or Yuri Alek's guide on Single GPU passthrough or 4chan's /g/ wiki on it as these assume prior knowledge.

Prerequisites

UEFI Options

Enable VT-d and VT-x (or AMD equivalent)

Kernel Config

Enable KVM and VFIO

you can set VFIO as builtin but as a module is more flexible Also add "iommu=pt intel_iommu=on" to your kernel command line (or in CONFIG_CMDLINE)

Current Options

...
CONFIG_IOMMU_IOVA=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
CONFIG_IOMMU_DEFAULT_PASSTHROUGH=y
# use the respective AMD options if using an AMD CPU
CONFIG_INTEL_IOMMU=y
CONFIG_INTEL_IOMMU_SVM=y
CONFIG_INTEL_IOMMU_DEFAULT_ON=y
CONFIG_INTEL_IOMMU_FLOPPY_WA=y

CONFIG_KVM_VFIO=y
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
...

Packages Required

app-emulation/qemu (actual program)
sys-firmware/edk2-ovmf (UEFI firmware for Nvidia GPU)
media-sound/scream (audio)
looking-glass-client (compile from source if no package, or make your own)

app-emulation/libvirt can be used as well for easier configuration and autostart but I have had problems with it:

Service not starting properly, workaround is restarting service after it starts (Gentoo)
Networks and domains not autostarting, workaround is starting them manually (CRUX)

Gentoo USE Flags

app-emulation/qemu gtk opengl sdl sdl-image usb # (spice, ssh, vhost-user-fs, virgl, and virtfs are optional I think)
media-libs/libsdl2 X gles opengl # for Looking Glass

note to self (2020-10-17): check how minimal you can make qemu to run vfio

IOMMU

Run dmesg | grep -E 'DMAR' and see if DMAR: IOMMU enabled or something similar is in output

QEMU Script

All code blocks in this section go in the qemu script file unless specified otherwise

Environment Variables

IMG=/path/to/windows-image-file
VIRTIO=/path/to/virtio-iso
WINDOWS=/path/to/windows-install-iso
OVMF=/usr/share/edk2-ovmf/OVMF_CODE.fd
RAM=16G
ULIMIT=$(ulimit -l)
ULIMIT_TARGET=$(( $(echo $RAM | tr -d 'G')*1048576+100000 ))

GPU_VIDEO=01:00.0
GPU_AUDIO=01:00.1
VIDEOID="10de 13c0"
AUDIOID="10de 0fbb"
VIDEOBUSID="0000:${GPU_VIDEO}"
AUDIOBUSID="0000:${GPU_AUDIO}"

VFIO Detaching and Attaching

vfio_on() {
    # for nvidia card with proprietary drivers
    rmmod nvidia_drm
    rmmod nvidia_modeset
    rmmod nvidia

    # disable bumblebee service or use bbswitch to detach card if using bumblebee

    modprobe vfio-pci

    echo $VIDEOID > /sys/bus/pci/drivers/vfio-pci/new_id
    echo $VIDEOBUSID > /sys/bus/pci/devices/$VIDEOBUSID/driver/unbind
    echo $VIDEOBUSID > /sys/bus/pci/drivers/vfio-pci/bind
    echo $VIDEOID > /sys/bus/pci/drivers/vfio-pci/remove_id

    echo $AUDIOID > /sys/bus/pci/drivers/vfio-pci/new_id
    echo $AUDIOBUSID > /sys/bus/pci/devices/$AUDIOBUSID/driver/unbind
    echo $AUDIOBUSID > /sys/bus/pci/drivers/vfio-pci/bind
    echo $AUDIOID > /sys/bus/pci/drivers/vfio-pci/remove_id

    # add rest of gpu devices if they are in the same group (I think 4 devices in 1000 or 2000 series nvidia)
}

vfio_off() {
    rmmod vfio_iommu_type1
    rmmod vfio_pci
    rmmod vfio_virqfd
    rmmod vfio

    modprobe nvidia
}

Networking

net_on() {
    ip tuntap add dev tap0 mode tap group kvm
    ip link set dev tap0 up promisc on
    ip addr add 0.0.0.0 dev tap0

    ip link add br0 type bridge
    ip link set br0 up
    ip link set tap0 master br0
    echo 0 > /sys/class/net/br0/bridge/stp_state
    ip addr add 192.168.123.1/24 dev br0

    sysctl net.ipv4.conf.tap0.proxy_arp=1 > /dev/null
    sysctl net.ipv4.conf.enp0s31f6.proxy_arp=1 > /dev/null
    sysctl net.ipv4.ip_forward=1 > /dev/null

    iptables -t nat -A POSTROUTING -o enp0s31f6 -j MASQUERADE > /dev/null
    iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT > /dev/null
    iptables -A FORWARD -i br0 -o enp0s31f6 -j ACCEPT > /dev/null
}

net_off() {
    sysctl net.ipv4.conf.tap0.proxy_arp=0 > /dev/null
    sysctl net.ipv4.conf.enp0s31f6.proxy_arp=0 > /dev/null
    sysctl net.ipv4.ip_forward=0 > /dev/null

    ip link set dev br0 down
    ip link del br0

    ip link set dev tap0 down
    ip tuntap del mode tap name tap0
}

Also add this to /etc/conf.d/net if using Gentoo (source)

replace enp0s31f6 with the host/master interface

...
tuntap_tap0="tap"
config_tap0="null"
bridge_br0="enp0s31f6 tap0"

config_br0="192.168.123.2 netmask 255.255.255.0"
routes_br0="default via 192.168.123.1"
bridge_forward_delay_br0=0
bridge_hello_time_br0=10

depend_br0() {
    need net.enp0s31f6
    need net.tap0
}
...

Hugepages

hugepages_on() {
    PAGES=$(( $(echo $RAM | tr -d 'G') * 1048576 / 2048))
    mkdir -p /dev/hugepages
    mount -t hugetlbfs hugetlbfs /dev/hugepages
    echo $PAGES > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
}

hugepages_off() {
    echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
    umount /dev/hugepages
}

QEMU Command

Before installing guest OS (Windows 10 used as example)

ulimit -l $ULIMIT_TARGET

qemu-system-x86_64 \
    -name 'vfio-vm' \
    -vga qxl \
    -nodefaults -enable-kvm -machine q35 \
    -m $RAM -mem-path /dev/hugepages \
    -cpu host,kvm=off,svm=off,topoext,hv_relaxed,hv_spinlocks=0x1fff,hv_time,hv_vapic,hv_vendor_id=novideobad43,hv_vpindex,hv_synic,hv_stimer,hv_frequencies \
    -smp 8,sockets=1,cores=4,threads=2 \
    -rtc clock=host,base=localtime \
    -boot menu=on -boot d \
    -nic tap,ifname=tap0,script=no,downscript=0,model=virtio-net-pci \
    -drive if=pflash,format=raw,readonly,file=$OVMF \
    -drive file="$VIRTIO",id=cd1,media=cdrom \
    -drive file="$WINDOWS",id=cd2,media=cdrom \
    -device virtio-scsi-pci,id=scsi0 \
    -device scsi-hd,bus=scsi0.0,drive=rootfs \
    -drive file="$IMG",id=rootfs,index=0,format=qcow2,media=disk,if=none

ulimit -l $ULIMIT

After installing guest OS

ulimit -l $ULIMIT_TARGET

qemu-system-x86_64 \
    -name 'vfio-vm' \
    -vga none -nographic \
    -nodefaults -enable-kvm -machine q35 \
    -m $RAM -mem-path /dev/hugepages \
    -cpu host,kvm=off,svm=off,topoext,hv_relaxed,hv_spinlocks=0x1fff,hv_time,hv_vapic,hv_vendor_id=novideobad43,hv_vpindex,hv_synic,hv_stimer,hv_frequencies \
    -smp 8,sockets=1,cores=4,threads=2 \
    -rtc clock=host,base=localtime \
    -boot menu=on -boot c \
    -nic tap,ifname=tap0,script=no,downscript=0,model=virtio-net-pci \
    -device vfio-pci,host=$GPU_VIDEO,multifunction=on,x-vga=on \
    -device vfio-pci,host=$GPU_AUDIO \
    -device ivshmem-plain,memdev=ivshmem,bus=pcie.0 \
    -object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M \
    -device virtio-keyboard-pci \
    -device virtio-mouse-pci \
    -object input-linux,id=kbd0,evdev=/dev/input/by-id/usb-Corsair_Corsair_K70R_Gaming_Keyboard-if02-event-kbd,grab_all=on,repeat=on \
    -object input-linux,id=mouse0,evdev=/dev/input/by-id/usb-Logitech_Gaming_Mouse_G502_0E5F335C3236-event-mouse \
    -object input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Logitech_Gaming_Mouse_G502_0E5F335C3236-if01-event-kbd,grab_all=on,repeat=on \
    -drive if=pflash,format=raw,readonly,file=$OVMF \
    -drive file="$VIRTIO",id=cd1,media=cdrom \
    -device virtio-scsi-pci,id=scsi0 \
    -device scsi-hd,bus=scsi0.0,drive=rootfs \
    -drive file="$IMG",id=rootfs,index=0,format=qcow2,media=disk,if=none

ulimit -l $ULIMIT

Extra

Adding USB Devices

Get vendor and product id from lsusb and add them to your QEMU command arguments:

    -device qemu-xhci,id=xhci0 -device usb-host,bus=xhci0.0,vendorid=0x<yourvendorid>,productid=0x<yourproductid>

Example for my USB bluetooth receiver:

$ lsusb
...
Bus 001 Device 004: ID 0b05:17cb ASUSTek Computer, Inc. Broadcom BCM20702A0 Bluetooth
...

My vendorid is 0x0b05 and productid is 0x17cb, so in QEMU it would be:

    -device qemu-xhci,id=<usb-bus-id> -device usb-host,bus=<usb-bus-id>.0,vendorid=0x0b05,productid=0x17cb

Set CPU Affinity

While libvirt makes this more simple, it appears we need a script/function to do it in bare QEMU Borrowed from here

note: uses bash-isms so that's why I put it in a separate file

#!/bin/bash
THREAD_LIST="0,4,1,5,2,6,3,7"
NAME="vfio-vm"

sleep 20 &&
HOST_THREAD=0
# for each vCPU thread PID
for PID in $(pstree -pa $(pstree -pa $(pidof qemu-system-x86_64) | grep $NAME | awk -F',' '{print $2}' | awk '{print $1}') | grep CPU |  pstree -pa $(pstree -pa $(pidof qemu-system-x86_64) | grep $NAME | cut -d',' -f2 | cut -d' ' -f1) | grep CPU | sort | awk -F',' '{print $2}')
do
    let HOST_THREAD+=1
    # set each vCPU thread PID to next host CPU thread in THREAD_LIST
    echo "taskset -pc $(echo $THREAD_LIST | cut -d',' -f$HOST_THREAD) $PID" | sh
done

Additional Disk

You can add another disk by simply copying the arguments for adding the rootfs and slightly modifying Example for a qcow2 image:

    -device virtio-scsi-pci,id=<scsi-id> \
    -device scsi-hd,bus=<scsi-id>.0,drive=<drive-id> \
    -drive file=<location>,id=<drive-id>,index=0,format=qcow2,media=disk,if=none

No Drives During Installation

Make sure virtio driver is loaded:

Click Load Driver
Choose virtio-cd disc > amd64 > w10 and press enter
Load Red Hat Virtio SCSI driver

Looking Glass Not Starting

Make sure no virtual display like QXL is loaded too (-nographic -vga none in QEMU)

JACK Support

To use JACK instead of Scream, you can use these QEMU arguments

-audiodev jack,id=snd0,in.client-name=default,out.client-name=default,in.start-server=off,out.start-server=off,in.exact-name=on,out.exact-name=on,in.connect-ports=system,out.connect-ports=system,in.frequency=48000,out.frequency=48000,timer-period=2048,out.buffer-length=5120 \
-device ich9-intel-hda \
-device hda-output,audiodev=snd0 \

You might need to change the timer-period and buffer-length if experiencing crackling. Also you might have to change the controller (ich9-intel-hda) and codec (hda-output) to something else.

To list controller and codecs, run:

qemu-system-x86_64 -device help | grep hda