Supported hardware

For development, the GL.iNet GL-MT300A is an attractive choice as it has a builtin “debrick” procedure in the boot monitor and is also comparatively simple to attach serial cables to (soldering not required), so it is lower-risk than some devices.

For a more powerful device, something with an ath10k would be the safe bet, or the Linksys E8450 which seems popular in the openwrt community.

Belkin RT-3200 / Linksys E8450

This device is based on a 64 bit Mediatek MT7622 ARM platform, and is mostly feature-complete in Liminix but as of Dec 2024 has seen very little actual use.

Hardware summary

  • MediaTek MT7622BV (1350MHz)

  • 128MB NAND flash

  • 512MB RAM

  • b/g/n wireless using MediaTek MT7622BV (MT7615E driver)

  • a/n/ac/ax wireless using MediaTek MT7915E

Installation

Liminix on this device uses the UBI volume management system to perform wear leveling on the flash. This is not set up from the factory, so a one-time step is needed to prepare it before Liminix can be installed.

Preparation

To prepare the device for Liminix you first need to use the OpenWrt UBI Installer image to rewrite the flash layout. As of Jan 2025 there are two versions of the installer available: the release version 1.0.2 and the pre-release 1.1.3 and for Liminix you nee the pre-relese. The release version of the installer creates UBI volumes according to an older layout that is not compatible with the Linux 6.6.67 kernel used in Liminix.

You can run the installer in one of two ways: either follow the instructions to do it through the vendor web interface, or you can drop to U-Boot and use TFTP

MT7622> setenv ipaddr 10.0.0.6
MT7622> setenv serverip 10.0.0.1
MT7622> tftpboot 0x42000000  openwrt-mediatek-mt7622-linksys_e8450-ubi-initramfs-recovery-installer.itb
MT7622> bootm 0x42000000

This will write the new flash layout and then boot into a “recovery” OpenWrt installation.

Building/installing Liminix

The default target for this device is outputs.ubimage which makes a ubifs image suitable for use with ubiupdatevol. To write this to the device we use the OpenWrt recovery system installed in the previous step. In this configuration the device assigns itself the IP address 192.168.1.1/24 on its LAN ports and expects the connected computer to have 192.168.1.254

Warning

The ubi0_7 device in these instructions is correct as of Dec 2024 (dangowrt/owrt-ubi-installer commit d79e7928). If you are installing some time later, it is important to check the output from ubinfo -a and make sure you are updating the “liminix” volume and not some other one which had been introduced since I wrote this.

$ nix-build -I liminix-config=./my-configuration.nix  --arg device "import ./devices/belkin-rt3200" -A outputs.default
$ cat result/rootfs | ssh root@192.168.1.1 "cat > /tmp/rootfs"
$ ssh root@192.168.1.1
root@OpenWrt:~# ubimkvol /dev/ubi0 --name=liminix --maxavsize
root@OpenWrt:~# ubinfo -a
[...]
Volume ID:   7 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        851 LEBs (108056576 bytes, 103.0 MiB)
State:       OK
Name:        liminix
Character device major/minor: 250:8
root@OpenWrt:~# ubiupdatevol /dev/ubi0_7 /tmp/rootfs

To make the new system bootable we also need to change some U-Boot variables. boot_production needs to mount the filesystem and boot the FIT image found there, and bootcmd needs to be told _not_ to boot the rescue image if there are records in pstore, because that interferes with config.log.persistent

root@OpenWrt:~# fw_setenv orig_boot_production $(fw_printenv -n boot_production)
root@OpenWrt:~# fw_setenv orig_bootcmd $(fw_printenv -n bootcmd)
root@OpenWrt:~# fw_setenv boot_production 'led $bootled_pwr on ; ubifsmount ubi0:liminix && ubifsload ${loadaddr} boot/fit && bootm ${loadaddr}'
root@OpenWrt:~# fw_setenv bootcmd 'run boot_ubi'

For subsequent Liminix reinstalls, just run the ubiupdatevol command again. You don’t need to repeat the “Preparation” step and in fact should seek to avoid it if possible, as it will reset the erase counters used for write levelling. Using UBI-aware tools is therefore preferred over any kind of “factory” wipe which will reset them.

GL.iNet GL-AR750

Hardware summary

The GL-AR750 “Creta” travel router features:

  • QCA9531 @650Mhz SoC

  • dual band wireless: IEEE 802.11a/b/g/n/ac

  • two 10/100Mbps LAN ports and one WAN

  • 128MB DDR2 RAM

  • 16MB NOR Flash

  • supported in OpenWrt by the “ath79” SoC family

The GL-AR750 has two distinct sets of wifi hardware. The 2.4GHz radio is part of the QCA9531 SoC, i.e. it’s on the same silicon as the CPU, the Ethernet, the USB etc. The device is connected to the host via AHB and it is supported in Linux using the ath9k driver. 5GHz wifi is provided by a QCA9887 PCIe (PCI embedded) WLAN chip, supported by the ath10k driver.

Installation

As with many GL.iNet devices, the stock vendor firmware is a fork of OpenWrt, meaning that the binary created by mtdimage can be flashed using the vendor web UI or the U-Boot emergency “unbrick” routine.

Flashing over an existing Liminix system is not possible while that system is running, otherwise you’ll be overwriting flash partitions while they’re in use - and that might not end well. Configure the system with Adding packages if you need to make it upgradable.

Vendor web page: https://www.gl-inet.com/products/gl-ar750/

OpenWrt web page: https://openwrt.org/toh/gl.inet/gl-ar750

GL.iNet GL-MT300A

The GL-MT300A is based on a MT7620 chipset.

For flashing from U-Boot, the firmware partition is from 0xbc050000 to 0xbcfd0000.

WiFi on this device is provided by the rt2800soc module. It expects firmware to be present in the “factory” MTD partition, so - assuming we want to use the wireless - we need to build MTD support into the kernel even if we’re using TFTP root.

Installation

The stock vendor firmware is a fork of OpenWrt, meaning that the binary created by mtdimage can be flashed using the vendor web UI or the U-Boot emergency “unbrick” routine.

Flashing over an existing Liminix system is not possible while that system is running, otherwise you’ll be overwriting flash partitions while they’re in use - and that might not end well. Configure the system with Adding packages if you need to make it upgradable.

Vendor web page: https://www.gl-inet.com/products/gl-mt300a/

OpenWrt web page: https://openwrt.org/toh/gl.inet/gl-mt300a

GL.iNet GL-MT300N-v2

The GL-MT300N-v2 “Mango” is is very similar to the GL.iNet GL-MT300A, but is based on the MT7628 chipset instead of MT7620. It’s also marginally cheaper and comes in a yellow case not a blue one. Be sure your device is v2 not v1, which is a different animal and has only half as much RAM.

Installation

The stock vendor firmware is a fork of OpenWrt, meaning that the binary created by mtdimage can be flashed using the vendor web UI or the U-Boot emergency “unbrick” routine.

Flashing over an existing Liminix system is not possible while that system is running, otherwise you’ll be overwriting flash partitions while they’re in use - and that might not end well. Configure the system with Adding packages if you need to make it upgradable.

Vendor web page: https://www.gl-inet.com/products/gl-mt300n-v2/

OpenWrt web page: https://openwrt.org/toh/gl.inet/gl-mt300n_v2

OpenWrt One

Hardware summary

  • MediaTek MT7981B (1300MHz)

  • 256MB NAND Flash

  • 1024MB RAM

  • WLan hardware: Mediatek MT7976C

Status

  • Only tested over TFTP so far.

  • WiFi (2.4G and 5G) works.

  • 2.5G ethernet port works.

Limitations

  • adding he_bss_color=”128” causes Invalid argument for hostap

  • nvme support untested

  • I don’t think the front LEDs work yet

Installation

TODO: add instructions on how to boot directly from TFTP to memory and how to install from TFTP to flash without going through OpenWrt.

The instructions below assume you can boot and SSH into OpenWrt:

Boot into OpenWrt and create a ‘liminix’ UBI partition:

root@OpenWrt:~# ubimkvol /dev/ubi0 –name=liminix –maxavsize

Remember the ‘Volume ID’ that was created for this new partition

Build the UBI image and write it to this new partition:

$ nix-build -I liminix-config=./my-configuration.nix –arg device “import ./devices/openwrt-one” -A outputs.default $ cat result/rootfs | ssh root@192.168.1.1 “cat > /tmp/rootfs” $ ssh root@192.168.1.1 root@OpenWrt:~# ubiupdatevol /dev/ubi0_X /tmp/rootfs # replace X with the volume id, if needed check with ubinfo

Reboot into the U-Boot prompt and boot with:

OpenWrt One> ubifsmount ubi0:liminix && ubifsload ${loadaddr} boot/fit && bootm ${loadaddr}’

If this works, reboot into OpenWrt and configure U-Boot to boot ubifs by default:

root@OpenWrt:~# fw_setenv orig_boot_production $(fw_printenv -n boot_production) root@OpenWrt:~# fw_setenv boot_production ‘led white on ; ubifsmount ubi0:liminix && ubifsload ${loadaddr} boot/fit && bootm ${loadaddr}’

Troubleshooting

The instructions above assume you can boot and SSH into the (recovery) OpenWrt installation. If you have broken your device to the point where that is no longer possible, you could re-install OpenWrt, but probably you could also install directly from U-Boot:

QEMU MIPS

This target produces an image for QEMU, the “generic and open source machine emulator and virtualizer”.

MIPS QEMU emulates a “Malta” board, which was an ATX form factor evaluation board made by MIPS Technologies, but mostly in Liminix we use paravirtualized devices (Virtio) instead of emulating hardware.

Building an image for QEMU results in a result/ directory containing run.sh vmlinux, and rootfs files. To invoke the emulator, run run.sh.

The configuration includes two emulated “hardware” ethernet devices and the kernel mac80211_hwsim module to provide an emulated wlan device. To read more about how to connect to this network, refer to Networking in the Development manual.

QEMU Aarch64

This target produces an image for the QEMU “virt” platform using a 64 bit CPU type.

ARM targets differ from MIPS in that the kernel format expected by QEMU is an “Image” (raw binary file) rather than an ELF file, but this is taken care of by run.sh. Check the documentation for the QEMU MIPS target for more information.

QEMU ARM v7

This target produces an image for the QEMU “virt” platform using a 32 bit CPU type.

ARM targets differ from MIPS in that the kernel format expected by QEMU is an “Image” (raw binary file) rather than an ELF file, but this is taken care of by run.sh. Check the documentation for the QEMU MIPS (MIPS) target for more information.

Turris Omnia

This is a 32 bit ARMv7 MVEBU device, which is usually shipped with TurrisOS, an OpenWrt-based system. Rather than reformatting the builtin storage, we install Liminix on to the existing btrfs filesystem so that the vendor snapshot/recovery system continues to work (and provides you an easy rollback if you decide you don’t like Liminix after all).

The install process has two stages, and is intended that you should not need to open the device and add a serial console (although it may be handy for visibility, and in case anything goes wrong). First we build a minimal installation/recovery system, then we reboot into that recovery image to prepare the device for the full target install.

Installation using a USB stick

First, build the image for the USB stick. Review examples/recovery.nix in order to change the default root password (which is secret) and/or the SSH keys, then build it with

$ nix-build -I liminix-config=./examples/recovery.nix \
  --arg device "import ./devices/turris-omnia" \
  -A outputs.mbrimage -o mbrimage
$ file -L mbrimage
mbrimage: DOS/MBR boot sector; partition 1 : ID=0x83, active, start-CHS (0x0,0,5), end-CHS (0x6,130,26), startsector 4, 104602 sectors

Next, copy the image from your build machine to a USB storage medium using dd or your other most favoured file copying tool, which might be a comand something like this:

$ dd if=mbrimage of=/dev/path/to/the/usb/stick \
  bs=1M conv=fdatasync status=progress

The Omnia’s default boot order only checks USB after it has failed to boot from eMMC, which is not ideal for our purpose. Unless you have a serial cable, the easiest way to change this is by booting to TurrisOS and logging in with ssh:

root@turris:/# fw_printenv boot_targets
boot_targets=mmc0 nvme0 scsi0 usb0 pxe dhcp
root@turris:/# fw_setenv boot_targets usb0 mmc0
root@turris:/# fw_printenv boot_targets
boot_targets=usb0 mmc0
root@turris:/# reboot -f

It should now boot into the recovery image. It expects a network cable to be plugged into LAN2 with something on the other end of it that serves DHCP requests. Check your DHCP server logs for a request from a liminix-recovery host and figure out what IP address was assigned.

$ ssh liminix-recovery.lan

You should get a “Busybox” banner and a root prompt. Now you can start preparing the device to install Liminix on it. First we’ll mount the root filesystem and take a snapshot:

# mkdir /dest && mount /dev/mmcblk0p1 /dest
# schnapps -d /dest create "pre liminix"
# schnapps -d /dest list
ERROR: not a valid btrfs filesystem: /
    # | Type      | Size        | Date                      | Description
------+-----------+-------------+---------------------------+------------------------------------
    1 | single    |    16.00KiB | 1970-01-01 00:11:49 +0000 | pre liminix

(not a valid btrfs filesystem: / is not a real error)

then we can remove all the files

# rm -r /dest/@/*

and then it’s ready to install the real Liminix system onto. On your build system, create the Liminix configuration you wish to install: here we’ll use the rotuer example.

build$ nix-build -I liminix-config=./examples/rotuer.nix \
  --arg device "import ./devices/turris-omnia" \
  -A outputs.systemConfiguration

and then use min-copy-closure to copy it to the device.

build$ nix-shell --run \
  "min-copy-closure -r /dest/@  root@liminix-recovery.lan result"

and activate it

build$ ssh root@liminix-recovery.lan \
  "/dest/@/$(readlink result)/bin/install /dest/@"

The final steps are performed directly on the device again: add a symlink so U-Boot can find /boot, then restore the default boot order and reboot into the new configuration.

# cd /dest && ln -s @/boot .
# fw_setenv boot_targets "mmc0 nvme0 scsi0 usb0 pxe dhcp"
# cd / ; umount /dest
# reboot

Installation using a TFTP server and serial console

If you have a U-Boot and serial shenanigans console connection and a TFTP server, and would rather use them than fiddling with USB sticks, the examples/recovery.nix configuration also works using the tftpboot output. So you can do

build$ nix-build -I liminix-config=./examples/recovery.nix \
  --arg device "import ./devices/turris-omnia" \
  -A outputs.tftpboot

and then paste the generated result/boot.scr into U-Boot, and you will end up with the same system as you would have had after booting from USB. If you don’t have a serial console connection you could probably even get clever with elaborate use of fw_setenv, but that is left as an exercise for the reader.

Zyxel NWA50AX

Zyxel NWA50AX is quite close to the GL-MT300N-v2 “Mango” device, but it is based on the MT7621 chipset instead of the MT7628.

Installation

This device is pretty, but, due to its A/B capabilities, can be a bit hard to use completely.

The stock vendor firmware is a downstream fork of U-Boot: <https://github.com/RaitoBezarius/uboot-nwa50ax> with restricted boot commands. Fortunately, OpenWrt folks figured out trivial command injections, so you can use most of the OpenWrt commands without trouble by just command injecting atns, atna or atnf, e.g. atns “; $real_command”.

From factory web UI, you can upload the result of the zyxel-nwa-fit output. From another operating system, you need to dumpimage -T flat_dt -p 0 $zyxel-nwa-fit -o firmware.bin, flash_erase $(mtd partition of the target partition firmware or zy_firmware) 0 0, then you complete by nandwrite -p $(mtd partition of the target partition firmware or zy_firmware) firmware.bin.

How to put the firmware.bin on the machine is left to you as an exercise, e.g. SSH, TFTP, whatever.

From serial, you have two choices:

  • Flash this system via U-Boot: same reasoning as from an existing Linux system, two choices: - ymodem the binary, perform the write manually, you can inspire yourself from the script contained in the vendor firmware, those are just a FIT containing a script. - prepare a FIT containing a script executing your commands, tftpboot this.

  • boot from an existing Liminix system, e.g. TFTPBOOT image.

  • boot from an OpenWrt system, i.e. follow OpenWrt steps.

Once you are in a Linux system, understand that this device has A/B boot.

OpenWrt provides you with zyxel-bootconfig to set/unset the image status and choice.

The kernel is booted with bootImage=<number> which tells you which slot are you on.

You should find yourself with 10ish MTD partitions, the most interesting ones are two:

  • firmware: 40MB

  • firmware_1: 40MB

In the current setup, they are split further into kernel (8MB) and ubi (32MB).

Once you are done with first installation, note that if you want to use the A/B feature, you need to write a _secondary_ image on the slot B. There is no proper flashing code that will set the being-updated slot to new and boot on it to verify if it’s working. This is a WIP.

Upgrading your system can be achieved via:

  • liminix-rebuild for the userspace.

  • flash_erase + nandwrite for the kernelspace to the other slot than the one you are booted on, note that you can just nandwrite the mtd partition corresponding to the kernel and not the whole firmware.

If you soft-bricked your AP, i.e. you cannot boot anything in U-Boot, no worries, just plug the serial console, prepare a TFTP server (via tufted for example), download vendor firmware, set up atns, atnf, etc. and run atnz.

This will reflash everything back to normal via TFTP.

If you hard-bricked your AP, i.e. U-Boot is telling you to transfer a valid bootloader via ymodem, just extract a U-Boot from the vendor OS, send it via ymodem and use the previous operations to perform a full flash this time of all partitions.

Note that if you erased your MRD partition, you lost your serial and MAC address. There’s no way to recover the original one except by reading the physical label on your… device!

If you super-hard-bricked your AP, i.e. no output on serial console, congratulations, you reached one of the rare state of this device. You need an external NAND flasher to repair it and write the first stage from Mediatek to continue the previous recovery operations.

Development TODO list:

  • Better support for upgrade automation w.r.t. to A/B, e.g. automagic scripts.

  • Mount the logs partition, mount / as overlayfs of firmware ? rootfs and rootfs_data for extended data.

  • Jitter-based entropy injection? Device can be slow to initialize its CRNG and hostapd will reject few clients at the start because of that.

  • Defaults for hostapd based on MT7915 capabilities? See the example for one possible list.

  • Remove primary/secondary hack and put it in preinit.

  • Offer ways to reflash the bootloader itself to support direct boot via UBI and kernel upgrades via filesystem rewrite.

Vendor web page: https://www.zyxel.com/fr/fr/products/wireless/ax1800-wifi-6-dual-radio-nebulaflex-access-point-nwa50ax

OpenWrt web page: https://openwrt.org/inbox/toh/zyxel/nwa50ax OpenWrt tech data: https://openwrt.org/toh/hwdata/zyxel/zyxel_nwa50ax