Tiny 2″ TFT Composite Video on Raspberry Pi Zero

This weekend I wanted to test composite video on the Raspberry Pi Zero. I had a Raspberry Pi Zero W and a NTSC/PAL (Television) TFT Display – 2.0″ Diagonal from Adafruit.

This display is ridiculously small. It’s quite something to boot up Raspbian with the PIXEL desktop environment on a Pi Zero with this little 2″ display.

Hardware Wire Up

The display only needs 4 connections.

  • Power (I’m using 5v from a Adafruit Power Boost 1000 basic module)
  • Ground (ground connected to pin 14 / ground on Pi Zero and negative on the boost module)
  • Positive (yellow) from TFT display board to TV on the Pi Zero
  • White from the TFT display board to the other pin, next to the TV pin.
connecting composite video on raspberry pi zero with the TV pin

Here’s what everything looked like after connecting the boost module (3.7v to 5v conversion), the battery charge controller, LiPo, and 2″ TFT display.

boost module, battery charge controller, LiPO, and TFT display all connected to the Raspberry Pi Zero

Powering up and Configuration

Power up and enjoy the tiny display outputting the boot up sequence.

The display is meant to run at 320×240, so after booting up I edited /etc/config.txt to set this up along with some overscan tweaks.

sudo nano /etc/config.txt

Set the following in the /boot/config.txt file:

# these overscan settings are what worked well for me
overscan_left=-26
overscan_right=-26
overscan_top=-16
overscan_bottom=-24

framebuffer_width=320
framebuffer_height=240

Although it is really expensive for what it is, the 2.0″ TFT display is great for small electronics projects that call for full display output. It’s simple and easy to connect, and doesn’t take up too much space either.

Cheap Minecraft Server in AWS with Docker and Traefik

minecraft-like figure on the grass

According to the Minecraft Realms plan pricing page, you can get a realms server at around ¬£5.59 per month. You get some nice conveniences there but… I refuse to pay much at all when I can throw some infrastructure together myself in the cloud to create the ultimate cheap Minecraft server.

Considering my Docker instance running Traefik hosts another 3 or 4 of my personal services along with a Minecraft server, then this solution only costs me around £1.50 a month.

I chose to go with a single AWS EC2 instance that runs Docker. Minecraft runs in a container and sits alongside other personal websites and services that I host there too.

I use Traefik to route traffic coming in to this single host for various TCP ports as well as HTTP(s) on different hostnames. This essentially levels up the cost savings even further as I don’t need multiple EC2 instances (one for each service), and I don’t even need to pay for something like an application or network load balancer, as Traefik does this for me.

A Quick Review of Alternatives

There are other alternatives to consider if you’re looking for a cheap Minecraft server, so don’t take this as being the only option. Here is what I’ve used in the past before settling on my current solution:

  • Minecraft on a dedicated cloud VM. If you just want a dedicated Minecraft VM in the cloud, then DigitalOcean is a good, cheap option. You can also get fairly cheap instances Vultr.
  • Running Minecraft on my own personal Raspberry Pi Kubernetes Cluster. I was even able to expose it over the internet for friends to play on by leveraging a Pi device as a dedicated router. I then used port forwarding to get it working through my double NAT setup. The ARM container was a little slow as a server for more than 2 or 3 players on Raspberry Pi hardware though.
  • Minecraft Server on a home PC / Workstation, with port forwarding to allow other players to connect. This is not ideal, especially on Windows machines or systems that you don’t want to leave running 24/7 as you would for a dedicated server.
  • Various other Minecraft-as-a-service providers. These are decent options in some cases. However for me price and control are important, and I much prefer to self host in this case.

Cheap Minecraft Server in AWS EC2 with Traefik

I used my Cheap Traefik EC2 Docker Hosting solution as the base. You can read that article to get access to the CDK resources required to deploy it yourself.

The cost benefits to using this particular recipe are:

  • EC2 Graviton2 ARM based processor – slightly cheaper to run than Intel and AMD. The downside is more limited software choices. You need to make sure you use ARM compatible packages or Docker images.
  • Spot instance – this has massive savings over a normal lifecycle EC2 instance. The downside is that it can be terminated at any time with only a couple of minutes of notice. When using these you need to make sure you have good data persistence that is not local to the EC2 instance. I personally use a mounted EFS volume. It is re-attached to a new instance from the autoscaling group if the old instance is terminated.

If you don’t use the CDK solution I mentioned above, then alternatively deploy yourself an EC2 instance. Give it an elastic IP address, set up the Security Group ingress rules accordingly, and get shell access. First thing you’ll want to install is Docker, then you’re pretty much good to go.

Minecraft Docker Image

I found a great Minecraft Docker image that is well maintained and has the correct ARM image builds for use on Graviton2 hardware. Check out itzg/minecraft-server. There are other arch builds there that’ll run on just about any other platform.

Docker Compose Service

If you use docker-compose, then here is the simple service definition to get things running.

version: "3"

networks:
  web:
    external: true
  internal:
    external: false

services:
  mc:
    image: itzg/minecraft-server:2021.1.0-multiarch-latest
    environment:
      EULA: "TRUE"
      VERSION: "1.16.5"
      ENABLE_AUTOPAUSE: "TRUE"
      OVERRIDE_SERVER_PROPERTIES: "TRUE"
      MAX_TICK_TIME: "-1"
      TYPE: "BUKKIT"
    labels:
      - traefik.tcp.routers.mc.rule=HostSNI(`*`)
      - traefik.port=25565
    networks:
      - web
    volumes:
      - /data/mc:/data

The docker-compose definition will run a Docker container using the latest multiarch image (which will run on ARM devices). When starting, the container will prepare and run a Minecraft 1.16.5 server. It will also use Bukkit and enable auto pause. The game server does not tick over when there are no players connected.

Traefik Configuration

In the docker-compose definition above, you might have noticed the container labels. The labels prefixed with traefik are used to inform Traefik of how to route network traffic.

the cheap minecraft server uses a Traefik TCP router with HostSNI *
The TCP router using HostSNI on *

In our case, TCP connections are required on port 25565 and HostSNI is used to route those coming in for * (all hosts). The TCP connections on port 25565 go to Traefik, and based on this rule, directed to the Minecraft container.

There is one limitation to be aware of here, and that is that you can only use HostSNI with * for connections that do not use TLS. This is because Server Name Indication (SNI) is an extension of the TLS protocol.

I don’t believe Minecraft supports TLS in any case though. It just means that you won’t be able to have more than one Minecraft server container using the same port running on the single Docker host.

Finishing Off Configuration

Lastly, you might want to point a convenient Host record (A record) to your AWS EC2 Elastic IP address. For example: yourmcserver.example.com -> 1.2.3.4.

All being well, you should now be able to find and connect to your server.

minecraft server listing

WSL2 GUI X-Server Using VcXsrv

wsl2 gui desktop

I needed to set up a WSL2 GUI recently on my machine (WSL2 running uBuntu 20.04.1 LTS). I found a guide that runs through the process but found that a few tweaks needed to be made. Specifically, the communication to VcXsrv was being blocked by Windows Firewall.

There were also a couple of extra tweaks needed for audio passthrough using PulseAudio and setting a windowed resolution.

Setting up a WSL2 GUI X-Server in Windows

Start by installing xfce4 and goodies.

sudo apt install xfce4 xfce4-goodies

If you’re running Kali you should use:

sudo apt install kali-desktop-xfce

During the install you’ll be prompted about which display manager to use. This is up to you, though I personally chose lightdm.

Download this .zip package which contains VcXsrv and PulseAudio along with some configuration and a shortcut to launch.

Extract it to the root of your C:\ drive. You should end up with contents under C:\WSL VcXsrv.

WSL2 GUI vcxsrv package contents

Run the vcxsrv-64.1.20.8.1.installer.exe installer in this folder, choosing defaults for the install.

Once installed, you’ll want to enable High DPI scaling for VcXsrv in Windows.

  • Navigate to C:\Program Files\VcXsrv
  • Right-click xlaunch.exe and go to Compatibility
  • Click Change high DPI settings and choose Override high DPI scaling behavior. Ensure Application is in the dropdown.

Next, edit the startWSLVcXsrv.bat batch file and change the last line that reads ubuntu.exe run to one of:

  • ubuntu2004.exe run in the case you are using uBuntu 20.04 from the Microsoft Store for WSL
  • ubuntu1804.exe run if you are using uBuntu 18.04 from the Microsoft Store for WSL
  • ubuntu.exe run for when you are using standard uBuntu from the Microsoft Store for WSL
  • kali.exe run if you installed Kali-Linux from the Microsoft Store for WSL

Pin the WSL VcXsrv shortcut somewhere convenient like the taskbar.

Opening Windows Firewall for VcXsrv and PulseAudio

Next you need to allow inbound traffic to Windows for VcXsrv and PulseAudio.

Open Windows Defender Firewall with Advanced Security and add two new Inbound Rules as follows:

  • Type: Program
  • Program path: %ProgramFiles%\VcXsrv\vcxsrv.exe for VcXsrv and %SystemDrive%\WSL VcXsrv\pulseaudio-1.1\bin\pulseaudio.exe for PulseAudio
  • Allow the connection
  • Profile: Domain, Private
  • Name: vcxsrv or pulseaudio depending which rule you are adding

I personally added the following to ExtraParams under the XLaunch node of config.xlaunch. This sets windowed mode to 1920×1080 for monitor #1 on my machine.

-screen 0 1920x1080@1

Viewing your WSL2 GUI

With all of that setup out of the way, you should be able to simply launch VcXsrv from the pinned shortcut and everything should work.

Try it out and you should get Desktop up and running for your WSL2 environment.

WSL2 gui example with audio settings open

PulseAudio passthrough should also be available if you check your sound / volume settings. Try an audio test using alsa-utils:

sudo apt install alsa-utils
speaker-test

Kudos to this guide on reddit for most of the setup instructions. As mentioned before, I needed to configure my firewall and also added some tweaks for windowed mode.

Troubleshooting

If you find your VcXsrv Server display window is blank when launching, try the following:

  • Double-check your firewall rule is allowing inbound connections for vcxsrv.exe for the domain and private scopes.
  • With the black X-server / display window from VcXsrv still open, launch a WSL shell separately, and run the following to set your DISPLAY environment variable:
export DISPLAY=$(grep -m 1 nameserver /etc/resolv.conf | awk '{print $2}'):0

This takes the IP address of your host machine (conveniently used as a nameserver in your WSL Linux environment for DNS lookups) and sets it as the Display remote location (with :0 for the display number appended).

Now, try to launch a xfce4 session with:

xfce4-session

If all goes to plan, the session should target your machine where VcXsrv Server is running and your display window should come to life with your WSL environment desktop.

ZFS ARC Assisting Frequent, Random IO from Web Traffic

ZFS includes some great features that help deliver lightning fast read speeds for storage. Two such features are Adapative Replacement Cache (ARC), and a second possible cache tier, L2ARC. Together these features can dramatically decrease the number of reads required from magnetic storage.

Usually ZFS will assign all but 1GB of system memory to ARC. When files are retrieved they are cached in system RAM dedicated to ARC. This allows them to be retrieved very quickly the next time they are read.

A second tier of cache named L2ARC is also possible. This is where you can dedicate solid state disks to act as another tier of cache. L2ARC only really makes sense when you are fronting slower magnetic spinning disks with cache, and you won’t have enough RAM to support a large enough ARC either. L2ARC doesn’t make a lot of sense if you are already running storage pools across solid state drives.

Another point to consider is that if your requirements mandate a large ARC size, system memory will start to get very expensive. If you are using magnetic storage tiers, then using SSD for L2ARC becomes a much cheaper option than simply dumping piles of money into more memory.

My personal FreeNAS server has 16GB of RAM, with much of that dedicated to ARC.

Analyzing ZFS ARC hit rates for Web Traffic to this Blog

I’ve been running this blog on my personal Kubernetes Raspberry Pi Cluster, which has its storage provisioned via NFS from my FreeNAS storage server.

The storage and cache breakdown is:

  • 4 x 3TB SATA spinning disks, RAIDZ1
  • 2 x 250GB SSD SATA disks, ZFS Mirror
  • 16GB RAM, around 4GB used for jails and VMs, the remaining for ARC
  • No L2ARC. NFS storage is provided from the SSD based storage pool

This blog has a bunch of static files along with it’s WordPress installation, plus a MySQL database. Both of these sets of files are provisioned from NFS on the ZFS mirror storage pool, utilizing Kubernetes PVs.

The SSD storage is already quite fast, but the cache features of ZFS really help here when serving frequent random IO generated from web traffic. Just take a look at these two charts:

ZFS ARC hit ratio with ARC requests chart.
ARC Hit Ratio and ARC Requests. Notice the really high (almost 100%) ARC hit ratio.

Instead of constantly reading from SSDs and causing unecessary wear, web traffic file requests are mostly served from ARC.

From these two graphs, I can tell that right now 16GB of system memory is perfectly fine for my home storage and web traffic serving needs. ARC is handling these workloads perfectly, with a very high cache hit ratio too.

Secondly, I certainly don’t need L2ARC. Web content is already sitting on SSD storage. Additionally, even if files were on magnetic storage, the main ARC would still be able to serve almost all web traffic.

Conclusion

ZFS ARC impresses all around. On it’s busiest day of web traffic last year, this blog saw 12000 sessions, and around 13500 page views in just a few hours. ZFS ARC happily served the site’s storage needs from system memory.

In fact, the ARC hit ratio actually increased to just over 99% at this point in time!

ZFS ARC hit ratio above 99%
Over 99% ARC hit ratio

Puck.js Duplo Block Police Siren Build

soldering iron, pexels

I picked up a Puck.js a while ago and after trying out a few basic bits of code, sadly let it start to gather dust on my shelf. That changed this weekend as I browsed the sample project listings for something simple to build, picking up the Puck.js Duplo police siren build to try.

It should be a fun toy for my youngest to play with, as he really enjoys Duplo.

What is the Puck.js?

puck.js duplo build start - the puck.js out of it's case.
The Puck.js and it’s button case.

The Puck.js is an open source JavaScript microcontroller. It has a variety of features such as:

  • Button
  • Magnetometer
  • Accelerometer
  • Gyro
  • IR & RGB LEDs
  • Temperature and light sensor
  • FET output
  • Programmable NFC tag
  • 9 IO pins

The best part about the Puck.js for me is how accessible it is to run and deploy code to. Using Web Bluetooth and it’s included puck.js library, you can write code in a Web IDE, connect over Bluetooth, and have your code running in seconds.

Building the Puck.js Duplo Police Siren Project

If you want to try it out yourself, the actual tutorial page itself is the best resource to begin with. There is a video available there to follow along with.

Here is the Thingiverse page where you can get the model. It currently only lists a scad version of the file, so you’ll need to download OpenSCAD and open it there.

Here is a gist for the file including the cut-out operation that subtracts the innards from the block to make room for the puck.js and piezo to fit into.

I printed the block using my Elegoo Mars resin 3D printer. My first go seems to be slightly loose fitting, so I might shrink the model to 99% size and try again for a second iteration.

I used a new resin that is meant to be easily rinsed/washed after printing with water. The quality on the top of the block doesn’t look as good as usual, so I’m not sure if this resin is to blame for that or not.

puck.js duplo block 3D print

Connecting the Components

The LEDs connect to D1/D2 and D30/D31. The piezo goes on D28/D29. I used RGB LEDs, so I snipped the other legs off, leaving just the blue and common cathode terminals to connect up.

After a bit of dodgy soldering, it works!

Installing Everything Into the Block

With a little bit of coercion, the whole lot fits in. I used a bit of hot glue on four edges of the Puck.js to keep it in place, but keep it easy to remove if needed.

After that, I added a layer of scrap paper with the piezo in-between, and glued that in too.

Here is the final result.

The Puck.js may be fairly pricey, but it includes a lot of IO. It’s battery and use of Bluetooth LE make it ideal for projects where battery life is a concern. The battery is super cheap and can last up to around a year if used carefully.

I had fun making this project. It’s a great project to get started with the Puck.js, and hopefully I’ll find some more use cases soon where I can use more of these great little devices.

Quick and Easy ffmpeg Cheat Sheet

mixing board

This post contains my ffmpeg cheat sheet.

ffmpeg is a very useful utility to have installed for common audio and video conversion and modification tasks. I find myself using it frequently for both personal and work use.

Got an audio or video file to compress or convert? Or maybe you’re looking to split off a section of a video file. Did you do a screen recording of a meeting session and find you need to share it around, but it’s way too large in filesize?

ffmpeg is the clean and easy command line approach. I’ve put together a ffmpeg cheat sheet below that has some quick and easy usages documented.

Skip ahead to the ffmpeg Cheat Sheet section below if you already have it installed and ready to go.

Installing ffmpeg (macOS / Windows)

On macOS, you’re best off using homebrew. If you don’t have homebrew already, go get that done first. Then:

macOS

brew install ffmpeg

Windows

You can grab a release build in 7-zip archive format from here (recent as of the publish date of this post). Be sure to check the main ffmpeg downloads page for newer builds if you’re reading this page way in the future.

If you don’t have 7-zip, download and install that first to decompress the downloaded ffmpeg release archive.

Now you’ll ideally want to update your user PATH to include the path that you’ve extracted 7-zip to. Make sure you’ve moved it to a convenient location. For example on my Windows machine, I keep ffmpeg in D:\Tools\ffmpeg.

Open PowerShell (Windows key + R), type powershell, hit enter.

To ensure that the path persists whenever you run powershell in the future, in powershell, run:

notepad $profile

This will load a start-up profile for powershell in notepad. If it doesn’t exist yet, it’ll prompt you to create a new file. Choose yes.

In your profile notepad window, enter the following, replacing D:\Tools\ffmpeg with the path you extracted ffmpeg to on your own machine.

[Environment]::SetEnvironmentVariable("PATH", "$Env:PATH;D:\tools\ffmpeg")

Close Notepad and save the changes. Close Powershell, then launch it again. This time if you type ffmpeg in the powershell window you’ll run it, no matter which directory you’re in.

ffmpeg Cheat Sheet

This is a list of my 10 most useful ffmpeg commands for conversion and modification of video and audio files.

Task DescriptionCommand
Convert a video from MP4 to AVI formatffmpeg -i inputfile.mp4 outputfile.avi
Trim a video file (without re-encoding)ffmpeg -ss START_SECONDS -i input.mp4 -t DURATION_SECONDS -c copy output.mp4
Convert WAV audio file to compressed MP3 formatffmpeg -i input.wave -acodec libmp3lame output.mp3
Mux video from input1.mp4 with audio from input2.mp4 ffmpeg -i input1.mp4 -i input2.mp4 -c copy -map 0:0 -map 1:1 -shortest output.mp4
Resize or scale video from current size to 1280×720.ffmpeg -i input.mp4 -s 1280x720 -c:a copy output.mp4
Extract audio to MP3 from a video fileffmpeg -i inputvideo.mp4 -vn -acodec libmp3lame outputaudio.mp3
Add watermark or logo to the top-left of a video. (Change overlay parameter for different positions).ffmpeg -i inputvideo.mp4 -i logo.png -filter_complex "overlay=5:5" -codec:a copy outputvideo.mp4
Covert video to GIFffmpeg -i inputvideo.mp4 output.gif
Change the frame rate of a videoffmpeg -i inputvideo.mp4 -filter:v 'fps=fps=15' -codec:a copy outputvideo.mp4
ffmpeg cheat sheet example - logo overlay.
Yes, that is a banana logo overlayed onto this video using ffmpeg. What of it?

This list doesn’t even scratch the surface of the capabilities of ffmpeg. If you dig deeper you’ll find commands and parameters for just every audio and video modification process you need.

Remember, while it’s often easy to find a free conversion tool online, there’ll always be a catch or risk of using these. Whether it’s being subjected to unecessary advertising, catching potential malware, or being tracked with 3rd party cookies, you always take risks using free online tools. I guarantee almost every single free conversion website is using ffmpeg on their backend.

So do yourself a favour and practice and using CLI tools to do things yourself.