Better Image Building

I’ve managed to recently upgrade my image building process for this website, using some of what I have been learning about Nix. For a while while I was first learning to use Nix I thoguht of the language, the packages, and the OS as being one thing. And this did a disservice to both the breadth of what Nix offers as well as the capabilities it offers.

Old Paradigm

So previously with this site I used a fairly straightforward and manual process. I installed Hugo on my local machine with dnf, ran the dev server, and edited my pages. When I pushed the commits into GitHub, I had a Containerfile in the repository that would get built with Docker or Podman, pushing the resulting image to my image repository. I used a multi-stage file, first pulling the Hugo image to build the files and then grabbing the resulting build produts to place them into the Nginx image.

When I first learned how to use NixOS I spun up Gitea, added a flake.nix file that would offer me Hugo locally through direnv, and wrote a shell script that would test building the Containerfile locally to give myself a sanity test. I felt that this was a pretty snappy way to handle it. Now, anywhere I had Nix, I could just check out my code and run the development bits of it.

More Nix

Although I have since moved my self-hosting to Gitlab instead of Gitea, I was operating under the same basic flow. However, recently, I beefed up my use of Nix in a few nifty ways that stray far from just leveraging the pacakages and the OS. Now, the whole cycle of the build process is managed by Nix.

In my flake.nix file, I have added a few enhancements. First, I added devshell. This is not to be confused with the built-in devShells from flakes themselves. While there is nothing in devshell that you cannot do with the built-in tool, there is just so much boiler-plate built in. For one, it gives me a nice listing of all the tools I want to highlight in this development. Secondly, it produces a very nice motd by default to help alert the developer that they are entering the shell (this is helpful for me, as I use direnv to automate activating my shell environments).

The second enhancement I added was using a flake to wrap process-compose. This eliminates the need for me to write a shell script to start up Hugo or remember the exact command to use. Plus, process-compose has a very nice Curses tui to show the status of the applications you are running. While starting up the hugo server is not exactly rocket science, process-compose is capable of far more than just starting one service in a tui. You can start up databases, caching services, and more with a single command and see them all nicely displayed in the tui. The available options are quite extensive. While they might be slight overkill for just this site, the power that can be shown for more complex development work is impressive. Spinning up whole fleets of VMs or other servies, allowing for dependencies and readiness checks. The ability to spin up a reproducible environment for development work with such a tool is very impressive.

Image Building with Nix

I was reading someone else’s blog post a few weeks ago about building OCI container images with Nix and how the images you build are even better than the ones you can build with a Dockerfile or Containerfile. I had my doubts, at first, but as I looked into the subject, it turns out this is actually pretty true. On top of the reproducible ability for packages that are identified with a flake.lock file which you do not get when using a different base for your container, a layered image from Nix creates one layer per package you add to the image. This means that every image I build and deploy with my Nix tools can share the same set of base packages. Yes, I hear you say, if you build a dozen different images from the same base image, all of those resulting images will share that base layer. And that is true. But when you start to install other packages on top of it, unless you install all of the pacakges in the same way and the same order in every separate Containerfile, you will not have the benefit of sharing the layers above that base image. With a Nix built container, this part is already handled by the tooling directly.

So how do I build this container?

Build the site

First, I create a derivation that builds and packages my site’s static version:

site = pkgs.stdenv.mkDerivation {
	name = "Greg's homepage";
	nativeBuildInputs = [ pkgs.hugo ];
	buildPhase = ''
		hugo -v
	'';
	installPhase = ''
		mkdir -p $out
		cp -R public/* public/.* $out
	'';
	src = ./.;
};

This derivation is simple. It uses pkgs.hugo as a native build input, executes hugo against the repository, and copies the resulting files to the $out target. It isn’t important now, but I build this as a package in the flake outputs. I could easily have put it as just local variable, or such. But this example shows easily how to build the local package directly in the included flake.

Create Nginx Config

I still plan to create a static site that is hosted by Nginx in my container. To accomplish this, Nginx will need a very minimal configuration. At least for now. So I add another package to the flake outputs that just creates the Nginx file:

nginx-config = pkgs.writeText "nginx.conf" ''
	user root root;
	worker_processes 10;
	pid /tmp/nginx.pid;
	daemon off;

	error_log stderr;

	events {
	}
	
	http {
		access_log stdout;
		include ${pkgs.nginx}/conf/mime.types;
		server {
			listen 80;
			root ${self'.packages.site};
			autoindex off;
		}
	}
'';

This derivation is even simpler than the previous one. I leverage the pkgs.writeText function to create the configuration file for Nginx. Unlike in the official version of the Nginx container, there is no need here to copy my resulting site files to some path. I can directly pass the path of my site to this package. You can see this happen at the line root ${self'.packages.site}. In this case, the value of self' is just what I use to reference the flake itself. Since site is the derivation that includes the built site products, I can just reference this directly.

Create the Image

Again, this is just a static site, so the resulting image is fairly mundane and straightforward. However, it is telling for how easy it is to build up the resulting packages and derivations into a simple image. I choose to make the container image the default package for this particular flake repo. And I am sure to build a layered image so that I can make the most of caching and reusing shared packages with any other containers I might run. Here is how I build the image:

default = pkgs.dockerTools.buildLayeredImage {
	name = "gregs-homepage";
	tag = "latest";
	contents = [
		self'.packages.site
		(pkgs.buildEnv {
			name = "image";
			paths = with pkgs; [
				dockerTools.fakeNss
				nginx
			];
			pathsToLink = [ "/bin" "/etc" "/var" ];
		})
	];
	fakeRootCommands = ''
		mkdir tmp
		chmod 0777 tmp
	'';
	config = {
		Cmd = [ (lib.getExe pkgs.nginx) "-c" self'.packages.nginx-config "-e" "stderr" ];
		Port = [ 80 ];
	};
};

Here we can see a simple image with Nginx to serve up the base image that is needed. Now, with this added into the flake package output, I can run nix build . and Nix will grab the exact version of Hugo that I use, build my site, and package it into an OCI container image. By default, nix build drops a symlink to the final derivation in the current directory with the name result. Using Podman or Docker, I can import my image with podman load -i result and my image gregs-homepage:latest will be loaded into my local images. I can then tag and push it anywhere I want.

Conclusion

I know there is nothing Earth shattering here. There is not anything that cannot be done manually or with some amount of effort using other tools. What is so nice about Nix is that it wraps up all that manual and boiler plate bits that I would probably never have spent the time to write all for myself and gives it to me in very nice, neat packages.

What do you use to simplify your development, building, and deployment process? So far, Nix is great for development and building, and I use NixOS on all my servers. But I’m still hoping to get even deeper into the Nix world and find or create some good deployment tooling.