Taskfile: a modern alternative to Makefile

Introduction

I meant to write this post for a while now, but I never thought many people would read it. I'm writing it now for those few people brave enough to try something new around automation tools.

After being unhappy with Makefile for years now, last year I decided that I had enough and I started looking around for alternatives. None of the projects I found seemed to fit my needs until I discovered Taskfile. After using it successfully for the past year, I'm happy to say that I found a better way to write automation scripts than Makefile and I'm not going back.

In this post, I would like to introduce Taskfile and a few other tools that combined make a nice automation framework. My intention is not for this post to be an exhaustive guide to those tools. People can read the official documentation if they are interested.

I just want to introduce some basic features that I use every day and maybe spark the reader's curiosity. Just use 20% of those tools' features to reap 80% of the benefits. Thanks, Pareto for your principle.

This is just my first post on this topic. I'm planning to write soon a more in-depth one where I describe my current use cases.

Stuck in the past

What I don't get is how we make so much progress in AI, data science, and machine learning, but we are still stuck in the past whenever it comes to automation and developer tools. How can we innovate fast enough in AI (or any other fields) if technology hasn't caught up with our modern needs for developer tools?

As anyone that writes or interacts with software these days, I need to run automation scripts every day but I am still stuck with a tool that (according to Wikipedia) has been written 47 years ago. Don't get me wrong, Makefile was really useful 20 years ago but I think we can do better than that in 2023.

My take on why people are still using Makefile?

Everyone got used to the fact that Makefile is the only alternative available for a standard (not language-specific like Grunt or Gulp) task runner and there is nothing they can do about it.

Well, not everyone. Someone decided that they had enough of Makefile and wrote an alternative in Golang called Taskfile.

My personal experience

Like everyone here, I used to just write makefiles or simple bash scripts and then lengthy Readme in Markdown to explain how to run those scripts or how to install all the required tools necessary.

It was a long and excruciating, but necessary, process not just for other people but mostly for myself. I have a very bad memory, and I tend to forget what was that command argument or the script that I run last week. Things don't get better with age, I had to find a better alternative.

Also, as an engineer, I hate writing documentation:

It is a tedious process.
It gets out of date the minute that you finish writing it.
There are no refactoring tools for a Readme in Markdown that keep your documentation up to date with your code.
Documentation is not executable. I mean there are tools now that allow you to run code from Markdown documentation but they are not a widespread practice.

So What do I do now instead?

Instead of writing documentation as an after-math, if I run a shell command that I think I might use in the future, I add it to my now big list of tasks, make it reusable, and attach a one-line comment to the code, if I need it. My automation framework makes it very easy to pick it up next time when I need to run that command again.

In order to make those automation scripts reusable, I separate the state (the part that might change at each use case) from the reusable code (the part that is fixed every time).

How do I do that? I use a combination of three different tools: Taskfile, Direnv, Devenv.

My documentation is much shorter now, I still embed notes on how to use those scripts but I tend to write them next to the code as comments. The proximity of documentation and code makes the shelf life of my docs a lot longer.

My automation framework

I just want to point out, that this is not a simple setup. In order to make this happen, I am using three different tools and I had to do quite some experimentation. During this time, I bumped into many bugs (some of which are now fixed) and feature missing due to the new nature of those tools and the fact they are not widely used.

Having said that, I see the development of those tools going in the right direction and the number of Github stars rising exponentially. I know, GitHub stars as a metric for adoption is not everything, but that's what we are stuck with.

Also, you don't have to go through the same pain. You can just reap the benefit of my experience.

Furthermore, the learning curve of those tools is quite gentle and you can just start with the basics and add more stuff over time. I'm not even saying that you should adopt my full framework. If you don't feel comfortable you can just start playing with each single tool in isolation.

I like to think that by combining these three somehow small tools I am following the teaching of the Unix Philosophy, which I like to paraphrase to

Combine simple and small tools to create something powerful

And this described here is exactly it, a very powerful framework, that once correctly setup, can allow you to build a complex pipeline of tasks

Taskfile

Until now, I've been calling it Taskfile, but in reality, that's the name of the config file used by the tool Task. Given how generic is the word task, I believe that's a better way to identify the tool. I mean, it is kind of the same story with Make the tool and its format Makefile.

The official definition of Task from taskfile.dev is the following:

Task is a task runner / build tool that aims to be simpler and easier to use than, for example, GNU Make.

Some interesting facts about Task:

Written in Go
Single binary with no other dependencies
Taskfile is just a dialect of YAML format with a specific syntax
8k stars on GitHub at the time of writing

Between the features of this tool we can find:

You can build a pipeline of tasks in parallel or in sequence, call other tasks from other Taskfiles, and have tasks that run as dependencies.
Tasks can be exposed (aka public) or internal (aka private)
You can run a task's cleanup using Go's defer command
Prevent unnecessary work (similar to how Makefile works)
Reference environment variables already defined in the shell environment
Tasks can be templated using Go's template engine
Dry run mode

The list of features goes on and on. More on the official docs.

Here it is a simple Taskfile taken from taskfile.dev

version: '3'

tasks:
  build:
    cmds:
      - go build -v -i main.go

  assets:
    cmds:
      - esbuild --bundle --minify css/index.css > public/bundle.css

Please refer to the official documentation for how to install the tool and to see sample use of those features.

Direnv

The perfect one-liner to describe Direnv can be found in their official documentation direnv.net

direnv – unclutter your .profile

and then:

direnv is an extension for your shell. It augments existing shells with a new feature that can load and unload environment variables depending on the current directory

Some noticeable facts about Direnv:

Single binary
Written in Golang
Integrates with your shell (eg. bash, zsh, fish, and more)
It supports 12 factor apps where you store your configs in environment variables or config files
10k starts on GitHub at the time of writing

I mostly use it to define environment variables on .env files and then load them into my shell automatically when I cd into a directory with a .envrc file.

For example, if I have a .envrc with the following content

dotenv
dotenv ../golang/.env

I will be able to load environment variables both from a .env in the current directory and also from a file at ../golang/.env.

This way of splitting environment variables into different files allows for better reusability over the long term. If you only need Golang in your next project, you can just bring with you golang/.env file.

Devenv

Devenv is probably the most complicated of these three tools but also the most powerful. With "only" 2.2k stars on GitHub it is still in the early stages of its life but it is already very powerful.

The one-liner from the official documentation states

Use a simple unified configuration to configure packages, processes, services, scripts, git hooks, integrations.

Devenv is based on Nix a powerful package manager and system configuration tool, that comes with its own language. I'm not qualified to explain what Nix is, and this is not even the right place for it. Suffice it to say, I don't know how to write Nix code and I didn't need to learn it so far. I'm only using it here via a higher-level abstraction.

Devenv deserves an entire post to describe all its features and use cases. Here I'm only using it to describe dependencies (in the shape of command line tools) that I would like to have installed on my laptop.

{ pkgs, ... }:

{
    packages = [ 
        pkgs.git 
        pkgs.govulncheck
        pkgs.gofumpt 
        pkgs.go-swag
        pkgs.gocyclo
    ];
}

In this example, I use to install a few command line tools that I use to write Golang applications.

This file is enough to tell what dependencies I have in my project, Devenv will make sure those are installed in an isolated and reproducible environment.

A very basic project

It is finally time to put those three projects together and explain how I use them in my framework.

Structure of the project

Taskfile.yml
taskfiles/
- golang.taskfile.yml
- docker.taskfile.yml
envs/
- golang/
  - .env
- docker/
  - .env
  - .envrc
devenv.nix
... (other files omitted for brevity) ...

Here Taskfile.yml is just the entry point to all other taskfiles. Tasks are split into multiple files to achieve a nice separation of concerns:

version: '3'

includes:
  docker: taskfiles/docker.taskfile.yml
  go: taskfiles/go.taskfile.yml

Here we have a golang.taskfile.yml used to build a Golang application

version: '3'

tasks:  
  build:
    cmds:
      - GOOS={{.CMD_GOOS}} GOARCH={{.CMD_GOARCH}} go build -o build/hello cmd/hello.go

and a docker.taskfile.yml used to build a Docker image from the Golang binary and a Dockerfile

version: '3'

includes:
  go: taskfiles/go.taskfile.yml

tasks:  
  build:
    deps:
      - go:build
    cmds:
      - docker build -t {{.DOCKER_IMAGE}}:{{.DOCKER_TAG}} -f Dockerfile .

Here we have a file envs/golang/.env used to define the environment variable for the Golang application

CMD_GOOS=linux
CMD_GOARCH=arm64

and another file envs/docker/.env used instead for Docker environment variables

DOCKER_IMAGE=hello
DOCKER_TAG=v1.0

Furthermore we have a file envs/docker/.envrc that tides together the two .env files and load environment variables from both

dotenv

dotenv ../golang/.env

Finally a file devenv.nix used to install Taskfile and Golang

{ pkgs, ... }:

{
    packages = [ 
        pkgs.go-task
    ];

    languages.go.enable = true;
}

Devenv comes with more files than just devenv.nix. Here we haven't discussed about them since they don't really need human intervention.

In order for this framework to work, there are a couple of assumptions:

Devenv and Direnv needs to be installed
Taskfile will be installed via devenv but alternatively you can install it via brew
We assume that you have already installed Docker on your laptop.
No need to install Golang

If you want to build the docker image you only need to run the following commands

cd envs/docker
task docker:build

Changing the directory will instruct Direnv to load the environment variables from the relative .env file, while the second command will build the docker image after having built the binary from the source code.

It is that simple!

Now you have a reusable framework to build container images for your Golang applications. You can copy the content of these files to your Golang application and just change the content of the .env files without touching anything else.

I know this is a small artificial project, but I hope you can see the power of this framework and expand it for your use case.

Once you spend enough time with this framework, you are going to be able to run automation scripts that can save you hours of manual toil.

Conclusion

I'm sorry for the very long post but I couldn't make it shorter. There was a lot to unpack and I couldn't miss the opportunity to add those memes.

I'll probably write a more elaborate post where I provide some samples of how I am using this automation framework for real use cases.

I hope that you will adopt one of those tools or all of them. If you find any of those useful, please star them on GitHub to drive adoption.

This article was originally published at https://cloudnativeengineer.substack.com/p/ep-5-taskfile-a-modern-alternative

https://cloudnativeengineer.substack.com/p/ep-5-taskfile-a-modern-alternative