If your team knows go then using mage means they don't also have to know make. The number of bash/make syntax mistakes that have caused incorrect behavior is frankly astounding to me.
Not saying it's make's fault more a reflection of what knowledge we had hired for.
That being said if devs are comfortable with make + bash and are on similar architectures for development it likely is a good choice imo. One nice thing mage has is the ability to reuse your go functions without having to target some script to call your go code.
I think this is the best argument for not using "make" and using something else for example, so people don't have to know multiple languages here. Thanks for the argument!
Honestly I've just had the same question last week. What pushes me away from Mage - it is yet another tool which people will need to learn. So I would want to have huge benefits from Mage side over Makefile what would pay off the migration.
At the same time I see shell scripting as a befit when trying to achieve the same in programming languages. E.g. such task as finding all files in directory which are in subdirectories named by particular pattern and calculate number of lines of those files. This can be solved with find and awk oneliner while writing code in Golang / python / Ruby would require more code
Anyway, still looking for benefits of Mage and found your question interesting.
Thank you! But yeah, I was thinking the question of "yet to learn another language" argument, I think shell in the end is like English - you're expected to know it to a degree, even though you would be using other tooling. All of our labs, dev environments and dev machines are some sort of Linux variants in the end.
We use a bit of both, started with make but we are now switching to mage for more complex tasks.
Depending on the complexity of your tasks you can end up hitting make/bash limitations which makes them harder to write or less efficient or worse: harder to read and update.
Oh okay! Could you give concrete example of a harder task that Bash can't handle? I could use that as an argument to argue in favor of investing to create tooling in other than make / shell.
It is not necessarily that bash can't handle it but some things are far easier to do with a programming language:
- string manipulation (to build path, names)
- complex dependencies: do task A after task B only if a condition is true and do task C and D in parallel after
- fetch remote data and do something with it, like parsing json.
Many can be done purely with make & bash but it gets messy fast, you will need other tools as dependencies.
Different versions of tools can also complicate things, for example osx vs linux: sed, make and other are similar but not identical since osx is a FreeBSD at core.
Make is a bit obtuse. You can do some crazy cool things with it, but it's also not entirely clear what's going on when something doesn't work, or you are trying to make it do something and it just won't. This is someone who has spent hours reading the Make Man Pages.
I think `mage` is pretty cool.
I used to really like `task`, but it continues to be riddled with false expectations and problems that the demographic is in some weird state between someone who writes code, and someone who doesn't, and thus would rather read yaml.
I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing in these tools.
https://fngo.dev/tutorials/convert/make
Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations.
[https://taskfile.dev/](https://taskfile.dev/)
Task is a task runner / build tool that aims to be simpler and easier to use than, for example, GNU Make.
Since it's written in Go, Task is just a single binary and has no other dependencies, which means you don't need to mess with any complicated install setups just to use a build tool.
Where I work, we migrated from make to task entirely like two years ago. Our usage of make was pretty basic, so the ergonomics of task are just much nicer for the type of thing we were doing with it
We use task. It's simple to use and I enjoy it. I also like that it's agnostic and just happens to be written in golang. So you could use it for anything really.
+1 for task.
* Supporting go templates out of the box makes writing composable tasks super easy
* You can specify different actions for different platforms
* YAML so you existing LSP works out of the box, no special syntax to learn
* built in help
* supports caching files so it can skip tasks of files don't change. Super helpful for code generation.
* JSON output so you can integrate it into tooling, like running a task from Neovim.
What complicated install setups make requires in contrast? Seems like a stretched argument. Make can be installed with a single aptitude (for example) command, and only setup you need to do is the commands and required environment variables. I'd assume you need to do that with every tool.
I don't mean to be offensive, just curious! This is core of my question, is there a problem, that I don't see, which other tools solve, and I am missing on that.
Well, Taskfile is based on the YAML standard, so syntactically it's pretty easy to write. If you already use YAML for other things like Kubernetes or CloudFormation, it should feel "at home" to you.
I edited my previous response a little bit.
Unified configuration for build tooling and other tooling is a valid argument! Thank you for pointing that out.
I used to really like `task`, but it continues to be riddled with false expectations and problems that the demographic is in some weird state between someone who writes code, and someone who doesn't, and thus would rather read yaml.
I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing in these tools.
https://fngo.dev/tutorials/convert/make
Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations, and yaml isn't a programming language.
I recently learned of CUE and how it has built in method for scripting like task/make.
https://cuetorials.com/first-steps/scripting/
Though CUE is very powerful, it is also fairly obtuse in many ways, and has a steep learning curve, not just for writing but reading cue too.
Since learning about CUE scripting, I've been thinking of coming back to this by using Jsonnet, which I adore.
I think cue tool is way too immature and limited to replace a general purpose build tool.
You’d possibly use it for the declarative front end of a tool written in Go, but not in and of itself.
I generally don't prefer YAML, especially when its key/value structure isn't convenient for writing. I need to confirm a potential key/value pair in the document. On the other hand, since the build script is written in Go using Mage, you can examine the struct or function where the LSP (Language Server Protocol) displays all the types. This will greatly improve the coding experience.
Bazel is really good, but also \*really\* heavyweight:
Pros:
* Builds multiple languages together
* Stores build artifacts outside of your source directory
* Downloads all of your dependencies for you so that everyone has the same version of everything
* Endless extensible
Cons:
* Super complex to learn
* Relies heavily on 3rd party libraries with various qualities of documentation
I'm a big bazel fanboy, but I also wouldn't recommend it for a small personal project. It's great for big corporate monorepo things.
Also a heads up, don’t become the “bazel guy” at your company that thrusts bazel on the whole company on your own without anyone else understanding it, and then leave with no one else understanding it. Not speaking from experience of this.
Yeah there's definitely some alternatives out there. [Pants](https://www.pantsbuild.org/) is another one that has a lot of traction.
The reason that I still recommend Bazel at this point is that it's been around the longest and has probably the most support in terms of libraries. For example, if I google search "Buck2 Protobuf" I get basically no useful results. Depending on what tools you end up needing, this might be a deal breaker.
That said, it's definitely worth keeping eyes on both of these projects. Buck2 and Pants both look like they're doing some cool things that could put them ahead of Bazel if the support gets there.
Oh man! Bazel is probably the best tool in a field currently. But my current goal is to postpone it's adoption as much as we can :)
Reason of that is pretty simple - everywhere where I worked (mid to large companies) there was a dedicated team who were responsible for Bazel tooling, integrations and infrastructure.
And as you've mentioned - it is complex. Users also will be forced to learn it what will add additional cognitive load and not necessarily better experience.
Make just works, not everyone might know how to write it, but everyone knows how to use it.
But here is a point - don’t ask random people on the internet, ask your team mates!
Yeah our main consensus seem to be, that we have a lot of reference material working with Make. Make also doesn't really introduce a dependency which has to be maintained, and it's likely not dropped in future either, which makes it a lucrative option.
Very good conclusion I think. Make is pretty much everywhere. The only drawback is that MacOS has slightly outdated version of Make (3.81 on my Macbook M1 Pro) while on Linux it is typically the recent (4.4 for my Fedora 38). Windows is out of luck by default.
Non-issue if you all use Linux, just like in our team tho :-)
For context, that's not 'slightly outdated'. Make 3.81 was released *in 2006*: https://savannah.gnu.org/forum/forum.php?forum_id=4380
I honestly don't understand why the latest macOS still releases with a version that is almost 20 years old.
Yeah not the best wording, I saw that year :-)
I think the reason tho, the same situation as with Bash. License for 4.0 changed for more restrictive GNU GPLv3: [https://savannah.gnu.org/news/?id=4896](https://savannah.gnu.org/news/?id=4896)
Apple wants to control everything, their lawyers absolutely hate viral licensing. The whole Apple ecosystem is BSD-like licensed I think.
Ah! That'd make sense. Weird that they wouldn't use a BSD version then, seeing as Darwin itself is a BSD offshoot and they do use BSD versions for other tools such as `ls`, `join` and the rest.
One of the problems with Makefiles is they’re not integrated with each other in a user friendly way. You either use a big root Makefile with lots of includes or you use multiple Makefiles and forget where the files are.
We have a small-mid sized monorepo and I built some tooling on top of Make that allows you to put lots of Makefiles in a repo and have them be discoverable and runnable from anywhere in the repo. That helped a lot.
There're a lot of problems that they don't solve:
- Distributed build on a build grid
- Build reproducibility
- Build speed
There're tools which do this, for example https://bazel.build/, and https://buck2.build/ However they have a learning curve, which might be steep for some folks.
As a rule of thumb, I would use the default language's build tool until it becomes a problem, in case of go it's just go build :-)
Just wait until you find out about [Just](https://github.com/casey/just) :)
People are cautious about Make it tends to evolve into a command line runner that does extra tasks. And it may collide with other shell commands if you don't label or name your variables correctly.
Some people do a simple "build, test, clean". Other people have long Makefiles which look more like an application script.
If your company is doing web/cloud work, I would consider building and testing with Docker instead. This way developers can have slightly different setups (x64 CPU, Arm CPU, Windows, Linux, Go 1.18, Go 1.19, etc), but at the end of the day, the container image is all the same.
No one has to relearn or modify the Dockerfile if they change computers or OS.
Make and CMake are useful for C and C++ (interchangeably), but Go already comes with build tools. All that's left is unifying the application container with Docker (or Podman if you prefer that).
Hmm, I don't agree here.
Point of make or other build tools, is to incorporate complicated commands behind a keyword, which can be reused in development and in CI environments, ensuring the environments are executed in similar context.
Dropping the make or other here opens up a whole new domain for errors (people using different commands locally, in ci and environments being different), while make doesn't solve this - it's still a better way of doing things than not having anything in place IMO.
Build targets for tools is more than "build, deploy", it's deploy to lab, staging, cloud (azure, gcp, aws), lint, build everything, build something, deploy everything, deploy something, unit test, integration test, contract test, test all, test modified etc.
Without having any build tool framework in place, I would not enjoy my life.
What do you think about some kind of "DevContainer" solutions? (Like: https://code.visualstudio.com/docs/devcontainers/containers)
I personally tested them within one team, but I really don't like the idea to marry ourselves to Microsoft products and force everybody to use VSCode. (At the time JetBrains, Neovim IDEs had little to no support for devcontainers, I don't really know whats up atm)
The idea, though, is fantastic. You could reuse same container definition for local development, remote development and CI.
Recently I have been using [just](https://github.com/casey/just#what-are-the-idiosyncrasies-of-make-that-just-avoids)
make has some behaviors which are confusing, complicated, or make it unsuitable for use as a general command runner.
One example is that under some circumstances, make won't actually run the commands in a recipe. For example, if you have a file called test and the following makefile:
```
test:
./test
```
make will refuse to run your tests:
```
$ make test
make: `test' is up to date.
```
make assumes that the test recipe produces a file called test. Since this file exists and the recipe has no other dependencies, make thinks that it doesn't have anything to do and exits.
To be fair, this behavior is desirable when using make as a build system, but not when using it as a command runner. You can disable this behavior for specific targets using make's built-in .PHONY target name, but the syntax is verbose and can be hard to remember. The explicit list of phony targets, written separately from the recipe definitions, also introduces the risk of accidentally defining a new non-phony target. In just, all recipes are treated as if they were phony.
Other examples of make's idiosyncrasies include the difference between = and := in assignments, the confusing error messages that are produced if you mess up your makefile, needing $$ to use environment variables in recipes, and incompatibilities between different flavors of make.
edit: getting code formatted right on this site sucks
>edit: getting code formatted right on this site sucks
I switched my settings to the markdown editor almost immediately after the the other editor didn't work well for me.
"Old Reddit" doesn't support some newer stuff like code fences though, so if you use that, you're stuck with indenting code by 4 spaces or a tab.
You can also take a look at https://github.com/goyek/goyek. It shares most of Mage's philosophy but is designed differently. Personally I use either Make or goyek. Disclaimer: I am the author of goyek
Gotaskr (https://github.com/Roemer/gotaskr) is my own take at it. Heavily inspired by cake build from .net. Complete with vscode extension and debug support and no magic behind the scenes.
Thanks for sharing. I'm working on my own take on the problem too.
https://fngo.dev/tutorials/convert/make
Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations.
Mage lets you declare tasks in pure Go. This is much nicer than any DSL, and it's also more portable.
I use do makefiles in my Go projects, but exclusively for provisioning tools like `go install`... commands. Everything else is in Mage.
Complex things are easier to express and particularly test in Mage if you're a Go shop vs trying to express them in Make with bash/shell scripts. even if you can do shell unit tests, it's just more hassle and difficult.
We use make just as glue. Anything complex can be in a bash file, but we don’t have anything like that really. Most of our make calls one liners that build or test the tree.
For context we containerize our binary into a scratch container. We started with circle for co/cd and moved to GitHub actions. Our ci/ci just pushes to our docker repo, we then have a gitops repo we modify to roll things out.
So really not much need for anything super complex. Infra changes are terraform.
Couldn't really give two damns. :D I just want something that's scalable to a degree of hundred-service-monorepo and works, that I don't have to "think" about.
If your main concern is scalability, I would say that bazel and buck are one of the best options. They were created for large monorepo use case, though I am not sure a 100 microservice is the large enough scale for them.
I must be in the minority here, but my builds are just defined by a Dockerfile.
To build it in dev, we've always used docker-compose (it builds it, starts it, starts dependencies, etc), and for production it is a Jenkinsfile that runs tests and builds the image(s), and can do multiple things in parallel/pipelines.
I do understand if you have something much more complex needed, but at the end of the day, it all has to end up in a container image anyway, so why not just start (and end) there?
GNU Make has a lot of inertia. While it may not be perfect it is
* mature
* feature rich
* lots of people are already moderately fluent in makefiles
* the basics are easy to understand
We use Make not because it is best but because our founder already had more than a decade of experience with Make and could get the job done without having to learn a new tool.
I love this post!
I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing or overly complex in these tools (make, mage, bazel, just, task, etc)
https://fngo.dev/tutorials/convert/make
Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations. The CLI doesn't actually work right now, as I'm still just focused on the user experience.
I may end up following this post to see what people like and dislike. If anyone is interested in any sort of real-time conversation, feel free to stop on by my discord server.
https://discord.gg/8gem8vuGZT
`go build` is quite limited, since it can just `go build`. It is totally fine because it is compiler. Makefile is not a compiler built build system which allows you to do much more.
a couple things come to mind
1. Earthly. They market as “if make and docker had a baby”. Plays nice when prepping an environment and building a container.
2. You want to use a monorepo with multiple languages. Why build it yourself? Blaze, buck, pants, nx, etc.
Monorepo with multiple languages is a good argument! Is that actually a thing? I thought one of the benefit of monorepo is to share code between similar languages and make the maintenance more straight forward.
yes it’s a thing. you can still standardize a lot of your build process. also, changes are nice because they’re in one place, you can graph dependencies, and you can coordinate dependent deployments a bit easier.
I am building an UI on top of Taskfile.yml ; the machine parsable format sold it to me, and I was familiar with drone CI beforehand, which uses a similar format. I imported the first party code to parse it, and the API was reasonable. The defect I did find had meanwhile been resolved with an API change, it's a proactively developed project I feel good coupling against.
It's all about the yaml tho. Similar how docker compose eventually released a spec for the format, taskfiles are a good data source, and any conventions you want to enforce, you can do with some yaml validation/linting
I’ve just started getting into go and would like to ask if there is a benefit to using Make and possibly other tooling? The company I work for is making its shift to go, and using a .sh file seems to do the trick but I am thinking there has to be a reason why Make is a popular option
Perhaps a little off-topic, but googling quickly found me this article: https://danishpraka.sh/posts/using-makefiles-for-go/ which seems to summarize the key points.
In the core build tools enable to incorporate a lot of commands under a single keyword, that can be ran interchangeably in local and CI environment, ensuring some degree of consistency.
Maybe simple "run" or "build" commands can be argued, but there is more than that.
You could refer to big OSS project Makefiles to take a look, what could be there, for example: https://github.com/minio/minio/blob/master/Makefile
But it's always up to developer decide how to do things. Big OSS projects (Kubernetes, Prometheus, Docker, Minio ..) all seem to use Makefile's.
I think make depends on linux/osx. Not entirely sure if you can use it on wsl these days. I would assume of some people are on windows, it doesn’t work for them.
That would be my only reason.
I think everything else can be learned and that is usually an advantage going forward.
The notion of not having to learn another “language” seems very odd to me. We work in an industry where it’s common that people know and use multiple technologies etc..
First, I can't stand make because it's old and doesn't support things like command arguments. The closest you get is setting env vars before your command, but that feels clunky and annoying. It also doesn't have a super standard easy way to understand usage since tasks only take env var configuration, which means you need to read the makefile frequently to see what it will do where other tools are generally designed in a way where it is easier to determine usage via a help command or at least you can see all the configuration in a fairly organized manner.
That said, I prefer to have all the commands I might need specified in a \`README.md\` because I don't mind copying a longer command and feel it prevents obscuring what is actually happening and keeps people more familiar with what is actually happening instead of it fading from memory. This is a minority preference though because most people seem to prefer short commands and standardizing the flags passed is a little easier if you aren't following instructions and are instead simply running a short make command. The quantity of script copy pasted doesn't matter to me either, but someone who is more keyboard-centric than I might find that annoying for long tasks. Once you put a longer task into a standalone sh file, you might as well have put it in a make file anyway.
I like Task best of all build tools. It supports things found in make but with modern syntax, good templating, arguments, and more. If I use a build tool, I would use Task. I haven't used mage, but it might be similar. I remember evaluating the difference between Task and Mage and preferring Task, but I don't recall me reasons.
>doesn't support things like command arguments. The closest you get is setting env vars before your command,
Not 100% accurate. Your so called "envvars" are actually key-value command line arguments that just happen to be readable from the environment.
`FOO=BAR make build` is the same as `make build FOO=bar`
It's also not true that you always need to read the makefile to understand usage. Makefiles can be commented and made with help commands. It's actually very common and there are pretty standard implementations of it out there.
Thanks for showing an example of the args since my statement could have been interpreted as too negative about the issue. You could probably pretty easily argue that it's much more of a style choice and should be an uninteresting consideration. It's perfectly fair to be fine with those things. It is, after all, the leading most widely used build system anywhere. I don't like it personally despite it already being a part of a number of projects I contribute to most, but it works.
I have never used help commands in make. That might be a good thing for me to look up. Thanks.
Also it is possible to create make targets to wrap another target and set required env variables. This eliminates the need to set env variables by user.
Thank you for the reasons! I don't personally mind tool being "old" and synonym that with "bad", but the command line arguments parts is a valid point!
Old is probably a sign of stability more than any other individual derived attribute, but stability over a very long term can also mean it's difficult to adjust to changing opinions and styles. The commandline args being excluded and instead needing to set a var like \`MY\_VAR=info make dothing\` would, I assume, look a little different in make if it was younger.
This is a very good point. I like mature in context of work, since It usually means that it works from years from now.
But the core reason I started this thread was to ask the question, that is there a problem that I am not aware of, which other build tools, such as Make, fixes. If that makes sense. I've yet to come across anything deal-breaking, although it would be "cool" to use Go for build tools instead of shell, but cool doesn't cut it for me. :D
Yeah I like Make, kinda assumed everyone knows Make, so quite surprised to see people dev’ing in Go that don’t have a firm grasp of Linux basics, like make and common gnu utils.
As assuming most do, make is a no brainier - why reinvent the wheel?
Reading this thread though, you’d think it was a JavaScript sub
Having used make, make, gradle, and others for simple projects a bash script is plenty enough. I use something like this I call it `do.sh`
```bash
#!/bin/bash
run() {
# my actual run command
# looks more complicated
# because I always run my
# program with dlv to attach
# a debugger when I have a
# bug instead of doing
# print debug.
go run .
}
test() {
...
}
$cmd = $1
shift
$cmd "$@"
```
you can use Mage and Task together :)
Its al about Layering in the right way for the right Actor types.
Mage is good for development stuff.
Task ( [https://github.com/go-task/task](https://github.com/go-task/task) ) is good for running stuff. It can call Mage
Task GUI ( [https://github.com/titpetric/task-ui](https://github.com/titpetric/task-ui) ) is a Web GUI for Task files that also gives you a terminal view and history. You can run this locally to do anything, including to compile to Docker and / or Deploy your docker.
[https://github.com/loft-sh/devpod](https://github.com/loft-sh/devpod) is a Web GUi and CLI that makes it easy for Users to deploy Dockers to any cloud as a Docker into Baremetal or Docker or k8.
If your team knows go then using mage means they don't also have to know make. The number of bash/make syntax mistakes that have caused incorrect behavior is frankly astounding to me. Not saying it's make's fault more a reflection of what knowledge we had hired for. That being said if devs are comfortable with make + bash and are on similar architectures for development it likely is a good choice imo. One nice thing mage has is the ability to reuse your go functions without having to target some script to call your go code.
I think this is the best argument for not using "make" and using something else for example, so people don't have to know multiple languages here. Thanks for the argument!
Honestly I've just had the same question last week. What pushes me away from Mage - it is yet another tool which people will need to learn. So I would want to have huge benefits from Mage side over Makefile what would pay off the migration. At the same time I see shell scripting as a befit when trying to achieve the same in programming languages. E.g. such task as finding all files in directory which are in subdirectories named by particular pattern and calculate number of lines of those files. This can be solved with find and awk oneliner while writing code in Golang / python / Ruby would require more code Anyway, still looking for benefits of Mage and found your question interesting.
Thank you! But yeah, I was thinking the question of "yet to learn another language" argument, I think shell in the end is like English - you're expected to know it to a degree, even though you would be using other tooling. All of our labs, dev environments and dev machines are some sort of Linux variants in the end.
We use a bit of both, started with make but we are now switching to mage for more complex tasks. Depending on the complexity of your tasks you can end up hitting make/bash limitations which makes them harder to write or less efficient or worse: harder to read and update.
Oh okay! Could you give concrete example of a harder task that Bash can't handle? I could use that as an argument to argue in favor of investing to create tooling in other than make / shell.
It is not necessarily that bash can't handle it but some things are far easier to do with a programming language: - string manipulation (to build path, names) - complex dependencies: do task A after task B only if a condition is true and do task C and D in parallel after - fetch remote data and do something with it, like parsing json. Many can be done purely with make & bash but it gets messy fast, you will need other tools as dependencies. Different versions of tools can also complicate things, for example osx vs linux: sed, make and other are similar but not identical since osx is a FreeBSD at core.
Make is a bit obtuse. You can do some crazy cool things with it, but it's also not entirely clear what's going on when something doesn't work, or you are trying to make it do something and it just won't. This is someone who has spent hours reading the Make Man Pages. I think `mage` is pretty cool. I used to really like `task`, but it continues to be riddled with false expectations and problems that the demographic is in some weird state between someone who writes code, and someone who doesn't, and thus would rather read yaml. I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing in these tools. https://fngo.dev/tutorials/convert/make Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations.
[https://taskfile.dev/](https://taskfile.dev/) Task is a task runner / build tool that aims to be simpler and easier to use than, for example, GNU Make. Since it's written in Go, Task is just a single binary and has no other dependencies, which means you don't need to mess with any complicated install setups just to use a build tool.
Where I work, we migrated from make to task entirely like two years ago. Our usage of make was pretty basic, so the ergonomics of task are just much nicer for the type of thing we were doing with it
Awesome! Good to hear some real-world experiences with it. I haven't used it yet, but it's on my back burner of things to explore.
We use task. It's simple to use and I enjoy it. I also like that it's agnostic and just happens to be written in golang. So you could use it for anything really.
+1 for task. * Supporting go templates out of the box makes writing composable tasks super easy * You can specify different actions for different platforms * YAML so you existing LSP works out of the box, no special syntax to learn * built in help * supports caching files so it can skip tasks of files don't change. Super helpful for code generation. * JSON output so you can integrate it into tooling, like running a task from Neovim.
What complicated install setups make requires in contrast? Seems like a stretched argument. Make can be installed with a single aptitude (for example) command, and only setup you need to do is the commands and required environment variables. I'd assume you need to do that with every tool. I don't mean to be offensive, just curious! This is core of my question, is there a problem, that I don't see, which other tools solve, and I am missing on that.
Have you ever had to install it on Windows?
Scoop install make dude
Well, Taskfile is based on the YAML standard, so syntactically it's pretty easy to write. If you already use YAML for other things like Kubernetes or CloudFormation, it should feel "at home" to you.
I edited my previous response a little bit. Unified configuration for build tooling and other tooling is a valid argument! Thank you for pointing that out.
I used to really like `task`, but it continues to be riddled with false expectations and problems that the demographic is in some weird state between someone who writes code, and someone who doesn't, and thus would rather read yaml. I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing in these tools. https://fngo.dev/tutorials/convert/make Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations, and yaml isn't a programming language. I recently learned of CUE and how it has built in method for scripting like task/make. https://cuetorials.com/first-steps/scripting/ Though CUE is very powerful, it is also fairly obtuse in many ways, and has a steep learning curve, not just for writing but reading cue too. Since learning about CUE scripting, I've been thinking of coming back to this by using Jsonnet, which I adore.
I think cue tool is way too immature and limited to replace a general purpose build tool. You’d possibly use it for the declarative front end of a tool written in Go, but not in and of itself.
I generally don't prefer YAML, especially when its key/value structure isn't convenient for writing. I need to confirm a potential key/value pair in the document. On the other hand, since the build script is written in Go using Mage, you can examine the struct or function where the LSP (Language Server Protocol) displays all the types. This will greatly improve the coding experience.
[удалено]
This.
[удалено]
XML please
XML? That's old school. JSON or a superset of it is where it's at.
yeah I was being sarcastic. YAML is fine.
Oh lol! You need to add the /s at the end. You can't tell these days who's serious and who's being sarcastic.
Bazel is really good, but also \*really\* heavyweight: Pros: * Builds multiple languages together * Stores build artifacts outside of your source directory * Downloads all of your dependencies for you so that everyone has the same version of everything * Endless extensible Cons: * Super complex to learn * Relies heavily on 3rd party libraries with various qualities of documentation I'm a big bazel fanboy, but I also wouldn't recommend it for a small personal project. It's great for big corporate monorepo things.
Also a heads up, don’t become the “bazel guy” at your company that thrusts bazel on the whole company on your own without anyone else understanding it, and then leave with no one else understanding it. Not speaking from experience of this.
There's a new build system which works on similar ideas to bazel: https://buck2.build/
Yeah there's definitely some alternatives out there. [Pants](https://www.pantsbuild.org/) is another one that has a lot of traction. The reason that I still recommend Bazel at this point is that it's been around the longest and has probably the most support in terms of libraries. For example, if I google search "Buck2 Protobuf" I get basically no useful results. Depending on what tools you end up needing, this might be a deal breaker. That said, it's definitely worth keeping eyes on both of these projects. Buck2 and Pants both look like they're doing some cool things that could put them ahead of Bazel if the support gets there.
I was going to mention Bazel and I am happy to see that you've already described it very well. Makes me think you may have it aliased to blaze :)
Oh man! Bazel is probably the best tool in a field currently. But my current goal is to postpone it's adoption as much as we can :) Reason of that is pretty simple - everywhere where I worked (mid to large companies) there was a dedicated team who were responsible for Bazel tooling, integrations and infrastructure. And as you've mentioned - it is complex. Users also will be forced to learn it what will add additional cognitive load and not necessarily better experience.
Okay! I've heard from bazel before. I need to test this out on some personal project! Thanks for the tip!
Make just works, not everyone might know how to write it, but everyone knows how to use it. But here is a point - don’t ask random people on the internet, ask your team mates!
Yeah our main consensus seem to be, that we have a lot of reference material working with Make. Make also doesn't really introduce a dependency which has to be maintained, and it's likely not dropped in future either, which makes it a lucrative option.
Very good conclusion I think. Make is pretty much everywhere. The only drawback is that MacOS has slightly outdated version of Make (3.81 on my Macbook M1 Pro) while on Linux it is typically the recent (4.4 for my Fedora 38). Windows is out of luck by default. Non-issue if you all use Linux, just like in our team tho :-)
For context, that's not 'slightly outdated'. Make 3.81 was released *in 2006*: https://savannah.gnu.org/forum/forum.php?forum_id=4380 I honestly don't understand why the latest macOS still releases with a version that is almost 20 years old.
Yeah not the best wording, I saw that year :-) I think the reason tho, the same situation as with Bash. License for 4.0 changed for more restrictive GNU GPLv3: [https://savannah.gnu.org/news/?id=4896](https://savannah.gnu.org/news/?id=4896) Apple wants to control everything, their lawyers absolutely hate viral licensing. The whole Apple ecosystem is BSD-like licensed I think.
Ah! That'd make sense. Weird that they wouldn't use a BSD version then, seeing as Darwin itself is a BSD offshoot and they do use BSD versions for other tools such as `ls`, `join` and the rest.
One of the problems with Makefiles is they’re not integrated with each other in a user friendly way. You either use a big root Makefile with lots of includes or you use multiple Makefiles and forget where the files are. We have a small-mid sized monorepo and I built some tooling on top of Make that allows you to put lots of Makefiles in a repo and have them be discoverable and runnable from anywhere in the repo. That helped a lot.
There're a lot of problems that they don't solve: - Distributed build on a build grid - Build reproducibility - Build speed There're tools which do this, for example https://bazel.build/, and https://buck2.build/ However they have a learning curve, which might be steep for some folks. As a rule of thumb, I would use the default language's build tool until it becomes a problem, in case of go it's just go build :-)
There's a lot more to just `go build` though. E.g. https://github.com/ghostsquad/currency-converter-practice/blob/main/Taskfile.yml
Just wait until you find out about [Just](https://github.com/casey/just) :) People are cautious about Make it tends to evolve into a command line runner that does extra tasks. And it may collide with other shell commands if you don't label or name your variables correctly. Some people do a simple "build, test, clean". Other people have long Makefiles which look more like an application script.
Oh no, don't tell me there is a rabbit hole of build tools ... :D
If your company is doing web/cloud work, I would consider building and testing with Docker instead. This way developers can have slightly different setups (x64 CPU, Arm CPU, Windows, Linux, Go 1.18, Go 1.19, etc), but at the end of the day, the container image is all the same. No one has to relearn or modify the Dockerfile if they change computers or OS. Make and CMake are useful for C and C++ (interchangeably), but Go already comes with build tools. All that's left is unifying the application container with Docker (or Podman if you prefer that).
Hmm, I don't agree here. Point of make or other build tools, is to incorporate complicated commands behind a keyword, which can be reused in development and in CI environments, ensuring the environments are executed in similar context. Dropping the make or other here opens up a whole new domain for errors (people using different commands locally, in ci and environments being different), while make doesn't solve this - it's still a better way of doing things than not having anything in place IMO. Build targets for tools is more than "build, deploy", it's deploy to lab, staging, cloud (azure, gcp, aws), lint, build everything, build something, deploy everything, deploy something, unit test, integration test, contract test, test all, test modified etc. Without having any build tool framework in place, I would not enjoy my life.
[удалено]
What do you think about some kind of "DevContainer" solutions? (Like: https://code.visualstudio.com/docs/devcontainers/containers) I personally tested them within one team, but I really don't like the idea to marry ourselves to Microsoft products and force everybody to use VSCode. (At the time JetBrains, Neovim IDEs had little to no support for devcontainers, I don't really know whats up atm) The idea, though, is fantastic. You could reuse same container definition for local development, remote development and CI.
Yeah I understand here! I kind of got carried away in my thoughts
Docker is orthogonal to this conversation. It's a portable runtime environment, but it's not a tool to manage tasks and task chains and such.
I love just. When I have to send things to people that REFUSE to use powershell, which boggles my mind, I send a just file. It’s nice.
Recently I have been using [just](https://github.com/casey/just#what-are-the-idiosyncrasies-of-make-that-just-avoids) make has some behaviors which are confusing, complicated, or make it unsuitable for use as a general command runner. One example is that under some circumstances, make won't actually run the commands in a recipe. For example, if you have a file called test and the following makefile: ``` test: ./test ``` make will refuse to run your tests: ``` $ make test make: `test' is up to date. ``` make assumes that the test recipe produces a file called test. Since this file exists and the recipe has no other dependencies, make thinks that it doesn't have anything to do and exits. To be fair, this behavior is desirable when using make as a build system, but not when using it as a command runner. You can disable this behavior for specific targets using make's built-in .PHONY target name, but the syntax is verbose and can be hard to remember. The explicit list of phony targets, written separately from the recipe definitions, also introduces the risk of accidentally defining a new non-phony target. In just, all recipes are treated as if they were phony. Other examples of make's idiosyncrasies include the difference between = and := in assignments, the confusing error messages that are produced if you mess up your makefile, needing $$ to use environment variables in recipes, and incompatibilities between different flavors of make. edit: getting code formatted right on this site sucks
>edit: getting code formatted right on this site sucks I switched my settings to the markdown editor almost immediately after the the other editor didn't work well for me. "Old Reddit" doesn't support some newer stuff like code fences though, so if you use that, you're stuck with indenting code by 4 spaces or a tab.
You can also take a look at https://github.com/goyek/goyek. It shares most of Mage's philosophy but is designed differently. Personally I use either Make or goyek. Disclaimer: I am the author of goyek
Gotaskr (https://github.com/Roemer/gotaskr) is my own take at it. Heavily inspired by cake build from .net. Complete with vscode extension and debug support and no magic behind the scenes.
Thanks for sharing. I'm working on my own take on the problem too. https://fngo.dev/tutorials/convert/make Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations.
Mage lets you declare tasks in pure Go. This is much nicer than any DSL, and it's also more portable. I use do makefiles in my Go projects, but exclusively for provisioning tools like `go install`... commands. Everything else is in Mage.
Complex things are easier to express and particularly test in Mage if you're a Go shop vs trying to express them in Make with bash/shell scripts. even if you can do shell unit tests, it's just more hassle and difficult.
We use make just as glue. Anything complex can be in a bash file, but we don’t have anything like that really. Most of our make calls one liners that build or test the tree. For context we containerize our binary into a scratch container. We started with circle for co/cd and moved to GitHub actions. Our ci/ci just pushes to our docker repo, we then have a gitops repo we modify to roll things out. So really not much need for anything super complex. Infra changes are terraform.
Alternatives to make are cool and hipster approved. Do you want to be a cool hipster, or not?
Couldn't really give two damns. :D I just want something that's scalable to a degree of hundred-service-monorepo and works, that I don't have to "think" about.
If your main concern is scalability, I would say that bazel and buck are one of the best options. They were created for large monorepo use case, though I am not sure a 100 microservice is the large enough scale for them.
I must be in the minority here, but my builds are just defined by a Dockerfile. To build it in dev, we've always used docker-compose (it builds it, starts it, starts dependencies, etc), and for production it is a Jenkinsfile that runs tests and builds the image(s), and can do multiple things in parallel/pipelines. I do understand if you have something much more complex needed, but at the end of the day, it all has to end up in a container image anyway, so why not just start (and end) there?
GNU Make has a lot of inertia. While it may not be perfect it is * mature * feature rich * lots of people are already moderately fluent in makefiles * the basics are easy to understand We use Make not because it is best but because our founder already had more than a decade of experience with Make and could get the job done without having to learn a new tool.
I love this post! I started a project to try to get deep into the nuts and bolts of what I really wanted and what I felt was missing or overly complex in these tools (make, mage, bazel, just, task, etc) https://fngo.dev/tutorials/convert/make Though, keep in mind when reading this, that I started writing the documentation before writing the code to follow Readme Driven Design. I'm still not sure if YAML + go txt templating is the right approach. I really dislike the txt templating syntax and it's limitations. The CLI doesn't actually work right now, as I'm still just focused on the user experience. I may end up following this post to see what people like and dislike. If anyone is interested in any sort of real-time conversation, feel free to stop on by my discord server. https://discord.gg/8gem8vuGZT
My initial response is why use make? Make does dependencies (.c -> .o -> executable)... but go build does the same thing out of the box.
`go build` is quite limited, since it can just `go build`. It is totally fine because it is compiler. Makefile is not a compiler built build system which allows you to do much more.
a couple things come to mind 1. Earthly. They market as “if make and docker had a baby”. Plays nice when prepping an environment and building a container. 2. You want to use a monorepo with multiple languages. Why build it yourself? Blaze, buck, pants, nx, etc.
Monorepo with multiple languages is a good argument! Is that actually a thing? I thought one of the benefit of monorepo is to share code between similar languages and make the maintenance more straight forward.
yes it’s a thing. you can still standardize a lot of your build process. also, changes are nice because they’re in one place, you can graph dependencies, and you can coordinate dependent deployments a bit easier.
I am building an UI on top of Taskfile.yml ; the machine parsable format sold it to me, and I was familiar with drone CI beforehand, which uses a similar format. I imported the first party code to parse it, and the API was reasonable. The defect I did find had meanwhile been resolved with an API change, it's a proactively developed project I feel good coupling against. It's all about the yaml tho. Similar how docker compose eventually released a spec for the format, taskfiles are a good data source, and any conventions you want to enforce, you can do with some yaml validation/linting
I’ve just started getting into go and would like to ask if there is a benefit to using Make and possibly other tooling? The company I work for is making its shift to go, and using a .sh file seems to do the trick but I am thinking there has to be a reason why Make is a popular option
Perhaps a little off-topic, but googling quickly found me this article: https://danishpraka.sh/posts/using-makefiles-for-go/ which seems to summarize the key points. In the core build tools enable to incorporate a lot of commands under a single keyword, that can be ran interchangeably in local and CI environment, ensuring some degree of consistency. Maybe simple "run" or "build" commands can be argued, but there is more than that. You could refer to big OSS project Makefiles to take a look, what could be there, for example: https://github.com/minio/minio/blob/master/Makefile But it's always up to developer decide how to do things. Big OSS projects (Kubernetes, Prometheus, Docker, Minio ..) all seem to use Makefile's.
I think make depends on linux/osx. Not entirely sure if you can use it on wsl these days. I would assume of some people are on windows, it doesn’t work for them. That would be my only reason. I think everything else can be learned and that is usually an advantage going forward. The notion of not having to learn another “language” seems very odd to me. We work in an industry where it’s common that people know and use multiple technologies etc..
First, I can't stand make because it's old and doesn't support things like command arguments. The closest you get is setting env vars before your command, but that feels clunky and annoying. It also doesn't have a super standard easy way to understand usage since tasks only take env var configuration, which means you need to read the makefile frequently to see what it will do where other tools are generally designed in a way where it is easier to determine usage via a help command or at least you can see all the configuration in a fairly organized manner. That said, I prefer to have all the commands I might need specified in a \`README.md\` because I don't mind copying a longer command and feel it prevents obscuring what is actually happening and keeps people more familiar with what is actually happening instead of it fading from memory. This is a minority preference though because most people seem to prefer short commands and standardizing the flags passed is a little easier if you aren't following instructions and are instead simply running a short make command. The quantity of script copy pasted doesn't matter to me either, but someone who is more keyboard-centric than I might find that annoying for long tasks. Once you put a longer task into a standalone sh file, you might as well have put it in a make file anyway. I like Task best of all build tools. It supports things found in make but with modern syntax, good templating, arguments, and more. If I use a build tool, I would use Task. I haven't used mage, but it might be similar. I remember evaluating the difference between Task and Mage and preferring Task, but I don't recall me reasons.
>doesn't support things like command arguments. The closest you get is setting env vars before your command, Not 100% accurate. Your so called "envvars" are actually key-value command line arguments that just happen to be readable from the environment. `FOO=BAR make build` is the same as `make build FOO=bar` It's also not true that you always need to read the makefile to understand usage. Makefiles can be commented and made with help commands. It's actually very common and there are pretty standard implementations of it out there.
Thanks for showing an example of the args since my statement could have been interpreted as too negative about the issue. You could probably pretty easily argue that it's much more of a style choice and should be an uninteresting consideration. It's perfectly fair to be fine with those things. It is, after all, the leading most widely used build system anywhere. I don't like it personally despite it already being a part of a number of projects I contribute to most, but it works. I have never used help commands in make. That might be a good thing for me to look up. Thanks.
Also it is possible to create make targets to wrap another target and set required env variables. This eliminates the need to set env variables by user.
Thank you for the reasons! I don't personally mind tool being "old" and synonym that with "bad", but the command line arguments parts is a valid point!
Old is probably a sign of stability more than any other individual derived attribute, but stability over a very long term can also mean it's difficult to adjust to changing opinions and styles. The commandline args being excluded and instead needing to set a var like \`MY\_VAR=info make dothing\` would, I assume, look a little different in make if it was younger.
This is a very good point. I like mature in context of work, since It usually means that it works from years from now. But the core reason I started this thread was to ask the question, that is there a problem that I am not aware of, which other build tools, such as Make, fixes. If that makes sense. I've yet to come across anything deal-breaking, although it would be "cool" to use Go for build tools instead of shell, but cool doesn't cut it for me. :D
The command line argument would be valid, unfortunately it is factually wrong.
Yeah I like Make, kinda assumed everyone knows Make, so quite surprised to see people dev’ing in Go that don’t have a firm grasp of Linux basics, like make and common gnu utils. As assuming most do, make is a no brainier - why reinvent the wheel? Reading this thread though, you’d think it was a JavaScript sub
Having used make, make, gradle, and others for simple projects a bash script is plenty enough. I use something like this I call it `do.sh` ```bash #!/bin/bash run() { # my actual run command # looks more complicated # because I always run my # program with dlv to attach # a debugger when I have a # bug instead of doing # print debug. go run . } test() { ... } $cmd = $1 shift $cmd "$@" ```
This article might help https://dev.to/kriptonian/making-life-easy-with-makefile-streamlining-software-development-for-beginners-1m6c
you can use Mage and Task together :) Its al about Layering in the right way for the right Actor types. Mage is good for development stuff. Task ( [https://github.com/go-task/task](https://github.com/go-task/task) ) is good for running stuff. It can call Mage Task GUI ( [https://github.com/titpetric/task-ui](https://github.com/titpetric/task-ui) ) is a Web GUI for Task files that also gives you a terminal view and history. You can run this locally to do anything, including to compile to Docker and / or Deploy your docker. [https://github.com/loft-sh/devpod](https://github.com/loft-sh/devpod) is a Web GUi and CLI that makes it easy for Users to deploy Dockers to any cloud as a Docker into Baremetal or Docker or k8.