How to Use CMake Without the Agonizing Pain - Part 1

When age fell upon the world, and wonder went out of the minds of men; when grey cities reared to smoky skies tall towers grim and ugly, in whose shadow none might dream of the sun or of spring's flowering meads; when learning stripped earth of her mantle of beauty, and poets sang no more save of twisted phantoms seen with bleared and inward-looking eyes; when these things had come to pass, and childish hopes had gone away forever, there was a man who travelled out of life on a quest into the spaces whither the world's dreams had fled. — H.P. Lovecraft

I spent the better part of my off-hours last year rewriting Halide's CMake build.

I knew CMake had a polarizing reputation, but I needed to make Halide work easily on Windows. The existing build didn't work right in CLion, it couldn't find its dependencies (except on CI, somehow), and it didn't produce usable packages. I figured I'd roll up my sleeves and get to work, and so I started where anyone else would: by Googling "CMake tutorial".

I was nearly stricken blind.

There is so much bad information about CMake out there. It's pervasive. It's high in the search results. Just about every StackOverflow answer is out of date, wrong, or both. Heeding any of this advice will send you and your project careening down a road to madness, paved into the earth by the sweat and tears of those who have tried to port a project that hard-codes a library path.

If you don't want your builds to break, and your crops to die, you should learn to use CMake properly. This is the first in a series of blog posts that will attempt to teach you to use CMake effectively. My earlier post about whether CMake is a build system could be considered part 0 of this series.

So without further ado, let's talk about the most basic decision to make: what version of CMake to use in the first place.

Picking a CMake Version

If you're writing an open source project, you most likely want to make your code available to as many users as possible. So you might assume that you want to use a very old CMake version to build your project. This is nonsense. Recent versions of CMake are available absolutely everywhere. Your build's users are technical: C++ developers, not laypeople. They can upgrade CMake if for some reason they haven't yet. For every major platform, there are easy ways to get a recent CMake version installed and kept up to date. Don't believe me? See the table below.

I can't stress this enough: Kitware's portable tarballs and shell script installers do not require administrator access. CMake is perfectly happy to run as the current user out of your downloads directory if that's where you want to keep it. Even more impressive, the CMake binaries in the tarballs are statically linked and require only libc6 as a dependency. Glibc has been ABI-stable since 1997. It will work on your system.

We on the Halide team use the CMake 3.20.2 tarballs from Kitware on a variety of aging and new ARM hardware for our build infrastructure. We used to build CMake from scratch, which was a little painful, but since upstream started providing ARM binaries, it's been trivial.

There are good reasons for using modern CMake versions, too. Beyond broader compiler and platform compatibility, newer CMake versions offer many more features to help keep your builds simple and expressive. One of the best examples is CMake's CUDA support. It has gone through several evolutions from a find module to a full first-class language. Working with CUDA prior to CMake 3.17 is about as much fun as eating glass. The move away from package variables to targets with transitive, propagating properties has turned ugly, error-prone build scripts into simple, declarative build descriptions. We will touch on many of these features in the next few parts.

So there is no problem with taking a minimum version of 3.20 (the latest at time of writing). Maybe it's worth taking a minimum of 3.16 just because Ubuntu 20.04 LTS is such a hold-out, but anything earlier than that is plain masochism.

Another hard requirement is that you must never use a version of CMake older than your compiler. Older versions of CMake won't somehow know how to work with a compiler that was released later in time, and the command line defaults for GCC, Clang, and other major compilers changes frequently. The most basic example of this is the default language version and set of supported language versions. Other changes include the wording of errors and warnings that CMake matches to detect compiler capabilities.

Thus, if you intend to use C++17 on Linux, you will need to use at least Clang 5 (released Sep 7, 2017) or GCC 7 (released May 2, 2017), so you therefore cannot use a minimum CMake version prior to 3.9.3 (released Sep 20, 2017), and versions prior to 3.8 (released April 10, 2017) didn't even understand 17 as a possible value of the CXX_STANDARD target property, so there was no correct way to enable it. Rather than doing this tedious and ultimately pointless work of determining the oldest potentially compatible versions, just use the newest.

Validating Your Minimum Version

No matter what minimum version you pick for whatever reason, it would be a major mistake to simply set cmake_minimum_required(VERSION 3.X) and call it a day. You must also test with the actual CMake 3.X release on your local development machine and on CI.

Why? Simply because the policy mechanism ensures backwards compatibility, not forwards compatibility. If you use a more recent CMake version, nothing will stop you from using a feature that is too new for the declared minimum version. This is very, very common, too. Here are three examples off the top of my head that have bitten me:

1. You might use a generator expression that was not in the old CMake version. CMake will not even try to warn you about this, and many common and useful generator expressions were introduced later than you think.
2. You might rely on newer features of commands unintentionally. In particular, CMake 3.18+ searches lib64 directories when using HINTS arguments to find_library, but older ones don't. So code for old versions have to check CMAKE_SIZEOF_VOID_P and add those paths to HINTS manually. I don't think this is documented; I bisected to find that version number.
3. CMake's find modules change behavior pretty frequently. Old versions might not understand a newer library version's package layout.

So another basic rule is to never declare a minimum version lower than the one you actually test your build against. I have seen projects in the wild that claim compatibility with ridiculously old versions of CMake, like 2.6. Not only is it extremely unlikely that those builds actually work with 2.6, newer versions of CMake are soon dropping compatibility with versions before 2.8.12. So this "increased" compatibility will in fact cost you users who are doing the right thing by keeping up to date.

If you're setting up a CI pipeline, you should test your build with both the absolute newest version of CMake, and the minimum required version. This will allow you to very quickly catch backwards compatibility bugs and make upgrading the version a breeze. I do this on GitHub Actions with the jwlawson/actions-setup-cmake action. You can see such a workflow here on tinyxml2, whose CMake build I recently helped modernize.

Conclusion

These are the most important lessons from this post:

1. Use the most recent CMake version. It is trivial to install and keep up to date. If you must pick an older version, do it for a logical reason, not because you're copying some ancient StackOverflow answer that set 3.5 as a minimum.
2. Use a version of CMake at least as recent as your compiler version.
3. Always test your build with the actual CMake version you're taking as a minimum.

In part 2, we'll talk about the contract between a CMake build and its many consumers.

Until then, join the conversation here on Reddit!

Addendum

Distribution minimum versions

Since publishing this article, I have heard from several readers that they cannot upgrade their minimum versions because some particular Linux distribution (e.g. Ubuntu 18.04 LTS, RHEL 7, etc.) packages an older version of CMake, and so they must accept that version.

I stand by what I wrote. On the one hand, if the maintainers independently want to include your package, then it's up to them to figure out how to use a newer CMake version in their build process. If that means bootstrapping a newer CMake version from source, so be it.

On the other hand, if you want to ask the maintainers to include your package, and they won't let you use a newer version, you should instead ask yourself why your package needs to be included in the base distribution. There are many viable distribution methods on Linux these days. You could host your own APT or RPM repository; you could release your package on pip or snap. Agreeing to downgrade for the sake of one distro harms all of your users.

Lowest common denominator thinking is toxic to the progress of the C and C++ ecosystems. Distribution maintainers should periodically update CMake, even on LTS releases. CMake is incredibly backwards compatible, but when there are issues, there are also many recourses for a distribution maintainer: they can package multiple CMake versions, they can patch a problematic package (and maybe upstream the patch, which is better for everyone), or they can patch their distribution of CMake. The vcpkg team occasionally has to rewrite entire build systems for projects.

Resources

At the beginning of the article, I complained that there are no good learning resources. Fortunately, this isn't quite true. So far as I know, the best places to get high quality advice for writing CMake code are these:

1. The CMake Discourse forum. The actual developers hang out and answer questions here.
2. The #cmake channel on the CppLang Slack. This is a very friendly community for CMake users to think through build issues, ask beginner to intermediate level questions, and share wisdom.
3. The book "Professional CMake" by Craig Scott. Craig is a volunteer maintainer of CMake, and he sells his book for $30 through his consulting business. This is the most comprehensive and clearly written reference guide for CMake. Even better, your purchase also includes updates to new editions as the book is updated (and it is updated frequently). This is a must-have if CMake is part of your job; you should convince your employer to purchase copies for your team.

If you don't want to buy Professional CMake or can't afford it, here are some good free resources on the web:

1. Craig gave a talk at CppCon 2019, "Deep CMake for Library Authors" that covers issues including symbol visibility, library versioning, writing install rules, and RPATH pitfalls.
2. Deniz Bahadir gave a pair of talks called "More Modern CMake" and "Oh No! More Modern CMake" at Meeting C++ 2018 and 2019, respectively. These talks use CMake 3.12-3.14, so there are some things that are out of date, but his explanation of the modern CMake targets system is very good. We'll talk about dependencies soon, but I disagree with the approach here, which sets IMPORTED_GLOBAL on targets created by find_package calls. These talks are particularly valuable for showing the old, painful way of doing things next to the new(er), less-painful way.
3. Robert Schumacher is the lead developer of vcpkg and has a lot of experience with dealing with every type of problematic build system. He's also a great presenter and generally a smart guy, so I wholeheartedly recommend his talks:
   1. "Don't Package Your Libraries Write Packagable Libraries! Part 1".
   2. "Don't Package Your Libraries Write Packagable Libraries! Part 2". Note: I disagree with his use of globbing in CMake, but his point about projects being globbable is good.
   3. "How to Herd 1,000 Libraries"

Unless otherwise stated, all code snippets are licensed under CC-BY-SA 4.0.