Premake5 and Compilation of Sciter Engine.

Those famous “nightly builds”…

Not a secret that compilation/build times are somehow critical for projects of Sciter’s scale.

The fact: it takes from 4 to 6 hours to build Chrome/Chromium browser. And note – that is on dedicated build servers / farms, not on your work machine.

So devops, as a desperate move, are forced to run builds at nights. This is where “night build” term comes from.

But software development is about frequent change-compile-verify cycles. Just imagine that after a change you’ve made you will be able to verify it only next day… That’s horrible to be honest.

Sciter build times were significantly better but still not perfect…

Sciter projects needs

Main motivations for me to start using Premake5 in Sciter development process were:

1. Number of platforms where Sciter works have grown significantly: multiple build variants for Windows, MacOS and Linux. And Sciter.Lite alone adds many more of those for different devices and OSes. So I needed a tool that would allow me to configure various build targets from single source.
2. I suspected that I would need special build configurations optimized for development, I simply don’t want to waste my lifetime waiting for the builds.

So I decided to look around for solution for these problems and ended up with Premake5.

So why Premake5?

First of all: Premake5 provides simple set of configuration options that allows to generate solutions and projects for Microsoft Visual Studio on Windows (my primary dev platform), XCode on MacOS, CodeLite on Linux and makefile based build scripts on others.

And the second: while typical Premake5 file looks like as something written with special syntax it actually is a valid Lua script. And so we can use full power of normal procedural language (Lua) when needed.

Results so far

On Windows I have distinct (from other platforms) build configuration that uses multiple static libraries for each module Sciter is made of (see the image on the right). Each such static library uses its own PCH targeted header file. Such setup significantly improves compilation times:

• Full Debug re-build of Sciter Engine: 46 seconds.
• Full Release re-build: 65 seconds.

As you see, the numbers are very good and comfortable for development even on pretty average development PC that I am using (Intel Core i7-6700k, 4 cores/8 logical processors). I even don’t need any special development builds – standard Release configuration is good enough.

For the comparison, I have also “monolithic” configurations that do the same but with all files from all modules assembled into single static .lib file (for teams that link Sciter statically into their executables).  For obvious reasons such setup cannot use PCH file and so compilation time is worse: ~ 5 minutes.  But I do those only for public releases 1-2 times per month so that time is perfectly acceptable as it takes minutes – not hours.

Premake5 file configuration details

Consider the following task: you have set of source/header files and you need to build either static library from the set or to include it into monolithic assembly as just list of files. Conceptually you can copy paste the list into two places but that is not good – it must be single source of truth – the list shall be defined in single place.

So I wrapped the list into Lua function (Premake is Lua!) and call that function in all places where I need to inject that list. Here is an example of such function for libUV:

— libUV files

function include_files_uv() 
  
  includedirs { "engine/external/uv/include", "engine/external/uv/src" }

  files { "engine/external/uv/src/*.c", "engine/external/uv/include/*.h", "engine/external/uv/src/*.h" }

  filter "system:windows"
    files { "engine/external/uv/src/win/*.c" }
  
  filter "not system:windows"
    files { 
      "engine/external/uv/src/unix/async.c",
      "engine/external/uv/src/unix/atomic-ops.h",
      "engine/external/uv/src/unix/core.c",
      "engine/external/uv/src/unix/dl.c",
      "engine/external/uv/src/unix/fs.c",
      "engine/external/uv/src/unix/getaddrinfo.c",
      "engine/external/uv/src/unix/getnameinfo.c",
      "engine/external/uv/src/unix/internal.h",
      "engine/external/uv/src/unix/loop-watcher.c",
      "engine/external/uv/src/unix/loop.c",
      "engine/external/uv/src/unix/pipe.c",
      "engine/external/uv/src/unix/poll.c",
      "engine/external/uv/src/unix/process.c",
      "engine/external/uv/src/unix/signal.c",
      "engine/external/uv/src/unix/spinlock.h",
      "engine/external/uv/src/unix/stream.c",
      "engine/external/uv/src/unix/tcp.c",
      "engine/external/uv/src/unix/thread.c",
      "engine/external/uv/src/unix/tty.c",
      "engine/external/uv/src/unix/udp.c",
      "engine/external/uv/src/unix/timer.c",
      "engine/external/uv/src/unix/proctitle.c",
    }
  filter "system:macosx"
    files {
      "engine/external/uv/src/unix/bsd-ifaddrs.c",
      "engine/external/uv/src/unix/darwin.c",
      "engine/external/uv/src/unix/darwin-proctitle.c",
      "engine/external/uv/src/unix/fsevents.c",
      "engine/external/uv/src/unix/kqueue.c",
    }
  filter "system:linux"
    files {
      "engine/external/uv/src/unix/linux-core.c",
      "engine/external/uv/src/unix/linux-inotify.c",
      "engine/external/uv/src/unix/linux-syscalls.c",
      "engine/external/uv/src/unix/procfs-exepath.c",
      "engine/external/uv/src/unix/sysinfo-loadavg.c",
      "engine/external/uv/src/unix/sysinfo-memory.c",
    }
  filter {} — note: this ends scope of last filter 
end

And the libuv.a project declaration looks as:

project "uv"   
  kind "StaticLib"
  language "C"
  symbols "Off"
  include_files_uv() — includes list of UV files, sic!

Resume

Use of Lua in Premake5 was very good architectural decision as Lua’s syntax allows to define such de facto make DSL naturally – just using its core syntax.

Any descent make configuration tool (CMake, SCons, Premake, etc. ) allows you to easily define 90% of your project needs.

Problem is in rest 10%. that you may not be able to define by pure make means. Having procedural language easily available in configuration files is a great benefit in such cases. Trust me 🙂