Crossbuild is slow

[slrtbtfs]

Currently promu spins up a new docker container for each crossbuild target, with the respective native toolchain installed.

promu/cmd/crossbuild.go

Lines 198 to 203 in 2e745da

	ctrName := "promu-crossbuild-" + pg.Name + strconv.FormatInt(time.Now().Unix(), 10)
	err = sh.RunCommand("docker", "create", "-t",
	"--name", ctrName,
	pg.DockerImage,
	"-i", repoPath,
	"-p", platformsParam)

Since Go 1.5 this isn't necessary any more: https://dave.cheney.net/2015/08/22/cross-compilation-with-go-1-5

Using this "new" feature speeds up the crossbuild process significantly. For the PromQL language server the speed up is more than 10x.

Since crossbuilding is the slowest part of the Prometheus CI, it would make sense to update promu to use the crossbuild capabilities of newer go releases.

[simonpasquier]

Currently promu spins up a new docker container for each crossbuild target, with the respective native toolchain installed.

The later part isn't 100% accurate as it's only true for builds with CGO_ENABLED=1:

promu/cmd/crossbuild.go

Lines 152 to 163 in 2e745da

	// In non-CGO, use the base image without any crossbuild toolchain
	var allPlatforms []string
	allPlatforms = append(allPlatforms, mainPlatforms[:]...)
	allPlatforms = append(allPlatforms, armPlatforms[:]...)
	allPlatforms = append(allPlatforms, powerPCPlatforms[:]...)
	allPlatforms = append(allPlatforms, mipsPlatforms[:]...)
	allPlatforms = append(allPlatforms, s390xPlatforms[:]...)
	pg := &platformGroup{"base", dockerBaseBuilderImage, allPlatforms}
	if err := pg.Build(repoPath); err != nil {
	fatal(errors.Wrapf(err, "The %s builder docker image exited unexpectedly", pg.Name))
	}

When CGO_ENABLED=0 all builds happen in the same container. The problem is more that each GOOS/GOARCH build variant needs to recompile everything from scratch as there is nothing cached.

I think there are 2 things that can be improved:

• almost all the .promu.yml configurations (if not all) adds -a to the build command which forces rebuilding of all packages. I suspect it is a safety measure for cgo builds but we probably don't need it if cgo isn't enabled.
• we could save and reuse the build cache across CI runs.

[SuperQ]

Fixed with #208