ci: build the compiler image on a prebuilt LLVM image (#5671)

* ci: build the compiler image on a prebuilt LLVM image

The Docker workflow built LLVM as the first stages of the compiler image. The
only thing that prevented a rebuild was the BuildKit registry layer cache. A
layer cache is a best-effort optimisation, so CI sometimes built LLVM again
although llvm-version.txt did not change.

Move the LLVM stages to Dockerfile.llvm and make the compiler image start from
that image through an LLVM_IMAGE build argument. The compiler build now holds
no LLVM build step, so it cannot build LLVM again.

Tag the LLVM image with the LLVM revision and a hash of the files that set the
content of the image: llvm-version.txt, Dockerfile.llvm, GNUmakefile, and the
files in make/. tools/llvm-image-tag.sh prints the tag, and CI and developers
use the same script. The workflow builds and pushes the LLVM image only when
the registry does not hold that tag.

Remove the object files and the git history from the LLVM image in the same
layer as the build. The CI caches already link against that subset, see
.github/actions/setup-llvm/action.yml.

Delete llvm.yml. It made an image that nothing used, and it needed a push to a
special branch. The Docker workflow now does the same work when it is
necessary, and the force-llvm input of the manual trigger makes the image
again.

The LLVM image goes to GHCR only, because only CI uses it. The compiler image
continues to go to Docker Hub and GHCR.

* ci: remove the LLVM git history in the layer that makes it

The removal was in the tinygo-llvm-build stage. This stage is below the
layer that makes the shallow clone. A later layer only hides files from a
parent layer. Thus the pack files stayed in the published image.

* ci: link the LLVM image to the repository

The label puts the package in the repository package list, so the package
settings are easy to find. BUILDING.md now tells you to build LLVM locally
if you are not able to pull the image.

.github/workflows/docker.yml

@@ -2,18 +2,85 @@
 # If you are looking for the tinygo/tinygo "release" Docker image please see
 # https://github.com/tinygo-org/docker
 #
+# The compiler image is built on top of an LLVM image. The LLVM image is built
+# only when the registry holds no image for the current LLVM version and build
+# recipe, so a change to TinyGo alone does not build LLVM again. An LLVM build
+# takes at least 1-2 hours.
+#
+# Use the manual trigger with force-llvm to build the LLVM image again.
 name: Docker
 on:
   push:
-    branches: [ dev, fix-docker-llvm-build ]
+    branches: [ dev ]
+  workflow_dispatch:
+    inputs:
+      force-llvm:
+        description: Build the LLVM image again
+        type: boolean
+        default: false
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
+  llvm:
+    name: build-push-llvm
+    runs-on: ubuntu-latest
+    permissions:
+      packages: write
+      contents: read
+    outputs:
+      image: ${{ steps.tag.outputs.image }}
+    steps:
+      - name: Free Disk space
+        shell: bash
+        run: |
+          df -h
+          sudo rm -rf /opt/hostedtoolcache
+          sudo rm -rf /usr/local/lib/android
+          sudo rm -rf /usr/share/dotnet
+          sudo rm -rf /opt/ghc
+          sudo rm -rf /usr/local/graalvm
+          sudo rm -rf /usr/local/share/boost
+          df -h
+      - name: Check out the repo
+        uses: actions/checkout@v6
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+      - name: Log in to Github Container Registry
+        uses: docker/login-action@v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Get the image tag
+        id: tag
+        shell: bash
+        run: |
+          tag=$(sh tools/llvm-image-tag.sh)
+          echo "image=ghcr.io/${{ github.repository_owner }}/llvm-22:$tag" >> "$GITHUB_OUTPUT"
+      - name: Look for the image
+        id: check
+        shell: bash
+        run: |
+          if docker buildx imagetools inspect "${{ steps.tag.outputs.image }}" > /dev/null 2>&1; then
+            echo "exists=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "exists=false" >> "$GITHUB_OUTPUT"
+          fi
+      - name: Build and push
+        if: steps.check.outputs.exists != 'true' || inputs.force-llvm
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          file: Dockerfile.llvm
+          push: true
+          tags: ${{ steps.tag.outputs.image }}
+
   push_to_registry:
     name: build-push-dev
+    needs: llvm
     runs-on: ubuntu-latest
     permissions:
       packages: write
@@ -62,7 +129,9 @@ jobs:
         with:
           context: .
           push: true
+          build-args: |
+            LLVM_IMAGE=${{ needs.llvm.outputs.image }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
           cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/tinygo-dev:buildcache
-          cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/tinygo-dev:buildcache,mode=max
+          cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/tinygo-dev:buildcache,mode=max,ignore-error=true

.github/workflows/llvm.yml

@@ -1,63 +0,0 @@
-# This is the Github action to build and push the LLVM Docker image
-# used by the tinygo/tinygo-dev Docker image.
-#
-# It only needs to be rebuilt when updating the LLVM version.
-#
-# To update, make any needed changes to this file,
-# then push to the "build-llvm-image" branch.
-#
-# The needed image will be rebuilt, which will very likely take at least 1-2 hours.
-name: LLVM
-on:
-  push:
-    branches: [ build-llvm-image ]
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-
-jobs:
-  build-push-llvm:
-    name: build-push-llvm
-    runs-on: ubuntu-latest
-    permissions:
-      packages: write
-      contents: read
-    steps:
-      - name: Check out the repo
-        uses: actions/checkout@v6
-        with:
-          submodules: recursive
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v4
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v6
-        with:
-          images: |
-            tinygo/llvm-22
-            ghcr.io/${{ github.repository_owner }}/llvm-22
-          tags: |
-            type=sha,format=long
-            type=raw,value=latest
-      - name: Log in to Docker Hub
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.DOCKER_HUB_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
-      - name: Log in to Github Container Registry
-        uses: docker/login-action@v4
-        with:
-          registry: ghcr.io
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-      - name: Build and push
-        uses: docker/build-push-action@v7
-        with:
-          target: tinygo-llvm-build
-          context: .
-          push: true
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-          cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/llvm-22:buildcache
-          cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/llvm-22:buildcache,mode=max