Setup CI-CD Pipeline Using Github Actions to deploy a Containerized Application to Google App Engine

This article walks you through on establishing a Continuous Integration and Continuous Deployment (CI/CD) pipeline using GitHub Actions to facilitate the deployment of a containerized application onto the Google Cloud Platform(GCP). The process integrates automated steps that enable seamless software delivery.

This configuration I set up for a GitHub Action is triggered when a new release is made(the event is a choice of yours), and do the following steps upon triggering.

• Builds a Docker image
• Assigns it the corresponding version number as a tag
• Uploads it to the Google Artifact Registry.
• Populate workflow secrets as environment variables
• Deploy the container to the Google App Engine

Before starting, you should have,

• A GitHub repository that contains a working Dockerfile and the application source code. (Say a FastAPI backend application that does some CRUD operation with PostgreSQL)
• The Google Cloud SDK tool gcloud installed and authenticated. (You can try this on the free tier offered by GCP)

Obtain Gcloud Configurations and Parameters

1. Create a dedicated service account in Gcloud IAM

Login to gcloud CLI

gcloud auth login

Use the same terminal instance for all the following commands as we will be exporting some reusable values as environment variables.

export PROJECT_ID=your-project-id
export SERVICE_ACCOUNT=my-service-account

• The project id can be taken from the GCP’s cloud console once you have created a project.
• You can select your own name as the name of the service account. (eg: service-account-github)

gcloud iam service-accounts create "${SERVICE_ACCOUNT}" \
  --project "${PROJECT_ID}"

2. Create a Workload Identity Federation

a. enable GCP’s Identity and Access Management(IAM) API

gcloud services enable iamcredentials.googleapis.com \
  --project "${PROJECT_ID}"

b. Create a workload identity pool. This will manage the GitHub Action’s roles in Google Cloud’s IAM service.

export WORKLOAD_IDENTITY_POOL=my-pool

gcloud iam workload-identity-pools create "${WORKLOAD_IDENTITY_POOL}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --display-name="${WORKLOAD_IDENTITY_POOL}"

• You can give any name for my-pool

c. Get the unique identifier of that pool.

gcloud iam workload-identity-pools describe "${WORKLOAD_IDENTITY_POOL}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --format="value(name)"

d. Export the returned value to a new variable.

export WORKLOAD_IDENTITY_POOL_ID=whatever-you-got-back-in-step-c

e. Create a workload provider within the pool for GitHub to access.

gcloud iam workload-identity-pools providers create-oidc "${WORKLOAD_PROVIDER}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --workload-identity-pool="${WORKLOAD_IDENTITY_POOL}" \
  --display-name="${WORKLOAD_PROVIDER}" \
  --attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository" \
  --issuer-uri="https://token.actions.githubusercontent.com"

f. Allow a GitHub Action based in your repository to login to the service account via the provider.

export REPO=gh-username/my-repo

gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com" \
  --project="${PROJECT_ID}" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/${WORKLOAD_IDENTITY_POOL_ID}/attribute.repository/${REPO}"

g. Get the unique identifier of the created provider.

gcloud iam workload-identity-pools providers describe "${WORKLOAD_PROVIDER}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --workload-identity-pool="${WORKLOAD_IDENTITY_POOL}" \
  --format="value(name)"

Save the returned value from the step g. It will be used in the workflow file we will be created later.

3. Assign Permission to the Service account to manage required service APIs

gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member="serviceAccount:${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com" \
    --role="roles/artifactregistry.admin" \
    --role="roles/iam.serviceAccountUser" \
    --role="roles/cloudsql.admin" \
    --role="roles/appengine.serviceAdmin" \
    --role="roles/firebase.admin"

Verify assigned roles

gcloud projects get-iam-policy $PROJECT_ID \
    --flatten="bindings[].members" \
    --format='table(bindings.role)' \
    --filter="bindings.members:${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com"

4. Create a repository in Google Artifact Registry.

Navigate to the Google Artifact Registry interface located in your project. Generate a fresh repository by clicking the buttons located at the top section. Inform Google that the repository will be in Docker format, then proceed to choose a region. The specific region choice is not critical. Remember to save the name assigned to the repository as well as the abbreviation of the chosen region. This abbreviation will resemble something like “us-west1.” This region will be refered in the workflow file as gar-region.

5. Create a app.yaml file with relevant parameters and options.

The following config is specifically for a Python FastAPI application with Cloud SQL database.

You can place the app.yaml file in anywhere in the repository. Make sure that the correct path is given to the workflow file and the secrets are properly managed.

runtime: custom
env: flex

network:
  forwarded_ports:
    - 80:8080

env_variables:
  DB_USER: $SECRET_DB_USER
  DB_PASS: $SECRET_DB_PASS
  DB_PORT: $SECRET_DB_PORT
  DB_NAME: $SECRET_DB_NAME
  DB_HOST: $SECRET_DB_HOST
  INSTANCE_UNIX_SOCKET: $SECRET_INSTANCE_UNIX_SOCKET
  SOME_LICENSE_KEY: $SECRET_SOME_LICENSE_KEY
  SERVICE_API_KEY: $SECRET_SERVICE_API_KEY

beta_settings:
  cloud_sql_instances: 'CLOUD_SQL_CONNECTION_STRING'

6. Encrypt Secrets

You can store your secret token in the Github itself and refer those in the workflow. Refer this official documentation for learn how to encrypt secrets.

7. Create a workflow file for Github.

Now let’s geton to create a workflow.

You should add a new YAML file in the .github/workflows folder. In the authentication step you’ll want to fill in your provider id, your service account idand project id.

In the push step you’ll need to fill in your GAR repository name and region, as well as a name for your image, which you’ll need to make up on your own.

name: Docker-GCP-CI-CD
on:
  push:

jobs:
  docker-release:
    name: Build and Deploy REST API Container to Google App Engine
    runs-on: ubuntu-latest
    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')  # <-- Notice that I'm filtering here to only run when a tagged commit is pushed

    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
      - id: checkout
        name: Checkout
        uses: actions/checkout@v2

      - id: auth
        name: Authenticate with Google Cloud
        uses: google-github-actions/auth@v0
        with:
          token_format: access_token
          workload_identity_provider: ${{ secrets.SECRET_GCP_WORKLOAD_IDENTITY_PROVIDER }}
          service_account: ${{ secrets.SECRET_GCP_SERVICE_ACCOUNT }}
          access_token_lifetime: 300s

      - name: Login to Artifact Registry
        uses: docker/login-action@v1
        with:
          registry: <gar-region>-docker.pkg.dev
          username: oauth2accesstoken
          password: ${{ steps.auth.outputs.access_token }}

      - name: Get tag
        id: get-tag
        run: echo ::set-output name=short_ref::${GITHUB_REF#refs/*/}

      - id: docker-push-tagged
        name: Tag Docker image and push to Google Artifact Registry
        uses: docker/build-push-action@v2
        with:
          push: true
          tags: |
             <your-gar-region>-docker.pkg.dev/<your-project-id>/<your-gar-repo-name>/<your-docker-image-name>:${{ steps.get-tag.outputs.short_ref }}
             <your-gar-region>-docker.pkg.dev/<your-project-id>/<your-gar-repo-name>/<your-docker-image-name>:latest
      - id: populate-configs
        uses: 73h/gae-app-yaml-replace-env-variables@v0.3
        env:
          SECRET_AUTH_URL: ${{ secrets.SECRET_AUTH_URL }}
          SECRET_DB_USER: ${{ secrets.SECRET_DB_USER }}
          SECRET_DB_PASS: ${{ secrets.SECRET_DB_PASS }}
          SECRET_DB_PORT: ${{ secrets.SECRET_DB_PORT }}
          SECRET_DB_NAME: ${{ secrets.SECRET_DB_NAME }}
          SECRET_DB_HOST: ${{ secrets.SECRET_DB_HOST }}
          SECRET_INSTANCE_UNIX_SOCKET: ${{ secrets.SECRET_INSTANCE_UNIX_SOCKET }}
          SECRET_CLOUD_SQL_INSTANCE: ${{ secrets.SECRET_CLOUD_SQL_INSTANCE }}
          SECRET_SERVICE_API_KEY: ${{ secrets.SECRET_SERVICE_API_KEY }}
        with:
          app_yaml_path: ".github/configs/app.yaml"

      - id: deploy
        name: Deploy Docker image to App Engine
        uses: google-github-actions/deploy-appengine@v1
        with:
          deliverables: '.github/configs/app.yaml'

8. Verify the workflow functionality

Go to the releases panel for your repo on GitHub, punch in a new version tag like 0.0.1and hit the big green button. That should trigger a new process in your Actions tab, where the push of the tagged commit will trigger the release.

After the process completed on the actions tab, go to the GCP console’s App Engine section and check whether the latest version is serving the traffic.

References

GitHub Marketplace: actions to improve your workflow