Private GitHub Actions templates for your organization

The challenge of reusing workflows

My prior experience with GitLab CI made for a shorter learning curve with GitHub Actions, but the finer details proved to differ. For example, including a set of steps from one project in another is much different. GitHub Actions and GitLab CI both support reusing steps from one repository in another, but GitHub Actions Free and Team plans require the steps you want included to be publicly accessible. In Gitlab CI, one can use authentication to import a private repo’s jobs, stages, variables, etc., using the includes directive.

At the time of writing this, GitHub Actions had issues (make it two) tracking this desired functionality of private repos. These issues had been open for almost 18 months and it just recently got added for those on Enterprise plans. I’m not sure when (or if) this functionality will be available to everyone, but it seems like a core bit of functionality to support private reuse of workflows.

Syncing workflow templates 

I scoured the Stack Overflow posts and dug through GitHub Issues comment threads, but found no good workarounds for the missing ability to include workflows from other private repos in the same organization. 

I thought that maybe git submodules would be a possible solution. After all, it allows a repo to include another repo. Could we include our workflow templates as a submodule in our repo and use git’s submodule management to keep the templates up to date? We could, but not if we wanted to have templates irrelevant to some repos in the template repo. Git treats a repo as the smallest unit in submodule management, so our solution will either have a repo with only relevant workflows or avoid using submodules altogether.

We opted to not go with submodules because we want one workflow repo to hold Go and Python (among others) templates, but I gleaned some valuable information from that train of thought. Some people use symlinks to include parts of a repo in another repo instead of submodules.

So we tried to link our template to our app’s workflow. ln $DIR/workflows/go/build_and_push.yml $DIR/my-app/.github/workflows/build_and_push.yml works to keep our workflow template in sync with our app… until we run a git command that modifies the template (git changes the inodes of files it modifies). Since we want to be able to git pull for updates to the templates, we would need to find something that could automatically apply changes from the template repo to an app’s file.

GitHub Actions templates: An acceptable workaround for non-Enterprise plans

Without the native ability to include private templates in our workflows and us wanting to have a single repo storing our workflow templates, we opted for this simple solution that optimizes the ease of applying template updates. Until GitHub Free and Team plans support private templates, you can migrate, but this is one possible solution we have tried and tested.

We include git hooks samples in our template repository (e.g. hooks/post-checkout.sample) because any git commands that are going to break links need to re-establish them. The hooks for post-checkout and post-merge both look like this:

git merge and git checkout are common operations that break links, so we execute this script after those actions.

So, for users to apply workflow templates to their apps, they must:

1. Clone our workflows repo.

2. Edit the hooks/post-checkout.sample and hooks/post-merge.sample to match their app(s).

3. Copy them to their respective .git/hooks locations and give them execute permission.

4. Perform a git checkout or git pull (git checkout master from master works).

Unfortunately, this does not keep the template repo up-to-date on its own and requires some interaction from the project owners to adopt updates – the changes need to be applied to the remote, after all. They can do so via the following commands:

And after that, the workflow will run with the most up-to-date changes from the template repo.

Simplify setup using a Git-based package manager 

Since we optimized for ease of applying templates, we made other tradeoffs with our workaround. The most obvious is the setup required. We are asking users to organize their repos in a particular way and manually copy and paste files. These requirements contribute to a certain amount of fragility, where the system can break if files are moved or inodes change without our awareness.

However, there is an interesting benefit that we gain from this workaround. We’ve discovered a toolset and scripts that can work in a generic context. We created a git-based generic package manager that utilizes linking and git hooks to update files between git repos:

GitHub Actions pros and cons

So why GithHub Actions? Well, our current system requires modifications, pull requests, deployments, and more to onboard a new service. And as a GitHub Enterprise user, we have a large allocation of execution minutes within GitHub Actions, and so we decided to evaluate that as an alternative. 

Now that we’ve spent some time with GitHub Actions, here are my takeaways:

As you can see, the grass is not always greener. But we predict that GitHub, with its endless resources from Microsoft, will outpace Jenkins in the long run. 

Moving forward 

While our workaround is not pretty, it’s a solution that works until GitHub supports private templates for non-Enterprise plans. If this happens, you can migrate templates over into a single repository. 

With that said, we have a GitHub Enterprise account but don’t have the ability to turn on the private repo functionality that GitHub added yet. We’re still determining the source of this problem. Until then, we’re currently using the workaround and will check back in the future to see if we can get access to this new functionality. Watch this space!