Break Glass for Emergency Code Reviews in GitHub

a year ago
Written by
Fouad Matin
@fouadmatin

Break glass is a security measure that allows users to bypass normal access controls in emergency situations.

For GitHub, break glass access means allowing developers to perform code reviews and merge pull requests in an emergency — usually in the midst of an incident.

There's a few different ways for handling break glass access on GitHub:

  1. Make everyone an admin (not recommended)
  2. Creating teams with the ability to override branch protection rules
  3. Automate granting temporary access to a GitHub team

We'll also share an open-source template repo that demonstrates a DIY approach to implementing break glass access. This repo uses GitHub Issues and Actions to automate the process of granting and revoking access to a GitHub team.

It's a good starting point for implementing break glass access, but it's missing production-ready requirements like approvals and time-bounded access. We'll cover how Indent can help you solve this.

Make Everyone an Admin

The simplest method for granting break glass access is to make all developers administrators. However, this approach of persistent access is highly insecure and should be avoided.

Admin access grants users the ability to change organization settings, delete repositories, and perform other high-risk actions. This level of access should be reserved for a small group of individuals who need it as part of their day-to-day job function.

Usually this list includes:

  • Co-founders / CTO / VP of Engineering
  • Head of Infra / DevOps
  • Head of Security / IT

Create a Break Glass Team (RBAC)

A more secure method is to create a GitHub team that can override security or branch protection rules.

This team will have the ability to perform code reviews and merge pull requests even when other developers are unable to do so. To create a team with override access, follow these steps:

  1. Create a new team in your GitHub organization. If you want to allow emergency backend PRs, it could be named backend-BREAKGLASS.
  2. Add the appropriate developers to the team.
  3. If you want to implement using "CODEOWNERS" file:
    • In the CODEOWNERS file, you can put a team as a potential reviewer.
    • Add the team you created earlier as a collaborator with the "Write" permission to allow members to approve as a code owner.
  4. If you want to implement using "Admin" permission:
    • In the repository settings section, requiring break glass access, navigate to the "Collaborators and Teams" tab.
    • Add the team you created earlier as a collaborator with the "Admin" permission to allow overriding the approval.

While this method is more secure than making everyone an admin, it still has its drawbacks. Most notably, members of the team have persistent access, increasing the potential for a compliance violation, unauthorized action, or a simple mistake.

Better Break Glass Code Review using GitHub Issues and Actions

One approach for implementing break glass access on GitHub is to automate the process of granting and revoking access to the team. This ensures that access is only granted when necessary and is automatically revoked after a specified period.

To achieve this, you can use the GitHub API to create a script that:

  1. When an issue is opened, add them to backend-BREAKGLASS team
  2. When the issue is closed, remove the user from the team

Check out this open-source GitHub repository that uses GitHub Issues/Actions for a DIY solution: github-break-glass-template

The issue with the Issue/Action-based approach is that it has no approval process and it's up to the user to decide when to open/close the issue. It's better than nothing, but the next step is to make the access temporary.

Temporary GitHub Access with Indent

Indent provides temporary access and facilitates team approvals for systems like GitHub. With Indent, you can easily manage break glass access by setting up temporary access policies and automatically granting or revoking access as needed.

Here's an interactive demo of how Indent works:

We've covered a few different ways for break glass access on GitHub:

  1. Make everyone an admin (not recommended)
  2. Creating teams with the ability to override branch protection rules
  3. Automate granting temporary access to a GitHub team with Indent

Ready to start managing temporary break glass access?

Check out Indent and request a demo

Try Indent for free.