Fouad Matin

@fouadmatin

Elsie Phillips

When your service is down, who you gonna call? Outagebusters (aka your on-call engineers)! To be able to respond quickly, it’s crucial that the people on-call have all of the necessary permissions. While we might all wish that things only broke during business hours, unfortunately that isn’t reality.

So how do those folks get access?

Nobody wants to be woken up at 2 am just to grant production database access to an on-call engineer. You could manually add and remove privileges based on on-call schedule, but who has time for that? And just passing out standing admin access to sensitive systems is clearly not the answer.

So what do you do? Pair Opsgenie + Indent!

Opsgenie gives your scheduled Outagebusters a heads up when things are broken and gives them the critical information they need to jump in and get to fixin. More specifically, it facilitates on-call scheduling, notifications, and incident analytics.

How does it work? It ingests data from your monitoring, ticketing and chat tools and groups similar alerts into incidents. Yay less noise! It then references your on-call schedule and pings the appropriate parties that it’s time to rise and shine.

It also gives you visibility into the nuts and bolts of your on-call response. What’s breaking the most? How fast did folks respond? Who’s being pinged too much? Opsgenie adds clarity and accountability when outages happen.

Using Opsgenie + Indent you can:

• Notify the correct people to respond to an incident
• Give secure, on-demand production and customer data access in seconds based on Opsgenie on-call status
• Auto-revoke access after a configurable duration
• Record comprehensive access logs for compliance audits

So, your app blew up. How do you get access to it? You might be in an organization where you have standing access to Prod (yikes!), or an org where you need to contact someone in IT and get them to elevate your permissions (also yikes).

Here’s what getting access looks like with the Opsgenie + Indent integration. Indent confirms you’re on-call and need access. You now have access, and an auditable log of events is saved for future review or regulatory compliance. It’s that easy!

Or, in a bit more detail, here’s the chain of events from an incident kicking off:

1. You’re on-call and your app starts misbehaving
2. Opsgenie detects the issue, and alerts you
3. You type /access in Slack or click the “request access” button on Indent.com
4. Indent automatically elevates your privileges since the Opsgenie + Indent integration indicates you’re currently on-call
5. You SSH into the server (likely using a secure service like Tailscale or Twingate)

$ ssh ec2-user@prod-apac-kermit
…
$ ps aux | grep misbehaving_process

Cool, so what do you need to do to get that setup? You’ll obviously need Opsgenie and Indent. After that, it’s really your choice for how you prefer to grant access. Is it behind SSO with Okta? Maybe, through a secure networking service like Tailscale or Twingate? Or, possibly directly through something like AWS. Indent integrates with all of the above. What’s needed follows:

1. Create Indent space (choose Slack / email).
2. Go to indent.com/catalog/opsgenie to get the webhook secret for the integration.
3. Follow our documentation for instructions on setting up Indent with Opsgenie.
4. Pick your method of granting access, and follow the guide for setting that up on Indent. Okta Groups, Tailscale, and AWS are all great options.
5. Start automatically granting secure and temporary access during incidents.