Incidents

Record a fault, and tell your customers about it.

The panel opens some incidents for you, and you can log one yourself. An incident marked warning or error also puts a banner in front of your customers.

Where to find them

Incidents are not in the admin sidebar. Open them from the Nodes page.

1. Navigate to Admin > Nodes.
2. Click the Incidents tab at the top of the page.

What customers see

An incident is visible in two places outside the admin area.

The status page. Customers open it from the Status link in the header. The page shows a banner that reads All systems operational or Some services are degraded. Below the banner, the page lists the open incidents and the state of each node. The page refreshes every 30 seconds. Customers see the node name and its region only.

The panel banner. When the panel opens an incident with a severity of warning or error, it also posts a banner in the panel. Customers can dismiss the banner. When you resolve the incident, the banner disappears. An incident with a severity of info posts no banner.

To turn the status page off, or to make it sign-in only, go to Admin > Settings > General and find the Status Page section.

The list

The list shows these columns: Title, Severity, Status, Source, and Started. The newest incident is at the top.

Tick Open only to hide the resolved incidents.

The Source column tells you where the incident came from:

Source	Meaning
manual	An administrator logged it in the panel.
monitor	A node alert rule opened it.
server_monitor	A game server monitor opened it.

Incidents the panel opens itself

The panel opens an incident on its own in two cases.

A node alert rule fires. The rule must break for its full sustain time. The panel then opens an incident with the severity and the text of the rule. When the metric returns to normal, the panel resolves the incident. Set up these rules on the Monitoring page. All four default rules arrive disabled, so the panel opens nothing until you turn a rule on.

A game server monitor sees a change. Some eggs define a monitor that reads a value from the running game server. If the value changes to a state that the egg treats as a fault, the panel opens an incident with a severity of warning. You resolve this type of incident by hand.

Logging one yourself

1. On the Incidents tab, click New incident.
2. Enter a Title. The title is required, and customers see it.
3. Select a severity: info, warning, or error. The default is warning.
4. Write a Summary. The summary is optional.
5. Click Open.

The new incident starts with the status Open and the source manual.

Keep the title plain and short. It goes on your public status page.

Changing the status

An incident has three statuses:

Status	Meaning
Open	The fault is recorded. Nobody works on it yet.
Investigating	Somebody works on the fault.
Resolved	The fault is corrected.

Two links are on each row:

• Investigate moves an open incident to Investigating. The link is on open rows only.
• Resolve moves an incident to Resolved. The link is on every row that is not resolved yet.

A resolved incident leaves the status page and removes its banner.

WARNING

You cannot re-open a resolved incident. If the fault returns, log a new incident.

Notifications

The panel sends an alert when an incident opens. When the incident moves to resolved, the panel sends a second alert. The two events are incident.opened and incident.resolved.

The panel sends the resolution alert when you resolve the incident, and when the panel resolves it on its own.

To choose who gets these alerts, and on which channel, navigate to Admin > Settings. Click the Webhooks tab, then the Alerts tab. The channels are the in-panel alert, email, Discord, and a webhook.

The Webhooks tab requires the Webhooks license feature.

Subscribe to incident.resolved on the same channel as incident.opened. Your team then sees the start and the end of each fault.

Next steps

• Node Monitoring - the alert rules that open incidents
• Notifications - notification channels and templates