Notifications

Notifications are optional alerts sent to another source (e.g., email, PagerDuty, OpsGenie) when an event occurs. You will only receive notifications from policies that have a notification(s) configured.
While notifications are enabled by default, they will be automatically disabled if they are used in a policy and fail for a period of time.

Creating Notifications

Step 1: Choose a notification

  1. Click your username in the top right-hand corner.
  2. Click Account Profile.
  3. Click Notifications.
  4. Select the tab corresponding to the desired notification.
  5. Click the Add button in the top left-hand corner of the tab content.

Step 2: Configure a notification

Follow one of the tabs below to configure your desired notification.

Add an AWS SNS notification

  1. Type a name for the SNS notification.
  2. Ensure the Enabled checkbox is selected.
  3. Select an AWS Authentication method.
    If you have an existing role/user created from an AWS integration setup, you can just attach another policy to that role/user. Navigate to the role/user in the IAM Console, and under Managed Policies, click Attach Policy then skip to step ix in the IAM Role instructions, or skip to step viii if you chose Access Key.
    1. If you selected IAM Role:

      Creating a Read Only Role (with standard SNS permissions)

      1. In a separate tab from Metricly, log in to your AWS Identity & Access Management (IAM) Console.
      2. Once in the IAM dashboard, navigate to the Roles section.
      3. Click Create New Role.
      4. For Role Name, type Metriclyand click Next Step.
      5. For Role Type, select Role for Cross-Account Access.
      6. Click Select next to the “Provide access between your AWS account and a 3rd party AWS account” option.
      7. On the tab that has the AWS SNS Notification windowopen, copy the Account ID and the External ID provided to you.
      8. On the tab that has the AWS console open, paste your Account ID and External ID into the appropriate fields. Leave Require MFA unchecked. Click Next Step.
      9. For Attach Policy, search “sns”, then select AmazonSNSFullAccess.
      10. Click Next Step to review the role and access the Role ARN.
      11. Copy the Role ARN.
      12. After copying the Role ARN, click Create Role.
        Please make sure you save the role in the AWS console before you attempt the next step.
    2. If you selected Access Key:

      Creating a Read Only User (with standard SNS permissions)

      1. In a separate tab from Metricly, log in to your AWS Identity & Access Management (IAM) Console.
      2. Once in the IAM dashboard, navigate to the Users section.
      3. Click Add user.
      4. For a User Name, type Metricly. Select the Programmatic access checkbox in the Select AWS access type section.
      5. Click Next: Permissions.
      6. Click Attach existing policies directly.
      7. For Attached Policy, search “sns”, then select AmazonSNSFullAccess.
      8. Click Next: Review.
      9. Review the details to ensure you’ve selected all the correct options for the user, and then click Create User.
      10. Download and/or copy the User Security Credentials.
         

        You will not be able to access the Secret Access Key again unless you download the credentials.
      11. Click Close.
  4. Navigate to the SNS console.
  5. Click Topics on the left-hand menu.
  6. Copy the ARN from the ARN column next to the desired topic. Paste the value into the Topic ARN field in the SNS Notification window in Metricly.
  7. Back in the SNS console, select the same topic, and then click Edit topic policy in the Actions menu.
  8. Under the Allow these users to publish messages to this topic section, select Only these AWS users and add the Account ID from Metricly to the field.
  9. Click Update Policy.
  10. Return to Metricly and optionally select Custom from the Payload drop-down menu. A text field will open after selecting Custom. Create a custom JSON payload in the textbox. You can use the following variables to make your notification more dynamic. To test your template, go here or visit the GitHub page.
    Variable Description
    ${eventCategory.name} The event category (  (Info),  (Warning), or  (Critical)).
    ${elementFqn} The Fully Qualified Name (FQN) of the element.
    ${elementId} The type of element (e.g., SERVER, ELB, EC2, RDS, etc.).
    ${elementLocation} The location of the element.
    ${elementName} The friendly name for the element.
    ${policyId} The policy identification number.
    ${policyName} The name of the policy.
    ${eventTimestamp} The time (in UTC) the event occurred.
    ${policyDescription} The description of the policy that generated the event.

    Below is the default payload used in the SNS integration, but it’s a good starting place for creating a custom JSON payload.

    {
      "timestamp": "${eventTimestamp}",
      "category": "${eventCategory}",
      "element": {
        "fqn": "${elementFqn}",
        "name": "${elementId}",
        "location": "${elementLocation}"
      },
      "policy": {
        "name": "${policyName}",
        "description": "${policyDescription}"
      }
    }
  11. Click Save.

Add an Email notification

  1. Type the email address and ensure the Enabled checkbox is selected.
  2. Optionally, select Custom from the Template drop-down menu. Subject and Body text fields will open after selecting Custom. You can use the following variables to make your notification more dynamic. To test your template, go here or visit the GitHub page.
    Variable Description
    ${eventCategory.name} The event category (  (Info),  (Warning), or  (Critical)).
    ${elementFqn} The Fully Qualified Name (FQN) of the element.
    ${elementId} The type of element (e.g., SERVER, ELB, EC2, RDS, etc.).
    ${elementType} The type of element (e.g, SERVER, ELB, RUBY, etc.)
    ${elementLocation} The location of the element.
    ${elementName} The friendly name for the element.
    ${policyId} The policy identification number.
    ${policyName} The name of the policy.
    ${eventTimestamp} The time (in UTC) the event occurred.
    ${policyDescription} The description of the policy that generated the event.
  3. Click Test to test the email notification.
    Note   If the test is unsuccessful, an error message will appear noting what went wrong with the test.
  4. Click Save.

Add a HipChat notification

Setting up a new Hipchat notification is a two-step process:

  1. Generate a Hipchat Room token
  2. Link the created Hipchat room with Metricly
Metricly uses version 2 of the HipChat authentication API. See HipChat’s Rest API documentation for more info.

Step 1: Generate a room token

  1. Log in to your Hipchat account at hipchat.com.
  2. Navigate to the Rooms section.
  3. Under My Rooms, select or create the desired room.
  4. On the left panel, select Tokens.
  5. Type a Label for the room and select Send Notification from the scopes list.
  6. Click Create to generate a token.
  1. After clicking Add Hipchat, ensure the Enabled checkbox is selected.
  2. Copy the room name and token to the corresponding fields.
  3. Click Test and Save.
    If the test is unsuccessful, an error message will appear noting what went wrong with the test.

Add an OpsGenie notification

Setting up a new OpsGenie notification is a two-step process:

  1. Create a Metricly integration in OpsGenie
  2. Link the created OpsGenie integration with Metricly

Step 1: Create a Metricly integration in OpsGenie

  1. Log in to your OpsGenie account.
  2. Under Integrations, select Metricly.
  3. Copy the API key provided on this page to your clipboard. You will use this to set up a new OpsGenie notification in Metricly.
  4. Leave the Teams and Recipients fields as {{teams}} and {{recipients}}, respectively.
    By leaving the Teams and Recipients fields {{teams}} and {{recipients}}, you ensure that the Teams and Recipients you include when setting up a new OpsGenie notification in Metricly are passed to your OpsGenie account, and will receive alerts.
  5. Ensure that Enabled is selected.
  6. Click Save Integration.
  1. After clicking Add OpsGenie, type a name for the OpsGenie notification.
  2. Ensure the Enabled checkbox is selected.
  3. For API Key, paste the API key you copied from your Metricly Integration in OpsGenie in step 1.3.
  4. For Description, add a description of the notification.
  5. For Teams, type a comma-separated list of any OpsGenie teams that should receive notifications.
  6. For Recipients, type a comma-separated list of any OpsGenie recipients that should receive notifications.
    If you include no teams or recipients, and when setting up a Metricly Integration in OpsGenie, left the Teams and Recipients fields as {{teams}} and {{recipients}}, no parties will receive notifications for this policy. However, an alert will still appear in OpsGenie when an event for this policy is generated in Metricly.
  7. For Tags, type a comma-separated list of any tags you want to apply to the OpsGenie alert (e.g. tag1, tag2, tag3).
  8. Click Test and Save.

Add a PagerDuty notification

Setting up a new PagerDuty notification is a two-step process:

  1. Add a new Metricly service to your PagerDuty account
  2. Link the created service with Metricly

Step 1: Create a service in PagerDuty

  1. Log into your PagerDuty account.
  2. Under Services, click Add New Service.
  3. Type a name for the service and select an escalation policy.
  4. Select Metricly for the Service Type.
  5. Click Add Service.
  6. Copy the API key provided on this page to your clipboard. You will use this to set up a new PagerDuty notification in Metricly.
  1. After clicking Add PagerDuty, type a name for the PagerDuty integration and ensure the Enabled checkbox is selected.
  2. Add the service key from the Metricly service you created in PagerDuty.
  3. Click Test and Save.
    The endpoint URL must return an HTTP code 200 to pass the validation.

Add a Slack notification

Setting up a new Slack notification is a three-step process:

  1. Install the Incoming Webhooks app in your Slack account
  2. Add a custom integration to your Slack account
  3. Create a Slack notification in Metricly to push events to Slack

Step 1: Install the Incoming Webhooks app

If you’ve already installed the Incoming Webhooks app, you can skip to step 2.
  1. Go to Slack’s app directory.
  2. Search for Incoming Webhooks. It should dynamically update a drop-down beneath the search bar.
  3. Click Incoming Webhooks.
  4. Click Install next to the desired team.
  5. Select a channel or create a new one.
  6. Click Add Incoming WebHooks Integration.

Step 2: Add an integration to Slack

  1. Log into your Slack client.
  2. Click your username in the top left-hand corner. A menu will open.
  3. Click Apps & Integrations.
  4. Click Configure in the top right-hand corner.
  5. Click the Custom Integrations tab.
  6. Click Incoming WebHooks.
  7. Click Add Configuration.
  8. Select a channel to post your events to, or create a new one.
  9. Optionally, in the Customize Icon section, upload an image (or choose an emoji) to use for your event. If you want to upload an image, click Upload an Image and drag the image on to the screen and crop it as necessary.
     

    To distinguish your Metricly events in the chat, we recommend saving our logo to your computer and uploading it.
    You can also add an emoji or icon via Metricly in Step 3.
  10. In the Customize Name section, type your desired name. We recommend naming it Metricly Event.
  11. Click Copy URL in the Webhook URL section.
  12. Click Save Settings at the bottom of the page.

Step 3: Create a Slack notification in Metricly

If you want to display a custom message for your Slack notification, you’ll have to add and configure a Webhook notification instead, using the URL in 2.11 as your Webhook notification’s URL.
  1. Type a name for the Slack notification. We recommend naming it Slack-{yourChannelName} so it’s easy to find later.
  2. Ensure the Enabled checkbox is selected.
  3. For Webhook URL, copy and paste the URL from step 2.11.
  4. Optionally, input a bot username that will be used when Metricly posts to your Slack channel.
  5. Optionally, input a channel override. This can be a different channel (#other-channel), another user (@otheruser), or multiple users.
  6. Optionally, input either an Icon URL or an Emoji (if you didn’t do that in step 2). Emojis are passed in using Slack’s emoji variables (:smile:).
  7. Click Test and Save to test the Slack notification. The test must return an HTTP code 200 to pass the validation. If the test succeeds, the notification will automatically save.

Add a VictorOps notification

Configuring Metricly to receive VictorOps data and events is a two-step process:

  1. Copy the REST API URL for inbound webhooks in your VictorOps account
  2. Create a Webhook notification in Metricly using the URL from VictorOps

Step 1: Copy the REST API URL for inbound webhooks in your VictorOps account

  1. Login to your VictorOps account.
  2. Click on the Settings button in the top left corner.
  3. Click the Integrations button.
  4. Scroll down to find the REST Endpoint link.
     

    The Webhooks link at the bottom of the page is for outbound messages; we need to enable inbound messages.
  5. Click the REST Endpoint link. A Post URL will be generated on the page.
  6. Create a routing key at the bottom of the page.
  7. Copy the Post URL. You will use this URL in Step 2.

Step 2: Create a Webhook notification in Metricly using the URL from VictorOps

  1. Ensure you’ve created a blank Webhook notification in Metricly.
  2. Type a name for the webhook notification.
  3. Ensure the Enabled checkbox is selected.
  4. For URL, paste the URL you copied in step 1.g.
  5. If the URL you provided requires basic authentication, type the username and password.
  6. Optionally, add one or more headers (key-value pairs) to the webhook notification.
  7. Optionally, select Custom from the Payload drop-down menu. A text field will open after selecting Custom. You can use the following variables and/or VictorOps fields to make your notification more dynamic. To test your template, go here or visit the GitHub page.
      • VictorOps fields (visit the VictorOps knowledge base for more information):
        Field Name Description
        message_type String value. This field allows the following values: INFO, WARNING, ACKNOWLEDGMENT, CRITICAL, RECOVERY.

        This field is required.
        entity_is_host String value. Whether the alerting entity is the host.
        entity_id String value. The name of the alerting entity.
        timestamp Number value. Timestamp of the alert in seconds since epoch.
        state_start_time Number value. The time the entity entered its current state (in seconds since epoch).
        state_message String value. Any additional status information from the alert.
        monitoring_tool String value. The name of the monitoring system software/application.
        entity_display_name String value. Name used in VictorOps to display a human-readable name for the entity.
        ack_msg String value. User-entered comment for acknowledgment.
        ack_author String value. The name of the user that acknowledged the incident.

        The following is an example custom JSON payload using only VictorOps fields.

        {
          "message_type":"INFO",
          "entity_is_host":"Yes",
          "entity_id":"Metricly Test",
          "state_message":"host is up"
        }

        Here’s what the custom payload looks like in VictorOps:

      • Variables:
        Variable Description
        ${eventCategory.name} The event category (  (Info), (Warning), or  (Critical)).
        ${elementFqn} The Fully Qualified Name (FQN) of the element.
        ${elementId} The type of element (e.g., SERVER, ELB, EC2, RDS, etc.).
        ${elementLocation} The location of the element.
        ${elementName} The friendly name for the element.
        ${policyId} The policy identification number.
        ${policyName} The name of the policy.
        ${eventTimestamp} The time (in UTC) the event occurred.
        ${policyDescription} The description of the policy that generated the event.

        The following is an example custom JSON payload using a combination of VictorOps fields and variables.

        {
          "message_type":"INFO",
          "entity_is_host":"Yes",
          "entity_id":"${elementId}",
          "state_message":"${elementName} is up, 
        but an event occurred at ${eventTimestamp}."
        }

        Here’s what the custom payload looks like in VictorOps:

  8. Click Test to test the webhook.
    The endpoint URL must return an HTTP code 200 to pass the validation.
  9. Click Save.

Add a Webhook notification

  1. Type a name for the webhook notification.
  2. Ensure the Enabled checkbox is selected.
  3. For URL, copy and paste the webhook’s payload URL that should receive notifications.
  4. If the URL you provided requires basic authentication, type the username and password.
  5. Optionally, add one or more headers (key-value pairs) to the webhook notification.
  6. Optionally, select Custom from the Payload drop-down menu. A text field will open after selecting Custom. Create a custom JSON payload in the textbox. You can use the following variables to make your notification more dynamic. To test your template, go here or visit the GitHub page.
    Variable Description
    ${eventCategory.name} The event category (  (Info),  (Warning), or  (Critical)).
    ${elementFqn} The Fully Qualified Name (FQN) of the element.
    ${elementId} The type of element (e.g., SERVER, ELB, EC2, RDS, etc.).
    ${elementLocation} The location of the element.
    ${elementName} The friendly name for the element.
    ${policyId} The policy identification number.
    ${policyName} The name of the policy.
    ${eventTimestamp} The time (in UTC) the event occurred.
    ${policyDescription} The description of the policy that generated the event.

    Below is the default payload used in the PagerDuty integration, but it’s a good example of a custom JSON payload. You can modify this and use it as a custom Webhook if you desire.

    {
      "service_key": <your service key here from pager duty>",
      "incident_key": "${policyName} ",
      "event_type": "trigger",
      "description": "${category}: ${elementFqn} : ${policyName}",
      "client": "Metricly Cloud Service",
      "client_url": "https://app.Metricly.com",
      "details": {
        "category": "${category}",
        "elementFqn":"${elementFqn}",
        "elementType": "${elementType}"
      },
      "contexts": [{
        "type": "link",
        "href": "https://app.Metricly.com/#/element/${elementId}/events"
      }]
    }
  7. Click Test to test the webhook.
    The endpoint URL must return an HTTP code 200 to pass the validation.
  8. Click Save.

Using Notifications

In the Policy Editor

Set up notifications by opening the desired policy in Policy Editor. To enable or disable existing notifications, see below.

  1. In the Policy Editor, under Notifications, click Add Notification.
  2. Select the desired notification type.
  3. Once a type is selected, choose a notification you have set up for the selected type. You can also create a new notification directly from here.
  4. Select how often—between 5 minutes and 24 hours or never—the policy should re-notify you if a policy is creating events.
  5. Enable Notify on Clear if you’d like to receive a notification when the policy stops triggering.
  6. Save the policy.

Enabling and disabling notifications

  1. From the user account drop-down menu, select Notifications.
  2. Open the desired notification.
  3. Select (or clear) the Enabled checkbox.
  4. Click Save.