Forward Analytics Data to Splunk

Overview

Follow the steps in this guide to set up an NGINX Controller Integration that forwards data to Splunk.

Before You Begin

This guide assumes that you are already an active Splunk user. If you haven’t already done so, you will need to install and configure Splunk before you proceed.

Create a Splunk Metrics Index

(Optional) If you want to forward metrics and you haven’t already created a Splunk Index for metrics, take the steps below to add one. You can do the same for events, although you could also use the existing, default “main” index.

  1. Open the Splunk web interface and log in.
  2. Select Settings, then select Indexes.
  3. Select New Index.
  4. Add a Name.
  5. For the Index Data Type, select Metrics or Events.
  6. Select Save.

Set up Splunk to Monitor Data

  1. Open the Splunk web interface and log in.
  2. On the Explore Splunk Enterprise menu, select Add Data.
  3. Select Monitor as the data method.
  4. On the Add Data Select Source page, select HTTP Event Collector.
  5. Add a Name.
  6. Add a Description.
  7. Select Next.
  8. On the Add Data Input Settings page, select one or more of the available Splunk Indexes with the appropriate Index Data Type.
  9. Select Review.
  10. On the summary page, copy and save the token value. You’ll configure NGINX Controller with this value later.

Create an Integration

Take the following steps to create an Integration for Splunk:

  1. Open the NGINX Controller user interface and log in.
  2. Select the NGINX Controller menu icon, then select Platform.
  3. On the Platform menu, select Integrations.
  4. On the Integrations menu, select the Create Integration quick action.
  5. Add a name.
  6. (Optional) Add a display name.
  7. (Optional) Add a description.
  8. (Optional) Add tags.
  9. In the Integration Type list, select GENERIC_INTEGRATION.
  10. In the Endpoint URI box, add the Splunk collector URL.
  11. In the Credential Type list, select API_KEY.
  12. In the API Key box, add the Splunk token value.
  13. Select Submit.

Create a Forwarder

Use the NGINX Controller REST API to create a Forwarder.

To create a Forwarder, send a POST request to the /analytics/forwarders endpoint with a request similar to the following example.

Important:
You must reference the Integration in the JSON request body. Requests that do not contain a valid Integration reference will return an error message.

 

{
  "metadata": {
    "name": "splunk-forwarder",
    "displayName": "Splunk - Metrics",
    "description": "Metrics forwarder for Splunk HEC"
  },
  "desiredState": {
    "collectorType": "SPLUNK",
    "integrationRef": {
      "ref": "/platform/integrations/<name-of-splunk-integration>"
    },
  "streams": [
      {
        "inputDataType": "METRICS",
        "outputFormat": "SPLUNK",
        "selector": "names=http.request.count,client.network.latency.min&filter=http.request_method IN ('GET', 'POST')"
      }
    ]
  }
}

Parameters

metadata

  • name: The resource name to use for the Forwarder.
  • displayName (Optional): The friendly name to show for the Forwarder.
  • description (Optional): A brief description of the Forwarder.

desiredState

  • integrationRef.ref: The Integration reference for the Forwarder. Use the format /platform/integrations/<name-of-splunk-integration>. Use the name for the Integration that you created in the Create an Integration procedure.
  • collectorType: The type of collector. For Splunk collector, use SPLUNK.

streams

  • inputDataType: The type of data to forward. NGINX Controller can forward METRICS or EVENTS.

  • outputFormat: The format for the output. For Splunk-formatted output, use SPLUNK.

  • selector: Consists of the following query parameters (optional):

    • names (inapplicable for EVENTS): The list of metrics names that you want to forward.
    • excluded_names (inapplicable for EVENTS): The list of metric names that you don’t want to forward.
    • filter: The conditions to use to refine the metrics data.
    • Example usage when selecting metrics: "names=nginx.*&excluded_names=nginx.upstream.*filter=app='myapp'"
    • Example usage when selecting events: `“filter=type=‘security violation’ AND app=‘my-app’”
Important:

Each metric will be prefixed with a common namespace – such as nginx-controller – before it is sent to Splunk. This prefix is used by Splunk only and is not applied to any of the internal NGINX Controller metrics. Refer to the metrics catalog for the full list of valid metric names.

In case of events, the “nginx-controller” namespace will be placed in the “source” key and sent with each event.

See Also:
See the NGINX Controller Metrics docs for more information.

What’s Next


This documentation applies to the following versions of NGINX Controller Documentation: 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.13, 3.14, 3.15 and 3.16.