Manage Splunk KV Store using REST API

I’ve started working with Splunk KV store for one of my recent projects. It is a robust system with an extensive API. since I was learning and documenting my fundings anyway I thought, why not put up a blog post about how to manage Splunk KV Store using REST API.

I’ve used Postman to help me work with the API, but for better readability, the examples here are in form of curl.

But fear not, as a bonus you can also find at the end all the REST API commands exported from my Postman collection that I’ve used to manage Splunk KV Store using REST API.

I will be using these placeholders throughout this post

  • SPLUNK_SH – the hostname of your Splunk Search Head
  • SH_APP – Splunk Search Head Application name (could be as simple as “search” for you)
  • KVSTORE_COLL – your KV Store collection name

Set-up the Splunk KV Store Collection

Here we will be working with the  /storage/collections/config/ URI

Create KV Store

First things first, we need to create a KV Store.

curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/config/ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'name=KVSTORE_COLL'

This will create a KV Store collection called KVSTORE_COLL in the SH_APP Search Head app.

Configure the KV Store

Once we have a blank KVSTORE_COLL collection we need to configure it, define the fields .

That will define the fields of the collection and their type

curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/config/KVSTORE_COLL \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'enforceTypes=true&field.def_id=number&field.scheduled_time=time&field.earliest_time=time&field.latest_time=time&field.sla_status=number&undefined='

Create a Lookup for the KV Store

Once we have the KV Store created and defined, in order to start using it in Splunk searches we will need to create a lookup for it.

When creating the lookup we specify the below

  • name:KVSTORE_LOOKUP – a name for your lookup
  • external_type:kvstore – must be kvstore in this case
  • collection:KVSTORE_COLL – name of the collection that you are defining the lookup for
  • fields_list:_key, def_id, scheduled_time, earliest_time, sla_status – comma and space separated list of fields that are exposed via the lookup. Please note how I’ve exposed the “internal” _key field, to be able to use it in the Splunk searches later.
curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/data/transforms/lookups \

  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'name=KVSTORE_LOOKUP&external_type=kvstore&collection=KVSTORE_COLL&fields_list=_key%2C%20def_id%2C%20scheduled_time%2C%20earliest_time%2C%20sla_status&undefined='


Use the Splunk KV Store Collection

Note that here we are switching to the /storage/collections/data/ URI

Get all items

Getting all items

curl -X GET \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json' \

Get a single item

Here we are passing the key to the item that we want to get. Usually the KV Store key will be Splunk Generated GUID, but in my case I was overwriting it (to be it easier to search) so it looks more human-readable/meaningfull (1004_1552346100
in this case)

curl -X GET \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100 \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Get multiple items using a query

Sometimes you want to get only specific items and Splunk KV API provides you with some basic query language you can use.
In this case I used the following query { “$and”: [{“def_id”:{“$gt”:1049}},{“def_id”:{“$lt”:1060}},{“sla_status”:”0″}]}  which basically retrieved all the items that their def_id  is 1050-1060 AND sla_status = 0.

You can find more information about the query language syntax in the official documentation

curl -X GET \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100?query={%20%22$and%22:%20[{%22def_id%22:{%22$gt%22:1049}},{%22def_id%22:{%22$lt%22:1060}},{%22sla_status%22:%220%22}]}' \ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Update a single item

When you want to update an item you need to provide the key and the new value of the item. Note: It’s more of a replace then an update – if, let’s say, you’ve had 5 fields in the item and when updating you’ve provided value only for 3, the whole value of the item is replaced and now you have only 3 fields

curl -X POST \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1105_1551931200 \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'
  -d '{
    "def_id": 1105,
    "earliest_time": 1552338900,
    "schedule_time": 1552366207,
    "sla_status": "0",
    "_key": "1105_1551931200"

Delete items

Here we are deleting multiple items using a query {“earliest_time”:{“$lt”:1551877200}} (so all the items with earliest_time older than 1551877200). If you will not provide the query parameter all the items in the collection will be deleted,

curl -X DELETE \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100?query={%22earliest_time%22:{%22$lt%22:1551877200}}' \ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Postman export with the commands required to manage Splunk KV Store using REST API

You can find the Postman export below as well as here:

Related posts about Splunk

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.