kv_store Archives - ISbyR https://isbyr.com/tag/kv_store/ Infrequent Smarts by Reshetnikov Tue, 24 Jun 2025 15:18:21 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.4 Add new LLM models to Splunk MLTK https://isbyr.com/add-new-llm-models-to-splunk-mltk/ https://isbyr.com/add-new-llm-models-to-splunk-mltk/#comments Tue, 24 Jun 2025 14:57:26 +0000 https://isbyr.com/?p=1234 Splunk MLTK 5.6.0+ allows you to configure LLM inference endpoints, but the list is somewhat limited. Below, I’ll explain how you can add new LLM models to Splunk MLTK. The Issue You can configure any of the pre-added models in the Splunk UI by going to the MLTK App and then hitting the “Connection Manager” … Continue reading Add new LLM models to Splunk MLTK →

The post Add new LLM models to Splunk MLTK appeared first on ISbyR.

]]> Splunk MLTK 5.6.0+ allows you to configure LLM inference endpoints, but the list is somewhat limited. Below, I’ll explain how you can add new LLM models to Splunk MLTK.

The Issue

You can configure any of the pre-added models in the Splunk UI by going to the MLTK App and then hitting the “Connection Manager” tab.

When you select a service, you can see a list of pre-defined models. These are already somewhat outdated, for example, for Gemin, you don’t have any of the 2.5 models.

So, “how do we add new LLM models to Splunk MLTK?” you might ask.

The Solution

Easy-ish…

A bit of background

This configuration is managed in a Splunk KV Store collection (named mltk_ai_commander_collection), and in essence, it’s a big JSON that has all the providers and the models.

For example, here is the snippet for the Gemini Service and the first of its models

        "Gemini": {
            "Endpoint": {
                "value": "https://generativelanguage.googleapis.com/v1beta/models",
                "type": "string",
                "required": false,
                "description": "The API endpoint for sending chat completion requests to Google's Gemini language model."
            },
            "Access Token": {
                "value": "",
                "type": "string",
                "required": true,
                "hidden": true,
                "description": "The authentication token required to access the Gemini API."
            },
            "Request Timeout": {
                "value": 200,
                "type": "int",
                "required": false,
                "description": "The maximum duration (in seconds) before a request to the Gemini API times out."
            },
            "is_saved": {
                "value": true,
                "type": "boolean",
                "required": false,
                "description": "Is Provider details stored"
            },
            "models": {
                "gemini-pro": {
                    "Response Variability": {
                        "value": 0,
                        "type": "int",
                        "required": true,
                        "description": "Adjusts the response's randomness, impacting how varied or deterministic responses are."
                    },
                    "Maximum Result Rows": {
                        "value": 10,
                        "type": "int",
                        "required": false,
                        "description": "The maximum number of result entries to retrieve in a response."
                    },
                    "Max Tokens": {
                        "value": 2000,
                        "type": "int",
                        "required": false,
                        "description": "The limit on the number of tokens that can be generated in a response."
                    },
                    "Set as default": {
                        "value": false,
                        "type": "boolean",
                        "required": false
                    }
                },

So if we want to add a new model, all we need to do is add another element to the models array.

Now, here’s a big disclaimer: There might be a better/easier way to do that. This is probably not supported. Some models might not necessarily work with this approach, etc….

While there is a Loolup Editor app, it will only help you (to edit KV store collections) if there is a lookup configured for it. Which is not the case for the mltk_ai_commander_collection one.

High-level steps

Another way (and the one we will take) is to use Splunk REST API, and at a high level, it consists of the following steps:

  1. Get the current configuration (and the _key of the collection item) in a JSON format
  2. Update in a text editor the JSON payload
  3. Update the KV collection with the new JSON

Detailed steps

I will provide examples using Postman, but you can use curl or any other method of your choice for interacting with the REST API.

Get the current configuration

Run a GET call to the collection/data endpoint

The actual URL is https://localhost:8089/servicesNS/nobody/Splunk_ML_Toolkit/storage/collections/data/mltk_ai_commander_collection

Copy the results and take a note of the _key at the end of the JSON.

Update the JSON

Paste the JSON in a text editor of your choice.

Go to the Provider for which you want to add a new Model (Gemini) in our case,

Duplicate the model object inside the Service object and change the model name.

For example, here I copied/pasted the gemini-2.0-flash to the end of the Gemini service object and renamed it to be gemini-2.0-flash.

NOTE: You must ensure that the model name you provide here is exactly the same as it would appear when calling the inference API for the LLM Service.

For example, for Gemini

Update the KV collection

Now we need to update the collection with the updated JSON payload.

Send a POST request to the collection/data endpoint

  • replace the _key part of the URL with the value that you have in your JSON
  • remove the square brackets ([]) that surround the JSON

The actual URL is something like that: https://localhost:8089/servicesNS/nobody/Splunk_ML_Toolkit/storage/collections/data/mltk_ai_commander_collection/68540d2d0d2a214efd0d3b61.

Now, refresh the Connection Management page and enjoy a fresh new model at your disposal

Simply use the new model in the | ai command

And here is a sneak peek into an LLM Telemetry dashboard I’m working on

I hope that helped you to understand how to add new LLM models to Splunk MLTK.

The post Add new LLM models to Splunk MLTK appeared first on ISbyR.

]]> https://isbyr.com/add-new-llm-models-to-splunk-mltk/feed/ 2 Manage Splunk KV Store using REST API https://isbyr.com/splunk-kv-store-using-rest-api/ https://isbyr.com/splunk-kv-store-using-rest-api/#respond Sun, 17 Mar 2019 19:17:57 +0000 http://isbyr.com/?p=433 I’ve started working with Splunk KV store for one of my recent projects. It is a robust system with an extensive API. since I was learning and documenting my fundings anyway I thought, why not put up a blog post about how to manage Splunk KV Store using REST API. I’ve used Postman to help me work … Continue reading Manage Splunk KV Store using REST API →

The post Manage Splunk KV Store using REST API appeared first on ISbyR.

]]> I’ve started working with Splunk KV store for one of my recent projects. It is a robust system with an extensive API. since I was learning and documenting my fundings anyway I thought, why not put up a blog post about how to manage Splunk KV Store using REST API.

I’ve used Postman to help me work with the API, but for better readability, the examples here are in form of curl.

But fear not, as a bonus you can also find at the end all the REST API commands exported from my Postman collection that I’ve used to manage Splunk KV Store using REST API.

I will be using these placeholders throughout this post

  • SPLUNK_SH – the hostname of your Splunk Search Head
  • SH_APP – Splunk Search Head Application name (could be as simple as “search” for you)
  • KVSTORE_COLL – your KV Store collection name

Set-up the Splunk KV Store Collection

Here we will be working with the  /storage/collections/config/ URI

Create KV Store

First things first, we need to create a KV Store.

curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/config/ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'name=KVSTORE_COLL'

This will create a KV Store collection called KVSTORE_COLL in the SH_APP Search Head app.

Configure the KV Store

Once we have a blank KVSTORE_COLL collection we need to configure it, define the fields .

That will define the fields of the collection and their type

curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/config/KVSTORE_COLL \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'enforceTypes=true&field.def_id=number&field.scheduled_time=time&field.earliest_time=time&field.latest_time=time&field.sla_status=number&undefined='

Create a Lookup for the KV Store

Once we have the KV Store created and defined, in order to start using it in Splunk searches we will need to create a lookup for it.

When creating the lookup we specify the below

  • name:KVSTORE_LOOKUP – a name for your lookup
  • external_type:kvstore – must be kvstore in this case
  • collection:KVSTORE_COLL – name of the collection that you are defining the lookup for
  • fields_list:_key, def_id, scheduled_time, earliest_time, sla_status – comma and space separated list of fields that are exposed via the lookup. Please note how I’ve exposed the “internal” _key field, to be able to use it in the Splunk searches later.
curl -X POST \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/data/transforms/lookups \

  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'name=KVSTORE_LOOKUP&external_type=kvstore&collection=KVSTORE_COLL&fields_list=_key%2C%20def_id%2C%20scheduled_time%2C%20earliest_time%2C%20sla_status&undefined='

 

Use the Splunk KV Store Collection

Note that here we are switching to the /storage/collections/data/ URI

Get all items

Getting all items

curl -X GET \
  https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json' \

Get a single item

Here we are passing the key to the item that we want to get. Usually the KV Store key will be Splunk Generated GUID, but in my case I was overwriting it (to be it easier to search) so it looks more human-readable/meaningfull (1004_1552346100
in this case)

curl -X GET \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100 \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Get multiple items using a query

Sometimes you want to get only specific items and Splunk KV API provides you with some basic query language you can use.
In this case I used the following query { “$and”: [{“def_id”:{“$gt”:1049}},{“def_id”:{“$lt”:1060}},{“sla_status”:”0″}]}  which basically retrieved all the items that their def_id  is 1050-1060 AND sla_status = 0.

You can find more information about the query language syntax in the official documentation

curl -X GET \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100?query={%20%22$and%22:%20[{%22def_id%22:{%22$gt%22:1049}},{%22def_id%22:{%22$lt%22:1060}},{%22sla_status%22:%220%22}]}' \ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Update a single item

When you want to update an item you need to provide the key and the new value of the item. Note: It’s more of a replace then an update – if, let’s say, you’ve had 5 fields in the item and when updating you’ve provided value only for 3, the whole value of the item is replaced and now you have only 3 fields

curl -X POST \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1105_1551931200 \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'
  -d '{
    "def_id": 1105,
    "earliest_time": 1552338900,
    "schedule_time": 1552366207,
    "sla_status": "0",
    "_key": "1105_1551931200"
}'

Delete items

Here we are deleting multiple items using a query {“earliest_time”:{“$lt”:1551877200}} (so all the items with earliest_time older than 1551877200). If you will not provide the query parameter all the items in the collection will be deleted,

curl -X DELETE \

https://SPLUNK_SH:8089/servicesNS/nobody/SH_APP/storage/collections/data/KVSTORE_COLL/1004_1552346100?query={%22earliest_time%22:{%22$lt%22:1551877200}}' \ \
  -H 'Authorization: Basic bWU6bXlwYXNz' \
  -H 'Content-Type: application/json'

Postman export with the commands required to manage Splunk KV Store using REST API

You can find the Postman export below as well as here: https://gist.github.com/ilyaresh/0d15b73229771fd0315e1f7e5954ae5e

Related posts about Splunk

The post Manage Splunk KV Store using REST API appeared first on ISbyR.

]]> https://isbyr.com/splunk-kv-store-using-rest-api/feed/ 0