Tag Archives: Splunk

Predicting multiple metrics in Splunk

Splunk has a predict command that can be used to predict a future value of a metric based on historical values. This is not a Machine Learning or an Artificial Intelligence functionality, but a plain-old-statistical analysis.

So if we have a single metric, based on historical results we can produce a nice prediction for the future (of definable span), but predicting multiple metrics in Splunk might not be as straightforward.

Continue reading Predicting multiple metrics in Splunk

How to collect StatsD metrics from rippled server using Splunk

The XRP Ledger (XRPL) is a decentralized, public blockchain and rippled server software (rippled in future references) powers the blockchain. rippled follows the peer-to-peer network, processes transactions, and maintains some ledger history.

rippled is capable of sending its telemetry data using StatsD protocol to 3rd party systems like Splunk.

Continue reading How to collect StatsD metrics from rippled server using Splunk

Plotting Splunk with the same metric and dimension names shows NULL

When you try plotting on a graph Splunk metric split by a dimension with the same name (as the metric itself) will show NULL instead of the dimension.

Splunk timechart visualisation with breakdown by dimension with the same metric and dimension names will show NULL
Continue reading Plotting Splunk with the same metric and dimension names shows NULL

Splunk Eventgen Jinja templating

I’ve recently dipped my toes into Splunk Eventgen (Jinja templating). It’s an awesome app that allows you to generate sample events that can be ingested by Splunk (or for any other reason).

EventGen has two ways of configuring the event content generation:

  • Traditional – where you specify a sample file and provide regexes that will be used to replace static content in the sample file with the required values
  • Jinja Templating – where you use Jinja templating engine to create the events.

While the traditional way is quite straightforward, the event’s format that I was after had a few nuances that made it not suitable for me, thus I had to fiddle with Splunk Eventgen Jinja templating.

Continue reading Splunk Eventgen Jinja templating

Splunk Connect for Kafka

My journey with Splunk Connect for Kafka.

Splunk Connect for Kafka (aka SC4K) allows to collect events from Kafka platform and send them to Splunk. While the sending part (to Splunk) was pretty straight forward to me, the collection part (from Kafka) was very new, as I’ve had no experience with Kafka eco-system. So I guess will start with it.

Continue reading Splunk Connect for Kafka

Splunk – List REST API users and their IPs

Want to get a list REST API users and their IPs?

Run this search

index=_internal
host IN(SH1,SH2,SH3)
sourcetype=splunkd_access
user != "-"
clientip != "IP_of_SH1" clientip != "IP_of_SH2" clientip != “IP_of_SH3”
NOT TERM(127.0.0.1)
NOT TERM(splunk-system-user)
| stats values(clientip) by user

The limitation is if the users are going via a Load Balancer, you will see  Load Balancer’s IP as the clientip