This dashboard will show the server or infrastructure specs of your Splunk environment. This is not intended to replace the Monitoring console, but rather augment as sometimes we need a condensed version of what is going on inside our Splunk environment. I’ve had fun with it on my homelab, so if you find something not […]
Disk Usage per Index by Indexer
Summary: Instead of grabbing data from all time, using the dbinspect command will allow administrators to quickly determine how big an index is. There are additional fields in the dbinspect, so explore that to gain other data pivots. |dbinspect index=_internal | stats sum(sizeOnDiskMB) by splunk_server
Searches to check search concurrency for historical or real time
The following Splunk search will output historical or real time concurrency in a timechart by host. *NOTE* Change the text <search_head> to your search heads name, alternatively use a *. index=_internal host= source=*metrics.log group=search_concurrency “system total” NOT user=* | timechart max(active_hist_searches) by host index=_internal host= source=*metrics.log group=search_concurrency “system total” NOT user=* | timechart max(active_realtime_searches) by […]
Internal Splunk User Stats
This simple Splunk query will show us unique Splunk user logged into Splunk per day, as well as total count of log-ons. index=_audit info=succeeded | timechart span=1d dc(user) as unique_users count(user) as logons_all_users
Number of Hosts Associated with a Serverclass
The following query will list the number of hosts associated with all serverclasses on your Splunk Deployment server. This query should be run on your Deployment Server. | rest /services/deployment/server/clients splunk_server=local | table hostname applications.*.serverclasses | untable hostname applications | rex field=applications “applications\.(?<apps>.+)\.serverclasses” | stats dc(hostname) as hostname by apps
Show Searches with Details (Who | When | What)
The following Splunk search will show a list of searches ran on a splunk server with the following details: Who ran the search What sourcetype was used What index was used What the search string was When the search was last ran index=_audit action=search sourcetype=audittrail search_id=* NOT (user=splunk-system-user) search!=”‘typeahead*” | rex “search\=\'(search|\s+)\s(?P<search>[\n\S\s]+?(?=\’))” | rex field=search […]
Splunk Admin Account Activity – Role Modifications
This Splunk query shows when the admin account performed Create or Modify Roles actions: index=”_audit” action=edit_roles operation=* | table _time user operation object*
Splunk Admin Account Activity – Account Modifications
This Splunk query shows when the admin account performed Account Modification / Deletion / Creation actions: index=_audit user=admin action=edit_user operation=* | table _time user operation object*
Index Modifications
This Splunk query should show which users attempted to modify an index and if that action was successful: index=_audit user=* action=indexes_edit | stats count by index info user action
REST API response time
This is a Splunk query to measure REST API response time from the various rest URI’s in Splunk. index=_internal sourcetype=splunkd_access source=*splunkd_access.log | rex “- – – (?P<Response_Time>.*)” | rex “\”(?<REST_uri>[^\”]+)” | table _time, REST_uri, Response_Time Credit goes to somesoni2 on answers.splunk.com! Query found here: https://answers.splunk.com/answers/112073/splunk-query-to-measure-rest-api-response-time.html
Get Sourcetype and Index Info via TSTATS
Use the following simple tstats query to return the latest time events came in for a given index as well as list all sourcetypes for each index: |tstats values(sourcetype) as Sourcetype latest(_time) as Time groupby index | convert ctime(Time)
Show Splunk User to Role mapping
The following Splunk REST query shows all roles, number of capabilities, and landing app for each user. | rest /services/authentication/users | eval name=coalesce(realname, title) | stats values(roles) as Role first(defaultApp) as “Landing App” count(capabilities) as “Number of Capabilities” by name
List All Hosts Associated with All Indexes
Using the Splunk Tstats command you can quickly list all hosts associated with all indexes: |tstats values(host) where index=* by index
Count of Host added to Splunk by Month
Can we get a Splunk Query that list of hosts added to Splunk in a month. Like Month 1 : 200 devices added Month 2: 400 devices added You would do this: host=* | stats dc(host) as host by date_month (Edits Made and query provided by the GoSplunk Ninja)
Count of Splunk Errors Per Host
The following Splunk query will list the number of errors associated with each host over a given time range: index=_internal sourcetype=”splunkd” log_level=”ERROR” host!=splunk_server | stats count by host | sort – count
Traffic Volume by Forwarder
This Splunk search query will show you the top 10 “chattiest” forwarders on your network. I’ve used this query to determine why some forwarders were sending more data than others. The results are displayed in kilobits, you could use an eval to change it to the appropriate size for your network. index=”_internal” source=”*metrics.lo*” group=tcpin_connections NOT […]