Here’s an incredibly simple Splunk query to count the number of characters in an event:
|
1 |
index=* | eval CharCount=len(_raw) |
Dashboard for Splunk Infrastructure/Server Specs at a Glance
This dashboard will show the server or infrastructure specs of your Splunk environment. This is not intended to replace the Monitoring console, but rather augment as sometimes we need a condensed version of what is going on inside our Splunk environment. I’ve had fun with it on my homelab, so if you find something not […]
Detect Indexers in Maintenance Mode
This query will show any clustered indexers that are currently in maintenance mode. For it to work as an alert you will need to schedule it. It will not work if you run it in real time.
|
1 2 3 4 5 |
sourcetype=splunkd reason="'Maintenance mode*" | dedup host | eval maintenance_mode_enabled=if(reason="'Maintenance mode started'", "true", "false") | where maintenance_mode_enabled="true" | table _time, host, reason, maintenance_mode_enabled |
Breathing Fire Dragon when Starting dbx_task_server
|
1 |
index=_internal sourcetype=dbx_server Starting dbx_task_server |
Will return events that display a little dragon ascii art:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|\___/| (,\ /,)\ / / \ (@_^_@)/ \ W//W_/ \ (//) | \ (/ /) _|_ / ) \ (// /) '/,_ _ _/ (~^-. (( // )) ,-{ _ `. (( /// )) '/\ / | (( ///)) `. { } ((/ )) .----~-.\ \-' ///.----..> \ ///-._ _ _ _} |
svchost Injection
The following Splunk Search will highlight svchost injections:
|
1 |
sourcetype=wineventlog EventCode=4688 OR EventCode=592 New_Process_Name=*svchost.exe Creator_Process_Name!=*services.exe | table ComputerName New_Process_Name, Creator_Process_Name user | sort count |
Windows Dashboard showing Who (was) logged on to ?
Dashboard with 3 separate columns which allow you to drill into 3 separate assets to find out who was logged on, when they logged on, and how they logged on. Accounts for remote logins, local logins, and unlocks/reconnects accounted for but not Type 3 (network logons for shared file access etc). Time picker set so […]
Multiple Malware Detections on a Single Host
This is a simple enough query for detecting a host with multiple infections/detections. The reason for the bucket and incorporating a search over a longer time span (say 60m) is I found it to provide better results and less false negatives if the infrastructure isn’t setup to ingest data in near real-time.
|
1 |
index=malware category="something_high_fidelity" | bucket _time span=15m | stats count by dest | where count>=3 |
Baselining Dashboard
This is better and more flexible option then timewrap in my opinion. Performance ain’t too shabby either.
|
1 |
index=foo earliest=-1d latest=now | timechart span=10m count as Current | appendcols [search index=foo earliest=-1mon-1d latest=-mon | timechart span=10m count as "-1 Month"] appendcols [search index=foo earliest=-1w-1d latest=-w | timechart span=10m count as "-1 Week"] |
IPS Traffic Increase
You can use this for any type of baselining alerts around a predefined standard deviation. I used the IDS data model but the same logic can be applied to any random index.
|
1 |
|`tstats` count from datamodel=Intrusion_Detection.IDS_Attacks where IDS_Attacks.ids_type="network" by IDS_Attacks.dest,_time span=10m | stats count by IDS_Attacks.dest,_time| eval threshold=relative_time(now(),"-10m") | stats max(eval(if(_time>=threshold, count null()))) as latest | stats avg(eval(if(_time<threshold, count null()))) as avg | stats stdev(eval(if(_time<threshold, count null()))) as stdev by IDS_Attacks.dest | eval outlier=if((latest>(avg+(3*stdev))), "yes", "no") | search outlier="yes" |
You should be able to do something similar on a single sourcetype as such
|
1 |
sourcetype=foo bin _time span=10m | stats count by dest,_time| eval threshold=relative_time(now(),"-10m") | stats max(eval(if(_time>=threshold, count null()))) as latest | stats avg(eval(if(_time<threshold, count null()))) as avg | stats stdev(eval(if(_time<threshold, count null()))) as stdev by dest | eval outlier=if((latest>(avg+(3*stdev))), "yes", "no") | search outlier="yes" |
Nessus Security Center Dashboard
Description: This dashboard is intended make it easier to search the results from Nessus Security Center. It doesn’t require any additional addons.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
<form> <label>Nessus Scan Results</label> <fieldset submitButton="true" autoRun="false"> <input type="checkbox" token="t_severity"> <label>Severity</label> <choice value="Critical">Critical</choice> <choice value="High">High</choice> <choice value="Medium">Medium</choice> <choice value="Low">Low</choice> <prefix>(</prefix> <suffix>)</suffix> <initialValue>Critical,High,Medium,Low</initialValue> <valuePrefix>severity.name=</valuePrefix> <delimiter> OR </delimiter> </input> <input type="multiselect" token="t_scan_name"> <label>Scan Name</label> <choice value="*">All</choice> <fieldForLabel>Scan Name</fieldForLabel> <fieldForValue>Scan Name</fieldForValue> <search> <query>sourcetype=tenable:sc:vuln $t_severity$" | dedup scan_result_info.name | rename scan_result_info.name as "Scan Name" | table "Scan Name" |sort "Scan Name"</query> <earliest>$t_time_selector.earliest$</earliest> <latest>$t_time_selector.latest$</latest> </search> <prefix>(</prefix> <suffix>)</suffix> <valuePrefix>scan_result_info.name="</valuePrefix> <valueSuffix>"</valueSuffix> <delimiter> OR </delimiter> <default>*</default> </input> <input type="radio" token="t_search_type" searchWhenChanged="false"> <label>Search Systems By</label> <choice value="netbiosName">netbiosName</choice> <choice value="ip">ip</choice> <initialValue>netbiosName</initialValue> </input> <input type="multiselect" token="t_system_search"> <label>$t_search_type$</label> <fieldForLabel>$t_search_type$</fieldForLabel> <fieldForValue>formatted_$t_search_type$</fieldForValue> <search> <query>sourcetype=tenable:sc:vuln $t_severity$ $t_scan_name$ | dedup $t_search_type$ | eval formatted_$t_search_type$=$t_search_type$ | replace "*\\*" with "*\\\\*" in formatted_$t_search_type$ | table $t_search_type$, formatted_$t_search_type$ | sort $t_search_type$</query> <earliest>$t_time_selector.earliest$</earliest> <latest>$t_time_selector.latest$</latest> </search> <choice value="*">All</choice> <delimiter> OR </delimiter> <prefix>(</prefix> <suffix>)</suffix> <valuePrefix>$t_search_type$=</valuePrefix> <default>*</default> </input> <input type="checkbox" token="t_time_range"> <label>Publish Date Time Ranges</label> <choice value="(plugin_age<=30)">< 30 days</choice> <choice value="(plugin_age>=30 AND plugin_age<=90)">30-90 days</choice> <choice value="(plugin_age>=90 AND plugin_age<=365)">90-365 days</choice> <choice value="(plugin_age>=365)">>365 days</choice> <delimiter> OR </delimiter> <initialValue>(plugin_age<=30),(plugin_age>=30 AND plugin_age<=90),(plugin_age>=90 AND plugin_age<=365),(plugin_age>=365)</initialValue> </input> <input type="time" token="t_time_selector"> <label>Nessus Scan Age</label> <default> <earliest>-7d@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel> <title>Search Results</title> <table> <search> <query>sourcetype=tenable:sc:vuln $t_system_search$ $t_severity$ $t_scan_name$ severity.id>0 | eval scan_time=strftime(lastSeen,"%m/%d/%y %H:%M:%S") | eval plugin_age=tostring(now()-pluginModDate, "duration") | eval pluginModDate=strftime(pluginModDate,"%m/%d/%y %H:%M:%S") | rex field=plugin_age mode=sed "s/\+.*$//" | where $t_time_range$ | table scan_time, netbiosName, ip, pluginID, pluginName, pluginInfo, scan_result_info.name, port, severity.name |sort severity.name pluginName</query> <earliest>$t_time_selector.earliest$</earliest> <latest>$t_time_selector.latest$</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">100</option> <option name="dataOverlayMode">none</option> <option name="drilldown">none</option> <option name="percentagesRow">false</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="totalsRow">false</option> <option name="wrap">true</option> </table> </panel> </row> </form> |
FireEye Internals Monitoring
Summary: FireEye produces 2 types of logs: security event logs (the primary function of FireEye), and internal system logs (Logs about the appliance). Most users do not use the internal system logs, or are even aware that they are available. Sometimes, the appliances are configured to send both logs via syslog, and the messages are […]
Disk Usage per Index by Indexer
Summary: Instead of grabbing data from all time, using the dbinspect command will allow administrators to quickly determine how big an index is. There are additional fields in the dbinspect, so explore that to gain other data pivots.
|
1 |
|dbinspect index=_internal | stats sum(sizeOnDiskMB) by splunk_server |
Apdex Score
Apdex Score Apdex is a measure of response time based against a set threshold. It measures the ratio of satisfactory response times to unsatisfactory response times. The response time is measured from an asset request to completed delivery back to the requestor. It determines user satisfaction, and is based on request type & response time. All […]
Data Usage for Indexer and Forwarders
In my previous role I created this dashboard to identify how much data a Splunk forwarder had sent to my indexers. This was a daily check that either myself of someone on my team would review. This check helped us identify a misconfiguration across all of my production Windows servers. I was able to drilldown […]
Windows Sysmon Process Dashboard
Working with a customer I started this dashboard to give a high level overview of Windows Sysmon data. I have been evolving the dashboard in my home environment and will take any feedback to improve the effectiveness of this dashboard. First is getting sysmon data into your splunk environment. My home computers are running Windows […]
Get KV Store Metrics
This Splunk REST query will return KV Store Metrics:
|
1 2 3 4 5 6 7 |
| rest /services/server/introspection/kvstore/collectionstats | mvexpand data | spath input=data | rex field=ns "(?<App>.*)\.(?<Collection>.*)" | eval dbsize=round(size/1024/1024, 2) | eval indexsize=round(totalIndexSize/1024/1024, 2) | stats first(count) AS "Number of Objects" first(nindexes) AS Accelerations first(indexsize) AS "Acceleration Size (MB)" first(dbsize) AS "Collection Size (MB)" by App, Collection |
List of installed non-core applications
This Splunk REST query will return all non-core applications:
|
1 |
| rest /services/apps/local | search disabled=0 core=0 | table label title version |
Searches to check search concurrency for historical or real time
The following Splunk search will output historical or real time concurrency in a timechart by host. *NOTE* Change the text <search_head> to your search heads name, alternatively use a *.
|
1 |
<span class="il">index</span>=<span class="il">_internal</span> host=<search_head> source=*metrics.log group=search_concurrency "system total" NOT user=* | timechart max(active_hist_searches) by host |
|
1 |
<span class="il">index</span>=<span class="il">_internal</span> host=<search_head> source=*metrics.log group=search_concurrency "system total" NOT user=* | timechart max(active_realtime_searches) by host |
LDAP Search Dashboard
Description: This dashboard is designed to simplify Splunk’s LDAPSEARCH command. LDAP must be configured in your Splunk instance for this to work.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
<form> <label>LDAP objectClass/CN/OU Search</label> <description>LDAPSEARCH Dashboard.</description> <fieldset submitButton="true" autoRun="false"> <input type="radio" token="objectClass_field"> <label>objectClass</label> <default>*</default> <choice value="*">Any objectClass</choice> <choice value="user">Users</choice> <choice value="computer">Computers</choice> </input> <input type="text" token="cn_field"> <label>CN</label> <default>*</default> </input> <input type="text" token="ou_field"> <label>OU</label> <default>*</default> </input> </fieldset> <row> <panel> <table> <search> <query>| ldapsearch search="(&(cn=$cn_field$)(objectClass=$objectClass_field$))" | rex field=distinguishedName max_match=50 "(?<ou_extraction>OU=([^,]+))" | search ou_extraction="OU=$ou_field$" | rename ou_extraction AS "ou" | table cn, dNSHostName, operatingSystem, ou, objectClass, isCriticalSystemObject, lastLogon, pwdLastSet, description</query> <earliest>-15m</earliest> <latest>now</latest> </search> <option name="drilldown">none</option> <option name="refresh.display">progressbar</option> </table> </panel> </row> </form> |
Show your triggered alerts
This search shows all the alerts that where triggered in your splunk environment:
|
1 |
index=_audit action=alert_fired ss_app=* | eval ttl=expiration-now() | search ttl>0 | convert ctime(trigger_time) | table trigger_time ss_name severity | rename trigger_time as "Alert Time" ss_name as "Alert Name" severity as "Severity" |
