List Inputs using REST

As the title says. Pretty nice Splunk Search if you’ve forgotten what inputs you have configured and need a central place to list them. | rest /services/data/inputs/all | convert ctime(starttime) AS “Start Time”  | convert ctime(endtime) AS “End Time” | table index interval source sourcetype title updated starttime endtime “Start Time” “End Time”

Continue Reading →

REST Call for Memory & CPU usage on Splunk Servers

This Splunk search will show you use and available CPU and Memory statistics. Depending on your environment you may see multiple Splunk servers: | rest /services/server/status/resource-usage/hostwide | eval cpu_count = if(isnull(cpu_count), “N/A”, cpu_count) | eval cpu_usage = cpu_system_pct + cpu_user_pct | eval mem_used_pct = round(mem_used / mem * 100 , 2) | eval mem_used = […]

Continue Reading →

IIS Response Time

host=”*”  sourcetype=iis (insertIISurl) | eval time_taken = time_taken/1000  | stats  max(time_taken) AS “Highest Response Time” host=”*”  sourcetype=iis (insertIISurl) | eval time_taken = time_taken/1000  | stats  avg(time_taken) AS “Average Response Time” host=”*”  sourcetype=iis (insertIISurl) | eval time_taken = time_taken/1000  | stats  fastest(time_taken) AS “Fastest Response Time”     Above is 3 panels , Fastest, Average, and Longest response time. […]

Continue Reading →

Splunk Objects With Permissions Granted to Non-existent Roles

Useful search to show a bit of detail on roles and user permissions. | rest /servicesNS/-/-/admin/directory count=0 splunk_server=local | fields eai:acl.app, eai:acl.owner, eai:acl.perms.*, eai:acl.sharing, eai:location, title | eval perms=mvappend(‘eai:acl.perms.read’,’eai:acl.perms.write’) | fields – eai:acl.perms.* | mvexpand perms | where perms!=”*” AND NOT [ | rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title | rename title as perms […]

Continue Reading →

Every index explicitly granted to a role

Self explanatory, maps roles to indexes. Useful if you have a lot of indexes! | rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title,srchIndexesAllowed | rename srchIndexesAllowed as index title as role | mvexpand index | where NOT match(index,”.*\*.*”) I found this at: https://gist.github.com/acharlieh/3254a7ab13297c760376 Credit goes to acharlieh!

Continue Reading →

Average Splunk Web requests by hour

This query is pretty awesome! It helped enlighten us to exactly when our splunk infrastructure is being hit with users index=_internal sourcetype=splunk_web_access [ rest / splunk_server=local | fields splunk_server | rename splunk_server as host ] | bin _time span=1d | stats count by date_hour _time | appendpipe [ fields _time | dedup _time | eval […]

Continue Reading →

All indexes not explicitly granted to a role

| rest /servicesNS/-/-/data/indexes count=0 | stats max(isInternal) as internal, max(disabled) as disabled, max(isReadOnly) as readonly by title | fillnull | where internal=0 AND disabled=0 AND readonly=0 | fields title | rename title as index | join index type=left [ rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title,srchIndexesAllowed | rename srchIndexesAllowed as index title as role | […]

Continue Reading →

Pearson Coefficient of Two Fields

The following SPL query calculates the Pearson coefficient of two fields named x and y. index=* | fields x y | eval n=1 | eval xx=x*x | eval xy=x*y | eval yy=y*y | addcoltotals | tail 1 | eval rho_xy=(xy/n-x/n*y/n)/(sqrt(xx/n-(x/n)*(x/n))*sqrt(yy/n-(y/n)*(y/n))) | fields rho_xy

Continue Reading →

Top Header cpu & memory status

I didn’t like the CPU input from the Splunk TA Nix app, so I created this small ingest from top. The script takes a snapshot of the top command, and looks directly at the header: top -b -n 1 | sed -n ‘1,5p’ and comes back with the first 5 lines of Top: top – 15:20:55 […]

Continue Reading →

Linux Free Disk Space

The following Splunk query shows a percentage of free disk space over a period of time using timechart: index=os sourcetype=df PercentFreeSpace=* mount=”/” | timechart latest(PercentFreeSpace) by host

Continue Reading →

Linux Memory Usage

The following Splunk Search will show memory usage on a linux machine over a period of time using timechart: index=os sourcetype=top pctMEM=*| transaction host _time | streamstats window=1 global=f sum(pctMEM) as MEM | timechart latest(MEM) by host

Continue Reading →

Linux CPU Usage

The following query will output CPU usage per host over a period of time using timechart: index=os sourcetype=top pctCPU=* | transaction host _time | streamstats window=1 global=f sum(pctCPU) as CPU | timechart latest(CPU) by host

Continue Reading →

Show uptime in Days

The following query shows uptime of all systems over a certain period of time (days_uptime). Replace my indexes w/ yours.  index=os OR index=idx_appdev sourcetype=Unix:Uptime OR sourcetype=”WMI:Uptime” |dedup host |eval DaysUp=round(SystemUpTime/86400,2) |eval Years=round(DaysUp/365,2) |eval Months=round(DaysUp/30,2)|search DaysUp > $days_uptime$ | table host DaysUp Years Months SystemUpTime |sort – SystemUpTime |   Looks like: hostname | DaysUP | […]

Continue Reading →