Dashboard: Splunk Insights – Users & Roles

Quick glance into who’s who in the zoo for users, capabilities, roles, and what indexes are searchable. Also calls out users with can_delete capabilities. Mileage may vary, please comment if there are any issues! <dashboard version=”1.1″> <label>Splunk Insights – Users and Roles</label> <row> <panel> <title>Number of Roles</title> <single> <title>Click to Expand</title> <search> <query>| rest splunk_server=local […]

Continue Reading →

Truncated Data Issues

Displays sourcetypes being truncated on ingest, then on selection, shows the related _internal message & the an event that caused it to trigger. <form> <label>Data Issues</label> <description>Truncation, Date Parsing and Timestamp issues</description> <fieldset submitButton=”false”> <input type=”time” token=”field1″> <label></label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel> <title>Choose a problematic sourcetype</title> <table> <search> <query>index=_internal sourcetype=splunkd component=LineBreakingProcessor […]

Continue Reading →

Deployed application status

Created this dashboard to see when or if an application was deployed successfully. Close to splunkninja’s query, this will also show if the host in question also restarted to apply the new app.   <form> <label>Deployed Applications</label> <fieldset submitButton=”false”> <input type=”checkbox” token=”loglevelpicker” searchWhenChanged=”true”> <label>Log Level</label> <choice value=”INFO”>INFO</choice> <choice value=”WARN*”>WARNING</choice> <choice value=”ERROR”>ERROR</choice> <default>INFO,WARN*,ERROR</default> <valuePrefix>log_level=</valuePrefix> <delimiter> OR […]

Continue Reading →

REST API: Table all Splunk User Email Addresses

The following simple Splunk query will put all Splunk User accounts with an email address into a panel for copy and paste purposes (such as copying all email addresses to send in an email). I’ve added a semi colon delimiter in order to literally be copy and paste into an application such as Microsoft Outlook. […]

Continue Reading →

Bucket Status Dashboard

Shows status of buckets per indexer host, when they rolled from warm to cold, and cold to frozen. Gives a timechart and table of each, as well as detailed bucket names per index & host. <form> <label>Bucket Status</label> <fieldset submitButton=”false”> <input type=”time” token=”field1″> <label></label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> <input type=”multiselect” token=”hostpicker” searchWhenChanged=”true”> <label>Which Host</label> […]

Continue Reading →

List of Forwarders that are Deployment Clients

Need a list of Forwarders that are talking to a Deployment Server? Try this: index=_internal sourcetype=splunkd component=DC* Handshake | stats count by host Additional REST query (performed on the DS) will return desired results (Thanks to Lyx for pointing this out!): | rest /services/deployment/server/clients splunk_server=local | table hostname applications.*.serverclasses | untable hostname, applications, serverclass | […]

Continue Reading →

Host not sending logs for x days

This Splunk Query will show hosts that stopped sending logs for at least 48 hours. You’ll want to change the time range to be relevant to your environment, and you may need to tweak the 48 hour range to something that is more appropriate for your environment. | tstats count as countAtToday latest(_time) as lastTime […]

Continue Reading →

List All Splunk Users & Associated Roles

The following Splunk query will show a table of all users and their roles: | rest /services/authentication/users | stats values(roles) as Roles by user *Admin Notes* I’ve found the following query to work better in my environment: | rest /services/authentication/users | stats values(roles) as Roles by title

Continue Reading →

Dashboard and App views by user

This Splunk query / search shows historical access to dashboards and apps on a local splunk server. index=_internal sourcetype=splunk_web_access host=* user=* | rex field=uri_path “.*/(?<title>[^/]*)$” | join title [| rest /servicesNS/-/-/data/ui/views splunk_server=* | search isDashboard=1 isVisible=1 | rename eai:acl.app as app | fields title app ] | rename title as dashboard | stats count by […]

Continue Reading →

Show how much disk space is used by _internal

The following Splunk query will return disk space used by the _internal index. index=_internal source=*license_usage.log type=Usage | eval gb=b/1024/1024/1024 | timechart span=1d sum(gb) as GB by host useother=false | untable _time host gb | top limit=1 host | join time [ search index=_internal source=*license_usage.log type=Usage | eval gb=b/1024/1024/1024 | timechart span=1d sum(gb) as GB by […]

Continue Reading →

Find queues that are nearly full

Easily find Splunk Instances (both forwarders and full installs) that have queues that are over 80% full! index=_internal group=queue | eval percfull=((current_size_kb/max_size_kb)*100) | search percfull>80 | dedup host, name | table _time host name current_size_kb max_size_kb

Continue Reading →

REST Call for Memory & CPU usage on Splunk Servers

This Splunk search will show you use and available CPU and Memory statistics. Depending on your environment you may see multiple Splunk servers: | rest /services/server/status/resource-usage/hostwide | eval cpu_count = if(isnull(cpu_count), “N/A”, cpu_count) | eval cpu_usage = cpu_system_pct + cpu_user_pct | eval mem_used_pct = round(mem_used / mem * 100 , 2) | eval mem_used = […]

Continue Reading →

Splunk Objects With Permissions Granted to Non-existent Roles

Useful search to show a bit of detail on roles and user permissions. | rest /servicesNS/-/-/admin/directory count=0 splunk_server=local | fields eai:acl.app, eai:acl.owner, eai:acl.perms.*, eai:acl.sharing, eai:location, title | eval perms=mvappend(‘eai:acl.perms.read’,’eai:acl.perms.write’) | fields – eai:acl.perms.* | mvexpand perms | where perms!=”*” AND NOT [ | rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title | rename title as perms […]

Continue Reading →

Every index explicitly granted to a role

Self explanatory, maps roles to indexes. Useful if you have a lot of indexes! | rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title,srchIndexesAllowed | rename srchIndexesAllowed as index title as role | mvexpand index | where NOT match(index,”.*\*.*”) I found this at: https://gist.github.com/acharlieh/3254a7ab13297c760376 Credit goes to acharlieh!

Continue Reading →

Average Splunk Web requests by hour

This query is pretty awesome! It helped enlighten us to exactly when our splunk infrastructure is being hit with users index=_internal sourcetype=splunk_web_access [ rest / splunk_server=local | fields splunk_server | rename splunk_server as host ] | bin _time span=1d | stats count by date_hour _time | appendpipe [ fields _time | dedup _time | eval […]

Continue Reading →

All indexes not explicitly granted to a role

| rest /servicesNS/-/-/data/indexes count=0 | stats max(isInternal) as internal, max(disabled) as disabled, max(isReadOnly) as readonly by title | fillnull | where internal=0 AND disabled=0 AND readonly=0 | fields title | rename title as index | join index type=left [ rest /servicesNS/-/-/authorization/roles count=0 splunk_server=local | fields title,srchIndexesAllowed | rename srchIndexesAllowed as index title as role | […]

Continue Reading →