Triggered Alert Analytics

• _audit/ Dashboards
• riparino
• 1 0

Primary Dashboards Contains alert analytics for both triggered alerts and saved searches. Please replace $name$ with the saved search naming convention you utilize (ie. 0001 – AlertName). You will need an outputlookup to generate the bottom two tables; it will be based on the query that generates the second table in the dashboard.

<form theme="dark"> <label>Triggered Alert Analytics</label> <description>Metrics tracker for triggered alerts.</description> <fieldset submitButton="false"></fieldset> <row> <panel> <title>alert_fired count for Triggered Alerts</title> <input type="time" token="upperTime" searchWhenChanged="true"> <label></label> <default> <earliest>-7d@h</earliest> <latest>now</latest> </default> </input> <chart> <title>Dashboard for counting per-alert totals, contingent on alerts having the 'Alert Action: Add to Triggered Alerts'</title> <search> <query>index=_audit action=alert_fired ss_app=* ss_name="$name$" | timechart span=1d sum(triggered_alerts) by ss_name useother=f limit=0 | sort -count</query> <earliest>$upperTime.earliest$</earliest> <latest>$upperTime.latest$</latest> <sampleRatio>1</sampleRatio> </search> <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option> <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option> <option name="charting.axisTitleX.visibility">visible</option> <option name="charting.axisTitleY.visibility">visible</option> <option name="charting.axisTitleY2.visibility">visible</option> <option name="charting.axisX.abbreviation">none</option> <option name="charting.axisX.scale">linear</option> <option name="charting.axisY.abbreviation">none</option> <option name="charting.axisY.scale">log</option> <option name="charting.axisY2.abbreviation">none</option> <option name="charting.axisY2.enabled">0</option> <option name="charting.axisY2.scale">inherit</option> <option name="charting.chart">line</option> <option name="charting.chart.bubbleMaximumSize">50</option> <option name="charting.chart.bubbleMinimumSize">10</option> <option name="charting.chart.bubbleSizeBy">area</option> <option name="charting.chart.nullValueMode">gaps</option> <option name="charting.chart.showDataLabels">none</option> <option name="charting.chart.sliceCollapsingThreshold">0.01</option> <option name="charting.chart.stackMode">default</option> <option name="charting.chart.style">shiny</option> <option name="charting.drilldown">none</option> <option name="charting.layout.splitSeries">0</option> <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option> <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option> <option name="charting.legend.mode">standard</option> <option name="charting.legend.placement">right</option> <option name="charting.lineWidth">2</option> <option name="refresh.display">preview</option> <option name="trellis.enabled">0</option> <option name="trellis.scales.shared">1</option> <option name="trellis.size">medium</option> <option name="trellis.splitBy">ss_name</option> </chart> </panel> </row> <row> <panel> <title>All Modified Alerts (-7d)</title> <input type="time" token="lowerTime" searchWhenChanged="true"> <label></label> <default> <earliest>-7d@h</earliest> <latest>now</latest> </default> </input> <table> <search> <query>| rest /servicesNS/-/-/saved/searches | search title="$name$" | rename dispatch.earliest_time AS "frequency", title AS "title", eai:acl.app AS "app", next_scheduled_time AS "nextRunTime", search AS "query", updated AS "lastUpdated", action.email.to AS "emailTo", action.email.cc AS "emailCC", action.email.subject AS "emailSubject", alert.severity AS "SEV" | eval severity=case(SEV == "5", "Critical-5", SEV == "4", "High-4",SEV == "3", "Warning-3",SEV == "2", "Low-2",SEV == "1", "Info-1") | table title lastUpdated, nextRunTime, emailTo action.lookup.filename, query, severity | fillnull value="" | sort -lastUpdated</query> <earliest>-24h@h</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">10</option> <option name="dataOverlayMode">none</option> <option name="drilldown">cell</option> <option name="percentagesRow">false</option> <option name="refresh.display">preview</option> <option name="rowNumbers">false</option> <option name="totalsRow">true</option> <option name="wrap">false</option> </table> </panel> </row> <row> <panel> <table> <title>Daily Alert Modification Report</title> <search ref="Daily Alert Modification Report"></search> <option name="drilldown">none</option> </table> </panel> <panel> <table> <title>From 'all_modified_alerts.csv'</title> <search> <query>| from lookup:all_modified_alerts.csv</query> <earliest>0</earliest> <latest></latest> </search> <option name="count">1</option> <option name="drilldown">cell</option> <option name="totalsRow">true</option> </table> </panel> </row> </form>

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117

<form theme="dark"> <label>Triggered Alert Analytics</label> <description>Metrics tracker for triggered alerts.</description> <fieldset submitButton="false"></fieldset> <row> <panel> <title>alert_fired count for Triggered Alerts</title> <input type="time" token="upperTime" searchWhenChanged="true"> <label></label> <default> <earliest>-7d@h</earliest> <latest>now</latest> </default> </input> <chart> <title>Dashboard for counting per-alert totals, contingent on alerts having the 'Alert Action: Add to Triggered Alerts'</title> <search> <query>index=_audit action=alert_fired ss_app=* ss_name="$name$" | timechart span=1d sum(triggered_alerts) by ss_name useother=f limit=0 | sort -count</query> <earliest>$upperTime.earliest$</earliest> <latest>$upperTime.latest$</latest> <sampleRatio>1</sampleRatio> </search> <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option> <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option> <option name="charting.axisTitleX.visibility">visible</option> <option name="charting.axisTitleY.visibility">visible</option> <option name="charting.axisTitleY2.visibility">visible</option> <option name="charting.axisX.abbreviation">none</option> <option name="charting.axisX.scale">linear</option> <option name="charting.axisY.abbreviation">none</option> <option name="charting.axisY.scale">log</option> <option name="charting.axisY2.abbreviation">none</option> <option name="charting.axisY2.enabled">0</option> <option name="charting.axisY2.scale">inherit</option> <option name="charting.chart">line</option> <option name="charting.chart.bubbleMaximumSize">50</option> <option name="charting.chart.bubbleMinimumSize">10</option> <option name="charting.chart.bubbleSizeBy">area</option> <option name="charting.chart.nullValueMode">gaps</option> <option name="charting.chart.showDataLabels">none</option> <option name="charting.chart.sliceCollapsingThreshold">0.01</option> <option name="charting.chart.stackMode">default</option> <option name="charting.chart.style">shiny</option> <option name="charting.drilldown">none</option> <option name="charting.layout.splitSeries">0</option> <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option> <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option> <option name="charting.legend.mode">standard</option> <option name="charting.legend.placement">right</option> <option name="charting.lineWidth">2</option> <option name="refresh.display">preview</option> <option name="trellis.enabled">0</option> <option name="trellis.scales.shared">1</option> <option name="trellis.size">medium</option> <option name="trellis.splitBy">ss_name</option> </chart> </panel> </row> <row> <panel> <title>All Modified Alerts (-7d)</title> <input type="time" token="lowerTime" searchWhenChanged="true"> <label></label> <default> <earliest>-7d@h</earliest> <latest>now</latest> </default> </input> <table> <search> <query>| rest /servicesNS/-/-/saved/searches | search title="$name$" | rename dispatch.earliest_time AS "frequency", title AS "title", eai:acl.app AS "app", next_scheduled_time AS "nextRunTime", search AS "query", updated AS "lastUpdated", action.email.to AS "emailTo", action.email.cc AS "emailCC", action.email.subject AS "emailSubject", alert.severity AS "SEV" | eval severity=case(SEV == "5", "Critical-5", SEV == "4", "High-4",SEV == "3", "Warning-3",SEV == "2", "Low-2",SEV == "1", "Info-1") | table title lastUpdated, nextRunTime, emailTo action.lookup.filename, query, severity | fillnull value="" | sort -lastUpdated</query> <earliest>-24h@h</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">10</option> <option name="dataOverlayMode">none</option> <option name="drilldown">cell</option> <option name="percentagesRow">false</option> <option name="refresh.display">preview</option> <option name="rowNumbers">false</option> <option name="totalsRow">true</option> <option name="wrap">false</option> </table> </panel> </row> <row> <panel> <table> <title>Daily Alert Modification Report</title> <search ref="Daily Alert Modification Report"></search> <option name="drilldown">none</option> </table> </panel> <panel> <table> <title>From 'all_modified_alerts.csv'</title> <search> <query>| from lookup:all_modified_alerts.csv</query> <earliest>0</earliest> <latest></latest> </search> <option name="count">1</option> <option name="drilldown">cell</option> <option name="totalsRow">true</option> </table> </panel> </row> </form>

Report […]

Continue Reading →

Internal Splunk User Stats

• _internal
• SplunkNinja
• 4 0

This simple Splunk query will show us unique Splunk user logged into Splunk per day, as well as total count of log-ons.

index=_audit info=succeeded | timechart span=1d dc(user) as unique_users count(user) as logons_all_users

1	index=_audit info=succeeded | timechart span=1d dc(user) as unique_users count(user) as logons_all_users

Continue Reading →

List All Splunk Users & Associated Roles

• REST
• john117
• 1 1

The following Splunk query will show a table of all users and their roles:

| rest /services/authentication/users | stats values(roles) as Roles by user

1	| rest /services/authentication/users | stats values(roles) as Roles by user

*Admin Notes* I’ve found the following query to work better in my environment:

| rest /services/authentication/users | stats values(roles) as Roles by title

1	| rest /services/authentication/users | stats values(roles) as Roles by title

Continue Reading →

Splunk Admin Account Activity – Role Modifications

• _internal
• john117
• 1 0

This Splunk query shows when the admin account performed Create or Modify Roles actions:

index="_audit" action=edit_roles operation=* | table _time user operation object*

1	index="_audit" action=edit_roles operation=* | table _time user operation object*

Continue Reading →

Splunk Admin Account Activity – Account Modifications

• _internal
• john117
• 2 0

This Splunk query shows when the admin account performed Account Modification / Deletion / Creation actions:

index=_audit user=admin action=edit_user operation=* | table _time user operation object*

1	index=_audit user=admin action=edit_user operation=* | table _time user operation object*

Continue Reading →

Index Modifications

• _internal
• john117
• 0 1

This Splunk query should show which users attempted to modify an index and if that action was successful:

index=_audit user=* action=indexes_edit | stats count by index info user action

1	index=_audit user=* action=indexes_edit | stats count by index info user action

Continue Reading →

Splunk Server Restart Duration

• _internal/ audittrail/ splunkd
• ItsJohnLocke
• 5 0

As titled, the following Splunk search query will show the restart duration (using the transaction command) of the Splunk service itself.  

index=_audit (action="splunkShuttingDown" OR action="splunkStarting") | eval Date=strftime(_time, "%Y/%m/%d") | transaction splunk_server startswith=action="splunkShuttingDown" endswith=action="splunkStarting" | eval duration=round(duration/60, 2) |table Date splunk_server duration| rename duration as "Splunk Restart Duration" splunk_server as "Splunk Server"

1

index=_audit (action="splunkShuttingDown" OR action="splunkStarting") | eval Date=strftime(_time, "%Y/%m/%d") | transaction splunk_server startswith=action="splunkShuttingDown" endswith=action="splunkStarting" | eval duration=round(duration/60, 2) |table Date splunk_server duration| rename duration as "Splunk Restart Duration" splunk_server as "Splunk Server"

Continue Reading →

Splunk Query Count by users

• _internal
• databeastmaster
• 0 0

index=_audit search=* NOT (search_id='scheduler* OR search_id='Summary*) user=admin | timechart span=1d count by user usenull=f

1	index=_audit search=* NOT (search_id='scheduler* OR search_id='Summary*) user=admin | timechart span=1d count by user usenull=f

Continue Reading →

Failed Attempts to Logon to Splunk Web

• _internal
• SplunkNinja
• 4 Comments
• 2 0

The following Splunk Search Query will return all users who have failed to logon to the Splunk Web console. This query will also include an average (from eventstats).  

index=_audit action="login attempt" info=failed | timechart count(user) as Failed_Attempts| eventstats avg(Failed_Attempts) as Average

1	index=_audit action="login attempt" info=failed | timechart count(user) as Failed_Attempts| eventstats avg(Failed_Attempts) as Average

Continue Reading →