Summary:
FireEye produces 2 types of logs: security event logs (the primary function of FireEye), and internal system logs (Logs about the appliance). Most users do not use the internal system logs, or are even aware that they are available. Sometimes, the appliances are configured to send both logs via syslog, and the messages are mixed. In an attempt to monitor the health of all security appliances, this dashboard was created to monitor the health of FireEye beyond the basic stoplight health that is provided by the vendor.
Use Case:
Understand what is going on with the FireEye appliances, monitor detailed appliance health, and determine when vendor support is needed. Provide positive confirmation that the system is performing as expected.
Solution:
Configure FireEye to send logs via HTTPS instead of syslog. This will provide data separation at the ingest pipeline. Inside the FireEye CLI, turn on syslog logging. The CLI Syslog setting enables system level logging. Point this data feed to sourcetype=fireeye.
Dashboard Code:
<form> <label>FireEye Device Management</label> <description></description> <fieldset submitButton="false"> <input type="time" token="time_selector"> <label>TimeRange</label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel> <chart> <title>Workorder Completion Times</title> <search> <query>sourcetype=fireeye | timechart max(FE_WO_Completion_Time) AS max, avg(FE_WO_Completion_Time) AS avg, min(FE_WO_Completion_Time) AS min</query> <earliest>$time_selector.earliest$</earliest> <latest>$time_selector.latest$</latest> </search> <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option> <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option> <option name="charting.axisTitleX.visibility">visible</option> <option name="charting.axisTitleY.visibility">visible</option> <option name="charting.axisTitleY2.visibility">visible</option> <option name="charting.axisX.scale">linear</option> <option name="charting.axisY.scale">linear</option> <option name="charting.axisY2.enabled">0</option> <option name="charting.axisY2.scale">inherit</option> <option name="charting.chart">line</option> <option name="charting.chart.nullValueMode">gaps</option> <option name="charting.chart.sliceCollapsingThreshold">0.01</option> <option name="charting.chart.stackMode">default</option> <option name="charting.chart.style">shiny</option> <option name="charting.drilldown">all</option> <option name="charting.layout.splitSeries">0</option> <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option> <option name="charting.legend.placement">right</option> <option name="refresh.display">progressbar</option> </chart> </panel> </row> <row> <panel> <chart> <title>System Events</title> <search> <query>sourcetype=fireeye | timechart count by FE_SUBSYS</query> <earliest>$time_selector.earliest$</earliest> <latest>$time_selector.latest$</latest> </search> <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option> <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option> <option name="charting.axisTitleX.visibility">visible</option> <option name="charting.axisTitleY.visibility">visible</option> <option name="charting.axisTitleY2.visibility">visible</option> <option name="charting.axisX.scale">linear</option> <option name="charting.axisY.scale">linear</option> <option name="charting.axisY2.enabled">0</option> <option name="charting.axisY2.scale">inherit</option> <option name="charting.chart">line</option> <option name="charting.chart.nullValueMode">gaps</option> <option name="charting.chart.sliceCollapsingThreshold">0.01</option> <option name="charting.chart.stackMode">default</option> <option name="charting.chart.style">shiny</option> <option name="charting.drilldown">all</option> <option name="charting.layout.splitSeries">0</option> <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option> <option name="charting.legend.placement">right</option> <option name="refresh.display">progressbar</option> </chart> </panel> </row> <row> <panel> <table> <title>System Event Types</title> <search> <query>sourcetype=fireeye NOT FE_SUBSYS="postfix/cleanup" | chart values(FE_SUBSYS) sparkline count by FESYS_EVENT_TYPE | sort - count</query> <earliest>$time_selector.earliest$</earliest> <latest>$time_selector.latest$</latest> </search> <option name="count">10</option> <option name="dataOverlayMode">none</option> <option name="drilldown">cell</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="wrap">true</option> </table> </panel> <panel> <table> <title>Event Count by Host</title> <search> <query>sourcetype=fireeye | chart sparkline count by host | sort - count</query> <earliest>$time_selector.earliest$</earliest> <latest>$time_selector.latest$</latest> </search> <option name="count">14</option> <option name="dataOverlayMode">none</option> <option name="drilldown">cell</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="wrap">true</option> </table> </panel> </row> <row> <panel> <table> <title>Errors by System</title> <search> <query>sourcetype=fireeye ("err" OR "warn" OR "fail" OR "crash" OR "fatal") NOT "URL"| chart values(FE_SUBSYS) values(FESYS_EVENT_TYPE) sparkline count by host | sort - count</query> <earliest>$time_selector.earliest$</earliest> <latest>$time_selector.latest$</latest> </search> <option name="count">17</option> <option name="dataOverlayMode">none</option> <option name="drilldown">cell</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="wrap">true</option> </table> </panel> </row> </form>