Monday, October 27, 2014

Troubleshooting No Syslog Data in Stealthwatch from ISE Appliances


This is part of a “tales from the trenches” series by K.LeBlanc

Purpose

When faced with an issue where the ISE server appears to not be providing the Syslog data to the SMC and you are certain that all your configurations are correct there are certain troubleshooting paths that can be followed to determine where the issue lies. This can be difficult when the ISE devices and Stealthwatch appliances are maintained by different teams. And both teams are certain of their own configurations…

Recently in a client case where an existing ISE deployment that spanned internationally was being integrated with a new Stealthwatch platform, there was a need to do shared troubleshooting between the ISE technician and the Stealthwatch Implementation Engineer. The troubleshooting steps that will be discussed throughout this blog were found to be very useful in identifying the root cause. The purpose of this blog is to save someone else some steps when they are faced with a need for troubleshooting an integration between ISE and Stealthwatch. Without a foundational understanding of what is the expected normal, locating the root cause can be made much more difficult.

To understand how the SMC uses the data it is necessary to know what data the SMC is specifically interested in seeing from the ISE server. The SMC requires two items to display Auth-Session from ISE; IP Address and Audit Session ID. Why the Auth-Session is important to Stealthwatch is actually key to a major benefit of the platform to security experts that are using Stealthwatch for investigations. The Auth-Session data represents an individual user in your organization and includes fields for such critical items as the data and time the session was created, the login type, if there is a parent session plus more, and all this tied to a user in the organization. From security point of view this is some really nice data to have in the documents.

Stealthwatch first looks to get all active sessions in ISE. Stealthwatch uses four specific fields to create its second query of the data. These fields are: user_name, Nas_ip_address, audit_session_id and framed_ip_address. If Stealthwatch successfully receives this data it well then look for each individual host. This results in the data that the investigator will see in his document queries.
There are some basic requirements that need to be recognized prior to even the first troubleshooting steps so at this point I am assuming these requirements are met. In this engagement these requirements were met so we will move on to the next phase taken in troubleshooting the lack of data in the SMC.

Requirements and Assumptions:

  • -          The Cisco LAN and/or WLAN infrastructure needs to be configured to support 802.1x authentication and accounting prior to the StealthWatch and ISE integration to work.
  • -          ISE needs to be configured with a Helpdesk Admin account for StealthWatch to use.-          StealthWatch integrates with the ISE appliance designated as the Monitor. This is important to know when ISE is deployed with multiple appliances

These troubleshooting steps assume that 802.1x has been configured.


After these requirements were confirmed and the configurations had been gone over to ensure no spelling errors or fields left blank, the first step both ISE and Stealthwatch Engineers decided on was a packet capture. Since both sides were certain that they issue was on the other end, a capture from both exit points was decided upon. The capture showed that the encrypted handshake was sent, but was failing and then the ISE server was responding with a RST packet. The .172 address is the SMC and the .56 is the ISE server in this capture. (not actual addresses, these were changed for confidentiality).




At first glance this appears to be related to a certificate trust issue. Recall that the SMC must have the same root certificate trusted that generated the certificate in use by ISE.  None of the traces showed the certificate exchanged so the systems were not developing a trust.

We were able to confirm that the same root certificate was installed on the SMC so that the communication should have been authenticated. The fact that no Syslog data was appearing from the ISE server was puzzling as we were able to confirm that the current configuration was a match for the golden switch configuration (monitor mode) according to TrustSec 2.0 Design and Implementation Guide. Through following other troubleshooting steps it was determined that in this existing ISE deployment, the PEC’s were actually generating the Syslog data. So the obvious step was to add those three PEC’s under the Identity Services folder in the Enterprise tree on the SMC. However, the SMC did not allow this, it actually denied that the PEC’s were part of a valid ISE structure.

At this time Lancope stepped in with a patch strictly for the SMC appliance which when applied, allowed us to add the PEC’s and through that we began to see the Syslog data appear and fill in the user data. With the Stealthwatch platform no longer relying on API calls, there is no longer as simple as a troubleshooting step as there once was. Where a simple query in a web browser would return the answer as to if the ISE server was providing the data. Being able to capture the traffic in a packet capture to analyze and understanding the way the devices were supposed to be talking to each other was key in resolving this situation.

Hopefully, understanding the need for a troubleshooting flow or process will assist the readers in getting through a similar situation, and if not, you can always call upon the expertise of the Priveon consultants to expedite getting out of a situation like the one experienced in this instance.



No comments:

Post a Comment