Structure of PIX syslog messages

Hi, Is there a definiton of keywords used by PIX Syslog messages?

I want to extract important information from these messages, like source-port, destination-port, ...

Best regards, tobi

Reply to
tobi
Loading thread data ...

In article , tobi wrote: :Is there a definiton of keywords used by PIX Syslog messages?

Not really, no.

:I want to extract important information from these messages, like :source-port, destination-port, ...

That was reasonably easy up to 6.1. From 6.2 onward, the messages changed in such a way as to lose important contextual information, making it necessary to actively keep state on each connection in order to figure out which is the source and which is the destination [at least if one wants to do accounting by incoming vs outgoing.]

Also, if one has multiple interfaces, the relationship between the interfaces can only be infered, with the inference being potentially incorrect if one is using "reverse nat".

There is also the issue that if one has created a 'name' for an IP address, then the name will be used in Deny messages, but the IP address will be used in Translation and Build messages.

VPN SA messages use a mix of the two, and unless one has crypto isakmp debugs turned on, it is not possible to figure out which device the peer is for most VPN messages.

There is a 6.3 bug/oddity in which translation information is torn down too quickly after a TCP connection is reset, resulting in Deny reports for the trailing packets from the destination towards the source -- making it appear as if the destination was attempting to contact the source.

If you don't need accounting, and you don't need VPNs, and you are prepared to hard-code knowledge of interface security levels for each monitored device, then you can get by with monitoring a fairly small number of messages.

Unless, that is, you want to be able to track -successful- connections other than UDP or TCP (e.g., ICMP or GRE) -- if you need to track those, then it gets messy again.

:(

Reply to
Walter Roberson

Yes. reading PIX logs is quite simple. For ever tcp or udp connection here are two detailed records for buiilding and summarising connection. Cisco website has a detailed page about an every single instance in PIX log and how to get any info you want out of it

Reply to
IgorZ

In article , IgorZ wrote: :Yes. reading PIX logs is quite simple. For ever tcp or udp connection here :are two detailed records for buiilding and summarising connection. :Cisco website has a detailed page about an every single instance in PIX log :and how to get any info you want out of it

Sorry, but -every- part of what you say is incorrect.

1) denied connections have no Build or Teardown message.

2) UDP and TCP use -different- Build and Teardown messages.

3) The only way to be sure that one is associating the correct Build with the correct Teardown is by tracking the connection number.

4) Connection numbers are independant for udp and tcp.

5) Unless you are using the TCP version of PIX's syslog, one cannot be sure that the Build or Teardown message made it into the log.

6) Connections must be tracked independantly for each device monitored, because the interface names, IP address spaces, and connection numbers can all overlap.

7) It is not possible to tell from the Teardown message which side initiated the connection. Therefore one must track that information from the Build message.

8) The Build message always puts parameters in the same order, so one must look for the keyword 'inbound' or 'outbound' in order to determine the direction and hence to know which of the IP addresses one should record in the "source" field of one's accounting structure.

9) The Teardown message always puts parameters in the same order, so one must carry around the directional flag in the accounting structure in order to get the accounting right.

10) There are at least 7777777rent formats of Deny messages which must be processed differently -- some of which do not even mention the word 'Deny' (nor 'Denied'). PIX-4-406002 for example.

11) Several of the variant deny messages give no interface information. PIX-2-106007 for example.

12) The PIX SYSLOG reference manuals have numerous places in which the manual indicates simply that at a particular point in the message, there is a "string". The possible content and semantic meaning of the string is not described.

13) In order to properly break out the interface information from the IP information, one must be aware of the fact that interface names may contain arbitrary number of colons. One must also be aware that named hosts may contain underscore and dash, and one must know all the details of when names are used and when IPs are used.

14) Every time the PIX is reset, the counters start all over, so one must know how to detect a restart and flush out one's accounting buffers.

15) The PIX simply never sends Teardown messages in response to some Build conditions. One must therefore watch out for re-use of the connection number, and for restarts, and at the earliest time that one can deduce that the connection died, one must emit a conditional accounting record that indicates that the connection lasted "up to" the deduced time. This requires, naturally, that carry around the connection times.

16) Considering (5) and (15), when one finds a Teardown message, one must cross-check the IPs involved to be sure that the Teardown is referring to the same event.

17) Connection numbers can wrap while connections are still alive, so for the purposes of (14) one cannot simply watch to see that a new connection has a lower connection number than a previous one.

18) Syslog being UDP, connection information can arrive out-of-order: the teardown may arrive before the build.

19) Some of the Deny messages are shared between various protocols. One must thus examine the protocol field in order to know how to parse the parameters -- including, for example, that some protocols have ports after the IPs and others don't.

20) One cannot use fixed field offsets for finding the message content in the syslog, as there are three different time formats in use, plus one other field if a particular sysop is chosen.

21) One cannot use fixed relative field offsets for extracting information from some of the messages, as they may use (e.g.) "protocol 47" instead of a simple protocol name such as "udp" or "ESP".

22) In 7.0, there are sysops that allow the device identification to be changed in ways that permit spaces, thus throwing the field counts off again.

23) Some icmp messages put the icmp type and subcode information into positions normally occupied by port numbers, rather than by the [e.g.] (type 11, code 0) format.

24) Any actual attempt to use port number 0 in a udp or tcp packet will generate a message with a completely different format.

25) None the less, in some instances, 0 can show up in the port number slots.

26) In some instances, the IP address emitted by the PIX to identify itself in syslog is 0.0.0.0 .

27) "Deny inbound (No xlate)" has two completely different causes, which can only be [approximately] distinguished by keeping connection histories and examining timestamps.

28) Standard deny-by-acl messages can have two different causes, again requiring connection histories and timestamp tracking. In PIX 6.3, the alternative interpretation occurs fairly often, and must not be misinterpeted as an attack or ACL failure or IP forgery.

29) In both Build and Deny messages, it is possible for the same interface name to occur in both the left and right hand sides. One must know how to interpret this.

30) One must understand how translations are noted in the PIX messages. This is NOT documented in the SYSLOG reference.

31) In order to know about connections other than TCP or UDP, one must track Translation messages -- no other messages are produced for successful connections for other protocols (unless one puts in explicit 'log' keywords.)

32) There are various circumstances under which Translation messages are not emitted and so NO trace is made of successful non- TCP/UDP connections.

33) The exact set of messages possible depends on which subrelease one is using. There were major transitions at 6.2 and again at 7.0, and extensions in pretty much every subrelease that introduced new features.

34) Now... do you know exactly how the messages differ when Reverse Nat is being used?

And there are other tricks one has to know as well.

I don't know about you, but I would not call this "simple". I know for sure that two records are not produced for each TCP and UDP connection. There is clearly a lot about the messages that the SYSLOG reference doesn't tell you that you need in order to extract information properly.

Reply to
Walter Roberson

Hi, thank you very much for all these answers.

But my problem haven't been solved yet. The problem is that Cisco uses a different syntax for the same semantic. Example: In log messages (type: IDS) the source IP address followed the keyword "from". This isn't the case if you have a look at log messages of the SEC type - now there isn't any keyword for the source ip address.

How should I parse these different information? I want to extract - from all Cisco Syslog Messages - the source/target ip address, ports, ....

The question in matter is, how can I do this quiet efficient???

Best regards, Tobi

Reply to
tobi

:But my problem haven't been solved yet.

Yup.

:The problem is that Cisco uses a different syntax for the same :semantic.

Yup.

:How should I parse these different information? :I want to extract - from all Cisco Syslog Messages - the source/target :ip address, ports, ....

What do you want -done- with the information? How accurate do you need the results to be? How quickly do you need to be able to process the messages?

For example, if the message is the one complaining that the wrong port number was used in an FTP command, then considering that the message itself does not contain information about interfaces, control port, is always missing one of the two IP addresses, and lists the untranslated IP instead of the NAT'd IP, should the analysis program be keeping around enough context and do a sophisticated enough parsing to be able to emit a log entry that contains the source and destination IPs and ports, by matching the entry back through active transactions?

:The question in matter is, how can I do this quiet efficient???

If you need per-source and per-destination accounting, and you need the outputs to be in a consistant format such as "source first then destination", and if you want the IP adddresses to be looked up into names, and if you want features like ability to select a range of times or to quickly to a file block and start searching from there, and if you need the code to be -efficient- and to handle a series of devices simultaneously...

Then you can expect to spend 3+ months writing the program and fine-tuning it and featurizing it... and that's if you already know Perl decently well.

Network Intelligence used to have a commercial program that did some of this work... but it made a number of mistakes and it was slow, and it was not exactly inexpensive. And it's now discontinued anyhow.

I know of two non-commercial programs that do more than a superficial analysis. As far as I know, neither of them is available to the public.

I don't know what the other one is like; it sounds to be faster than mine, but I do not have information about it's features or accuracy. I had the -impression- that it was considered Proprietary Software.

Mine would need probably a month or more additional work before a public release could be realistic. I'm not likely to have that month or more available this year. My management doesn't perceive the program as having any value beyond keeping me amused, and it doesn't have anything to do with our Mandate...

For more information on this topic, and to find the reference to the author of the other program I mentioned, see the subthread starting from:

formatting link

Reply to
Walter Roberson

Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.