Not quite ready for prime-time, we have been testing our home-grown UNIX domain socket adapter using Coral8 Java APIs. We are using this adapter to evaluate and demonstrate stream processing with intrusion detection systems (IDS) using event stream processing to reduce false alarms, detect derived situations from the raw intrusion event data, and feed a security management visualization dashboard.
You can click on the teaser image below to see more of our first IDS screenshots from Coral8’s Studio stream visualization tool.
If you click on the image above, you will four additional event stream properties. For this part of the demo, there are 14 total IDS properties in the event stream, but we only show 5 properties in this cropped screen capture.
I am quite sure that we could do similar integration with other event stream processing engines, but fortunately Coral8 makes it easy to download, start developing and testing.
The Complex Event Processing Blog 
Hi Tim and happy new year!!!
Just a “couple” of questions:
(1) the timestamp field is the timestamp of what, according to which clock?
(2) Could you put some examples of complex/composite events you need to detect? (3) Will you be using hierarchies of events ala Luckham?
(4) Would it make sense to furtherly type the events (i.e. refactor what now is the “message” field” into a new field acting as the type of the event and have, for instance, a type for SNMP event which would have subtypes SNMP trap tcp and SNMP request tcp, etc.) Does Coral( provide support for event type inheritance/subtyping?
It would be nice to know some details about this use case…
Thank you very much!
cheers,
Claudi
Hi Claudi and Happy New Year to YOU!!
I’ll try to answer in bullet format:
(1) In Coral8 Studio, the timestamp is selectable, using the message (when it was sent of course) or the receiving server. In our very crude case, so far, we are using the message timestamp – but I can see benefits and tradeoffs from the other configuration, depending on the time / clock / latency / bandwidth architecture of the network(s), etc.
(2) Yes, an example of a composite event would be a scan from a particular network or range of IP addresses followed by more targeted events from the same network, indicating more than just a scan, perhaps the derived situation of a more concentrated attack. Some folks, by the numbers, classify scans as “attacks” or “penetrations” which is something I tend to disagree with.
(3) Regarding hierarchies of events, I tend to follow the JDL multi-sensor data fusion model as an inference hierarchy since this model is mature, works in practice and is well developed (at least in the military, but I think this crosses over to commercial classes of problems quite easily). One of the first objectives is to accurately track-and-trace low level events from a single IP address or network in the same vicinity. After solid track-and-trace is established, then it is easier to think about more advance composite events. We are not there yet, have only just got the adapter working 🙂
(4) Yes, the events are currently typed with 14 (13 or 14) properties. So far, I have only exposed 5 properties in the screen shots. I will expose more at a later date. We are simply parsing the original data from the IDS.
I don’t think that Coral8 (today) supports inheritance, subtyping and polymorphism (but I could be wrong). So far, the user is left to build / construct an XML schema for the event properties and assign it to an input stream. I have not seen any facility for treating these XML schemas as objects to provide the “goodies” that come from object-oriented programming.
Yours faithfully, Tim
If you have more detailed example of typical situation that one is interested in this type of applications I would love to hear about them. I’m always looking for more use-cases and examples to add into our ruleCore example folder and this area is not what we normally work with so some pointers to what kind of situations are worth detecting would be nice.
Hi Marco!
We would love to share and plan to!
Draft plans on the drawing board are to share IP with the various sponsors of the CEP COE we are organizing.
Further plans are to provide the domain-specific intellectual property from the COE via a subscription-based licensing service.
It is too early to sponsor, BTW.
We are still working on the organization and details. Things are moving slowly but surely. Lot’s of folks in key positions are just learning about event processing and CEP.
Yours faithfully, Tim