Coral8: Event Stream Processing and Intrusion Detection

January 3, 2008

Not quite ready for prime-time, we have been testing our home-grown UNIX domain socket adapter using Coral8 Java APIs.   We are using this adapter to evaluate and demonstrate stream processing with intrusion detection systems (IDS) using event stream processing to reduce false alarms, detect derived situations from the raw intrusion event data, and feed a security management visualization dashboard.

You can click on the teaser image below to see more of our first IDS screenshots from Coral8’s Studio stream visualization tool.

Coral8 IDS Example

If you click on the image above, you will four additional event stream properties.  For this part of the demo, there are 14 total IDS properties in the event stream, but we only show 5 properties in this cropped screen capture.

I am quite sure that we could do similar integration with other event stream processing engines, but fortunately Coral8 makes it easy to download, start developing and testing. 


Executives are Risk Adverse and Favor Large, Stable Companies

January 2, 2008

Marco Seiriö asks, To Integrate Or Not? And How? with an underlying message that he thinks it is unwise for RuleCore, as a CEP vendor, to spend development resources on integration and adapters.

I think most small companies in RuleCore’s position would make similar statements for a number of reasons, including Marco’s observation that they are resource constrained.

Unfortunately for these small companies, the flip side of that position is that large software companies with an event processing offering and a complimentary integration platform are favored by most large companies and government organizations.     Remember the old saying that goes something like, “An executive has never been fired by hiring IBM!”   This tongue-in-cheek perspective mirrors the risk adverse position of most company executives.

If you are an executive in a large company you tend to want less contracts to manage, less software licenses to negotiate,  and less companies to try to integrate.   You want large stable companies who will still be in business in 5 years.   You want companies with a proven track record that are part of a larger business ecosystem.   You want companies with a strong professional services organization.  You want companies that can survive the “an executive has never been fired for buying IBM” test.

There are only two companies that fit the executive litmus test that have referenceable customers in the CEP/EP space.  Therefore, it is not by accident that the same two companies happen to be at the top of the list of CEP/EP Reference Customers 2005-2007.

A First Glance at Coral8 Studio

December 22, 2007

My recent prior experience with an event processing (EP) product was with TIBCO BusinessEvents, which is a fine product, so I’ll talk about my experience so far with Coral8 and compare and contrast with TIBCO’s BE along the way.

First of all, I like the fact that folks can download Coral8’s EP software, plus all the documentation, and starting prototyping.   I’m not one to spend much time reading manuals.  Like most of you, we install the software and jump in feet first, working with the examples or writing our own Hello World applications.

One of the first things I noticed is that Coral8 Studio is much “lighter” than TIBCO’s Designer (design-time environment).   Is this a good thing or a bad thing?    So far my experience is that this is both good and bad.

One of the things I really like about Coral8 Studio is the way they structured environments and workspaces.   I think it is quite cool that a colleague and I can be on opposite parts of the world and independently create projects, compile and start them on the same Coral8 EP server.   We can, for example, bind to the same input adapter and write stream processing queries against the same data.  This is ideal for projects where a team might be collaborating to run various queries across the same data, which is the vast majority of projects.   It is much easier to do this type of distributed collaboration with Coral8 than TIBCO’S BE.   

Coral8 Studio, like TIBCO’s BusinessEvents, comes with numerous sample projects.  BusinessEvents projects tend to bind to a working event (messaging) infrastructure (like  JMS or RV).   Coral8 example projects, for the most part, bind to CSV files for their input.    I prefer examples that work with real live network communications, so I immediately looked at Coral8’s LiveJournalAlert sample project.   

Afterwards, I noticed that the Coral8 output adapter, SendEmailOut, does not permit the user to authenticate to an SMTP server.  In other words, there were no fields in the design time studio for sending the userID and password of the SMTP account; therefore, this version of the Coral8 SendEmailOut adapter only appears to work with SMTP services that do not require authentication (see notes).  TIBCO’s design-time environment supports SMTP authentication in a variety of ways, as I recall.

Moving on to my first Hello World example with Coral8, I realized that with TIBCO’s design time environment you can attach to data, use an XPATH graphical tool to visualize the data and then assign properties to the data.   You can then map the input properties to the outputs using the the TIBCO design-time XPATH graphical tool.   This can be a bit tricky when you first use it, if you have never used an XPATH tool, but it certainly make integration a snap. 

On the other hand, with Coral8 Studio, it appeared to me, at first glance, that you must define the structure of the incoming data with an XML schema and then use regular expressions (for example) to parse the data into processable event properties.     Maybe there is an XPATH tools with Coral8 Studio, but I missed it?   Coral8 is lighter weight using XML schemas and regular expressions; but the heavier weight of TIBCO’s XPATH tool (that creates the underlying XML schemas) is something I would like to have.

In a nutshell, my first glance at Coral8 Studio has been both interesting and satisfactory.  I am moving forward with questions to Coral8 support, who have been very helpful so far.  Coral8 has made it very easy for the user to download their product, read the docs, and go right to work.   TIBCO has a much more formal approach, including requesting exactly what software you want, getting approval for evaluation licenses and formalities for email Q&A with product support.

Coral8 is significantly “lighter weight” than TIBCO.   What does this mean in actual testing and evaluation?  I’ll post more on my Coral8 experience and observations in future posts.

Have a Happy Holiday Season, wherever you are!


(1) Coral8’s SMTP server authentication (userID and password) are provided by editing a flat file, coral8-services.xml, on the server side (not in the design-time studio).

CEP Center of Excellence for Cybersecurity at Software Park Thailand

December 16, 2007

In July 2007, at InformationSecurityAsia2007,  I unveiled an idea to create a cybersecurity CEP Center of Excellence (COE) in Thailand.  Under the collaborative guidance of Dr. Rom Hiranpruk, Deputy Director, Technology Management Center, National Science and Technology Development Agency (NSTDA), Dr. Prinya Hom-anek, President and Founder, ACIS Professional Center, and Dr. Komain Pipulyarojana, Chief National Security Section, National Electronics and Computer Technology Center (NECTEC), this idea continues to move forward.

Today, in a meeting with Mrs. Suwipa Wanasathop, Director, Software Park Thailand, and her executive team, we reached a tentative agreement to host the CEP COE at Software Park.   

The mission of Software Park Thailand is to be the region’s premier agency supporting entrepreneurs to help create a strong world-class software industry that will enhance the strength and competitiveness of the Thai economy.

Since 2001, Thailand’s software industry has experienced approximately 20% year-over-year (YOY) growth.  Presently, Software Parks Thailand supports a business-technology ecosystem with over 300 active participants employing over 40,000 qualified software engineers across a wide range of technology domains.

I am very pleased that Software Park Thailand is excited about the potential benefits of CEP in the area of cybersecurity and detection-oriented approaches to cyberdefense. The COE will be working with best-of-breed CEP vendors to build, test and refine rule-based (RBS), neural network (NN) based and Bayesian network (BN) based approaches (as well as other detection methods) for cybersecurity.

I will be announcing more details in the future, so stay tuned.  Please feel free to contact me if you have any questions.

Coral8 Developer’s License

December 9, 2007

I agree with Marc Adler regarding Coral8 and Transparency.   

Marc discusses the fact that Coral8 encourages you download their full development environment without any requirement to get a license key.   A Coral8 license is only needed when you move into production.  This is really great for kicking the tires and looking under the hood.

There is something I wish Coral8 would change, however.

Coral8’s development license, as I read it, is only valid for a single CPU machine.   However, most folks today run multiple CPUs even in their development environment, especially if they are running Linux.  If you want to do any development on a multiple CPU machine you have to contact Coral8.

In my opinion, it would be good if Coral8 would open their “easy” development license up to multiple CPU Linux servers since most of us run multiple CPU Linux servers even in the development environment.