On the Maturity of CEP

Deciphering the Myths Around Complex Event Processing  by Ivy Schmerken stimulated a recent flurry of blog posts about the maturity of CEP, including; Mark Palmer’s CEP Myths: Mature or Not? and Opher Etzion’s On Maturity.

I agree with Ivy.  CEP is not yet a mature technology by any stretch of the imagination.  In fact, I agree with all three of Ivy’s main points about CEP.

In 1998 David C. Luckham and Brian Frasca published a paper, Complex Event Processing in Distributed Systems on a new technology called complex event processing, or CEP (Postscript Version).  In that seminal paper on CEP, the authors said, precisely:

“Complex event processing is a new technology for extracting information from message-based systems.”

Ten years later there are niche players, mostly self-proclaimed CEP vendors,  whom do very little in the way of extracting critical, undiscovered, information from message-based, or event-based, systems.  

A handful of these niche players have informally redefined CEP as “performing low latency calculations across streaming market data.”  The calculations they perform are still relatively straight forward and they focus on how to promote white-box algo trading with commercial-off-the-shelf (COTS) software.  In this domain, we might be better off not using the term CEP at all, as this appears to be simply a type of new-fangled COTS algo trading engine.

The real domain of CEP, we thought, was in detecting complex events, sometime referred to as situations, from your digital event-driven infrastructure – the “event soup” for a lack of a better term.    In this domain, CEP, as COTS software, is still relatively immature and the current self-styled COTS CEP software on the market today is not yet tooled to perform complex situational analysis.

This perspective naturally leads to more energy flowing in-and-around the blogosphere, as folks “dumb down” CEP to be redefined as it benefits their marketing strategy, causing more confusion with customers who want CEP capabilties that have zero to do with low latency, high throughput algo trading, streaming market data processing, which maybe we should call “Capital Market Event Stream Processing” or CESP – but wait we don’t really need more acronyms!

Hold on just a minute!  Wasn’t it just a short couple of years ago that folks were arguing that, in capital markets, it was really ESP, not CEP, remember?  Now folks are saying that it is really CEP and that CEP is mature?   

CEP is mature?  CEP is really not ESP?  CEP is really event-driven SOA?  CEP is really real-time BI?  CEP is really low latency, high throughput, white-box COTs algo trading?  CEP is really not a type of BPM?  CEP is not really for detecting complex events?   Complex does not really  mean complex? 

Come on guys, give us a break! 

(Anyway, no one is going to give us a break….  so stay tuned!)

  

Is CEP a Service or a Process? Reloaded

In Is CEP a Service or a Process? Paul Vincent of TIBCO blogs that any classification of CEP depends on the application, concluding that CEP is both a process and a service. 

Well (sorry Paul!), I disagree.  CEP is neither a process nor a service; CEP is a concept architecture for processing complex events.   (I have advocated a CEP functional reference architecture, as most readers know.)

To illustrated this point, let’s take a quick look at another functional reference architecture (or, if you perfer, a conceptual architecture), distributed computing.

Is distributed computing a service or a process?

Of course, it is neither a process nor a service, distributed computing is a generic architectural pattern (or style) for processing distributed data, generally across a network.

The same question can be asked of SOA. 

Is SOA a process or a service?

Again, the answer is almost identical. 

SOA is an architectural style (subclass) of distributed computing.

Now, is CEP a product or a service?

CEP is an architectural style (or pattern) for processing complex events.

CEP is neither a process nor a service. 

On the other hand, there are component of a CEP solution that can be represented as a stand alone process or a service.   The same can be said of EAI, SOA, and other subclasses of distributed computing architectural styles and patterns.

The (ISC)2 Blog

(ISC)²^® is globally recognized for certifying information security professionals throughout their careers and the kind folks there have asked me to blog in information security topics.  If interested, here is the link to my posts over at the (ISC)2 blog.  

Open Service Event Management

One of the benefits of working in different countries is to get the perspectives of various client’s event processing problems.    Of interest to event processing professionals, companies are moving away from expensive software solutions and increasingly moving toward experimenting with economical and open software packages to solve complex problems.   

Recently, I was talking with a client about their experience with commercial security event management (SEM) solutions, for example ArcSight.   In his opinion, ArcSight was not an economically viable solution for his company, so he recommended I take a look at Open Service Event Management (OSEM). 
 
OSEM helps organizations collect, filter, and send problem reports for supported systems (ProLiant and Integrity) running compatible agents.   OSEM automatically sends service event notifications when system problems are detected.

I have not had a chance to look under the hood of OSEM and see how it can be used to collect and send events to emerging rule-based event processing engines.    However, this looks like an interesting lab project and I would like to hear from readers who have experimented with this systems architecture.