Epilogue on CEP Maturity

June 4, 2008

In On the Maturity of Complex Event Processing, the author concludes:

“I think [… the. …] comment at the end of [… the. …] post “we shouldn’t feel compelled to thwart that growth with a claim that the products are not ‘mature’ when they actually are in a lot of ways” is quite revealing. The fact that such a level of debate about CEP’s maturity is taking place, and the fact that [… someone …] is concerned that the debate might stifle growth, is itself indicative of an immature market segment in my opinion.”

This quote is compelling.  When vendors disagree with the direction and tone a debate is going and they call to end the debate, labelling the discussion “a distraction” – it tends to prove the premise of the original post Deciphering the Myths Around Complex Event Processing  by Ivy Schmerken;  the CEP market, both exciting and promising, is today, mostly immature and brittle. 

For more conclusive evidence, I turn our readers attention to this post, An Overture to the 2007 CEP Blog Awards,  That analysis was based, in part, on CEP/EP Reference Customers 2005-2007 where we documented 18 public “CEP reference clients” in 2007 (25 for the entire period 2005 – 2007).

Twenty five public reference clients over a three year period with 18 last year (2007) do not demonstrate a mature market or technology domain.



Here were the results of the CEP/EP Reference Customers Survey for 2005-2007:

Apama 5
StreamBase   4
AptSoft  (purchased by IBM)   4
Coral8   2
Aleri   2
Agent Logic   1
BEA   1
Total CEP/EP Reference Customers (2005-2007)   25
Looking only at 2007, the total CEP/EP reference customers available in the public domain were as follows:
Apama 4
StreamBase   4
AptSoft (purchased by IBM)   2
Coral8   2
Aleri   2
Agent Logic   1
BEA   1
Total CEP/EP Reference Customers (2007)  18

More on CEP Maturity: Capability Versus Reliability

June 3, 2008

Louis Lovas of Progress Apama wrote a complimentary blog entry on the topic at hand, CEP Maturity Models.   In his post, Louis says:

“What a CEP platform has tracks independently of what it is capable of doing. ….. What CEP does, is likely what Tim is referring to when he states we’re in the Technology Trigger phase.”

Peter Lin’s comment, in reply to Louis, concurs:

“Given that COTS CEP has only been around a few years, I think it is safe to say it’s still in the early phase. If we compare it to messaging middleware, which has been around for more than 15 years, CEP isn’t as mature. Another comparison is business rule engines and expert systems. The earliest business rule engines date back to late 80’s. All things considered, I would agree with Tim. COTS CEP still has a lot of time to mature.”

Louis was spot on when he said that I was focused on overall CEP functionality; not individual product reliability.

Independent of how reliable a particular CEP-type application might appear; the overall state-of-the-art of CEP is really quite immature.



On CEP Maturity and the Gartner Hype Cycle

June 1, 2008

In reply to Mark Palmer’s rebuttal, What Does it Mean to be Mature?, the figure below illustrates the popular Gartner Hype Cycle.  You can click on the illustration to get a clearer image.

In context to the Gartner Hype Cycle, CEP is closer to the “Technology Trigger” phase than anywhere else in the hype cycle.  CEP has not yet reached the “Peak of Inflated Expectations”, but is inching closer and closer.

In addition, as a correlating reference point, if you look at a recent Gartner Hype Cycle that covers EDA, for example, you will find that EDA  (Event Driven Architecture) is at a similar phase, the “Technology Trigger” phase. 

Open Service Event Management

May 17, 2008

One of the benefits of working in different countries is to get the perspectives of various client’s event processing problems.    Of interest to event processing professionals, companies are moving away from expensive software solutions and increasingly moving toward experimenting with economical and open software packages to solve complex problems.   

Recently, I was talking with a client about their experience with commercial security event management (SEM) solutions, for example ArcSight.   In his opinion, ArcSight was not an economically viable solution for his company, so he recommended I take a look at Open Service Event Management (OSEM). 
OSEM helps organizations collect, filter, and send problem reports for supported systems (ProLiant and Integrity) running compatible agents.   OSEM automatically sends service event notifications when system problems are detected.

I have not had a chance to look under the hood of OSEM and see how it can be used to collect and send events to emerging rule-based event processing engines.    However, this looks like an interesting lab project and I would like to hear from readers who have experimented with this systems architecture.

CEP Product Complexity at Coral8

April 5, 2008

In What makes a Coral8 Expert?, Coral8 CTO Mark Tsimelzon outlines nearly 60 subject areas that a customer must master to become a Coral8 expert. 

While this complexity is impressive, it tends to demonstrate why CEP is, today, more hype than reality.

I can hear the team at Techrotech in my mind, “Yea! Greg purchased Coral8 for our CEP solutions yesterday!   Holy Cow!! Let’s go out and learn 60 topics in depth so we can become experts in using and deploying Coral8!”

So, let’s say you are intelligent and can master a subject in a single weeks time (if you have nothing else to do), so you can become a Coral8 expert in only one year if you don’t have a day job!!

I don’t know about you, but the way Mark describes their product, Coral8 sounds more like a lab tool for the engineering department of Caltech or Stanford than a tool for everyday business users.  Mark concludes his post;

Not too scary, is it?  – Mark Tsimelzon, President & CTO, Coral8

Hmmmm.   I think I’ll ask the software team at Techrotech to write some event processing applications in C since we have a strong team of C programmers coming off another project next week….  then again, I heard some Java programmers will be free in two weeks …. 

Please Welcome Dr. Rainer von Ammon to The CEP Blog

February 12, 2008

Today is an especially joyful occasion on The CEP Blog.    I am pleased to announce that one of the world’s top experts on CEP, Dr. Rainer von Ammon, has joined the blog.

Dr. Rainer von Ammon is managing director of the Centrum für Informations-Technology Transfer (CITT) in Regensburg. Until October 2005 he was Professor for Software Engineering, specializing in E-Business infrastructures and distributed systems, at the University of Applied Sciences Upper Austria. Rainer is still teaching there and at the University of Applied Sciences of Regensburg. From 1998 to 2002, he worked as Principal Consultant and Manager for R+D Cooperations at BEA Systems (Central and Eastern Europe). Prior to this, he was Professor for Software Engineering in Dresden with a focus on development of applications with event driven object oriented user interfaces and component based application development. Before this Rainer was acting as manager of the field Basic Systems at the Mummert + Partner Unternehmensberatung, Hamburg. After finishing his studies of Information Sciences at the University of Regensburg, he started as project leader of Computer Based Office Systems (COBIS) from 1978 to 1983 and afterward founded a start up company with some of his colleagues.

Some of you may recall my recent musings, A Bitter Pill To Swallow: First Generation CEP Software Needs To Evolve.   When you read Rainer’s excellent reply, you will quickly see why we are very pleased to have his thought leadership here at The CEP Blog.  Dr. von Ammon and his team are leading experts in CEP and related business integration domains.  Not only does he provide thought leadership, his team  researches, develops, implements and tests CEP solutions.   

In another example of  his thought leadership, some of you might recall this post, Brandl and Guschakowski Deliver Excellent CEP/BAM Report, where Hans-Martin Brandl and David Guschakowski of the University of Applied Sciences Regensburg, Faculty of Information Technology/Mathematics, advised by Dr. von Ammon, completed an excellent CEP thesis, Complex Event Processing in the context of Business Activity Monitoring

Please join me in extending a warm welcome for Dr. Rainer von Ammon to The CEP Blog.

A Bitter Pill To Swallow: First Generation CEP Software Needs To Evolve

February 8, 2008

Frankly speaking, the CEP market is now saturated with hype about all the great things CEP can do, detecting opportunities and threats in real time and supporting the decision cycle.  However, in my opinion, it is time for the software vendors and analysts to move beyond the marketing hype and demonstrate real operational value with strong end user success, something seriously lacking today.

I have advocated this evolution for two years, including the notion of expanding CEP capabilities with proven techniques for event processing that have worked well long before current “Not yet CEP but called CEP” software hit the marketplace and airwaves.

For example, in my first CEP/EP presentation in New York in 1Q 2006, I presented Processing Patterns for Predictive Business and talked about how the US military has implemented high performance detection-oriented systems for many years (in the art-and-science of multisensor data fusion, MSDF), and how every day, when we sit at home (or at work or in transit), we are comforted to know we are safe from missile attacks because of what I would also call “complex event processing.”   There is a very rich history of “CEP but not called CEP” behind the scenes keeping people safe and warm. (The same thing can be said with many similar examples of complex event processing in use today, but not called “CEP” by CEP software vendors.)

This is one reason, when I read the “CEP history lessons,” I am amused at how, at times, the lessons appear self-serving, not end user serving.  There is so much rich event processing history and proven architectures in “CEP but not called CEP” (CEP that actually works, in practice everyday, long before it was called CEP).  It continues to puzzle me that a few people the CEP/EP community continue to take the “we invented EP” view.  Quite frankly, the history we read is missing most, if not all, of the history and practice of MSDF.

When we take the current CEP COTS software offerings and apply it to these working “CEP but not called CEP” applications, the folks with real operational “CEP but not called CEP” detection-oriented experience quickly cut through the hype because they are, based on their state-of-the-practice, now seeking self-learning, self-healing “real CEP type” systems.  They are not so excited about first generation technologies full of promises from software vendors with only a few years of experience in solving detection-oriented problems and very few real success stories.

The same is true for advanced fraud detection and other state-of-the-art detection-oriented processing of “complex events” and situations.  The state-of-the-art of complex event processing, in practice, is far beyond the first generation CEP engines on the market today. 

This is one of the reasons I have agreed with the IBM folks who are calling these first generation “CEP orchestration engines” BEP engines, because that view is closer to fact than fiction.  Frankly speaking again, process orchestration is much easier than complex detection with high situation detection confidence and also low false alarms.

Customers who are detection-savvy also know this, and I have blogged about a few of these meetings and customer concerns.  For example, please read my blog entry about a banker who was very sceptical in a recent wealth management conference in Bangkok.  I see this reaction all the time, in practice. 

Complex problems are not new and they still cry out for solutions.  Furthermore, many current-generation event processing solutions are already more advanced that the first generation CEP engines on the “call it CEP” market today.  This is a very real inhibitor, in my opinion, to growth in the “call it CEP” software space today – and credibility may ultimately be “at risk.”  Caution is advised.

Candidly speaking again, there are too many red-herring CEP-related discussions and not enough solid results given the time software vendors have been promoting CEP/EP (again, this is simply my opinion).  The market is in danger of eventually losing credibility, at least in the circles I travel and complex problems I enjoy solving, because the capabilities of the (so called) CEP technologies by software vendors in the (so called) CEP space have been over sold; and, frankly speaking, I have yet to see tangible proof of “real CEP capabilities” in the road maps and plans of the current CEP software vendors.  This is dissappointing.

This pill is bitter and difficult to swallow, but most of my life’s work has been advising, formulating and architecting real-time solutions for the end user (the C-level executives and the operational experts with the complex problems to solve).   CEP software must evolve and there needs to be more tangible results, not more marketing hype.