One of the brillant minds in the CEP community, Claudio Paniagua Macia, recently posted, Event Stream Processing != Complex Event Processing. In his post, Claudi draws a bold conclusion:
(1) SQL-based approaches to ESP might have a hard time doing CEP.
(2) No real CEP engine exists today in the marketplace, perhaps not even “off” the marketplace.
Friend, colleague, and co-chair Opher Etzion replied, On Event Stream Processing:
“CEP engines do exist today, none is perfect, but probably sufficient for big majority of the existing applications today.”
Respectfully, I find it necessary to agree with Claudi and disagree with Opher. Most of the so-called CEP engines today are solving quite simple event processing problems. If the CEP engines on the market were truly solving a “majority of the exisiting applications today” then sales would be orders of magnitudes larger.
The fact-of-the-matter is that the current “simple rules-based approach” dominate in today’s marketplace are used to solve problems where rules-based approaches are useful. Unfortunately, this is just a small fraction of the true potential of the CEP market.
For example (just one example of many), the vast majority of intrusion or fraud detection systems available today use rule-based approaches, and their detection capability, and the confidence in the detection, is quite elementary (poor quality). If these systems worked well, cyberspace would be a very different and much safer place.
Yes, it is useful to add another layer of rules, but rules alone will not solve the vast majority of CEP-domain classes of problems. In addition, the CEP applications that have made the press recently are quite simple, certainly nothing scientifically earth shattering.
So, the sad truth of the matter, from an architectural, scientific and solutions perspective, is exactly as Claudi boldly offered, no real CEP engine exists today. Furthermore, the vast majority, if not all, CEP applications sold today are used in very simple event processing (SEP) applications. This is not very “advanced,” but it is a good start.
What is holding the CEP market back is quite straight forward; the current “engines” are quite elementary (We should call them SEP engines.), relatively speaking, and SEP engines do not have the capability to solve difficult detection-oriented CEP problems in cyberspace. These difficult problems compose the vast majority of the applications where “true complex event processing” is required.