The Predictive Battlespace

June 11, 2008

Friend and colleague Don Adams, CTO World Wide Public Sector, TIBCO Software, explains how CEP can be used to sense, adapt and respond to complex situations in The “Predictive” Battlespace: Leveraging the Power of Event-Driven Architecture in Defense


Probabilistic Complex Event Triggering

June 8, 2008

Here is an interesting paper, Probabilistic Complex Event Triggering, Daisy Zhe Wang, Eirinaios Michelakis, and Liviu Tancau, Computer Science Division, University of California at Berkeley, circa 2005.

One of the first things I noticed about the paper was the discussion of probability in the content of complex event processing, including Hidden Markov processes, Bayesian Belief Networks, and inference models.  

The second thing I noticed was that David Luckham’s work on CEP at Stanford was not referenced anywhere in the Berkeley paper.


Update on the LinkedIn CEP Users Group

June 5, 2008

There are now 234 members of the CEP Users Group on LinkedIn.  So, if you have not yet joined the CEP Users Group, please do so by clicking here.

Algo Trading on Your BlackBerry?

June 5, 2008

In E*Trade (ETFC) seizes control of the BlackBerry, Douglas A. McIntyre asks “How long will it be before the Blackberry and other devices will have enough intelligence to trade stocks without their owner’s permission?”

Check out E*Trade’s Mobile*Pro where customers can view all their accounts and watch lists; trades stocks and options;  view streaming quotes, orders, and breaking news; monitor their portfolio and view account alerts; stay on top of markets as well as transfer money between accounts – all from their BlackBerry!

I can’t want to run a next generation algo trading engine on my BlackBerry, or perhaps configure my personal algo engine embedded in my E*Trade account!

Check out the Mobile*Pro demo.

Epilogue on CEP Maturity

June 4, 2008

In On the Maturity of Complex Event Processing, the author concludes:

“I think [… the. …] comment at the end of [… the. …] post “we shouldn’t feel compelled to thwart that growth with a claim that the products are not ‘mature’ when they actually are in a lot of ways” is quite revealing. The fact that such a level of debate about CEP’s maturity is taking place, and the fact that [… someone …] is concerned that the debate might stifle growth, is itself indicative of an immature market segment in my opinion.”

This quote is compelling.  When vendors disagree with the direction and tone a debate is going and they call to end the debate, labelling the discussion “a distraction” – it tends to prove the premise of the original post Deciphering the Myths Around Complex Event Processing  by Ivy Schmerken;  the CEP market, both exciting and promising, is today, mostly immature and brittle. 

For more conclusive evidence, I turn our readers attention to this post, An Overture to the 2007 CEP Blog Awards,  That analysis was based, in part, on CEP/EP Reference Customers 2005-2007 where we documented 18 public “CEP reference clients” in 2007 (25 for the entire period 2005 – 2007).

Twenty five public reference clients over a three year period with 18 last year (2007) do not demonstrate a mature market or technology domain.



Here were the results of the CEP/EP Reference Customers Survey for 2005-2007:

Apama 5
StreamBase   4
AptSoft  (purchased by IBM)   4
Coral8   2
Aleri   2
Agent Logic   1
BEA   1
Total CEP/EP Reference Customers (2005-2007)   25
Looking only at 2007, the total CEP/EP reference customers available in the public domain were as follows:
Apama 4
StreamBase   4
AptSoft (purchased by IBM)   2
Coral8   2
Aleri   2
Agent Logic   1
BEA   1
Total CEP/EP Reference Customers (2007)  18

More on CEP Maturity: Capability Versus Reliability

June 3, 2008

Louis Lovas of Progress Apama wrote a complimentary blog entry on the topic at hand, CEP Maturity Models.   In his post, Louis says:

“What a CEP platform has tracks independently of what it is capable of doing. ….. What CEP does, is likely what Tim is referring to when he states we’re in the Technology Trigger phase.”

Peter Lin’s comment, in reply to Louis, concurs:

“Given that COTS CEP has only been around a few years, I think it is safe to say it’s still in the early phase. If we compare it to messaging middleware, which has been around for more than 15 years, CEP isn’t as mature. Another comparison is business rule engines and expert systems. The earliest business rule engines date back to late 80’s. All things considered, I would agree with Tim. COTS CEP still has a lot of time to mature.”

Louis was spot on when he said that I was focused on overall CEP functionality; not individual product reliability.

Independent of how reliable a particular CEP-type application might appear; the overall state-of-the-art of CEP is really quite immature.



More on CEP: Process, Service or Reference Architecture?

June 2, 2008

In reply to Paul Vincent’s post Is CEP a Service or a Process? I posted Is CEP a Service or a Process? Reloaded.  This post is a follow-up to my dialog with Paul and the CEP community, as a whole.

Some of the more remarkable critical comments on the book “The Power of Events” was that the book did not (for the most part) discuss architecture. 

As we all know, there are many definitions of “architecture;” however, one definition that is easy to discuss, in this context, is that an IT systems “architecture” represents the components of an IT system and the relationships between the various components in the architecture. 

An architecture can be “technical” or “functional” or “operational” or “data” centric.  For example, an architecture can be based on an orchestration of service-components, like an SOA.  In another example, an architecture can be represented by the semantics of the data.  In yet another example, an architecture can be represented by the functionality of the components.

Because David’s book on CEP did not address architecture, folks have been free to use any “tool” or “technique” they like, and call it “CEP”.   My focus has been on overall CEP functionality and reference architectures that depict this functionality for solving CEP classes of problems.

This was one of the first topics (issues) with CEP we identified a few years ago; and is why we, including me at my good ole’ days at TIBCO until now, created a functional reference architecture for CEP (also in this blog and the TIBCO CEP blog).

In that functional reference architecture, we discussed and illustrated how CEP should operate as a cooperative (distributed) functional reference architecture to solve most “real” CEP classes of problems.

Therefore,  CEP should not be, generally speaking, considered as a “process” or a “service”,  per se,  because CEP, as a functional reference architecture, depicts the methodologies (functionaility) required to solve complex detection-oriented problems.  This abstract permits CEP to have meaning in a broad context of event processing applications.

Naturally, a functional reference architecture can be viewed as a “service” if all the components in the architecture cooperate to solve a problem and are encapsulated as a service.  In addition, a functional reference architecture can be viewed as a “process” when solving problems in a specific domain.  So, a “process,” in this case, is an instance of the functional reference architecture; and if the instance is packaged as a solution, this solution can be encapsulated as a service.

So, it is misleading, at least in my opinion, to reduce CEP to a “process” or a “service” unless we are discussing a particular solution to a domain problem within a (functional) reference architecture (functional context).

This confusion also manifests itself in the lively debate between Mark Palmer and the blogosphere regarding the maturity of CEP.   Mark and others have created an instance of event processing in capital markets and call it “CEP,” when in fact, what they are doing is COTS algo trading and using one or more functional components of CEP to realize their solution.

The is an important distinction, in my opinion.