Happy New Year 2008!

 

Dear Friends,

Thank you so much for all the support and kindness you have given me in 2007.     Your words of encouragement, support and readership are very much appreciated.

Peace on Earth and Good Will Toward All People in 2008.

Yours Sincerely,

Tim Bass

www.thecepblog.com

Motor Vehicle Crashes and Complex Event Processing

The Research and Innovative Technology Administration (RITA) coordinates Department of Transportation’s (DOT) research programs.  RITA’s mission is to advance the deployment of multi-disciplinary technologies to improve transportation system in the U.S.

Shaw-Pin Miaou, Joon Jin Song and Bani K. Mallick wrote a detailed paper, Roadway Traffic Crash Mapping: A Space-Time Modeling Approach, in RITA’s Journal of Transportation and Statistics.    In their paper, the authors state that, “motor vehicle crashes are complex events involving the interactions of five major factors: drivers, traffic, roads, vehicles, and the environment.”

Maiou, Song and Mallick go on to say that “studies have shown that risk estimation using hierarchical Bayes models has several advantages over estimation using classical methods.”    They also point out that “the overall strength of the Bayesian approach is its ability to structure complicated models, inferential goals, and analyses. Among the hierarchical Bayes methods, three are most popular in disease mapping studies: empirical Bayes (EB), linear Bayes (LB), and full Bayes methods.”

Maiou, Song and Mallick directly reference two important problems that David Luckham recently mentioned during his keynote presentation at the 2007 Gartner Event Processing Symposium, traffic congestion management and global epidemic warning systems.  In addition, Jean Bacon, professor of distributed systems in Cambridge University’s computer laboratory, was recently mentioned in the article Fusing data to manage traffic.

As Maiou, Song and Mallick point out, motor vehicle crashes are complex events requiring the correlation of five situational objects (drivers, traffic, roads, vehicles, and the environment).  Each one of these five situational objects may also be a complex event.  The representation of each object requires complex event processing.

We often see these discussions and articles across the wire, for example, “Is BAM (Business Activity Monitoring) Dead?”  or “Is it CEP or Operational BI (Business Intelligence)?” or “Is it Event-Driven SOA or Just Plain Old SOA?”  Frankly speaking, these debates and discussions are red-herrings.

What is important to solving real problems, indicated by the complex event processing paper by Maiou, Song and Mallick, are real solutions not buzzwords and three letter acronyms.  Please keep this in mind when using the term “complex event processing.”  

OpenCourseWare: Get Smart for Complex Event Processing!

Ready to move beyond the basics of event processing?   Perhaps you would like to beef up your Java skills?   The Basics of Signal Processing?  Or maybe you are interested in Advanced Complexity Theory?   Artificial Intelligence?  Computer Language Engineering?  Queueing Theory?

Well then, put your feet up, relax and click on over to the Department of Electrical Engineering and Computer Science at MIT OpenCourseWare  (OCW) and enjoy their courses, freely available to anyone, anywhere. 

MIT’s OCW program freely shares their lecture notes, exams, and other resources from more than 1800 courses spanning MIT’s entire curriculum, including many fields related to event processing.     There are even RSS feeds on new courses as they hit the wire, so you don’t have to miss a thing!  Also, check out the OSC Consortium.

Complex event processing is a multi-discipline approach for detecting both opportunities and threats in real-time cyberspace.   Make it your New Years Resolution to review a few OCW lectures and help advance the state-of-the-art of CEP!

CEP/EP Reference Customers 2005-2007

On November 8th 2007, after compiling a list of CEP/EP reference customers from the open literature and public press releases, I issued a call for CEP reference customers in the Yahoo! CEP-Interest Group and on the professional networking site LinkedIn. During the past 7 weeks I provided an opportunity for all concerned to view and respond to my public worksheet.

Here are the results of the “CEP/EP Reference Customers Survey” for 2005-2007:

Apama	5
TIBCO	5
StreamBase	4
AptSoft	4
Coral8	2
Aleri	2
Agent Logic	1
BEA	1
Total CEP/EP Reference Customers (2005-2007)	25

~~~

Looking only at 2007, the total CEP/EP reference customers available in the public domain were as follows:

~~~

Apama	4
StreamBase	4
TIBCO	2
AptSoft	2
Coral8	2
Aleri	2
Agent Logic	1
BEA	1
Total CEP/EP Reference Customers (2007)	18

~~~

The criteria for inclusion in the survey were based on the following marketing and sales criteria (purely marketing criteria, not technical):

~~~

– Must be a (CEP/EP) software vendor;
– Must be an end user / customer;
– Must NOT be a partnership or OEM announcement;
– Must mention complex event processing (CEP) or event processing (EP) in the public statement; and,
– Reference must be available on the Internet and in English

~~~

Based on the results of this public survey, comments received, and experiences with the actual products by end users, I plan to followup with The CEP Blog Event Processing Awards for 2007.

~~~

Please stay tuned!

Visualization Reloaded

Enjoying visualizing a better world, world peace and global cooling in my meditative serene holiday at Divana Nurture Spa, I could not help but transcend into thoughts about a recent post, Visualizations Update.

In his thoughtful post above, Marc Adler briefly mentions heatmaps and visualization in the context of event processing adapters, analytics, and the ecosystem in general.    

In an earlier post, I blogged about SL’s Architecture for CEP Visualization.  SL has been a friend of the CEP community for quite some time supporting TIBCO, Apama, StreamBase and more.  

Speaking to everyone interested in visualization, including blogosphere colleague Marc,  I kindly recommend you check out the very cool heatmap in this post, CEP Use Case: Stream Processing in Multiplayer Online Gaming.  

I thought that StreamBase, partnering with SL for visualization, did a fantastic job in their online gaming use case for Simultronics, demonstrating the event processing ecosystem that has Marc’s attention these days.

Happy New Year!

Adapters and Analytics: COTS? NOT!

Marc Adler shows why his musings are rapidly becoming one of my “must read” blogs in his post, CEP Vendors and the Ecosystem.

We have been making similar points in the event processing blogosphere, namely the important of adapters and analytics.   Today, event processing vendors are surprisingly weak in both areas. 

For one thing, there was way much emphasis on rules-based analytics in 2007.  Where are the rest of the plug-and-play commercial analytics end users need for event processing??

And another thing….. 🙂

Why are there so few choices of adapters and why do we have to write our own??

Sometimes I think that if I read another press release on 500,000 events per second I’m going to shout out – the event processing software on the market today cannot even connect to a simple UNIX domain socket out-of-the-box, so how about ZERO events per second!

The bottom line is that the market is still wide open for a software vendor to come to the party with a wide array of plug-and-play, grab-and-go, adapters and analytics.  

Folks are talking COTS, but more often it is NOTS.

Middleware and Event Processing Expenditures

Paul Vincent of TIBCO Software in his post, Outside CEP: the infrastructure stack, makes a statement that “every $ a CEP vendor spends on middleware integration is a $ less on interesting CEP functionality.”

Opher Etzion of IBM, in turn, agrees with Paul in his post, On the envelope for CEP, and discusses how there is much overlap between the capabilities in middleware and CEP.

I agree with both Paul and Opher, from a purely technical perspective.

On the other hand, if we sail a different tact and look deeper into the business aspects, we will see that, more often than not, EAI and CEP projects are (and will be) funded out of different part of the business organization.   There indeed is an overlap with CEP in the pure middleware applications, as Paul and Opher mentioned; but there is also a quite specific business domain aspect of CEP that is not traditionally (or purely) middleware related.

Recall that most detection-oriented CEP applications (the reason that CEP exists – detecting opportunities and threats in real-time) are domain specific applications.   These applications may, or may not, be funded out of the middleware side-of-the-house (often the CIO).  

Middleware is sometimes a difficult pill for organizations to swallow. 

Organizations know they need to integrate, but many have glued together their business processes with seemingly free protocols such as SMTP, FTP, SQL and HTTP for so long, that it hard to justify a large capital expenditure to rebuild the infrastructure.    Most of these organizations realize that it costs a lot of money to maintain this glueware, but getting everyone on board to paddle the ship in the same direction for integration is easier said than done.

CEP applications do not necessarily require such a broad coalition of people to work together because CEP is not purely about integration, it is about event processing – detecting opportunties and threats in real-time.    

The security team, working with the web team, can work together on an event processing project.  

The marketing team, working with the web team, can work together on an event processing project.

In other words, it may be easier to make business cases for specific event processing applications versus making a business case for a “revolutionize the enterprise” integration effort.

CEP is not purely a middleware technology.  Indeed, CEP can certainly be used in the middleware space,  but it can also be used in domain specific applications that are typically not funded from the same pot of gold as middleware.