More on CEP Maturity: Capability Versus Reliability

Louis Lovas of Progress Apama wrote a complimentary blog entry on the topic at hand, CEP Maturity Models.   In his post, Louis says:

“What a CEP platform has tracks independently of what it is capable of doing. ….. What CEP does, is likely what Tim is referring to when he states we’re in the Technology Trigger phase.”

Peter Lin’s comment, in reply to Louis, concurs:

“Given that COTS CEP has only been around a few years, I think it is safe to say it’s still in the early phase. If we compare it to messaging middleware, which has been around for more than 15 years, CEP isn’t as mature. Another comparison is business rule engines and expert systems. The earliest business rule engines date back to late 80’s. All things considered, I would agree with Tim. COTS CEP still has a lot of time to mature.”

Louis was spot on when he said that I was focused on overall CEP functionality; not individual product reliability.

Independent of how reliable a particular CEP-type application might appear; the overall state-of-the-art of CEP is really quite immature.

 

 

On Time-Series Analysis with Strict Determinism

Like the predictable ebb and flow of ocean tides, we see the rise, falling and passing away of lively debates about event processing languages (EPLs).   For example, you might recall that Louis Lovas, Progress Apama,  did an excellent job in his post, Bending the Nail, where he described why SQL or Extended SQL is not the optimal EPL for event processing.  

A few of us in the “SQL is not necessarily the best EPL” choir started singing with Louis which motivated a counter voice the choir with the post, Fair and unfair criticism of an SQL EP approach only to have the same author counter that post with, One down side to an SQL EP approach.   

Many technologists, including some of my team members at Techrotech, enjoy focusing on linear event processing problems with strict determinism, for example, processing a stream of market data and looking for opportunities to enter or exit the market (algo trading).    These same technologists tend champion event processing problems that are basic transformations of simple streams of time-series data.  

Many of the so-called CEP cybertrading examples (I would argue that these are simple event processing, not complex event processing examples) are not rooted in event processing scenarios that require looking for causal linkages between seemingly unrelated events; for example, debugging complex distributed systems or detecting fraud over long periods of time where sliding time windows on continuous streaming data are only a part of the solution in the uncertain world of  cloudy event-causality relationships.

Time-series analysis with strict determinism are interesting, but I would not go so far at to call this processing “complex event processing” relative to the myriad challenging complex problems in the real-world.

IBM Says Business Event Processing is Not CEP

Sandy Carter, IBM’s vice president of SOA and WebSphere strategies, said something in IBM Buys AptSoft To Boost BPM-SOA Line I completely agree with, relative to most of the technologies folks are calling “CEP” these days:

“In the marketplace today, everybody talks about complex event processing,” Carter said. “We actually are trying to rename that category, because we believe the real value is in business event processing, with a focus on the business.”

For example, none of the current CEP vendors are doing “complex event processing” as many of us have said, repeatedly.

TIBCO and AptSoft, for example, are examples of companies that are really implementing, business event processing. You can easily confirm this in TIBCO’s press release, TIBCO BusinessEvents 2.2 now shipping…, where Paul Vincent blogs:

The main change with this [TIBCO BusinessEvents 2.2] release is the inclusion of new deployment options:

+ deploy BusinessEvents within a BusinessWorks container: great for using BusinessEvents as a decision engine for SOA integration processes, choreography, transaction flow monitoring, etc, or for using BusinessWorks as a ruleflow tool.

+ deploy BusinessEvents as a BusinessWorks container: great for exploiting SOA orchestration and services under the control of CEP, such as invoking complex adapters.

This is absolutely, “business event processing” just as IBM’s Sandy Carter stated, correctly in my opinion, not CEP.

The same is true for event stream processing (ESP). ESP technology from companies like Apama, Coral8 and StreamBase, is much more closely aligned with the “business event processing” than anything that is truly CEP.

IBM Will Acquire AptSoft

I was wondering when IBM would actually jump into the event processing market.  

Well, it was announced today that IBM will acquire Aptsoft, adding an event processing platform to the IBM WebSphere porfolio.  IBM will also gain AptSoft’s event processing reference customers.  This was a smart move by IBM.

Oracle is acquiring BEA which uses Esper under the hood, another stream processing engine. 

However, AptSoft has a more advanced user interface and graphical design-time environment when compared to the Oracle/BEA/Esper platform,; so the IBM/WebSphere/AptSoft offering will propel IBM to the top of the event processing market.

Now, it’s TIBCO’s turn to acquire an event stream (time series) processing platform to compliment their process-driven event processing offering.

BAM Solutions for CEP Engine Users

Today I noticed that SL Corporation has revamped their website with a new page, Solutions for CEP Engine Users.    The page is well written, reinforcing some of my earlier posts on the value proposition for CEP; so I hope the folks at SL don’t mind if I repost their excellent thoughts on BAM and CEP here. 

Solutions for CEP Engine Users by SL Corporation

© 1999-2008 Sherrill-Lubinski Corporation. All rights reserved.

Complex Event Processing (CEP) is a relatively new technology that is used to help companies detect both opportunities and threats in real-time with minimal coding and reusable key performance indicators (KPIs) and business models. Just as services are shared and reused in a SOA, CEP permits the sharing and reuse of KPIs in business activity monitoring while efficiently processing events so businesses can act on situations that impact business and take advantage of real-time processing.

Business activity monitoring, often referred to as BAM, is the capability that Gartner and other distinguished analysts use to describe this visualization capability in the business world. BAM introduces a human element to CEP. It is well-established that the human mind is, today and for the foreseeable future, far superior to machine intelligence in making sense out of complicated situations and events. Therefore, BAM is critical to the success of any complex event processing (CEP) solution.

Depending on an organization’s mission, BAM can be used in various levels within an event processing solution to help users visualize and understand the dynamics behind rapidly changing situations and critical business events. In other words, BAM plays a key role wherever there is a need for better insight into the myriad events that effect your business operations.

BAM provides real-time visualization and alerting capabilities for users to better understand how business events impact their organization. BAM software permits users to quickly prototype, build and deploy event processing business solutions. For example, a telecommunications company would find BAM useful to achieve event-driven SLA monitoring and management; and a large retailer would find BAM important as they stay on top of business-critical events in their supply chain.

Insight gained from BAM, in concert with event processing solutions, enable organizations to make better and faster business decisions so they can rapidly sense and respond to threats, problems and opportunities. BAM solutions permit applications to be designed, deployed and modified rapidly with minimal or no coding resulting in significantly lower development costs. Therefore, a key benefit of BAM in real-time event processing solutions is that KPIs can be deployed, monitored, revised, reused and utilized, economically and rapidly.

Depending on the business application, BAM-enabled visualization is required at numerous levels in an event processing architecture. For example, events from across the enterprise are typically processed by a CEP software platforms from companies such as TIBCO, BEA (soon to be Oracle), Progress Apama, StreamBase, Aleri, and Coral8.

Long before KPIs are displayed to the business users, BAM tools can be configured to assist application developers to monitor and visualize the raw event stream. For the developer, their business is developing applications, and BAM can be very useful when designing KPIs for event processing applications.

Fine-tuned KPIs that have been derived from an event processing application are displayed to the business user. These KPIs can indicate risks, threats, problems, opportunities and other emerging business situations that impact the business.

BAM, in concert with state-of-the-art event processing software, provides the framework for a complete sense-and-respond capability for businesses. Processing raw events and event streams for business opportunities and threats requires robust and rapidly deployable visualization solutions. This is the reason that many distinguished analysts believe that BAM and CEP are complementary and critically interdependent core business capabilities. We at SL Corporation agree, and are pleased to be the leading BAM visualization platform in the event processing/CEP ecosystem today.

© 1999-2008 Sherrill-Lubinski Corporation. All rights reserved.

The 2007 CEP Blog Awards

Here are the CEP Blog Awards for 2007, based on the three categories outlined in An Overture to the 2007 CEP Blog Awards.

The CEP Blog Award for Rule-Based Event Processing

Winner: TIBCO Software

TIBCO has a very robust and sophisticated progress-oriented event processing product, TIBCO BusinessEvents, with a proven event processing customer base. TIBCO has a rich complimentary software suite for business process and enterprise integration, management, visualization, personalization and optimization. TIBCO has been in business for many years and has a global reach for both sales and professional services.

The CEP Blog Award for Event Stream Processing

Winner: Progress Apama

Similar to TIBCO in middleware status, Progress Apama has a strong event stream processing product with a proven customer base.  Progress has complimentary software suites for business process and enterprise integration.  Progress also has been in business for many years and has a global reach for both sales and professional services.

The CEP Blog Award for Advanced Event Processing

Winner: Reserved.

None of the software companies currently marketing themselves as event processing platforms meet the our criteria for the Advanced Event Processing award in 2007.

The ART of Event Processing: Agility, Reuse, Transparency

The other day I discussed CEP in Layman’s Terms: Reuse and Agility. Today, our topic is CEP and transparency. One of the major benefits of “white box” event processing solutions is transparency, something not readily available or obvious in black-box solutions.

Friend and colleague John Bates, Progress Apama, often discusses the benefits of white-box algorithmic trading platforms in terms of increased time-to-market and other competitive advantages. I agree with John and would like to point out that there is another key benefit, in simple layman’s terms, transparency.

For example, let’s say you have designed an event processing solution for operational risk management (ORM). It is time for your favorite auditors to come by and they wish to take a look at what is going on with that proprietary black-box ORM application running quietly in the server room.

The nice auditors ask you, “What does that application do?” and you reply “Well, it looks for evidence of insider trading,” and they ask “Do you mind if we ask how?” and you respond “Good question, do you mind to wait a moment while I get you the contact info for the vendor because we don’t have access to the source code or the actual key indicators (KIs)?”

Now, let’s look at the white-box scenario:

Again, the nice auditors ask you, “What does that application do?” and you reply “Well, it looks for evidence of insider trading,” and they ask “Do you mind if we ask how?” and you respond “Yes, sit down and we will pull up our insider trading key indicator models. These models are stored in XML format and viewable in our graphical KI design studio. We can print out the KI models for insider trading if you like!” and the smiling auditor says “Thank you, your system is much more transparent than the last place we visited!”

This scenario also applies in looking for why certain KIs were not detected that should have been; or when performing a root cause analysis to see why the KI you used in your wrong business decision was inaccurate.

So, CEP in layman’s terms is what we might refer to as the ART of event processing:

• Agility
• Reuse
• Transparency

Please feel free to reuse these idea, but please don’t forget to reference the author and this blog 🙂

Kindly share and reuse by reference, because all content in The CEP Blog is ©2007-2008 Tim Bass – All Rights Reserved. Thank you!