More on CEP Product Complexity

Mark Tsimelzon, President & CTO, Coral8 replies to CEP Product Complexity at Coral8 with More on CEP and Complexity.

In Mark’s reply he gently reminds us that the Coral8 Engine is a developer’s tool, not a business users tool.  Mark also tacitly reminds us that his customers are a bit smarter than our development team at Techrotech.  Perhaps that is why we can never get our SOA project off the ground!  

Mark’s customers can learn a new concept in a single day; however, our developers need a full week to learn the same thing.   Making matters more difficult, our CIO at Techrotech, Jerry Fleck is clueless according to the marketing analysts.  Jerry has not yet figured out SOA; so concepts like windows, joins, design patterns, causal tracking, messaging layers, adapters, CCL, and persistance will cause Jerry to fall Off the Grid.

It took us years to get rid of most of our legacy C programmers, bring in a bunch of Java gurus, and at the same time, correct our stock option backdating “clerical errors”.    Maybe we should now replace our worthless legal and HR departments with a CEP Engine?

 

CEP Product Complexity at Coral8

In What makes a Coral8 Expert?, Coral8 CTO Mark Tsimelzon outlines nearly 60 subject areas that a customer must master to become a Coral8 expert. 

While this complexity is impressive, it tends to demonstrate why CEP is, today, more hype than reality.

I can hear the team at Techrotech in my mind, “Yea! Greg purchased Coral8 for our CEP solutions yesterday!   Holy Cow!! Let’s go out and learn 60 topics in depth so we can become experts in using and deploying Coral8!”

So, let’s say you are intelligent and can master a subject in a single weeks time (if you have nothing else to do), so you can become a Coral8 expert in only one year if you don’t have a day job!!

I don’t know about you, but the way Mark describes their product, Coral8 sounds more like a lab tool for the engineering department of Caltech or Stanford than a tool for everyday business users.  Mark concludes his post;

Not too scary, is it?  – Mark Tsimelzon, President & CTO, Coral8

Hmmmm.   I think I’ll ask the software team at Techrotech to write some event processing applications in C since we have a strong team of C programmers coming off another project next week….  then again, I heard some Java programmers will be free in two weeks …. 

On Time-Series Analysis with Strict Determinism

Like the predictable ebb and flow of ocean tides, we see the rise, falling and passing away of lively debates about event processing languages (EPLs).   For example, you might recall that Louis Lovas, Progress Apama,  did an excellent job in his post, Bending the Nail, where he described why SQL or Extended SQL is not the optimal EPL for event processing.  

A few of us in the “SQL is not necessarily the best EPL” choir started singing with Louis which motivated a counter voice the choir with the post, Fair and unfair criticism of an SQL EP approach only to have the same author counter that post with, One down side to an SQL EP approach.   

Many technologists, including some of my team members at Techrotech, enjoy focusing on linear event processing problems with strict determinism, for example, processing a stream of market data and looking for opportunities to enter or exit the market (algo trading).    These same technologists tend champion event processing problems that are basic transformations of simple streams of time-series data.  

Many of the so-called CEP cybertrading examples (I would argue that these are simple event processing, not complex event processing examples) are not rooted in event processing scenarios that require looking for causal linkages between seemingly unrelated events; for example, debugging complex distributed systems or detecting fraud over long periods of time where sliding time windows on continuous streaming data are only a part of the solution in the uncertain world of  cloudy event-causality relationships.

Time-series analysis with strict determinism are interesting, but I would not go so far at to call this processing “complex event processing” relative to the myriad challenging complex problems in the real-world.

A Page from Greg’s Diary: Nerwana Software

I started my career in IT many years ago and since that year have worked in enterprise IT for year and years.     Almost all of my odd career story evolves around working with end users, often advising, architecting and managing the complexity of large systems integration projects, from hands on implementation to strategic vision development.  My deep background is with Techrotech in network systems engineering.

A few years ago, years after I started my career at Techrotech, I grew a bit dismayed at enterprise software companies.   They would, for the most part, always come to us, the end users, and try to sell us large software packages.  Their sales and technical teams had very little domain knowledge of the problems they claimed they could solve – and they had little doubt that if we purchased their wares, our problems would be solved,

These software companies were keen on buzzwords and technology jargon but somewhat clueless on operational solutions or the challenges of implementation across a large federated organization with many powerful business units and “in name only” CIOs.  We often referred to these software sales guys, and their favorite systems integrators, as “drive by (or fly by) implementations” where they dump the software (and hardware) at your door and run like crazy!

So, I joined a very cool Silicon Valley company,  Nerwana Software, hoping to change all of that, or so I thought 🙂

Naturally, when I first came on board Nerwana , the entire organization, from executives to recent new hires out of school, heaped praise-upon-praise on my years of operational experience at Techrotech and elsewhere.   They cheered me on as I wrote papers and created slides on operational use cases and event processing solutions that the sales and solutions teams could take to market.   They sang my praises as I spoke to large audiences and evangelized their most innovative software and solutions.  They were pleased with the great reviews from customers.

As one would expect, I was destined to learn the face of the problems I experienced as an end-user “outsider,” now from an “insider’s” perspective. 

One of the interesting challenges that surfaced at Nerwana was the “let’s export our culture and business model to the world” mantra, maybe better referred to as “if it sells in New York, then we must sell it the same way in Tokyo or Bejing!”

Also, I really was surprised to find out how dependent Nerwana was on the opinion of analysts.   When I worked for the customers and end users, we rarely paid any special attention to the analyst’s opinions.   Sure, analysts provides a good data point, but that is all it was (or is), simply another data point.   

I soon found that software companies are often held hostage by “analyst chasing” which really was an eye opener for me, because we end-users, the people who actually buy the software, view analysts as mere mortals reading from the same foggy crystal ball as everyone else. 

Another one of the fasinating challenges I experienced at Nerwana was what some would call  “The Hero Culture.”  

I’ll elaborate on some these, hopefully interesting, observations and experiences in a future Page from Greg’s Diary.