Simple Event Processing != Complex Event Processing

December 16, 2007

One of the brillant minds in the CEP community, Claudio Paniagua Macia, recently posted, Event Stream Processing != Complex Event Processing.   In his post, Claudi draws a bold conclusion:

(1) SQL-based approaches to ESP might have a hard time doing CEP.

(2) No real CEP engine exists today in the marketplace, perhaps not even “off” the marketplace.

Friend, colleague, and co-chair Opher Etzion replied, On Event Stream Processing:

 “CEP engines do exist today, none is perfect, but probably sufficient for big majority of the existing applications today.”

Respectfully, I find it necessary to agree with Claudi and disagree with Opher.   Most of the so-called CEP engines today are solving quite simple event processing problems.   If the CEP engines on the market were truly solving a “majority of the exisiting applications today” then sales would be orders of magnitudes larger.

The fact-of-the-matter is that the current “simple rules-based approach” dominate in today’s marketplace are used to solve problems where rules-based approaches are useful.   Unfortunately, this is just a small fraction of the true potential of the CEP market.

For example (just one example of many), the vast majority of intrusion or fraud detection systems available today use rule-based approaches, and their detection capability, and the confidence in the detection, is quite elementary (poor quality).   If these systems worked well, cyberspace would be a very different and much safer place.  

Yes, it is useful to add another layer of rules, but rules alone will not solve the vast majority of CEP-domain classes of problems.   In addition, the CEP applications that have made the press recently are quite simple, certainly nothing scientifically earth shattering.

So, the sad truth of the matter, from an architectural, scientific and solutions perspective, is exactly as Claudi boldly offered, no real CEP engine exists today.    Furthermore, the vast majority, if not all, CEP applications sold today are used in very simple event processing (SEP) applications.  This is not very “advanced,” but it is a good start.  

What is holding the CEP market back is quite straight forward; the current “engines” are quite elementary (We should call them SEP engines.), relatively speaking, and SEP engines do not have the capability to solve difficult detection-oriented CEP problems in cyberspace.   These difficult problems compose the vast majority of the applications where “true complex event processing” is required.

Advertisements

CEP Center of Excellence for Cybersecurity at Software Park Thailand

December 16, 2007

In July 2007, at InformationSecurityAsia2007,  I unveiled an idea to create a cybersecurity CEP Center of Excellence (COE) in Thailand.  Under the collaborative guidance of Dr. Rom Hiranpruk, Deputy Director, Technology Management Center, National Science and Technology Development Agency (NSTDA), Dr. Prinya Hom-anek, President and Founder, ACIS Professional Center, and Dr. Komain Pipulyarojana, Chief National Security Section, National Electronics and Computer Technology Center (NECTEC), this idea continues to move forward.

Today, in a meeting with Mrs. Suwipa Wanasathop, Director, Software Park Thailand, and her executive team, we reached a tentative agreement to host the CEP COE at Software Park.   

The mission of Software Park Thailand is to be the region’s premier agency supporting entrepreneurs to help create a strong world-class software industry that will enhance the strength and competitiveness of the Thai economy.

Since 2001, Thailand’s software industry has experienced approximately 20% year-over-year (YOY) growth.  Presently, Software Parks Thailand supports a business-technology ecosystem with over 300 active participants employing over 40,000 qualified software engineers across a wide range of technology domains.

I am very pleased that Software Park Thailand is excited about the potential benefits of CEP in the area of cybersecurity and detection-oriented approaches to cyberdefense. The COE will be working with best-of-breed CEP vendors to build, test and refine rule-based (RBS), neural network (NN) based and Bayesian network (BN) based approaches (as well as other detection methods) for cybersecurity.

I will be announcing more details in the future, so stay tuned.  Please feel free to contact me if you have any questions.


The Asia Business Forum: Information Security Risk Assessment and Management (Day One)

December 11, 2007

Today is the opening day of the Information Security Risk Assessment and Management conference in Bangkok.   Mr. Charoon Boonsanong, Lecturer, Faculty of Economics, Chulalongkorn University, open the conference.  

Dr. Komain Pipulyarojana, Chief National Security Section, National Electronics and Computer Technology Center, will lead off with a presentation on the Latest Trends, Standards and Threats for Information Security & Future Direction.   Dr. Komain also serves as the lead for ThaiCERT.    

Police General Yanaphon Youngyuen, Deputy Commissioner, Department of Special Investigation, Royal Thai Police, will present Legal Updates: Interacting with Law Enforcement After a Cyber Crime or Systems Intrusion & Its Impact on Todays Business.

The last presentation before lunch is my presentation, CEP and SOA: An Event-Driven Architecture for Operational Risk Management.   

After lunch, Mr. Phillip Chong, Partner, Enterprise Risk Services, Deloitte Touche Tohmatsu Jaiyos Advisory Company Limited, will talk to us about Governance, Risk Management and Compliance (GRC) as a Model for the Management of Corporate Information.

The last presentation before I must rush off to fight the traffic in Bangkok for a cross-town meeting is, Mr. David Old, Partner, Information Risk Management, KPMG Poomchai Business Advisory Limited.


Type I and Type II Errors – The Heart of Event Processing

December 5, 2007

Opher Etzion begins to discuss one of the topics I consider to be the heart of event processing in his post, On False Positives and False Negatives.

Statistically speaking, false positives are called Type I errors (α errors) and false negatives are called Type II errors (β errors).

If you are interested in “getting to the heart” of complex event processing, visit this link.   I’ll discuss detection theory in more detail in a future post.