EPTS Report: Event Processing Reference Architecture Working Group (Slides)

The Event Processing Technical Society (EPTS) met in Orlando, Florida, collocated with the Gartner Event Processing Symposium. I reported on the activities of the Event Processing Reference Architecture Working Group (EPRAWG) in this set of slides:

Event Processing Technical Society: Event Processing Reference Architecture Working Group – Roll Call and Open Discussion

The full roll call results of the activities, summarized in the slides above for 2006-2007, may be found in this Google spreadsheet.

Please comment if you have any questions or follow-on comments.

Getting Started in CEP: How to Build an Event Processing Application (Slides)

Many of you have asked me for a copy of my slides from the Gartner Event Processing Symposium last week. Here they are:

Getting Started in CEP: How to Build an Event Processing Application

Thank you for all the kind words after the presentation!

I’ll post some of my reflections on the meeting soon.

What is Complex Event Processing? (Part 6)

Posted by: Tim Bass

In What is Complex Event Processing? (Part 5), we discussed situation refinement, the functional component of event processing that describes refining multiple event objects in order to estimate and identify business situations and scenarios in real-time. Today, in Part 6 of What is Complex Event Processing, we discuss impact assessment – where detected business situations are compared, correlated, and/or analyzed in “what if” type of scenarios to determine and predict business consequences.

After event detection and situation refinement, businesses are very concerned with ascertaining or predicting outcomes and financial gains or losses if a detected situational threat or opportunity materializes. Impact assessment is the functional component of event processing that is focused on the estimation and prediction of the priority, utility or cost of an estimated business situation, complex event or scenario.

At this stage of the CEP reference model (above), we estimate the impact of an assessed situation, which includes likelihood and/or cost/utility measures associated with potential outcomes. From this inference, loss projections and liabilities (or gains) may> In addition, resource allocation and processing priorities may be estimated.

Opportunities and threats in business generally need to be predicted based upon an estimate of the current situation, known plans and predicted reactions. Example of real-time predictive types of business use cases are:

– determining the expected consequences of a fraudsters actions in an ecommerce scenario given the current estimated threat to the business;

– estimate the consequence of a failure in a distributed computing application and the effects on other systems that interact with the failed component;

 

– estimating the potential profit if an algorithmic trade is executed on a tracked equity or basket of equities;

 

– predicting how delays in shipping effect the supply chain, including consumer choices and behavior;

– predicting network congestion and outages based on specific patterns of anomalous network behavior in real-time;

 

– assessing risk and losses in a potential aircraft collision based on information about the planes, the location and their cargo or passengers;

 

– predicting the impact of a viral epidemic on different geographic areas and populations;

 

– predicting costs saving based on optimizing network resources in a transportation or supply chain network; or,

 

– predicting potential losses if an identified class of missile reaches its projected target.

Impact assessment generally requires real-time correlation of historical data which resides in databases. This is represented by the Database Management component of the event processing reference architecture.

In my next post, What Is Complex Event Processing, Part 7, we will discuss another important area in CEP, process refinement – actions taken, parameters adjusted, resources allocated (for example) based on detected (and/or predicted) business situations and scenarios.

Copyright © 2007 by Tim Bass, All Rights Reserved.

What is Complex Event Processing? (Part 2)

(Originally Published by Tim Bass, TIBCO Software Inc. , April 23, 2007)

In a previous blog entry, What is Complex Event Processing? (Part 1), we introduced a few basic event processing concepts and a functional reference architecture for CEP based on the JDL model for multisensor data fusion. One of the most important concept in our reference architecture is the notion of events, which is the topic of this blog entry today.

What is an Event?

Similar to many topics in science and engineering, the term event has different meanings based on who is observing the event and the context of the observation. Let’s review of few of the different definitions from the point of various observers, keeping in mind that in CEP we are primarily interested in processing events related to business. First, we take a wider survey.

If you are a mathematician, you might view an event via the lens of event probability theory, which states than an event is a set of outcomes (a subset of the sample space) to which a probability is assigned. So, for example, if we were processing many banking application log files, in real-time, looking for fraud, there exists some conditional probability at any moment that a fraud is being orchestrated against the bank. The event is the fraud (detected or undetected outcome); and based on a number of factors, the probability of a fraudulent event against the bank changes over time.

On the other hand, if you are a particle physicist, an event is a single collision of two particles or a decay of a single particle! A collision, in particle physics, is any process which results in a deflection in the path of the original particles, or their annihilation. This view seems to imply that atomic and subatomic exceptions and state transitions are the foundation for events, which may be significant if you are a particle physicist. Assuming most of the readers of the blog are not particle physicists, you may be interested in the draft definition of an event from the Event Processing Technical Society (EPTS) CEP glossary working group, summarized below:

Event: Something notable that happens.

Examples:

– a financial trade

– an airplane lands

– a sensor outputs a reading

– a change of state in a database, a finite state machine

– a key stroke

– a natural or historical occurrence such as an earthquake

– a social or historical happening, e.g., the abolition of slavery, the battle of Waterloo, the Russian revolution, and the Irish potato famine.

Event (also event object, event message, event tuple): An object that represents, encodes or records an event, generally for the purpose of computer processing. Notes: Events are processed by computer systems by processing their representations as event objects. Events are immutable objects. However, more than one event may record the same activity.

Examples:

– a purchase order (records a purchase activity)

– an email confirmation of an airline reservation

– stock tick message that reports a stock trade

– a message that reports an RFID sensor reading

– a medical insurance claim document

Overloading: Event objects can contain data. The word “event” is overloaded so that it can be used as a synonym for event object. In discussing event processing, the word “event” is used to denote both the everyday meaning (anything significant that happens) and the computer science meaning (an event object or message). The context of each use indicates which meaning is intended.

As one can see, none of these definitions are completely satisfying! For example, if we look at financial market data, some might observe that it appears a bit pedestrian to say that each trade is an event. Why? Because the market data is the entire sample space and each trade is an element of the set of trades of that particular equity (for example) on a particular day. To call each trade an “event” may be unsatisfactory for some people.

On the other hand, when a business is processing market data using a VWAP algorithm, for some the event occurs, for example, when the price of a buy trade is lower than the VWAP. Conversely, if the price is higher than the VWAP the event would be an indication to sell.

This example tends to more closely align with the mathematicians view of events, an outcome from the sampled set with an assigned probability. The fact that the event is “significant” is due to the context of the theory and application of the VWAP strategy – without VWAP there might be no event, in this context. Similar analogies can be illustrated for fraud detection, supply-chain management, scheduling and a host of other CEP related business problems.

Events are Context Dependent

For example, if you have thousands of packages with RFID tags traveling the globe, is the event when the RFID reader registers the RFID tag? Or is the event when an exception occurs, for example, a lost package? One view is that the RFID reader is simply recording data and the associated RFID data is the sampled set (not necessarily the event). The outcome of interest, with an assignable probability based on the business context, are exceptions, which, in term, become business events. On the other hand, another view might be that each RFID recording is an event, and CEP is detecting “situations,” in this use case, the situation we refer to as “lost package”.

In you are interested in other terms related to CEP, please visit the Draft Event Processing Glossary. Your comments on the glossary are both welcome and much appreciated!

In What is Complex Event Processing (Part 3), I will begin to discuss the functional components of event processing based on the functional reference architecture introduced in What is Complex Event Processing? (Part 1).