Bending CEP for Rules

I think James Taylor and my good friend Paul Vincent should be mindful not to reduce CEP (accidentally or intentionally) to rule-based systems, and broaden their perspectives and blog entries. In the original work on CEP by Dr. Luckham, the point of CEP is to solve complex problems in many problem domains, many require backwards chaining, uncertainty principles, statistical methods and more. Rule-based systems are interesting and useful, congruent with expert-systems, but also have well documented limitations (see notes below) in the classes of complex problems they can efficiently address.

Both James and Paul have excellent backgrounds in rule-based systems and have worked together in this area; on the other hand, CEP is not simply “rules and events” or “rules with EDA” etc.

Dr. Luckam’s background as a distinquished professor at Stanford was AI, including debugging large-scale distributed systems and performing complex network security research for DARPA. In all of these application areas, there is a known limit to the usefulness of rule-based approaches to address complex classes of decision support systems that require statistical methods to mitigate uncertainty. Rule-based systems are very useful, but they are suboptimal for the challenges of more complex decision support services that are better addressed by statistical and stochastic methods designed for systems with uncertainty – the problem set addressed by Dr. Luckam’s original CEP work.

I enjoy reading James and Paul supporting each other in the area of rules-based approaches to CEP; but I hope the “business rules folks” will keep in mind that CEP was designed to be significantly broader than rule-based decision support.

Notes:

Reference 1: Rule-based systems are only feasible for problems for which any and all knowledge in the problem domain can be written in the form of if-then rules and for which this problem space is not large.

Reference 2: Abstract: “We demonstrate that classes of dependencies among beliefs held with uncertainty cannot be represented in rule-based systems in a natural or efficient manner. We trace these limitations to a fundamental difference between certain and uncertain reasoning. In particular, we show that beliefs held with certainty are more modular than uncertain beliefs. We argue that the limitations of the rule-based approach for expressing dependencies are a consequence of forcing non-modular knowledge into a representation scheme originally designed to represent modular beliefs….”

Reference 3: “Rule based systems have no ability to automatically learn from their mistakes, nor do they have any way of determining information from their environment. As such, their use is usually limited to very simple problems that have a finite, known set of possible states.”

Reference 4: Broad Google search on limitations of rule-based systems.

Advertisements

15 Responses to Bending CEP for Rules

  1. Paul Vincent says:

    Hi Tim: you are quite right of course:
    CEP != rules + events.
    but events + rules do provide a lot of useful CEP capability. And really a rule is simply a query with an action at the end!
    Note also you can google plenty of other references on:
    – combining rule-based processing with other algorithms (cf your reference 1)
    – uncertainty in rule-based systems (cf your reference 2)
    – machine-learning in rule-based systems (cf your reference 3)
    It will be interesting to see what technologies evolve to be most successful in CEP. Certainly the rules + events approach seems to have a head start, and complex event driven decisision support seems to be a very popular CEP subset at present…

  2. Tim Bass says:

    Hi Paul!

    The cited studies on using rule-based systems in problems where uncertainty dominates mostly conclude that rule-based approaches are much less efficient than other approaches (Bayes, NNs, Demster-Shafer, etc).

    Actually, the CEP vendors who specialize in fraud detection, at the first event processing symposium in 2006, NY, highlighted Bayes nets, neural nets and more. I am not sure I would agree with you that the rules-crowd has a head start – but I would agree that rule-based approaches are certainly important. FYI: IBM has many partners in their event processing ecosystem in the fraud area, most do not depend solely on rule-based analytics to solve complex problems.

    On another front, many (if not most) of the other CEP vendors in the market today are processing tosets; they are stream processing where uncertainty (and noise) does not appear to be dominating the software design principles.

    IMHO, the software vendors who eventually will “win” in the CEP space will be “white box” event processing experts who offer different analytical methods (as plugins where appropriate).

    Yours faithfully, Tim

  3. James Taylor says:

    Of course there is a widely deployed, well established and hugely successful fraud solution out there that uses a combination of business rules and neural networks to detect fraud. It’s called Falcon Fraud Manager and it’s been catching credit card fraud for many more than 10 years, dramatically reducing the rate of fraud around the world (some 700 Million accounts are protected using the product). In the interests of fair disclosure I should say that this is a Fair Isaac product!

    My focus is on decision management and in the context of CEP that has a number of related points. Firstly it means focusing on the decisions you are going to take (once you detect an event), not just on the detection of the event. The “P” in CEP I guess. Secondly it means that a business rules platform is ideal as this is one of the best, most transparent ways to automate decisions. Thirdly, though, it means applying analytic methods to improve the precision of these decisions. These might be neural nets, analytically dervied decision trees, predictive scorecards, regression models etc. These techniques turn uncertainty into probability and the rules can then act deterministically on these probabilities.

    So you are right, rules alone will not get it done. Rules and analytics and a focus on decisions, on the other hand, have already proven themselves in CEP and will continue to be critical anytime someone wants to take intelligent action in response to an event.

    JT

    PS Re your refs I would point out that they assume you don’t combine rules with analytics. They are also derived from AI people who, let’s face it, have a poor track record in terms of getting anything real delivered 🙂 Rules-based systems do a certain set of things really well. They can also be enhanced easily with more sophisticated kinds of analytical decisions. They are widely deployed and they work. Expert systems and AI, in contrast, have what can only be described as a dubious track record.

    I’m a pragmatist. As my recent book puts it, I would rather have a system that is “smart enough” than wait for some researcher to make it “intelligent”.

  4. Complex Event Processing – not just rules

    I promise this will be the last post on event processing for a while but Tim Bass posted a comment to me and this post on this blog – Bending CEP for Rules. Tim was making the point that CEP…

  5. peter lin says:

    I think the references are out-dated. The one about uncertainty is from 1987 and a lot of progressed has occured since then. Look at the recent success of Stanford’s Stanley autonomous vehicle. Stanford won the grand challenge and Sebastian Thrum. Here’s some links for those interested in it.

    http://robots.stanford.edu/papers/thrun.stanley05.pdf

    I agree that using prolog type system is far from optimal for CEP. In contrast, a mature business rule engine that supports forward chaining, backward chaining, custom modules, custom built-in functions and development tools can handle CEP without any problems.

    Is there a specific use case, which you believe to be suboptimal using a mature product like JRules, or Blaze? To my knowledge Dr. Forgy’s OPSJ is the fastest RETE engine and it supports both forward and backward chaining. It also provides a set of API for developers to extend and built analytics if they wish.

    In terms of scalability, I firmly believe using distributed RETE algorithm, a system could handle hundreds of millions of rows of data for real-time applications. I filed a patent for it back in 2003. The basic principle of distributed RETE is to replicate the betaNode indexes across a large cluster of rule engines. In concept this is very similar to distributed indexes in databases like Oracle10g or DB2.

    If you combine analytics with a distributed RETE rule engine, the engine can distribute the indexes and workload efficiently.

    peter lin

  6. Tim Bass says:

    Hi Peter,

    Bayesian analysis, one of the most popular analytics for a wide class of problems today cannot be implemented efficiently with a rules-engine.

    Yours faithfully, Tim

  7. peter lin says:

    I don’t know about Bayesian analysis, but I will read up on it. If it’s similar to bayesian filters used in spam filters and other types of bayesian filters, I would agree one wouldn’t want to attempt to model that with a rule.

    In those cases, why not simply integrate a rule engine with an existing bayesian analysis tool? If you look at the work Sebastian Thrum did for Stanley, he combined rule engines with statistical analysis. In the case of stanley, one of the biggest challengs was figuring out what data is noise. To do that, the stanley team used a variety statistical filters to weed out noise. The data was then aggregated and sent to the rule engine for further reasoning. When I read the paper, I thought their techniques could work well for real-time trading systems.

    I see no reason why these two technologies shouldn’t be combined. I have seen people try to write long complex calculations with a rule language. I don’t recommend people do that. In fact, I usually tell people to offload that type of work and integrate with a rule engine. The hard part is figuring where one stops and the other starts.

    One of the easiest ways to integrate analytic tools is to write a function to call out to it from the rule engine. That’s an old technique that has been around for a very long time. As much as I love rule engines, they aren’t magical and don’t solve every single problem. It’s just a tool like everything else.

    peter lin

  8. Tim Bass says:

    Hi Peter,

    Thanks again for bringing so much wisdon to my blog. I appreciate the comments.

    FYI, neural networks are another example of analytics useful in CEP applications. There are more examples (like HMM, Hidden Markov Models).

    The point of my message is that rules are great, but they should not be considered the sum total, end-all be-all, of CEP analytics, as some have been posting.

    In fact,, there are some CEP vendors (Aleri, as I recall, but don’t quote me) that use Bayesian analytics in their fraud detection offering.

    Yours faithfully, Tim

  9. peter lin says:

    I wouldn’t say I bring any wisdom. I enjoy learning about new things and figuring out the best way to use rule engines. part of that is figuring out when not to use rule engines. I’ve had dozens of people ask me “would you write a complex calculation using the rule language?” It might surprise others, but my response has been “why would you want to do that?”

    A rule language is good for declaring patterns. Although I could write a rule to calculate the Profit & Loss with the rule language, that would be mildly retarded. Makes much more sense to call out to an optimized C++ or Java library. I spent an hour googling “bayesian analysis” and it seems to share similarities with Case Base Reasoning.

    Do you know of any good resources for Bayesian analysis?

    peter

  10. Tim Bass says:

    Hi Peter,

    Bayesia is an interesting company and has some software you can download:

    http://www.bayesia.com/

    Classifier4J is one of many Java projects

    http://classifier4j.sourceforge.net/

    Also, Ruby has some nice Bayesian support:

    http://www.rubyinside.com/classifier-gem-bayesian-and-lsi-classification-for-ruby-507.html

    To name a few…

    BTW, I am not affiliated with any of the above.

    Yours faithfully, Tim

  11. peter lin says:

    thanks for the references. I did spent about 3 hours last night reading up on bayesian analysis. It’s interesting stuff and reminds me of Case Base Reasoning, which also uses the same techniques. I came across some papers for CBR that used Bayesian analysis. definitely fascinating stuff.

    peter

  12. Tim Bass says:

    Hi Peter,

    I am glad you found the links useful. Bayesian techniques are used in a wide-range of applications including, diagnostics, fraud detection, intrusion detection, and spam filtering, to name a few. RSS filtering is now done with Bayesian techniques.

    There is a good write-up on Wikipedia:

    http://en.wikipedia.org/wiki/Bayesian_inference

    Cheers.

    PS, I can’t access your blog.

  13. peter lin says:

    thanks for the link. if you want an invite, email me at woolfel AT gmail DOT com

    peter

  14. […] Engines and Bayes’ Theorem Charles Young kindly calls out my blog post, Bending Rules for CEP, and the discussion on rules and Bayesian analytics in his post, Rules Engines and Bayes’ […]

  15. […] and rules engines, specifically regarding RETE implementations. I particularly like this entry from Tim Bass on the […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: