Bare-Bones Requirements for an Event Processing Banking Application

I am working on a security-related  event processing banking application for one of the main banks in Thailand.     Here are the basic “must have, bare minimum” requirements:

  —  The event processing engine must run on Linux.

  —  The engine must be configurable and manageable remotely via a web-based interface. (Edit:  A Windows-based fat-client remote manager could also meet this requirement.)

  —  Must have a Windows-based modelling studio for building event logic / rules.

  —  Modelling studio should generate the running code to upload to the engine.

  —  Processing engine must have a UNIX sockets interface (adapter) out-of-the-box.

  —  Must have a data modelling / transformation, mapping tool, such as XPATH, for mapping raw input (in this case a UNIX socket) to event data structure(s).

These are only the bare minimum requirements.

Since I am an ex-TIBCO Principal Global Architect, I was hoping to find other CEP software vendors who have this very basic functionality, because I don’t want to appear to be biased toward TIBCO with the bank.

I tried BEA‘s WebLogic Event Server because they also have a presence in Asia, but their event processsing platform met only one (The Linux Requirement) of the six “bare minimum” requirements above. 

If any CEP vendor can meet the banks “very basic, bare-bones requirements” please comment here on the blog or email directly.

Thank you.

PS:  Latency is less critical than the bare-bones requirements above.  We can easily route the events to instances of the event processing engine, so the main requirements are based on the ease of design (modelling), remote configuration and management and deployment.

Advertisements

17 Responses to Bare-Bones Requirements for an Event Processing Banking Application

  1. Hans says:

    Although I don’t care what vendor is chosen, I have looked at Business Events and other vendors for a large monitoring project. I agree that BE is a good product but I would question whether XPATH mapping is a primary reason for this.

    XPATH mapping of input is nice, but is really just a replacement for some other code. XPATH syntax can be hard to understand for anything but the most simple paths. So for simple mappings, you are replacing simple code with simple code. And for complex mapping, you are replacing complex code with complex code. In this context, I have found that XPATH is no more likely to be understood by a less technical user than any other kind of code. And a regular programmer is still needed to define the XPATH, then it has not saved any money for the project.

    If this will be a big project, it will end up with far more effort put into the rules than into mapping fields from input. For example, if the project has to standardize the contents of fields containing heterogeneous data, this will be done in the EP rules and not in the mapping.

    So I suggest that another criteria for any EP evaluation is the ease with which the product handles the kinds of rules that the project will need. And I suggest that this criteria is significantly more important then whether the product has XPATH mapping of input.

    For example, if the project spends 40 staff-hours mapping input data to fields but 500 staff-hours on getting the rules to work right, then the overall cost will be affected much more by the effectiveness of the EP language than by the ability to input with XPATH.

    I wonder if, before choosing these requirements, someone looked at the alternatives. I would think that, for example, rather than mandating a web based interface, the requirements would be for the operations staff to understand and like the management and monitoring interface. After all, if a desktop GUI based monitoring interface works better than a web based interface, then why would someone insist on using the web?

  2. Tim Bass says:

    Hi Hans,

    If you notice, the requirements did not say must be XPATH, it said “must have a data modelling / transformation, mapping tool, such as XPATH” where XPATH is an example.

    Really Hans…. 🙂 you are quite the contrarian 🙂

    Yours faithfully, Tim

  3. Hans says:

    Hmm… my comment still stands about preferring a better language to a better input mapping system. If I’m working on a big EP project, I would rather have an EP language that does everything I want, but be forced to write a little code to map input to structures in the EP engine. To me this is a much better option than having an excellent data mapping tool but an EP language that doesn’t meet my needs.

  4. Tim Bass says:

    Hi Hans,

    Not sure why you are keying on “mapping” as the event language is already covered, from a requirements perspective, under the modelling studio requirement.

    The bank prefers a good graphical rule-building tool versus “the best” event processing language (EPL).

    You don’t address the real question of which CEP vendor meets the banks requirements. Perhaps you can address which EP vendor meets the requirements of this customer, per their stated bare-bones prerequisites?

    Yours faithfully, Tim

  5. Hans says:

    Ah, it was not clear to me that you meant a graphical language, it looked like you just meant a windows development studio.

    I was keying in on mapping because, to me, the ability of the language to solve the problems posed in the project seems critical, where a mapping framework seems like a nice-to-have.

    Just to let you know, my experience is that a graphical language can turn to spaghetti just as easily as a text language, only with the graphical language it actually looks like spaghetti.

    I will let the individual vendors discuss their product features, my interest lies more in discussion than in arguing for or against a particular product.

  6. Tim Galvin says:

    I believe Aptsoft Director for CEP meets the requirements you’ve outlined

    – The CEP server is written in java, so it will run on a variety of platforms including Linux.
    – There is a web-based adminstration UI to facilitate remote administration of the server.
    – The UI for building event logic and rules can be run in Windows.
    – We have a generic TCP/IP connector which should address your UNIX socket requirements.
    – We’ve abstracted our UI’s to a high level such that there is generally no coding required. Our built-in connectors take care of a lot of the data mapping requirements that you’d typically use something like XPath to address, but you can augment the connectors mapping functionality with both XSLT and javascript if need be

    Please let me know if there is any additional information I can provide.

  7. Tim Bass says:

    Hi Tim,

    This sounds interesting.

    The Aptsoft director can be completely installed and configured remotely?

    Also,

    I assume the Window’s based UI creates a Java EAR file that is uploaded to the Aptsoft Director via the web-based admin tool?

    Thank you for responding.

    Yours faithfully, Tim

  8. Tim Bass says:

    Hi Hans,

    We also think that EPL’s are important. However, we also assume that most, if not all, CEP/EP vendors have an underlying EPL, even when they use a graphical modelling tool or studio. This particular requirement list was only bare-bones and was not meant to outline every possible requirement.

    Take care.

    Yours faithfully, Tim

    PS:

    For example, Apama has a modelling studio and they also have their underlying EPL (Monitor Script), Aptsoft does as well; BEA and ESPER do not yet have this feature, unfortunately, since I like both of these products as well.

  9. Tim Bass says:

    Dear Folks,

    Thank you for your responses and email.

    It appears that a Windows-based remote configuration / management tool should satisfy the customer’s requirements if a web-based remote configuration / management tool is not available.

    Yours faithfully, Tim

  10. Hugh O'Hare says:

    Hi Tim,

    Tim Galvin’s suggestion suffers from one of the same problem as WLES which is support for UNIX sockets out of the box.

    It is a common misunderstanding that UNIX domain sockets are interchangable in some way with TCP/UDP sockets. If you are to look at a Java based platform you are probably going to run into this issue irrespective of vendor because java does not support UNIX domain sockets. Unless the vendor has implemented a JNI interface to a native handler for the sockets and make the event data accessible by some other means, an “adapter for the adapter” if you like, you will hit this.

    The other alternative on java EPs would be to add that layer, with is probabaly not a big issue if you are deploying to a single target environment, however if you forsee a situation where your adapters will be deployed across multiple *nix platforms you’ll have to have this compiled for each target platform.

  11. Tim Bass says:

    Hi Hugh!

    Good point! It looks like the “pure” Java-based EP vendors will need to incorporate some additional code to meet the UNIX sockets requirement.

    Ref: http://freshmeat.net/projects/j-buds/

    “Java-based Unix Domain Sockets (J-BUDS) provides a UnixDomainSocket class to address the need in Java for accessing Unix Domain Sockets. It provides a similar interface to the standard java.net.Socket class provided as part of the standard Java API. The source is provided for a shared library containing the compiled native C code, which is called into by the Java UnixDomainSocket class, via JNI, to open, close, read, and write to Unix domain sockets.”

    Thank you for pointing this out, Hugh.

    Yours faithfully, Tim

  12. Michael Zatl says:

    Hello Tim !

    Regarding your requirements list, perhaps our product may fit those requirements:

    Linux : Initially, we currently cannot comply to this point. But I´d like to elaborate on this a little bit further : our product is developed based on Microsoft .NET technology. Although we haven´t tested it on .NET for Linux up till now , it may also be ported on a Linux based .NET environment.

    Web-based interface/Windows Fat Client : yes, we have this ! This is part of our product and is called InTime Modelling Studio which can be deployed on the engine server or as a client on any other machine.

    Windows based modelling studio for logic & rules : yes, we have this ! Done in the
    InTime Modelling Studio. Correlations, Rules & Ruleset or Scoring services can be deployed and modified both via GUI, drag & drop and can also be implemented on code-level.

    Modelling studio generates the running code to upload : yes we have this ! You can develop your solution in the modelling studio and then deploy the application and all needed building blocks on the processing engine server. Changes can be made in the modelling studio and deployed to runtime on the engine, there´s no need to shutdown the processing engine.

    Unix sockets interface: yes, we also provide a TCP/IP adapter which allows us to receive event data via sockets. This adapter is appropriate for high-volume data exchange !

    Data modelling/mapping tool like XPATH: yes, we have this ! All events can be represented/accessed as XML documents and can be mapped to database tables. The mapping information is defined with XPATH expressions or alternatively with so-called “EA (EventAccess)-expressions” which provide more sophisticated mapping mechanisms.

    If you need any further information on our product, please feel free to take a look at our demo/trial version available on our website. If you have any further questions, don´t hesitate to contact me !

    with kind regards,
    Michael Zatl
    SENACTIVE Inc.

  13. Tim Bass says:

    Hi Michael!

    Thanks for posting!

    As a “gentle reminder” in this application, the requirement is *NOT* for TCP/IP sockets, as Hugh pointed out in an earlier comment. The requirement is for a UNIX domain socket.

    Ref:

    http://docsun.cites.uiuc.edu/sun_docs/C/solaris_9/SUNWdev/NETPROTO/p59.html

    “UNIX domain sockets are named with UNIX paths. For example, a socket might be named /tmp/foo. UNIX domain sockets communicate only between processes on a single host. Sockets in the UNIX domain are not considered part of the network protocols because they can be used to communicate only between processes on a single host.”

    Therefore, a TCP/IP socket out-of-the-box will not work. It must be a UNIX domain socket.

    Thank you for your understanding.

    Yours sincerely, Tim

  14. Hans says:

    If a product can only support TCP sockets, why not just use netcat to shuttle data back and forth between the domain socket and a TCP socket?

  15. Tim Bass says:

    Hi Hans,

    We already almost completed the development of a JNI-based UNIX domain socket adapter.

    More later ….

    Yours faithfully, Tim

    PS: netcat could work (seems a bit kludgy), but we prefer a native interface, that is why we have decided to go with JNI. I think the end use will like the JNI native interface better.

  16. Hans says:

    I’d do the same. At the very least, it’s one less process that can fail to start or be accidentally killed.

  17. Krishnendu says:

    Hi Tim,

    Actually I am also looking for an implementation of the Unix domain socket in Java .
    Did you try out JBUDS ? Since You are saying , you have created your own implementation, can you please share what problems you faced with JBUDS?

    Any pointers will be highly appreciated.
    Thanks ,
    Krish .

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: