CEP and BRE

I spent the last four months to dig into Complex Event Processing offerings and especially how BRMS and CEP can work together to support new type of business applications, and to move to agile IT architecture. As stated by Professor David Luckham in his book the ‘Power of events’: future applications need to address the following problems

• Monitor events at every level of the IT systems
• Detect complex patterns of events, consisting of events that are widely distributed in time and location of occurrence
• Trace causal relationship between events in real time.
• Take appropriate action when patterns of events are detected.

Adding CEP engine within our IT architecture will help bring agility as we can define complex patterns of events, hot deploy them, execute them, and improve their scope by time. CEP solution today uses a language close to SQL to define statement which filters, aggregates, joins events, and applies pattern matching on stream of events. Those statements are deployed to a CEP engine which will continuously evaluate them on a flow of events, when the conditions match subscriber applications (or code) will receive the complex or synthetic event and do something about it. If we consider those statements as rules, BRMS is a good candidate to manage them as it offers all the efficient tooling to consider those rules as asset with their own life cycle.

As of today BRMS uses Rete based engine to evaluate and fire if-then-else type of rule. An event processing statement looks more as a query. For example if we want to extract bid events on given car brand, event occurring in a time window of 15 seconds and we want to compute the average price (one of the attribute of the BidEvent). The statement may look like:

select *, avg(price) as avgprice

from BidEvent(itemName='BMW').win:time(15 seconds)

I’m using the Esper open source product EPL syntax to illustrate this example. This is a very simple ‘rule’ with the expressiveness of SQL. Easy to understand for a programmer but with big limitations when we need to communicate with the business user on what rule is doing. This is more true in real application. Most of the time those statements will become more complex to understand (even for programmer) when such statements combines joins, aggregation, multiple streams and database lookup. Putting a high level language on top of this SQL based approach will help on that matter. JRules for example offers a framework called BRLDF to define business language on top of lower programming language.

Those statements have a business motivation and CEP applications are really pushed by business users as those technologies are helping to identify pattern of events relevant to the business. We are moving to real time BI. So with this business dimension we can consider CEP statements as business rules. This means BRMS can support the management of those statements, and offers an integrated environment for business analysts and developers.

Now, one of the main questions is the deployment model. Does Rete based engine support well the high throughput and low latency of current CEP application requirements? The response is very close to no, if we speak about millions of events per second, rules to apply on sliding or jumping time window… If the constraint on the number of events decreases BRE may be a good solution. We are using JRules or Rule C++ in the telecoms industry since mid 90s for alarm filtering and correlation applications and it was the main driver of the BRE demand at that time. So we can use Rete for event processing. In fact in long term we can imagine having different engines under the same product or one engine which will pick up the more efficient algorithm to use according to its deployment model. I will post some example of rule using event in a next post.