There has been ongoing, and often controversial, discussions in the event processing community about standardized benchmarks for event processing. Experts tend to agree on at least one point – benchmarks should not be designed by commercial software vendors, but should be independently developed by a neutral third party.
Opher Etzion and I were discussing this point in Toronto at DEBS 2007 last week. We both agree, as David Luckham has often suggested, that we need an independent set of benchmarks for event processing. So, I was google’n around the net today, and stumbled upon NASA’s Spacecraft Health Inference Engine (SHINE). A bit of further investigation into SHINE lead me to this announcement from Viaspace, a spin-off of the NASA/Caltech Jet Propulsion Laboratory:
“Spacecraft Health Inference Engine (SHINE) is an ultra-fast rules engine originally designed for on-board monitoring, analysis and diagnosis of NASA’s spacecraft and ground data systems. It is intended for mission critical systems where inferencing speed, portability and reuse is of critical importance. SHINE embodies revolutionary concepts in knowledge representation and manipulation that go well beyond traditional expert systems. Standardized benchmarking studies have shown that SHINE can process over 30 million rules per second on desktop class computer processors. It delivers over 100X increase in inference speed and up to 1000X reduction in the size of the execution environment footprint over the best comparable commercial product.”
It is interesting to note that SHINE does not use Rete, which NASA considers to be non-deterministic, so it seems:
“Because SHINE does not use the RETE inference algorithm and uses a propriety Data Flow transformation, the performance of SHINE generated source code can always be determined in advanced.”
Any idea which standardized benchmarks they used to measure SHINE?
Note: With Viaspace trading around 0.28 per share this week, on loses of over $6M USD in 2006, should someone in the event processing space acquire Viaspace’s rights to commercialize SHINE?