From smart homes and smart cities to fitness devices and connected cars, Internet of Things (IoT) solutions are capturing new types of information. Not only do they provide insights into emerging trends; they also serve to trigger immediate alerts and actions based on conditions. At the same time, the sheer velocity and volume of data—with sensors generating new events every few seconds—bring new challenges to sift through events for meaningful information while keeping IoT applications from slowing to a crawl. To address this demand, WSO2 today unveiled the fully open source WSO2 Data Analytics Server (WSO2 DAS) 3.0 platform.

WSO2 DAS 3.0 is the next evolution of WSO2 Business Activity Monitor 2.5, which it replaces. It combines into one integrated platform real-time and batch analysis of data with predictive analytics via machine learning to support the multiple demands of Internet of Things solutions, as well as mobile and Web apps. As a result, it uniquely allows developers of IoT-enabled solutions to:

• Use the same published events to deliver business and technical insights, and trigger an action when a specific pattern is detected. Real-time analysis of events via complex event processing (CEP) can be used to trigger immediate actions, such as alerting homeowners on their mobile phones if the house temperature falls below a certain level. Meanwhile batch analysis can track trends over time, for example what time of day the heater temperature is adjusted higher.
• Apply a machine-learning model to the batch analytics pipeline, and then use the resulting predictive analytics with the CEP pipeline to, for example, predict traffic congestion based on highway sensors tracking traffic flow.
• Conduct analytics at the edge. With a footprint just 2 MB, the CEP engine in WSO2 DAS is light enough to run on distributed IoT devices or gateways, and it can be programmed to aggregate data on these edge systems. This facilitates the detection of important trends or aberrations—such as changes in temperature or failed access attempts—and significantly reduces network traffic to improve performance.
• Ensure high performance and availability in IoT applications where reliably rapid response is critical, such as connected cars, manufacturing equipment, and home fire and smoke detection. At the 2014 DEBS Grand Challenge requiring real-time analysis of a smart home electricity monitoring system, the CEP engine in WSO2 DAS took the first place award for achieving a rate of 400,000 events per second on a single node.
• Realize development and deployment flexibility. The same cloud-enabled WSO2 DAS software can run on servers, gateways and devices, as well as in the cloud. And since WSO2 DAS is fully open source and available under the Apache License 2.0—the enterprise-class product is also the community version.

“The vast amounts of data generated from the Internet of Things, mobile devices, and Web apps hold valuable keys to serving customers better, creating new business and revenue models, driving greater efficiency, and ensuring robust security,” said Dr. Sanjiva Weerawarana, WSO2 founder, CEO and chief architect. “With our WSO2 Data Analytics Server, we deliver a single, integrated open source platform to unlock the insights into these opportunities by combining the ability to analyze the same data at rest and in motion with predictive analysis. At the same time, our platform offers the flexibility to scale to millions of events, whether running on-premises and in the cloud, to support today’s high-volume, highly interactive IoT, mobile and Web apps, which demand immediate responses.”

“The goal is to build applications that can act on the insights gleaned from analyzing IoT data. That means the Internet of Things must become the Internet of Analytics to make sense of the rush of data emitted by the devices,” observes the Forrester Research report¹, Internet Of Things Applications Hunger For Hadoop And Real-Time Analytics In The Cloud. The report goes on to say, “IoT is nothing but a dazzling starfield of hype with no business value unless your firm builds it on a foundation that provides the requisite in-motion and at-rest analytical capabilities that are essential to building IoT or IoT-informed applications.”

Integrated Real-Time, Batch, Interactive Analysis and Predictive Analysis

WSO2 DAS 3.0 introduces the ability to analyze both data in motion and data at rest from the same software. It is architected around Apache Spark, the open source cluster computing framework for large-scale data processing, which the Apache Spark team has benchmarked as running up to 100x faster than Hadoop MapReduce in memory, or 10x faster on disk.

WSO2 DAS also builds on the fast performance of the open source Siddhi CEP engine developed by WSO2 by adding streaming regression and anomaly detection operators to facilitate fraud and error detection. Additionally, it leverages Apache Storm, the open source real-time computation system, which provides greater scalability by enabling multiple events and streams to run in parallel. WSO2 DAS works with relational databases and supports SQL queries via Spark SQL. Additionally, users can write SQL-like queries using the Siddhi Query Language.

WSO2 DAS adds predictive analytics through machine learning. Using a wizard to build machine-learning models, developers can then run these models on WSO2 DAS and WSO2 Enterprise Service Bus 4.9 (WSO2 ESB) to apply predictive analysis to their applications. The machine-learning functionality is based on the open source Spark MLlib distributed machine-learning framework running on Apache Spark.

Analysis from WSO2 DAS can be communicated via alerts, APIs and dashboards for visualization. Enhanced visualization capabilities in WSO2 DAS make it easy to build clear, uncomplicated, customizable dashboards that give users an at-a-glance view and let them drill down to uncover the details.

Extensible Analytics Toolboxes

WSO2 DAS can be used to implement a variety of use cases, from smart cities to targeted marketing. The platform also includes extensible toolboxes for common use cases, which users can extend to support their specific applications. The toolboxes cover how events will be received, aggregated and analyzed; how pattern detection will be handled; and how the results will be visualized.

The Activity Monitoring toolbox lets users correlate events related to the same transaction in order to visualize, analyze, and write queries on top of those activities. This is a useful tool to drill in, find out, and improve business activities.

WSO2 also provides an analytics toolbox specifically designed to work on top of each WSO2 middleware product, including the highly popular analytics functionality available with WSO2 API Manager.

Additionally, WSO2 has developed two sample use cases that customers can leverage:

• Fraud and Anomaly Detection supports fraud and anomaly detection through static rules, Markov chains, and scoring. When an alert is triggered, it allows the user to zoom in, interactively analyze each alert through visualizations, and to make appropriate decisions.
• GIS Data Monitoring takes a data stream tagged with geographical locations and support visualizations of that data in a map. Through the user interface, a user can set up speed monitoring, proximity monitoring, prediction, and geo fencing queries, as well as receive triggers.

Availability and Support

The WSO2 Data Analytics Server 3.0 platform, combining batch, real-time and predictive analysis is available today. Also available now are WSO2 Complex Event Processor 4.0 (WSO2 CEP) for users who only need to analyze streaming events in real time and WSO2 Machine Learner 1.0 for predictive analytics.

All three WSO2 analytics offerings are available as software downloads that can run directly on servers or on top of a private PaaS, and as a WSO2 Cloud Virtual Machine running on the Amazon Elastic Computing Cloud (EC2), Linux Kernel Virtual Machine (KVM), and VMware ESX. Additionally, customers can choose to have WSO2 host the software through the WSO2 Managed Cloud service. As fully open source solutions released under the Apache License 2.0, they do not carry any licensing fees.