5 Predictions for Continuous Intelligence in 2021

By Jay Shaver, Sr. Director of Product Management at Swim

With 2020 behind us, it's worth spending time to reflect on the major trends of the past year. A lot has changed, with previous assumptions being disrupted by a rapidly transforming world. The crisis from COVID-19 is not yet over and businesses across several industries are adapting to their own version of a new normal. While much of 2020 was spent reacting to events, now is the time to learn, grow, and plan for success in 2021. 

Forward-thinking companies will undoubtedly start the year evaluating how to build more resiliency into their organizations, starting with their data. The right initiatives in digital transformation can help companies become more situationally aware about their operations, so they can anticipate events and even project better outcomes into the near future. Unlocking their data will also lead many companies to become operationally responsive in everything they do - particularly when it involves their customers. Much of the flexibility companies are looking for comes through the adoption of Continuous Intelligence. 

Continuous Intelligence is a whole new approach in how data architectures are designed to understand and learn from data in real time. It means always having an answer in the moment, by processing and analyzing streaming and historical data simultaneously, so that insights generated are delivered in context. The journey for many organizations won't be easy, leading to my 5 Predictions for Continuous Intelligence in 2021: 

1.  Ubiquity will emerge between the Cloud and the Edge

Every company has its own internal views and dynamics, but a growing trend has been the organizational divide into two camps. One group believes "The Cloud is Eating the World" (Forbes, Reuven Cohen, 2012) and has focused on moving data into the platforms, tools, and technologies available through the major cloud providers. The other group believes "The Edge will Eat the Cloud" (Gartner, Tom Bittman, 2017) and has focused on Internet of Things (IoT) initiatives that move software processes closer to the sources of data. Regardless of appetite, each approach has its advantages and companies are wanting the flexibility to leverage both depending on the use case. 

My Prediction: Organizations will mandate that any new software technology adopted is flexible enough to run in the cloud, on the edge, or anywhere in between.

In 2021, companies will want to avoid committing to a single strategy and will only adopt technologies they believe are flexible enough to adapt with them. High costs of data storage and the desire for faster insights might suggest a preference for edge software that can interact with customers in the physical world. Yet most of these edge nodes still need to be connected to centralized data stores for companies to act and make informed business decisions. Future-proofing for both environments will require applications built on distributed compute practices that can seamlessly analyze and communicate insights regardless of where the processing occurs. This will give organizations the flexibility to run solutions on whatever compute resources are available now, but also enable them to evolve their strategy for future infrastructure investments as their business needs change.

2.  The Speed of Decision Analytics will be Prioritized

When asked about making real-time decisions from data analytics, many organizations confessed that the latency incurred across their data pipelines resulted in unacceptable delays.. The result of adopting and integrating several technologies together often means sacrificing latency and performance for familiar building blocks. The reality is that most technologies are purpose-built for specific scenarios, many of which center around database storage. However, the need for faster insights is driving an interest in alternative architectures that can deliver an answer in the moment.  

My Prediction: Organizations will prioritize efforts that reduce the time needed for decision makers to have access to relevant insights.

This year, business requirements for faster insights will drive many companies to explore data architectures that immediately transform, process, and analyze in real-time. It means a different approach than storing raw or partially transformed data, and then performing an analysis after the fact. These new architectures won't negate storage, as longer term trend analysis, reporting or governance requirements may require data and insights to be stored in a database or data lake. However, storage doesn't need to be the performance bottleneck it currently is for organizations needing ‘always-on' situational awareness.

3.  Artificial Intelligence will become Operational using Real-Time Data

Organizations are becoming more adept at building and training AI models that run on data in the cloud. This is a natural result of the better tools, training, and documentation that comes with keen market interest. Now, companies are asking how they make use of these models in real-time using live data. Changing the model inputs to a continuously streaming set of data, which is sometimes only ephemerally useful, introduces a new set of questions. Does ‘seed data' need to be used initially? How long does the solution need to train on ‘live data' before it's accurate enough? How often does the model update and can models be shared between instances? In order to learn more, the theoretical must be tried in practice. 

My Prediction: Organizations will need help in tackling the challenges of using real-time streaming data as they migrate AI/ML models into live operational practices.

Companies have been asking questions for a while about the readiness of AI/ML models to work with live data and their patience has worn thin. If anything, the past year has placed a new urgency on organizations to adopt AI/ML practices into part of their regular decision making operations. Continuous Intelligence applications that ‘listen to' continuously changing streams of data are ideal for building, training, and evolving a model from live data. They allow each instance within a distributed application to create an individual understanding of its own unique environment.   

4.  Companies will Reconcile with the High Costs of Data Storage

Current trends in data architectures are pushing companies to collect, transport, and store as much data as possible in the cloud for future analysis. The trend was further compounded in 2020 by remote work and video conferencing services, increased online shopping, the growth of streaming services, the continued adoption of IoT technologies, and finally, the introduction of 5G cellular services. Organizations needed a new way to interpret and understand major shifts in their markets, and for many, data became the "voice of the customer." However, all data storage comes at a cost - even though the value and usefulness of data may vary over time.

My Prediction: Organizations will start to reconcile with the significant cost of moving and storing data in the cloud.

This year, companies will make an effort to internally assess the usefulness of data in making business decisions. This will both inform and update cloud storage strategies and lead to new initiatives of when and where to process data in real time so as to maximize the use of data in the moment and in context with business operations. The adoption of Continuous Intelligence technologies will undoubtedly change the way organizations use and store their data going forward. 

5.  Continuous Intelligence will become the Nervous System of Decision Intelligence

In 2021, companies will show renewed vigor automating the decision making processes that give them a competitive edge. This means deploying technologies even more that immediately transform, process, and analyze raw data so that decisions can be made quickly. Incorporating business processes, to the extent they can handle best practices and fringe cases, will take time. Most platforms aren't yet sufficiently architected to handle the complexity and use judgement in a broad set of situations the same way staff can be trained. Nevertheless, companies know that differentiation in the future will come by automating contextual analysis of multiple sources of data into actionable insights for decision support. 

My Prediction: Organizations will adopt Continuous Intelligence as a critical component of a broader strategy towards Decision Intelligence.

In 2021, companies will start talking more broadly about Continuous Intelligence and how they're moving to understand data in real-time. Yet, Continuous Intelligence isn't just about real-time processing of event streams. Contextual information is essential to understanding the broader implications of any key findings. Connecting insights is also important for correlation analysis and informing the decision-making elements of a business or solution. With the addition of AI/ML capabilities in a solution, companies will start exploring a new level of Decision Intelligence for their business.

##

About the Author

Jamison (Jay) Shaver is the Senior Director of Product Management at Swim. He has years of leadership experience building iterative solutions to solve complex challenges. He enjoys developing new products, crafting business strategy, and building businesses with a lasting impact. He is well versed in the areas of AI, Software Technology and IoT.