Virtualization Technology News and Information
PROS 2020 Predictions: The Future of Data Science and What to Expect in 2020

VMblog Predictions 2020 

Industry executives and experts share their predictions for 2020.  Read them in this 12th annual series exclusive.

By Justin Silver, PhD, Manager of Data Science & AI Strategist, PROS

The Future of Data Science and What to Expect in 2020

The world of data science and analytics is a fast-moving space, and with each new advancement comes new capabilities across industries - be it in B2B or B2C, pricing or sales, distribution, manufacturing or logistics. At the heart of this progress is the need for businesses to provide their customers with the high-quality buying experience they have now come to expect and demand. Businesses are also motivated by the need to improve experiences for their employees and utilize data to augment internal operations. Data-derived insights can open up a world of opportunity, and while 2019 was an eventful year, these are the advancements we can look forward to in 2020.

Top Trends

Each year brings its own trends and, in some cases, disappointments when some highly anticipated advancements fall short. The first major trend we'll see in 2020 is more organizations exploring and implementing automated machine learning pipelines, which have become a pivotal component of data science platforms. The tasks that keep data scientists busy day-to-day, such as data preparation, feature engineering, and modeling, will be further augmented by tools that help automate these steps, and we will see more systematic implementations of business solutions to make ad-hoc analyses more efficiently repeatable.

Additionally, as the use of AI continues to permeate the business world, there will be significant attention given to interpretability of AI (i.e. Why is the system giving me this price and offer recommendation?) in order to support change management and organizational adoption of AI. If a system recommends pursuing a particular sales opportunity for sales reps, they'll need to understand its value.

The third trend we'll see in the coming year is rapid expansion and democratization of analytics tools, which will continue to bring data science to the masses in 2020. As these tools become more widely available and commoditized, there will be an even greater need for those with data science expertise to support the tailoring of these tools for effective use in particular business use cases. Organizations will continue to grow their data science teams and collaboration with external experts as they further adopt analytics tools. While "using data science and AI to solve this business problem" certainly sounds nice, there are many details that need to be considered for proper use of technology to address a business need. Demand will continue to grow for experts who can map out an end-to-end path to success with developing, implementing, and adopting data-driven solutions.

AI Systems and Chatbots

Many anticipated that in 2019 there would be wide adoption of chatbots in a business context, and it doesn't seem like that took off as quickly as might have been expected. However, in the coming year businesses will grow their use of natural language processing and building, or integrating chatbots, on top of their business AI systems, to improve customer and user experience. Companies will move beyond chatbots that rely on "if/then" commands that address customers' most frequently asked questions to smarter, more conversational chatbots that can better satisfy specific customer requests.

The major challenge for developing this smarter chatbot technology is being able to identify the customer's or user's intent. This requires gathering lots of training data from previous user interactions with the chatbot to understand whether the user's intent was properly identified for each interaction and learning from any mistakes to improve for future interactions. As a data scientist, I would certainly love a chatbot that I could use to help fetch me data from a large database, generate visualizations, and evaluate different models for me without me having to write any lines of code. If anyone has such a solution, please do send it my way!

Data Science Teams and their Challenges

Successful data science teams benefit from having a balance in skillsets. Many people talk about the elusive data scientist ‘unicorns' - great programmers, great at statistics, domain experts, excellent communicators - but it's rare to find one person with all these strengths (and even if you found this unicorn, how many different tasks would you really want one person to take on?). For that reason, leaders will look for data science talent across many academic disciplines and with varying areas of expertise. The more traditional data scientist backgrounds - statistics, math, computer science - will continue to be augmented with engineers, physicists, economists, psychologists, etc. Having a combination of different technical backgrounds provides a wide, rich set of perspectives for solving problems. It also makes collaboration more interesting and fun and encourages team members to effectively communicate their ideas, even at the earliest phases of research when ideas can be very fluid.

The data science field is never short on challenges. An ongoing obstacle that will continue into 2020 for many data science teams is keeping their research connected to practical applications. In running a data science team, it is important to create an environment that fosters creativity but also provides structure. Often, the nature of research is that it is open-ended. While there should be encouragement of creative, out-of-the-box thinking ("there are no crazy ideas"), there is also a need to achieve business objectives - it's not just academic.

In 2020, businesses will continue to sharpen the focus of their data science research and development initiatives by taking more AI-based solutions out of the lab and into production. In implementing AI solutions to solve their real-world business problems, companies can more tangibly measure the impact of these solutions through KPIs rather than having to rely only on theoretical or simulation-based assessments of these solutions. The learnings from tracking these KPIs can be used to directly inform the AI system on how to adjust in order to better achieve the targeted KPIs and provide a general sense of how the AI system is performing and how much value it is driving for the business. AI systems that learn from continuous feedback, driven by real user and customer interactions, will help propel businesses towards being able to deliver on the buying, selling, and operational experiences their customers and employees are increasingly expecting.


About the Author

Justin Silver 

Justin Silver, PhD is a manager of data science and research at PROS. He specializes in the application of data science to enable pricing and sales excellence. Dr. Silver's innovative contributions to the PROS solutions suite have helped customers to achieve substantial ROI through a scientific approach to commerce. Dr. Silver has a PhD in statistics from Rice University.

Published Tuesday, November 12, 2019 7:35 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<November 2019>