In this exclusive VMblog interview, Surajit Bhattacharjee, Senior Vice President, Technology & Global Lead for Generative AI at Virtusa, offers a comprehensive look at the current state and future of generative AI in enterprise environments. With Virtusa’s track record of implementing AI solutions across various industries, Bhattacharjee shares valuable insights on successful deployment strategies, the importance of data quality, and the evolving landscape of AI models. He discusses Virtusa’s new GenAI offering, Helio, and provides a balanced perspective on the challenges and opportunities in adopting generative AI technologies.
From cost considerations to the potential of open-source models, this Q&A provides IT leaders and decision-makers with crucial information to guide their AI strategies in an increasingly AI-driven business world.
VMblog: Tell us a bit about the generative AI solutions that Virtusa has helped its clients build and put into production.
Surajit Bhattacharjee: So far, we have put close to ten generative AI-powered solutions into production. The first solution was a learning coach for a professional education client that operates in more than 200 countries. It went live last year and is being used by more than 50,000 paid users every week. Another interesting solution we built takes drone footage of warehouses and detects damaged items; we used multiple types of AI models to solve the problem including a small generative AI model. Other solutions we took live include a couple of customer service chatbots, a real-time contact center assistant, an assistant for field technicians, and a handful of solutions in the assisted-engineering space.
VMblog: Businesses have allocated considerable budget to generative AI this year. What should they do in order to put more solutions into production?
Bhattacharjee: The space is still evolving but there are a few things that have been proven to work. Having a center of excellence (CoE) is one of them. We have seen clients with a genAI CoE make more progress towards productionization than those that did not have one. Recently, there was a survey done by a McKinsey analyst that validated the same.
Picking the right use case and what you promise to the rest of the organization are also equally important. Use cases that are likely to meet the organization’s legal and compliance requirements are more likely to see the light of day. With a few rare exceptions, businesses are uncomfortable with the idea of not having a human in the loop – so, proposing generative AI solutions that are assistive to human workers is met with less resistance.
Given this whole area of generative AI is riddled with concerns about hallucinations, safety, and security, another critical success factor is to develop an understanding of techniques that have emerged for dealing with those concerns in a responsible manner. The onus for this starts in the CoE but the CoE then needs to evangelize this with the broader business organization. One of the areas that is often an afterthought is – how will I test the solution? In the early days, like everyone else, we were focused on demos and proofs of concept, but as clients made rollout commitments, we had to figure out the assurance piece.
Last year, we heard a lot about cost, but with model providers bringing lower-cost options to market and competent open-source models now available, we are encouraging clients to look at those options if their business cases don’t work with the ‘biggest and the best’ model. Some of this boils down to the risk appetite of the business – for example, we saw a large healthcare payer preferring a large commercial model due to indemnification benefits. This was despite them knowing that an open-source model would perform just as well and make their business case stronger.
VMblog: We are seeing competent models now on all 3 cloud service providers (CSPs). Do you think we will eventually have the same model quality across all 3 major clouds? If so, how will clients choose which models to use?
Bhattacharjee: There was a time when OpenAI was the only game in town, but now we have comparable models available on both Google Cloud and AWS. So, I think we are getting to a point of parity, so to speak. It’s difficult to predict if we will get to full parity, but one thing is clear – we are in an AI race and any serious advantage enjoyed by a given CSP will be short-lived. This means the risk of missing out because you locked yourself into one CSP is going to be low.
We see more and more businesses now looking at models that are available on the CSP that happens to house most of their data. Avoidance of data egress costs is the main motivation there. So, that will be one way our clients decide which model to go with. On the other hand, we are seeing CSPs offer extremely favorable terms to large businesses that they have been looking to penetrate for a while. With generative AI being generally seen as an expensive bit of tech, businesses are more open to bite at these offers even if that means working with a new CSP. So, that’s the second way we see decisions being made as we move towards parity.
VMblog: Virtusa recently launched its new GenAI offering, Helio. Can you explain what exactly Helio is, as well as share some context on the market and business forces that drove its launch?
Bhattacharjee: When generative AI hit the scene, instead of doing a marketing splash, we focused on understanding how this new tech could benefit our clients. This didn’t go unnoticed and today clients appreciate how our conversations and advice around generative AI have been grounded in reality, relevant, and dependable. We wanted to preserve that identity by giving it a name and building a brand over time – that is the thinking behind Helio.
We often get asked why we choose ‘Helio’. There were in fact ten different names that our marketing team came up with. ‘Helio’ meaning ‘sun’ has two connotations that we found relevant to generative AI: one, it’s something that illuminates the path forward for our clients as they navigate this new world, and two, a rising sun marks the dawn of a new era of innovation.
Having a brand helps us connect initiatives across talent, capability, offerings, solutions, accelerators, and delivery, ensuring they are in line with the same identity. In the near term, it helps clients with quick recall of our focus on generative AI. Over time, our hope is they also associate the brand with relevance and dependability – we are clear that the onus to make that happen is on us.
VMblog: What are your thoughts on open-source models and how do you see this field developing?
Bhattacharjee: Over the years, we have been big fans of open-source tech and it’s no different when it comes to generative AI models. That said, it has been a tale of two cities, so to speak.
On one hand, we have seen enterprise clients show curiosity and interest in open-source models, but we are not seeing many commitments being made. The indemnification they can get from commercial model providers is one reason behind this lack of uptake; subsidies are another.
On the other hand, independent software vendor (ISV) clients looking to embed genAI powered capabilities in their products are first looking at open-source due to cost advantages – no one wants to erode margins by offering something that costs more than they can charge for it. ISVs, on average, are also better equipped than enterprises to do the extra leg work required to make open-source models perform.
All that said, we are still in the early innings, and enterprise adoption of open-source models is likely to go up as they roll out more solutions and get comfortable with the tech. This will be driven not just by cost, but also use cases that can benefit from heavy customization of a model.
VMblog: The garbage-in-garbage-out argument also holds true for gen AI, so getting data right is key. How should businesses go about this?
Bhattacharjee: Generative AI models work best with unstructured data. It used to be mostly text, and now they can process images, audios, and videos. This poses a challenge to data/analytics organizations in most enterprises since they have traditionally dealt with structured data. Business teams in those enterprises are often frustrated with this handicap when it comes to building generative AI solutions. So, there is an urgent need to acknowledge this gap and make necessary investments.
We have seen some businesses express concerns about the quality of their data and the need to fix that first before they can start building their first generative AI solution. Our advice to them is to first work backwards from their use case and determine the subset of data required by the use case; and then focus on improving the quality of just that dataset.
We have also learned that getting data right for a genAI use case can be an iterative process. For example, we launched a customer service chatbot for a healthcare client and saw a small population of agents use it in a way that we hadn’t anticipated. This required us to go back and add more metadata to the dataset so we could drive a better agent experience.
VMblog: A lot of use cases so far have used retrieval-augmented-generation, we have not seen much on fine-tuning and training. Do you see that changing?
Bhattacharjee: A vast majority of client use cases we have worked on so far have used retrieval-augmented-generation (RAG). With context windows expanding and the need to optimize cost, we will continue to see RAG dominate in the foreseeable future. That said, we are starting to see some demand in our client base for finetuning and alignment of open models for very specific use cases. We are very excited about this as we have been experimenting with finetuning, reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) in our lab, and the time has come to finally apply those in the real world. It’s a change we are seeing for sure, but we think we will see far fewer use cases in this space compared to RAG in the foreseeable future.
VMblog: How much is model latency a hurdle in adoption?
Bhattacharjee: It depends on the use case. For example, there are use cases where the time taken to show the first word of the response needs to be optimal, users then don’t mind waiting a little more for the rest of the response to form and show. Most such use cases are well served today. Then there are use cases where the full response is required for it to be usable. An example of that is a chatbot that a customer service rep is using while having a caller on the phone. Unless the full response is rendered, the agent can’t formulate what to tell the caller. Buying provisioned throughput gets you a predictable latency range, but there is no commitment on the actual latency. So, these use cases end up requiring additional work to tune performance. And finally, we have use cases which require many things to happen in real-time. For example, a contact center assistant solution needs to transcribe the live conversation, analyze the transcript, retrieve relevant information from a knowledge base, send a request to a model, and display the response – all in real-time. Fortunately, we can build solutions today that are quick enough, although not real-time.
It remains to be seen how much faster models get over time without compromising their abilities, but this can become a real impediment for scaled adoption.
VMblog: And finally, what about cost?
Bhattacharjee: We are seeing a range of behaviors when it comes to cost. Large enterprises that have struck favorable deals with a CSP and model provider are moving forward with their use cases. They know subsidies won’t last forever, but they believe the tech will get more affordable over time. Then there are CTOs that are reluctant to move forward with anything unless they have full clarity on all cost contributors. Their concern is less on the actual spend, but the lack of clarity. In the middle, we have some enterprises and ISVs that are taking the time to build comprehensive cost frameworks, evaluate different models, and then decide which choices to pursue. In this case, cost is a hurdle that is forcing a more thoughtful approach and exploration.
In all cases, it is important to realize that ROI will be impossible in certain situations, and those use cases will simply need to be set aside.
##
