Re:Adapt Data Science Insights

‍What is your Generative AI Strategy?

4 April 2024
Forward to a friend and subscribe to our regular newsletter with our latest insights.

Generated from DALLE-3.
The image required several prompt iterations as it repeatedly produced nonsensical words for the whiteboard content.

Business leaders are spending significant time and effort thinking about to what extent their organisations should embrace Generative AI (GenAI). While GenAI has showcased remarkable capabilities in generating content and facilitating work such as supporting customer contact centres, writing code, and generating marketing content, its reliability remains a concern. Gen AI can yield false outputs and may disregard privacy and copyright laws. This raises questions about whether its all-in return on investment is worthwhile, or whether a wait-and-see or fast-follower approach can mitigate some early implementation risks. Many business leaders are saying that the time for proofs-of-concept is up and now is the moment to realise value from GenAI. How to go about this? What are the corners that business leaders need to see around to capitalise on the potential of GenAI?

A Houlihan Lokey article noted that some investors are pulling out of certain private equity deals despite sound fundamentals in the target business. The argument is that if AI evolves over the next five years to render the target business unnecessary to customers, they cannot invest in it. This makes the case for leaders to understand the impact of GenAI on their businesses more important than ever.

What is GenAI and why is it relevant?

Gen AI models include Large Language Models (LLMs) which are language-specific models based on Foundation Models (FMs). LLMs take a natural language request as input and return natural language as output, hence their ability to generate dialogue for chatbots, summarise text, or even write poetry. This is possible due to their training on corpuses containing vast amounts of human generated text. What is remarkable is their versatility in being able to tackle specific tasks on which they were not explicitly trained. Whereas in the past, training a language model for a specific task would require a relevant labelled training set, which can be expensive to acquire, LLMs offer state of the art performance with little to no training effort (although improvements for certain tasks are possible by providing a small number of labelled examples to the LLM, or fine tuning the model if more data are available). There are already tens of thousands of open-source and proprietary models available, and more are being launched every week. AWS offers models from AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, in addition to their own models. Google provides FMs like Imagen, Muse, Codey, and Chirp, while OpenAI has GPT-4 Turbo and GPT 3.5 Turbo (which powers ChatGPT), as well as DALL-E, and Whisper.

This has led to chatbot-style applications such as ChatGPT and others:

1. Gemini - Multimodal LLMs from Google Deepmind, available for Google Workspace.
2. Copilot - An everyday AI companion by Microsoft.
3. ChatPDF - Upload and chat with PDFs such as contracts, legislation, and financial reports.
4. Character.AI - Create characters and chat to them for “personalised AI”.
5. FinChat - Q&A about public companies and investors.

And numerous productivity tools:

1. Otter.ai - Records audio, writes notes, captures slide content, and generates meeting summaries.
2. Cogram - Takes meeting notes and identifies action items.
3. Taskade - Outlines tasks and notes, creates lists and mind maps.
4. Notion AI - Upgrades its wiki and docs capabilities with Gen AI notes and docs.

Some GenAI tools may be adopted quickly because people interact with them as they would with another person. Many organisations have given their people access to various GenAI tools to support pieces of cumbersome work they already do. Augmenting work with GenAI is easier to execute than applying it across an entire cross-functional business process and can still generate value. For example, Morgan Stanley Wealth Management enhances its financial advisors’ recommendations to clients by using GenAI to “listen in” on live calls, matching client questions and data with firmwide knowledge on companies, sectors, asset classes and capital markets.

Many of the customer service use cases we see today do not enable a direct response to customers, with human agents free to apply their knowledge of the situation to decide whether to apply the LLM’s recommendations. This has the dual effect of capitalising on the most skilled employees’ experience, while more quickly ramping up the capabilities of new employees. This fits with Re:Adapt’s views on applying GenAI to help employees do what matters to them and their customers.

GenAI strategy in the context of machine learning

Whilst the general consciousness of GenAI exploded following the public launch of Chat GPT in November 2022, Machine Learning has been around for a long time. Looking at a Google Trends comparison of search terms, Machine Learning had a bump upwards in November 2015 and another in November 2021. This perspective highlights that the recent surge in awareness and excitement surrounding GenAI is part of a continuum of progress and innovation within the broader domain of artificial intelligence and machine learning, and a focus on the right use cases with the right technology is important to value creation and a high return on investment.

Google Trends Topic: GenAI and Machine Learning: GenAI searches rapidly increased after November 2022 while Machine Learning searches rapidly trended upwards 7 years earlier

Matching GenAI and machine learning to the appropriate use cases

There are different sets of business use cases which will benefit from machine learning versus GenAI and vice versa. The following are a few general considerations for when machine learning or GenAI should be used:

1. Creativity Requirement: GenAI is preferred when the task involves generating new content or creative outputs. Machine learning, on the other hand, is better suited for tasks focused on making predictions or recognising patterns within existing data.
2. Training Data Size: Machine learning models typically require large amounts of training data to learn patterns effectively. End users of LLMs need provide only small datasets (e.g., few shot learning) when using pretrained models whilst benefiting from the vast quantities of data used during training the foundation model.
3. Structured or Unstructured Data: Machine learning is generally associated with structured data (think tables of numbers that can be easily categorised and labelled) and GenAI is associated with unstructured data (think natural language text, images, and audio that may not have a clear format). However, ML can be applied to images for applications such as segmentation, whilst GenAI can be used to interact with structured data formats by writing code (such as SQL or python).

Use case examples

A few examples of use cases for both machine learning and GenAI follow. They are not an exhaustive list and there is also a bit of cross-over possible. For example we have used machine learning models for Natural Language Processing to count topics from unstructured data, but find that GenAI can often perform better with the right instructions and data preparation. The opposite is also true, where we have deployed GenAI to convert a natural language request for structured, tabular, financial data into a SQL query (possible with a combination of other methods and the data being set up the right way).

Sample use cases for machine learning and Generative AI

One way to improve on machine learning

A wrinkle in machine learning models is that they are based on associations among many variables and these associations are not necessarily causal. However, machine learning models can be further augmented with tools that enhance their explainability by adding an understanding of causality. Deeptech startup causaLens has launched enterprise software to help data scientists collaborate with business leaders to discover cause-and-effect relationships in their data, build “white-box causal models”, and translate model outputs into more actionable recommendations.

One way to improve on GenAI

The pretrained LLMs that enable GenAI can be enhanced with other technologies to be more accurate and relevant. LLMs can be set up to reference specific sources of information held by your organisation to ground the responses, making them more reliable. Adding this information allows the model to reference both the specific data provided and its pretrained knowledge to provide better responses. This is called Retrieval Augmented Generation (RAG).

RAG is enabled by connecting pretrained models to your own data sources. This is possible in AWS, Google Cloud, Microsoft Azure, and other cloud providers. For example, in the case of Microsoft’s Azure OpenAI, it uses a search service (Azure AI Search) to add the relevant data chunks to the user prompt. Once this data from your organisation’s documents is in an AI Search index, Azure OpenAI goes through the following steps:

1. Receive user prompt.
2. Determine relevant content and intent of the prompt.
3. Query the search index with that content and intent.
4. Insert search result chunk into the Azure OpenAI prompt, along with system message and user prompt.
5. Send entire prompt to Azure OpenAI.
6. Return response and data reference (if any) to the user.

The use cases for RAG are wide-ranging. Imagine people across an organisation having access to all of the relevant documents for a particular project, thereby improving cross-functional knowledge and speed of execution. If meetings are captured as text, this information can also be referenced to provide deeper context on written reports and other documents. Documents in other languages can be queried with relative ease, and research papers and long contracts summarised so they can be considered in the decision-making process. Permissions can allow access to the appropriate data by the relevant team members.

Factors of success for applying GenAI to a business strategy

So, you know a little about GenAI, when it should and should not be used, and one technique called RAG to explore. What are some factors to consider for a GenAI strategy to be successful?

1. Business strategy: Starting with the business strategy sounds obvious but it is not easy work. Business strategist Richard Rumelt refers to strategy dynamics, which studies how changes influence the dimensions of an industry, redefining competitive advantages and business opportunities. A small senior team needs to consider such strategy dynamics–how the organisation will re-invent its industry and business model given these complex changes, followed by coherent resourcing to match the aspiration.
2. Data strategy: A data strategy must consider technology, processes, people, and rules. Leaders need to understand the data needed for a GenAI strategy and ensure that data is accurate, complete, and reliable. They also need to ensure that the correct architecture archetype is chosen and the relevant infrastructure is in place. Five architecture archetypes include data lakes, cloud-native data warehouses, lakehouses, data mesh, and data fabric.
3. Talent strategy: Considering talent should be an investment commensurate to the technology changes involved. What are the correct positions needed to execute the roadmaps related to data, technology, and GenAI strategies? How will the organisation attract and match the right talent to these positions? How will the organisation develop its existing talent to lead the relevant GenAI initiatives?
4. Agile and systems thinking: Having the right culture and mindset in place is also important. Are you able to capture the relevant feedback from test-and-learn initiatives with rapid iteration while understanding how the business operates as an integrated system? How will management thinking and the resulting policies permit risk-taking, cooperation and learning from mistakes?
5. Adoption and improvement in the operating model: The willingness and means to change the operating model is also a factor in a GenAI strategy’s success. How is the leadership team investing in the ability to act on the business system as new learnings occur and opportunities to change arise? How will adoption of GenAI and the resulting new ways of working be encouraged? Which business leaders are accountable for GenAI adoption and its results?

Investments for organisations to enable GenAI

ING released an article which proposes that investment in digital infrastructure (which they describe as ultrafast chips, microprocessors, and data centres) is crucial if GenAI is to seriously boost economic growth. They note that major technology companies such as Amazon, Alphabet, Meta, and Microsoft have invested heavily in this area which is necessary for further development and deployment of GenAI. The article cites an estimate from Counterpoint Research that in 2023, about 13% of Microsoft’s digital infrastructure spending was specifically for GenAI. This may indicate more about the macro view of GenAI and an idea of Microsoft’s strategy to own the infrastructure which will create value with GenAI, but we all may be able to draw an analogy about investment in GenAI for growth.

All enterprises will need to invest, to a certain degree, in enabling value creation with GenAI–the question is where and when. This is not limited to tools and technologies; whilst there are excellent cloud platforms with myriad services to enable the mining of insights from company data, there needs to be comparable investment in talent (data scientists, data engineers, AI/ML Ops engineers, and the businesspeople that can understand how to leverage their efforts). An investment in data itself is also required – not only in third-party datasets, but also in the work to create fit-for-purpose datasets which have may have been on the back burner for some time. Organisations that take this structured approach are more likely to learn where their GenAI strategy works, and where it needs to be updated, all while seeing how it affects the creation of value from the customer’s point of view.

Forward to a friend and subscribe to our regular newsletter with our latest insights.

- Jason Frank is the Managing Director of Re:Adapt Data Science who has a passion for re-thinking how we manage and leverage data to make better decisions.