Blog Posts

Leveraging Generative AI

Introduction

Source: Pexels

What to Expect

This ebook provides an introductory overview  on Generative Artificial Intelligence (Gen AI) and a primer on how to prepare for your organization’s first GenAI project. This is not intended to be a technical document or geared towards any specific tech stack. If all you know about AI is what you may have read in the popular press and perhaps experienced while experimenting with the new AI Chatbots, then this is for you. You’ll walk away with a high-level understanding of the history, current landscape, and hype behind Generative AI.

Generative AI

/ˈdʒɛnəˌreɪtɪv eɪ aɪ/

Generative AI refers to the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content, such as images, videos, audio, text, and other digital content.

National Institute of Standards and Technology (NIST)

The AI Revolution

Over the last decade, there has been a revolution occurring within computing. However, unless you were actively following the changes, you probably didn’t realize how significant they were. Things such as autocomplete becoming smarter or photo apps being able to search by person or object may not seem particularly impressive now, but they were largely impossible to do only a decade ago—especially at scale or on device.

Source: Pexels

What was a mostly quiet revolution burst into public consciousness in 2022 with the release of ChatGPT by OpenAI and Stable Diffusion by Stability AI. Both products had capabilities far in advance of what had been previously seen and—more importantly—were accessible by the average person. It’s now possible to generate pictures, videos, stories or even code from a narrative prompt without any artistic or technical skill. This has also created challenges as forgeries, deep-fakes and misappropriation of IP can be done easier and faster than ever before. 

As we all know, there have been—and continue to be—countless startups all promising to harness the power and potential of what is termed “AI”. How do we separate the hype from the reality? As a decision-maker, how do you make sure you’re engaging in an effort that’s going to provide tangible benefits? Perhaps you think it’s all hype, but want to hedge in case it’s not; how do you position your company to take advantage of the tech that survives this hype cycle? 

We address these questions and more.

One Term, Many Meanings

 The term AI encapsulates several distinct techniques, such as neural networks, image diffusion, Generative AI, natural language processing (NLP), expert systems, and machine learning. We’re focusing on Generative AI, which refers to software that uses generative models to create new data based on their training and inputs.

Generatively Speaking

What exactly is meant when we talk about Generative AI? If we ask ChatGPT, it responds with the answer, “Generative AI is technology that creates new content, ideas, or data based on learned patterns and inputs”.  

If we follow up with the question, how does it do that, the system responds: “Generative AI analyzes vast data, learns patterns, then generates novel outputs resembling the data it was trained on.”  

Source: Pexels

Do you remember in the opening paragraph when we mentioned autocomplete becoming smarter? Generative AI can be viewed as an extremely sophisticated and advanced form of autocomplete. In fact, the current batch of large language models (LLMs) only predict the next word or part of a word (called a token). The software around the model feeds the complete statement plus the new token back into the model to get the next one until the model provides a stop token.  

This is important because the response is only going to be as intelligent as the input and training. Furthermore, the current consensus is that training should be reserved for learning new formats, whereas the input to the LLM is where the data should be. There are a couple of good reasons for this. Although LLMs contain vast amounts of data, the data is only up to date as of the time they were trained. They also have a tendency to hallucinate good-sounding answers—not necessarily correct ones. The news has been rife with examples of LLMs hallucinating court cases, providing fake medical advice, or even pretending to be alive.

Source: Reuters

One way to get around the problem of hallucinating is to use a technique called Retrieval-Augmented Generation (RAG). This technique, developed by Facebook AI research (https://arxiv.org/abs/2005.11401v4), essentially “packages” the knowledge you want the LLM to work on in the initial query. We’ll cover this in more detail later.

Reusing Your Data

“Data is the new Gold” – Salesforce Ad

Source: X

The above quote is only partially accurate. Data has always been gold, but until recently only the largest companies have been able to separate that gold from the dross. Regardless of what you want to do, the one thing you need is data – lots of it. However, the data you’re currently collecting is probably not sufficient enough. Even if you’re saving the data from some objects on a periodic basis, what are you going to do if you find out you need a different object? What do you do if you start a new project that needs a different subset of data? These are the problems you’re going to face in this new world where standard ETL and integration tooling isn’t going to cut it.  Traditional ETL is prescriptive when it comes to what objects and what fields are moved from one system to another.  If that information isn’t known in advance, a different technique is needed.  

If you want to reuse data for future AI and analyses, you need historical data as much as your current data. The only way to effectively train an AI model or populate a database for RAG is to have access to ALL of your historical data. This is because that data is what will be used to train the underlying model or store in the database. Without it, you would need to build a gathering step—also known as data collection—and wait for enough data to have accumulated.

Unlock the Power of Your Data

Discover how GRAX helped a national insurance services firm not just back up their data but turn it into a powerful asset for compliance, analytics, and decision-making.

Read the case study

High-Level Requirements for Generative AI Enablement

For Generative AI, you’re probably not going to have to do training – at least not initially.  Several of the LLM models on the market perform exceptionally well when combined with Retrieval-Augmented Generation to provide the initial context.  

A Short Primer on Retrieval-Augmented Generation

We’re not going to go into specific technical details here because, depending on the LLM stack you use, the approach will be different. In general, you’re going to use a search technique to get the data to feed into the LLM. 

One modern way of doing this is to use embeddings. Embeddings are vector representations of a word, sentence, or paragraph. As a simplified example (real embeddings are often hundreds of dimensions in size), dog might be [2,5,3] and cat could be [2,6,3]. Since the system has been trained in a particular way, those vectors that are semantically similar will be close to each other when you can do searches for neighbor vectors. So for instance, “dog” and “cat” may be close to each other, but “car” would be farther away. When you have a question for the system, you can turn the question into an embedding and then search to find similar vectors, get the context, and put it into part of the context block that gets sent to the LLM.

As an example of how RAG can help the LLM provide better content, think of getting asked to write a paragraph describing the appearance of the Titanic using only your knowledge. Now imagine getting asked to do that, but also being provided with the blueprints and newspaper articles describing the ship. The first way is like asking an LLM a question and just using the knowledge that’s contained within it because of the training data. The second way is like using a RAG approach to provide the model with the underlying information and letting the LLM summarize and synthesize from that information.

Source: AI-generated by OpenAI’s DALL·E based on a descriptive prompt emphasizing the Titanic’s grandeur and serene atmosphere.
Source: AI-generated by OpenAI’s DALL·E, designed to align more closely with the Titanic’s historical blueprints and documented features.
Source: OpenAI. (2025). Screenshot of an AI-generated summary about Generative AI. ChatGPT by OpenAI. https://chat.openai.com/

As you can see, this approach means the more applicable data you have, the better the possible result can be. The more of your history you have, the more opportunities you have.  

Managing your dataset

Since RAG begins with populating a database with data, it’s important to get your dataset in order. If you have access to all of your historical data, you can categorize/organize it appropriately. If you don’t have the right data, or your data hasn’t been stored correctly, you could wind up with a generative application that either doesn’t have the right data or returns incorrect data. Remember, the application is only as good as its data, so making sure the data is structured and cleaned correctly is the foundation to everything else.

Avoid the Hype

The following advice for leaders looking to evaluate AI while avoiding AI hype is similar to the advice for avoiding the pitfalls of any hype cycle.  

  • Look at any claims critically. Does it make sense? Does it seem realistic?
  • Even if a solution looks powerful in the proof of concept, it may not scale to your business. GenAI solutions are typically resource heavy, so you need to make sure as the product scales to production, that the performance and budgets can also scale.
  • Identify one or two concrete problems the new technology can solve and make sure it can solve them better than current approaches. Everyone likes the new shiny tool, but you want to make sure you’re actually moving the business forward.
  • The first-mover advantage is real, but never underestimate the advantage of being the second-mover. Second-movers can learn a lot from the successes and mistakes of first-movers.
  • Set yourself up for long-term success. Even if you’re not adopting Generative AI today, consider what you’ll need to be successful when you decide to roll out a generative solution.
  • Consider what the future of the market could look like in a year, and how you can prepare your company for that while not just waiting and losing ground.

The Future of AI

AI is here to stay.It is and will change your industry regardless of whether or when your company adopts AI. That being said, which technologies and techniques end up becoming widely adopted and which become niche offerings is at best a guess. Generative AI is a relatively new technology and, although it’s evolving very rapidly, there are many unknowns, including service costs, the regulatory environment, and consumer sentiment on GenAI.  

As with most new technologies, costs will come down as hardware and software are optimized to support the algorithms and techniques used. At the same time, there will eventually be a drop off in new startups as we move from the “we can do anything” to “here are proven solutions”. Prepare for the next wave of acquisitions, as enabling technologies and full solutions are bought by larger tech firms to build out their stack.

Where To Go From Here

The time to start planning using Generative AI is now—even if you’re not going to implement it right away. Like baking a loaf of bread, you need to gather the ingredients before you can cook.  You will want to identify your sources of data and make sure you have what you need for a successful project.

Source: OpenAI. (2025). Illustration combining Generative AI and bread baking [AI-generated image]. DALL·E by OpenAI. Retrieved January 8, 2025, from https://chat.openai.com/

Also, like baking bread, you need time. You should get ahead of the curve and have that complete data set. If you’re collecting your data now, then that clock has started and won’t need to be built into the timeline when you’re ready to take on the first Generative AI project. This will also enable you to begin the evaluation process using your own data.  As Generative AI accelerates the pace of change, you will be ready as you’ve built a solid foundation for your business.

Put Your Data to Work

With GRAX, you can get more value from your historical data for AI. Spot trends, make informed decisions, and take the guesswork out.

Discover how

Glossary of Terms

  • Artificial Intelligence (AI) — A field of research that develops methods and techniques for machines to learn, reason, problem solve, and understand knowledge.
  • Artificial Neural Network (ANN) — A cornerstone of machine learning and artificial intelligence (AI), which enables computers to learn and make decisions or predictions based on data.
  • Generative AI (GenAI) — A subset of the artificial intelligence field that focuses on generating content from a set of inputs.  There are several techniques to make this happen, but in general there
  • Retrieval-Augmented Generation (RAG) — A technique of storing information for the LLM in a separate database and using tooling to retrieve relevant information to feed back into the LLM along with the underlying question.  
  • Token — A word or component of a word represented by a numeric value.

See all
GRAX

Join the best
with GRAX Enterprise.

Be among the smartest companies in the world.