As of December 15th 2023, GenAI is now GA with Coveo. A lot of the steps I wrote in this blog post are not needed anymore, as the flow is now much simpler. You can find more information on Relevance Generative Answering (RGA) in our documentation.

As you may have heard, Coveo recently released its Generative Answering solution (also called “GenAI”). It’s been all the hype internally at Coveo, as well as externally with multiple customers and partners approaching us to play with this new product and implement it on their end.

Similarly, there’s recently been a lot of hype about the release of Baldur’s Gate 3, the Larian Studios video game based on Wizards of the Coast’s classic TTRPG Dungeons and Dragons.

As a big fan of D&D and a big fan of new tech, I thought it would be a great idea (and a great way to sink my teeth into a new Coveo product) to create a Coveo GenAI-powered bot that can answer questions about Dungeons and Dragons.

The Basics

I’ve taught the basics of Coveo many times to new employees, partners, or customers in the past, and I always say that a successful Coveo project comes in three main aspects: the content, the experience, and the relevance. GenAI is no exception to this, but the exact ways these interconnect is.

The Content refers to getting information from outside of Coveo inside of Coveo. How do we make sure we index the right things with the right metadata?

The Experience refers to the presentation of this information. How do people view this content using Coveo?

The Relevance refers to the order and value of the content. How do we make sure that the content people view using Coveo is the right content, for their specific use case and query, for this specific user.

Normally, in a non-GenAI scenario, I would say that the Experience and the Relevance are often built together, with the Content being built ahead of time. However, with GenAI, things are a bit different; the Relevance is built alongside the Content, and the Experience aspect is much more minimal.

I’ll explain what I mean in more details.

The Content

For my use case, I wanted to index content that explained the multiple races, classes, backgrounds, spells, etc., of Dungeons and Dragons (specifically 5th edition). Thankfully, there is a slew of different wiki’s and websites available that explain and list those for me. I simply chose one that was formatted in the easiest way for Coveo to ingest, and indexed it via the Sitemap connector, one of our most popular and simple connector.

I did need to tweak the sitemap a bit because it wasn’t exactly how I wanted it, but this required minimal effort, and in half an hour I was good to go.

It’s important to understand that Coveo’s GenAI model does not learn from everything ever posted online. It learns specifically from the content that you feed it via the Coveo index. This is great for use cases where there’s a lot of outdated content online, or where the content is hidden behind a log in or a firewall.

But, in my case, I was using content that was technically public, but at least I was able to control exactly which content I wanted to use.

With the Sitemap connector, I built my source and ended up with almost 2,500 items.

The Relevance

Normally, the bulk of the relevance aspect would come at a later point in the project, when you already have an experience. But, with GenAI, the relevance needs to happen concurrently with the content.

The way GenAI works requires new fields to be created on the documents, so they can store a numerical representation of that document in space. These are called Vector fields, and they are fundamental in getting anything Semantics-based to work at Coveo.

Since Coveo’s GenAI product is currently still in Beta, there were quite a few things that I needed to do via the API, such as creating these fields, adding my Indexing Pipeline Extension to populate those fields, and creating the GenAI Machine Learning models themselves. These steps will be made easier with time as we finalize the current beta program and move to general availability.

However, even with the slight complication of having to manually call the Coveo API’s, I was still able to finish all the steps in about half an hour, which did include having to rebuild my Sitemap source from earlier. Rebuilding in this case is crucial, as it is through it that the vector values can be added to the fields.

Finally, I created a dedicated query pipeline, added a few rules like changing all ranking weights to 7, enabling partial match, and associating my new ML models.

The Experience

Building the experience was probably the easiest of all three steps.

All I had to do was create a basic page using Coveo Atomic, a lightweight low-code framework to build search pages.

Then, I simply added my search box, and used the nifty Atomic component called atomic-generated-answer to display the response from GenAI on my page.

The whole code of the page ended up looking like this:

<script type="module" src="https://static.cloud.coveo.com/atomic/v2/atomic.esm.js"></script>
<link rel="stylesheet" href="https://static.cloud.coveo.com/atomic/v2/themes/coveo.css" />
<script type="module">
      await customElements.whenDefined("atomic-search-interface");
      const searchInterface = document.querySelector("#search");

      await searchInterface.initialize({
          accessToken: "my-api-key",
          organizationId: "my-org-id",
      });
      searchInterface.executeFirstSearch();
</script>
<atomic-search-interface id="search" search-hub="GenAI DnD">
    <atomic-search-layout>
        <atomic-layout-section section="search">
            <atomic-search-box></atomic-search-box>
            <atomic-generated-answer></atomic-generated-answer>
        </atomic-layout-section>
    </atomic-search-layout>
</atomic-search-interface>

I was done so quickly with my page that the Machine Learning models didn’t even have time to finish building, which means I did everything in under half an hour this time.

The Result

After just a little over an hour of work, I was ready to see what I was able to accomplish with GenAI. I’ve recently been thinking a lot about creating a new Paladin character, and so I entered my first query:

I was surprised at how quickly I was able to make everything work! I started making other queries, and each time was even more impressed.

For those of you reading that know Dungeons and Dragons, you’ll find that the answers to those questions are pretty accurate. Plus, if you didn’t trust the answer given by GenAI, you’d be able to click directly on the actual documents it used to come up with its answer, something every DM wish they could give to their players when they’re questioning your ruling.

And I was able to do all this without even reinforcing the model! It’s not a secret that, when it comes to Machine Learning, things improve over time as more and more people interact with the experience. In my case, I was able to accomplish a working and relevant GenAI model even before reaching this point. I can’t imagine what sort of amazing things would happen given more interactions.

The Conclusion

If I had to describe in one sentence my feelings towards testing Coveo’s GenAI product, I would say that I was amazed. Implementing GenAI is currently still a little rough around the edges, with many different individual API calls required to configure everything. But, these are details that can be improved on as we move towards a full GA release. What really matters is the results, and I can already see how amazing those results can be, especially considering the small amount of effort I needed to put in.

If you’re interested in getting your own GenAI model powered by Coveo, I strongly invite you to reach out to us. I think there is great potential for it to be the next big thing in the customer experience space.

If you also think GenAI is a game changer (hey see what I did there) and you’d like to be part of the party team making it happen, check out our careers page!