Build a Hybrid Search Feature with Storyblok Strata

The days of pure keyword search are long over, and users expect all search features to have semantic awareness—even outside dedicated AI interfaces. Strata is the new semantic search layer for Storyblok which, among other use cases, can serve as the backbone for hybrid search in Storyblok projects.

In this tutorial, you build a hybrid search feature for an Astro frontend that combines the precision of keyword search with the semantic superpowers of Strata. While this particular tutorial uses Astro to build the search UI, you can apply the core concepts and much of the implementation to every Storyblok project, no matter the scope or tech stack.

Strata in a nutshell

Strata is Storyblok’s semantic search layer, which searches spaces by meaning rather than exact keywords. Strata will over time become integrated into more Storyblok core features, but is currently confined to developer-facing REST APIs and the Storyblok MCP server.

For now, Strata is only available to Elite customers, who must explicitly enable it on a per-space basis. Once enabled, Strata can index the content in the space. After that, semantic search takes a single call to an endpoint on the Content Delivery API:

curl --request GET \
  --url 'https://api.storyblok.com/v2/cdn/strata/search?token=<ADD_ACCESS_TOKEN_HERE>&q=winter' \
  --header 'Accept: application/json'

The cdn/strata/search endpoint delivers search results with a score value that indicates how close a match each result is for the given query:

{
  "results": [
    {
      "excerpt": "Frostline Insulated Winter Boot",
      "score": 0.3911,
      "story": {
        "id": 227332818030170,
        "slug": "products/frostline-insulated-winter-boot",
        "name": "Frostline Insulated Winter Boot"
      }
    },
    {
      "excerpt": "Solstice 40° Summer Sleeping Bag",
      "score": 0.3556,
      "story": {
        "id": 228744493942051,
        "slug": "products/solstice-40-summer-sleeping-bag",
        "name": "Solstice 40° Summer Sleeping Bag"
      }
    },
    {
      "excerpt": "Frostbite Merino Balaclava",
      "score": 0.3377,
      "story": {
        "id": 227333793812401,
        "slug": "products/frostbite-merino-balaclava",
        "name": "Frostbite Merino Balaclava"
      }
    }
  ]
}

It’s apparent how Strata searches content by meaning rather than by keyword: the third result doesn’t mention “winter” in its title or excerpt, but a balaclava sold by the brand “Frostbite” is relevant to the query. Similarly, the “Solstice 40° Summer Sleeping Bag” is also semantically relevant (winter and summer are, after all, related concepts), but not exactly a great result in a search for “winter.” Since neither keyword search nor free semantic association yields perfect search results on their own, most search experiences are actually hybrid.

From keyword to hybrid search

Hybrid search is any search feature that combines different search result sets into a single, ranked list. This term applies to the meta search engines of the old internet as much as to the feature in this tutorial: a search feature that combines semantic and keyword matches. This ensures that the search results include all related content while ranking precise keyword matches particularly high.

Collecting search results from Storyblok’s REST APIs is rather straightforward: call an endpoint and collect the results. The challenge with building a hybrid search feature lies in combining the results from keyword and semantic searches. The available algorithms run the gamut from simple to sophisticated to outright scary, and many require extensive data processing or even entire machine learning pipelines. Reciprocal Rank Fusion (RRF), on the other hand, combines multiple ranked lists of results with only a Map and two nested loops. It’s fast, yields reasonably good results, and is therefore a great fit for a first foray into building with Strata.

Building a hybrid search function for Astro

The goal of this tutorial is to build a basic web form that performs a hybrid search powered by Strata. Astro is an implementation detail; you or your AI agent can adapt the following code snippets to your favorite frontend framework.

Astro setup

The example space for this tutorial contains content for a fictional outdoor activity company in categories like “Cycling,” “Running,” and “Fishing.” To bootstrap a basic Astro frontend, give an AI agent equipped with the Storyblok agent plugin the following prompt:

Create an Astro app for the Storyblok space 23421337 in the folder ‘app’. Add blocks and pages for browsing the different categories available in the space. Keep it simple and basic.

With access tokens in place and the app up and running, start building the hybrid search feature by adding old-school keyword search.

Building keyword search

Storyblok’s cdn/stories endpoint retrieves content matching a search_term parameter. To let users enter a search term, first add a search form to the project’s header in src/layouts/Layout.astro:

---
const { title = '' } = Astro.props;
---

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width" />
    <title>{title}</title>
  </head>
  <body>
    <header>
      <nav>
        <a href="/">Home</a>
        <form action="/search" method="get" role="search" class="search">
          <label>
            Search products
            <input
              type="search"
              name="q"
              value={Astro.url.searchParams.get('q') ?? ''}
            />
            <button type="submit">Search</button>
          </label>
        </form>
      </nav>
    </header>
    <slot />
    <hr />
    <footer>All rights reserved © {new Date().getFullYear()}</footer>
  </body>
</html>

Next, build the search result page in src/pages/search.astro. Here, the Storyblok API client calls the cdn/stories endpoint with search_term set by the q parameter passed by submitting the search form. Since this is a product search, constrain matches to stories with content type product:

---
import { useStoryblokApi } from '@storyblok/astro';
import Layout from '../layouts/Layout.astro';
import SearchResult from '../components/SearchResult.astro';
import { version } from '../lib/storyblok';

const q = Astro.url.searchParams.get('q')?.trim() ?? '';

const storyblokApi = useStoryblokApi();
const keywordResults = q
  ? await storyblokApi.getAll('cdn/stories', {
      version,
      content_type: 'product',
      search_term: q,
    })
  : [];
---

<Layout title={q ? `Search: ${q}` : 'Search'}>
  <main>
    <h1>{q ? `Results for “${q}”` : 'Search'}</h1>
    {
      q &&
        (keywordResults.length === 0 ? (
          <p>No products found.</p>
        ) : (
          <ul>
            {keywordResults.map((data) => (
              // SearchResult only reads full_slug (or, if that's not available,
              // slug) and name from data
              <SearchResult data={data} />
            ))}
          </ul>
        ))
    }
  </main>
</Layout>

This implements a functioning but bare-bones search whose results reflect neither relevance nor semantic meaning. The endpoint cdn/stories can sort the returned objects multiple ways, but there is no built-in way to rank by relevance. You can improve the basic search function in two ways:

• First, rank keyword results by relevance
• Then, apply RRF to fuse the ranked keyword search results with semantic search results from Strata

The MiniSearch package implements an in-memory full-text search engine that can re-rank keyword results based on relevance. To use it, install minisearch, import the library, then create and configure an instance of MiniSearch:

// https://www.npmjs.com/package/minisearch
import MiniSearch from "minisearch";

// Set up indexing and retrieval
const minisearch = new MiniSearch({
  fields: [
    "name",
    "content.brand",
    "content.title",
    "content.features",
    "content.materials",
    "content.seo_description",
    "content.short_description",
  ],
  storeFields: ["id", "full_slug", "content"],
  extractField: (doc, fieldName) => {
    const value = fieldName
      .split(".")
      .reduce(
        (obj, key) =>
          Array.isArray(obj) ? obj.map((o) => o?.[key]) : obj?.[key],
        doc,
      );
    return Array.isArray(value) ? value.flat(Infinity).join(" ") : value;
  },
});

The minisearch instance exposes methods that accept objects. MiniSearch then indexes the objects according to the instance configuration, and querying minisearch returns the objects ranked by relevance. The above configuration defines the following:

• fields lists the fields to consider for indexing and ranking
• storeFields lists the object properties to return when queried
• extractField supplies the logic to handle dot-separated paths in fields, which isn’t supported out of the box

Then use the instance to rank the keyword results and apply a final map() to extract the data relevant to the application:

minisearch.addAll(keywordResults);
const keywordResultsRanked = minisearch
  .search(q, { prefix: true, fuzzy: 0.2 }))
  .map(({ id, full_slug, content }) => ({
    id,
    slug: full_slug,
    name: content.title,
  }));

Passing { prefix: true, fuzzy: 0.2 } to MiniSearch’s search() method ensures that it finds partial matches as well as full matches. The above code also maps full_slug to slug to align with the shape of the Strata results that will soon become relevant.

MiniSearch can do much more (such as stemming and criteria-dependent rank boosting), but this tutorial sticks with basic keyword search and instead uses Strata’s semantic results to improve the results even further.

Retrieving semantic search results from Strata

For this tutorial, the relevant Strata API is the new Content Delivery API endpoint cdn/strata/search. It accepts the q parameter for its search term and can also constrain the results by story content type. Because the API client talks to it like any other endpoint, it integrates with the existing search page:

---
// ... imports

const q = Astro.url.searchParams.get('q')?.trim() ?? '';

const storyblokApi = useStoryblokApi();
const [keywordResults, strataResponse] = q
  ? await Promise.all([
      storyblokApi.getAll('cdn/stories', {
        version,
        content_type: 'product',
        search_term: q,
      }),
      storyblokApi.get('cdn/strata/search', {
        version,
        content_type: 'product',
        q,
      }),
    ])
  : [[], {}];

// minisearch...
---

<!-- template... -->

The Strata API response has a slightly different shape than the one from cdn/stories, so the data needs some massaging before you continue. Apart from the object types themselves, Strata response sets can also contain duplicates if a query matches multiple fields in a story.

To handle this, first extract id, slug, and name from the results, then deduplicate them:

// Extract { id, slug, name } from Strata results
const strataResults: any[] =
  strataResponse?.data?.results?.map(({ story }: any) => story) ?? [];

// Deduplicate items by storing them in a temporary map, indexed by id
const strataResultsDeduped = Array.from(
  new Map(strataResults.map((item) => [item.id, item])).values(),
);

This leaves two lists of search results: the re-ranked keywordResultsRanked and Strata’s deduplicated strataResultsDeduped, both containing objects of type { id, slug, name }. The last step is to combine the two lists into one with Reciprocal Rank Fusion.

Implementing Reciprocal Rank Fusion

Reciprocal Rank Fusion merges lists of ranked search results into one combined list. Each element receives a score, computed from the element’s ranks in their respective lists. Sorting the combined list by score yields the final ranking. RRF boils down to the formula score(d) = Σ(r ∈ R) 1 / (k + r(d)), in which

• d is a document in a search result
• R is a set of rankers (search result lists)
• k is a constant, usually set to 60
• r(d) is the rank of document d in ranker r

Each document’s score increases the higher it ranks in each result list. Translated into TypeScript code, the scary-looking formula turns into a rather mundane function:

type RRFResult<T extends { id: unknown }> = {
  id: T["id"];
  score: number;
  item: T;
};

function rrf<T extends { id: unknown }>(
  lists: Iterable<T[]>,
  k = 60,
): RRFResult<T>[] {
  const scores = new Map<T["id"], RRFResult<T>>();
  for (const list of lists) {
    for (let index = 0; index < list.length; index++) {
      const item = list[index];
      let entry = scores.get(item.id);
      if (!entry) {
        entry = { id: item.id, score: 0, item };
        scores.set(item.id, entry);
      }
      entry.score += 1 / (k + index + 1);
    }
  }
  return Array.from(scores.values()).sort((a, b) => b.score - a.score);
}

Every item in every list gets an entry in scores, indexed by its id property. The entries of type RRFResult<T> contain the id property alongside the item itself and a score. For each position index in a result list, the item’s score increases by 1 / (k + index + 1). After all scores have been tallied up, the entries are converted from a Map into an array and sorted in descending order by score.

To apply RRF to the results, pass the Strata and keyword results to rrf() as an iterable object. The second parameter k is optional and defaults to 60, which is its most commonly used value:

const hybridResults = rrf([strataResultsDeduped, keywordResultsRanked]).map(
  ({ item }) => item,
);

The results returned from rrf() are of type RRFResult<T> and contain information that the Astro templates don’t need, such as their score. The final .map(({ item }) => item) unwraps the RRFResult<T> objects; with that, you can build the search result UI from hybridResults and otherwise leave the template unchanged.

Browse the complete example on GitHub!

Search results in comparison

Searching the fictional outdoor activity company’s products for “boot” yields the following results using ranked keyword search, pure deduplicated semantic search, and RRF-powered hybrid search:

The downsides of pure keyword search are immediately apparent: only items that mention literal boots somewhere in their title or description appear in the result set. Semantic search casts a much wider net and also finds stories for related items, such as running shoes. This is precisely what semantic search is for, but on its own doesn’t create a compelling product search. Here, in a search for “boot,” a cook set outranks a product called “Frostline Insulated Winter Boot.” This may be explainable, but doesn’t make for an ideal user experience. Combining semantic and keyword search with RRF fixes this. The wide net cast by Strata’s semantic search remains intact, but the items that also appear in the keyword search results receive a boost according to their ranks in the source results.

Next steps

This tutorial showed how to build a hybrid search feature that combines keyword search with Strata’s semantic results. While this is a useful feature in and of itself, it only scratches the surface of what Strata has in store for you. What right now is only an API endpoint and an aspect of the MCP server will soon become a universal semantic content layer, touching every aspect of Storyblok—from advanced AI applications to mundane search features.

We’re excited to see what you build with Strata. If you have questions, have built something interesting, or want to hang out with the Storyblok community, join our Discord server.