How to index entries from Storyblok with Algolia

Contents
    Try Storyblok

    Storyblok is the first headless CMS that works for developers & marketers alike.

    In this article, we will check out how we can index the content of Storyblok in Algolia with just a few lines of code using node.js.

    Learn:

    You could be interested also in our article on how to integrate Algolia with your Headless CMS.

    Set up a node.js project

    Navigate into a folder you’re okay with creating a new Node project in.

    Console
    mkdir storyblok-to-algolia && cd storyblok-to-algolia

    Start a new node project using the command below:

    Console
    npm init

    Installation

    Please install storyblok-js-client, alogoliasearch and axios according to the following code block.

    Console
    # install Storyblok Universal JS client SDK to load data from Storyblok
    npm install storyblok-js-client
     
    # Install Algolia Search client to create the index
    npm install algoliasearch
     
    # Install axios so we can use axios.all to load all entries from Storyblok
    npm install axios

    Configuration of Algolia Indexer

    Create index.js with the following content. The comments will guide you through the whole implementation. You will need your space preview or public token to exchange the STORYBLOK_CONTENT_DELIVERY_API_TOKEN placeholder and your ALGOLIA_APP_ID as well as ALGOLIA_API_ADMIN_TOKEN.

    index.js
    const axios = require('axios')
    const algoliasearch = require('algoliasearch')
    const StoryblokClient = require('storyblok-js-client')
     
    // Important: make sure to use a proper way to secure your Algolia token information.
    const ALGOLIA_APP_ID = '1AseRHsdrh23'
    const ALGOLIA_API_ADMIN_TOKEN = '092adwawdj2l00af8wß009ß7awd7' 
    const ALGOLIA_INDEX_NAME = 'storyblok-articles'
    const STORYBLOK_CONTENT_DELIVERY_API_TOKEN = 'oKYQLDb7awddtWhcsmxAtt'
     
    const algolia = algoliasearch(ALGOLIA_APP_ID, ALGOLIA_API_ADMIN_TOKEN)
    const storyblok = new StoryblokClient({ accessToken: STORYBLOK_CONTENT_DELIVERY_API_TOKEN, })
     
    // define options for the API requests we will send to Storyblok.
    const options = {
        starts_with: 'articles/', // exchange with the folder you want to index, can be blank if you want to index everything.
        per_page: 100, // increases the default per page limit of 25 to 100 entries per page.
        page: 1,
        version: 'draft' // change to published after developing
    }
     
    // initial get requests that is used to retrieve the total amount of pages
    storyblok.get(`cdn/stories/`, options).then(async res => {
        // get total based on options
        const total = res.headers.total
        const maxPage = Math.ceil(total / options.per_page)
     
        // prepare necessary requests to retrieve all articles
        let contentRequests = []
        for (let page = 1; page <= maxPage; page++) {
            contentRequests.push(storyblok.get(`cdn/stories/`, { ...options, page }))
        }
     
        // init storyblok articles index in algolia
        const index = algolia.initIndex(ALGOLIA_INDEX_NAME)
     
        axios.all(contentRequests).then(axios.spread(async (...responses) => {
            // combine all article stories into one array
            let records = []
            responses.forEach((response) => {
                let data = response.data
                records = records.concat(data.stories)
            })
     
            records.forEach(record => {
                record.objectID = record.uuid // use Storyblok uuid as objectID in Algolia so we can auto update content.
            })
            
            // persist in Algolia
            // Recommended Batch sizes 1.000 - 10.000 per Batch according to documentation.
            await index.saveObjects(records, { autoGenerateObjectIDIfNotExist: false }).wait().catch(e => { console.log(e) })
            console.log('Index stored with ' + records.length + ' Entries.')
        })).catch(e => { console.log(e) })
    }).catch(e => { console.log(e) })

    Algolia Indexer

    Running the Indexer

    After adding the above code and adjusting the configuration with your own needs you can execute the following command to create an index in Algolia.

    Bash
    node index.js

    Filtering Content in Storyblok

    Instead of using Algolia to filter and query your content of Storyblok, we can highly recommend you to check out our Content Delivery feature called filter_query as this allows you to perform query actions on your content without any additional indexing step when entered in Storyblok.

    storyblok-client.js
    // use the universal js client to perform the request to get all products with a price greater than 100
    Storyblok.get('cdn/stories/', {
      "starts_with": "products/",
      "filter_query": {
        "price": {
          "gt_int": 100
        }
      }
    })
    .then(response => {
      console.log(response)
    }).catch(error => { 
      console.log(error)
    })

    Requesting filtered content from Storyblok.

    Summary

    With just a few lines of code, you can not only access your content from Storyblok but you can easily index it using Algolia including auto-update of existing indexes and entries.

    Resource Link
    How to integrate Algolia with your Headless CMS https://www.storyblok.com/tp/headless-algolia-integration
    Algoalia Indexing API https://www.algolia.com/doc/api-client/methods/indexing/

    Developer Newsletter

    Want to stay on top of the latest news and updates in Storyblok?
    Subscribe to Code & Bloks - our headless newsletter.

    An error occurred. Please get in touch with support@storyblok.com

    Please select at least one option.

    Please enter a valid email address.

    This email address is already registered.

    Please Check Your Email!

    Almost there! To confirm your subscription, please click on the link in the email we’ve just sent you. If you didn’t receive the email check your ’junk folder’ or