How to index entries from Storyblok with Algolia

Contents

In this article we will check out how we can index content of Storyblok in Algolia with just a few lines of code using node.js.

Set-up a node.js project

Navigate into a folder you’re okay with creating a new Node project in.

mkdir storyblok-to-algolia && cd storyblok-to-algolia

Start a new node project using the command below:

node init

Set-up & install dependencies

# install Storyblok Universal JS client SDK to load data from Storyblok
npm install storyblok-js-client

# Install Algolia Search client to create the index
npm install algoliasearch

# Install axios so we can use axios.all to load all entries from Storyblok
npm install axios

Creating the Algolia Indexer Node Script

Create a index.js with the following content. The comments will guide you through the whole implementation. You will need your space preview or public token to exchange the STORYBLOK_CONTENT_DELIVERY_API_TOKEN placeholder and your ALGOLIA_APP_ID as well as ALGOLIA_API_ADMIN_TOKEN.

const axios = require('axios')
const algoliasearch = require('algoliasearch')
const StoryblokClient = require('storyblok-js-client')

// Important: make sure to use a proper way to secure your Algolia token information.
const ALGOLIA_APP_ID = '1AseRHsdrh23'
const ALGOLIA_API_ADMIN_TOKEN = '092adwawdj2l00af8wß009ß7awd7' 
const ALGOLIA_INDEX_NAME = 'storyblok-articles'
const STORYBLOK_CONTENT_DELIVERY_API_TOKEN = 'oKYQLDb7awddtWhcsmxAtt'

const algolia = algoliasearch(ALGOLIA_APP_ID, ALGOLIA_API_ADMIN_TOKEN)
const storyblok = new StoryblokClient({ accessToken: STORYBLOK_CONTENT_DELIVERY_API_TOKEN, })

// define options for the API requests we will send to Storyblok.
const options = {
    starts_with: 'articles/', // exchange with the folder you want to index, can be blank if you want to index everything.
    per_page: 100, // increases the default per page limit of 25 to 100 entries per page.
    page: 1,
    version: 'draft' // change to published after developing
}

// initial get requests that is used to retrieve the total amount of pages
storyblok.get(`cdn/stories/`, options).then(async res => {
    // get total based on options
    const total = res.headers.total
    const maxPage = Math.ceil(total / options.per_page)

    // prepare necessary requests to retrieve all articles
    let contentRequests = []
    for (let page = 1; page <= maxPage; page++) {
        contentRequests.push(storyblok.get(`cdn/stories/`, { ...options, page }))
    }

    // init storyblok articles index in algolia
    const index = algolia.initIndex(ALGOLIA_INDEX_NAME)

    axios.all(contentRequests).then(axios.spread(async (...responses) => {
        // combine all article stories into one array
        let records = []
        responses.forEach((response) => {
            let data = response.data
            records = records.concat(data.stories)
        })

        records.forEach(record => {
            record.objectID = record.uuid // use Storyblok uuid as objectID in Algolia so we can auto update content.
        })
        
        // persist in Algolia
        // Recommended Batch sizes 1.000 - 10.000 per Batch according to documentation.
        await index.saveObjects(records, { autoGenerateObjectIDIfNotExist: false }).wait().catch(e => { console.log(e) })
        console.log('Index stored with ' + records.length + ' Entries.')
    })).catch(e => { console.log(e) })
}).catch(e => { console.log(e) })

Running the node script

After adding the above code and adjusting the configuration with your own needs you can execute the following command to create an index in Algolia.

node index.js

Filtering Content in Storyblok

Instead of using Algolia to filter and query your content of Storyblok we can highly recommend you to check-out our Content Delivery feature called filter_query as this allows you to perform query actions on your content without any additional indexing step when entered in Storyblok.

// use the universal js client to perform the request to get all products with a price greater than 100
Storyblok.get('cdn/stories/', {
  "starts_with": "products/",
  "filter_query": {
    "price": {
      "gt_int": 100
    }
  }
})
.then(response => {
  console.log(response)
}).catch(error => { 
  console.log(error)
})

Summary

With just a few lines of code you can not only access your content from Storyblok but you can easily index it using Algolia including auto update of existing indexes and entries.