How to generate a Sitemap with a headless CMS?

Some of you already asked to receive a tutorial on this topic. So we will demonstrate how to use the Storyblok APIs to create a sitemap according to content-entries. We will use JavaScript to load the content and display it on a website to get you the idea.

Goal

We want to access to Links API to generate a sitemap for our project according to our content entries in Storyblok. To do so I’ll use the content of the Storyblok space: NuxtDoc. NuxtDoc is a NuxtJS setup that helps you get started with your own documentation pages. It includes a basic NuxtJS setup, Storyblok content structure and components and also a deploy step for Netlify

Let’s start

We will use the Links API of Storyblok. The It allows you to easily access all your content-entries without actually loading their content. Instead, you will have access to all properties you would need to create all kind of hierarchy structures like:

• Breadcrumb
• RSS Feed
• Navigation Menus
• Pregenerate Pages
• …

and also a simple flat sitemap as we do in this tutorial. You can find a documentation for the Links API and all its parameters linked right here.

To get you started we will do a simple GET request to load a list of links. The structure of this is not an Array as the Stories collections since with the Links API you should be able to access all information of another content entries UUID instantly.

// view JSfiddle: https://jsfiddle.net/dominikangerer/ud2ydfah/4
let preview_token = 'bZhHNuvF3MwCX11cXstx4wtt'

axios.get(`https://api.storyblok.com/v1/cdn/links?token=${preview_token}`)
.then((result) => {
  document.querySelector('#linksresponse').innerHTML = JSON.stringify(result.data, null, 2)
})

Load all links

The result you can see above is an object with 25 properties. Each of those properties is one UUID (a reference to a content-entry or folder). To be able to load more than 25 we can either add the per_page parameter or (our recommendation) load it in a paged manner. Doing it like that you will never have to update the per_page parameter if you will have more than 1000 content-entries. You can check out the example code for this using Axios. We will use the total header of the response to generate all requests we need to perform and execute them to load our links. You can do the same with every other technology in a similar way.

// view JSfiddle: https://jsfiddle.net/dominikangerer/ud2ydfah/8
let preview_token = 'bZhHNuvF3MwCX11cXstx4wtt'
let per_page = 100
let page = 1
let all_links = []

// Call first Page of the Links API: https://www.storyblok.com/docs/Delivery-Api/Links
axios.get(`https://api.storyblok.com/v1/cdn/links?token=${preview_token}&per_page=${per_page}&page=${page}`).then((res) => {
  // push all links into our all_links variable
  Object.keys(res.data.links).forEach((key) => {
    all_links.push(res.data.links[key])
  })

  // Check if there are more pages available otherwise thats all to do.
  const total = res.headers.total
  const maxPage = Math.ceil(total / per_page)
  if (maxPage <= 1) {
    return
  }

  // Since we know the total we now can pregenerate all requests we need to get all Links
  let contentRequests = []

  // we will start with page two since the first one is already done.
  for (let page = 2; page <= maxPage; page++) {
    contentRequests.push(axios.get(`https://api.storyblok.com/v1/cdn/links?token=${preview_token}&per_page=${per_page}&page=${page}`))
  }

  // Axios allows us to exectue all requests using axios.spread. We will then push our links into our all_links variable.
  axios.all(contentRequests).then(axios.spread((...responses) => {
    responses.forEach((response) => {
      Object.keys(response.data.links).forEach((key) => {
        all_links.push(response.data.links[key])
      })
    })

	// now we have all our links
	document.querySelector('#linksresponse').innerHTML = JSON.stringify(all_links, null, 2)
  }))
})

Build the Sitemap

The final thing to do is to use those links to generate a sitemap. Since a basic sitemap will have elements that look like below. If you want to learn more about what you can do with a Sitemap make sure to read Google’s article: Learn about sitemaps and How to build a sitemap

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" 
  xmlns:image="http://www.sitemaps.org/schemas/sitemap-image/1.1" 
  xmlns:video="http://www.sitemaps.org/schemas/sitemap-video/1.1">
  <url><loc>https://www.example.org/</loc></url>
  <url><loc>https://www.example.org/home</loc></url>
  <url><loc>https://www.example.org/posts/post-1</loc></url>
  <url><loc>https://www.example.org/posts/post-2</loc></url>
</urlset>

In the next code example, I’ll output it with escaped HTML entities so you can see the result directly. However, you should make sure to output it as a proper XML or any other format you may choose after reading Google’s: How to build a sitemap.

// view JSfiddle: https://jsfiddle.net/dominikangerer/ud2ydfah/18
let preview_token = 'bZhHNuvF3MwCX11cXstx4wtt'
let per_page = 100
let page = 1
let all_links = []

// Call first Page of the Links API: https://www.storyblok.com/docs/Delivery-Api/Links
axios.get(`https://api.storyblok.com/v1/cdn/links?token=${preview_token}&per_page=${per_page}&page=${page}`).then((res) => {
  // push all links into our all_links variable
  Object.keys(res.data.links).forEach((key) => {
    all_links.push(res.data.links[key])
  })

  // Check if there are more pages available otherwise thats all to do.
  const total = res.headers.total
  const maxPage = Math.ceil(total / per_page)
  if (maxPage <= 1) {
    return
  }

  // Since we know the total we now can pregenerate all requests we need to get all Links
  let contentRequests = []

  // we will start with page two since the first one is already done.
  for (let page = 2; page <= maxPage; page++) {
    contentRequests.push(axios.get(`https://api.storyblok.com/v1/cdn/links?token=${preview_token}&per_page=${per_page}&page=${page}`))
  }

  // Axios allows us to exectue all requests using axios.spread. We will then push our links into our all_links variable.
  axios.all(contentRequests).then(axios.spread((...responses) => {
    responses.forEach((response) => {
      Object.keys(response.data.links).forEach((key) => {
        all_links.push(response.data.links[key])
      })
    })

    let sitemap_entries = all_links.map((link) => {
      // you got access to every property of those links here. Note the \n I've added to format it in the output - you don't need that in the real XML.
      return `\n<url><loc>https://www.example.org/${link.slug}</loc></url>`
    })

    // the actual sitemap with all it's entries.
    let sitemap = `<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" 
  xmlns:image="http://www.sitemaps.org/schemas/sitemap-image/1.1" 
  xmlns:video="http://www.sitemaps.org/schemas/sitemap-video/1.1">${sitemap_entries.join('')}
</urlset>`

    // let's output the Sitemap XML with encoded HTML characters
    document.querySelector('#linksresponse').innerHTML = sitemap.replace(/&/g, '&amp;')
      .replace(/</g, '&lt;')
      .replace(/>/g, '&gt;')
      .replace(/"/g, '&quot;')
      .replace(/'/g, '&apos;');
  }))
})

Summary

You’ve now learned how to access the links API and use it’s content to generate a sitemap. We use the same approach to generate our Sitemap but with another technology. The same setup is also used in NuxtDoc to generate the routes we need to build out a static version of it. Since you also have access to the parent_id you can also build out breadcrumbs and whole menus. The best thing about it is that if you’ve routes from an e-commerce solution you can add those routes to the all_links variable and add them to your sitemap directly.