Part 1- Indexing content in Algolia

On this page we'll walk you through an example of setting up customizable webhooks to connect Dynamic Content with the Algolia API. When the user creates or updates a content item in Dynamic Content, the specified search index will be updated in Algolia.

Note that if you've created a search index using the search wizard, then you don't need to follow this tutorial, but it does provide details of what's already been set up for you.

This example demonstrates several webhook customization features. It's structured for you to walk through step by step and to help you get started with your own Dynamic Content Algolia integration.

Note that this example shows how to index staged content, the content you'll use in your preview environment, because it uses the "content item created" and "content item updated" webhook events as triggers. To index published content for your production environment, see the extra steps required to support the "Snapshot Published" webhook event.

Features demonstrated

• Sending a request to the Algolia API to update a search index when a content item is created or updated in Dynamic Content.
• Using "PUT" as the webhook method
• Using a templated webhook URL to include id of the item to update
• Adding filters so you can index only content created from the specified content types
• Using custom and secret headers to authenticate access to the Algolia API
• Defining a custom payload to include the content item in the body of the request sent to the Algolia API. This will be the content of the record updated or created in your specified Algolia search index.

Pre-requisites

• An Algolia account
• A Dynamic Content account with the developer role assigned for the hub in which you want to create webhooks

Dynamic Content permissions

If your permissions are persona-based, your Dynamic Content account requires the developer persona for the relevant hub. To migrate to our more flexible role-based permissions, please contact your Amplience Customer Success Manager or Amplience support.

Before you begin

In order to set up your Dynamic Content Algolia webhook integration, you need the following information from your Algolia account:

• Application ID
• Index name
• Admin API key or Write API key (to create and update your index)

Step 1: Define the webhook URL and triggers

• Choose "Webhooks" from the Development menu and click the "Add webhook" button. Give the webhook a label.

• In the webhook triggers section, choose "Created" and "Updated". In this example a new record will be created in Algolia when a content item is created in Dynamic Content and the corresponding record will be updated when the content item is updated.

• Choose "PUT" as the webhook method. A PUT request is sent to the Algolia API to create or update the specified record.

• Add the webhook URL. Include your Algolia application ID and your chosen index.

When sending a PUT request to Algolia, you need to specify the id of the record to create or update. In the Algolia API this is referred to as the object ID. In this example you'll include the content item id in the webhook URL and this will be used as the Algolia object ID.

The standard payload sent by Dynamic Content when a "content item created" or content item updated" webhook event is triggered includes the content item id in {{payload.id}}.

Choose "content item id" from the "Add snippet" menu to the right of the webhook URL.

{{payload.id}} will be added to the end of your webhook URL so that when the webhook is triggered, it will be replaced by the id of the content item that has been created or updated.

Your webhook URL should look something like:

https://<yourAlgoliaAppId>.algolia.net/1/indexes/<yourIndexName>/{{payload.id}}

Replace <yourAlgoliaAppId> and <yourIndexName> with your Algolia application ID and the index you want to write to.

Note: if you are supporting snapshot published you would choose "snapshot content id" from the snippets menu.

Example PUT request

The following shows an example of a PUT request sent when the webhook is triggered. a2b1d192-04d5-4219-81cf-7cf2e6e1b3cf is the content id and will be used as the object ID by the Algolia API.

https://5051PKU5AX.algolia.net/1/indexes/amp-doc-example/a2b1d192-04d5-4219-81cf-7cf2e6e1b3cf

Example configuration

An example webhook configuration is shown in the image below.

Step 2: Filter on the blog content type

The next step is to add a filter so that you can choose which content types are indexed. This example uses the blog content type, but you can add a filter for your own content types if you choose.

To add a filter click the "Add Filter" button and enter the JSON path for the part of the standard payload you want to use as a filter. To filter on the content type use $.payload.body._meta.schema (1 in the image below). Choose the "Equals" operator and enter the content type schema URL (2) of the schema you want to use as a filter.

In the example below, the blog content type schema was created with a URL of http://example.com/blogpost.json. The webhook will only be triggered if content is created from a content type registered with this URL.

You can also add multiple filters. See the webhook customization page for more details.

Step 3: Add custom headers

To access the Algolia API you need to include the application ID and API key in the headers of each request.

• Click "Add custom header" and enter "X-Algolia-Application-Id" as the key (1 in the image below). Add your Algolia application ID as the value (2).

Add a secret header for the Algolia API Key

Add a secret header to ensure that your Algolia API key is encrypted. Click "Add secret header" and enter X-Algolia-API-Key as the key (1) and your API key (2) as the value.

Step 4: Define the custom payload

The final step to configure your webhook is to define a custom payload. This is the handlebars that is used to generate the body of the webhook request sent to the Algolia API. The custom payload determines what will be indexed in Algolia.

The custom payload in this step allows you to index the entire content item using the withDeliveryContentItem handlebars helper. This should generally be used as a starting point to test your webhook and determine the structure of your custom payload.

Algolia recommends that record sizes should not exceed 5KB, so we recommend that you refine your payload so that you only include the fields that your search needs to function. This is covered in detail in Part 2.

In the payload section choose "Custom" and choose "Snapshot with full delivery data" from the "Add snippet menu"

The following snippet will be added to the custom payload text box.

{{#withDeliveryContentItem contentItemId=payload.id account="myAccountId" stagingEnvironment="vseURL"}}
{{{JSONstringify .}}}
{{/withDeliveryContentItem}}

Replace "myAccountId" with your Amplience account name and "vseURL" with the virtual staging domain from your hub.

See finding your account name for how to find your Amplience account name. You can find the VSE URL by choosing "Preview" from the Settings menu in the Dynamic Content app and copying the "Default preview environment" setting.

You've now finished your webhook configuration. Click the "Save" button at the top of the window to create the webhook.

Step 5: Create a content item

To test out your webhook, create or update a content item from one of the content types you want to index- and that is included in the filters you set up in a previous step.

The content item in the image below uses the blog content type. When the item is saved the webhook will be triggered and the PUT request sent to the Algolia API.

Step 6: View the webhook details

Now it's time to check that the webhook was successfully processed. Find your Algolia webhook in the webhook list and choose "View" from its menu, or double-click the webhook label.

Choose the "Activity log" tab and find the most recent delivery. It should show a status of "200 OK", indicating that the webhook was successfully processed. Click "View Details" to examine the request and response.

If an error is shown then the webhook details will provide diagnostic information to help you track down the problem.

The webhook details window will include:

• The status. In the example below a status of "200 OK" indicates that the webhook was successfully processed and no error was returned by the Algolia API.
• The filters. This shows the filter that was matched in order for the webhook to be triggered. In this case a blog content item was updated.
• the custom payload used to generate the body of the request from this invocation of the webhook (1 in the image below). This is useful during development when you might be refining the handlebars you use.
• The webhook URL (2). This is the request that was sent to the Algolia API. Notice that the handlebars templated value was replaced by the id of the content item that was created or updated.
• The webhook request body (3). This contains the entire content item. You can use the body to help you refine the payload.
• The webhook response body (4). This confirms that a record has been added to the Algolia index and returns its objectID (the unique ID for the record in Algolia).

Content type schemas used in this example

The content type schemas used to demonstrate this webhook integration are included below. You can register these using the schema editor. These schemas are also used in the JS blog example.

The blog schemas are part of the content type schema templates included with the schema editor. See schema examples for more details.

Below is the blog content type schema used in this example:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.com/blogpost.json",
  "title": "Blog post",
  "description": "A blog post",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "title": {
      "title": "Title",
      "description": "Used for heading and SEO title tag",
      "type": "string",
      "minLength": 1,
      "maxLength": 150
    },
    "date": {
      "title": "Creation date",
      "description": "Creation date (YYYY-MM-DD)",
      "type": "string",
      "maxLength": 10,
      "minLength": 10
    },
    "description": {
      "title": "Description",
      "description": "Used for blog listing page and SEO description",
      "type": "string",
      "minLength": 1,
      "maxLength": 200
    },
    "image": {
      "title": "Image",
      "description": "Used for the blog post's thumbnail and banner",
      "allOf": [
        {
          "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content-link"
        },
        {
          "properties": {
            "contentType": {
              "enum": ["http://example.com/blog-image.json"]
            }
          }
        }
      ]
    },
    "urlSlug": {
      "title": "Url slug",
      "description": "Url friendly slug",
      "type": "string",
      "minLength": 1,
      "maxLength": 100
    },
    "tags": {
      "title": "Tags",
      "description": "Blog tags",
      "type": "array",
      "minItems": 0,
      "maxItems": 10,
      "items": {
        "type": "string",
        "minLength": 1,
        "maxLength": 100,
        "title": "Tag",
        "description": ""
      }
    },
    "readTime": {
      "title": "Read time",
      "description": "The time it takes to read the blog",
      "type": "integer"
    },
    "authors": {
      "title": "Blog author",
      "description": "Article author(s) - max 3",
      "type": "array",
      "minItems": 1,
      "maxItems": 3,
      "items": {
        "allOf": [
          {
            "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content-link"
          },
          {
            "properties": {
              "contentType": {
                "enum": ["http://example.com/blog-author.json"]
              }
            }
          }
        ]
      }
    },
    "content": {
      "title": "Content",
      "description": "",
      "type": "array",
      "minItems": 1,
      "maxItems": 20,
      "items": {
        "allOf": [
          {
            "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content-link"
          },
          {
            "properties": {
              "contentType": {
                "title": "Content",
                "enum": [
                  "http://example.com/blog-image.json",
                  "http://example.com/blog-video.json",
                  "http://example.com/blog-text.json"
                ]
              }
            }
          }
        ]
      }
    }
  },
  "propertyOrder": [
    "title",
    "authors",
    "date",
    "description",
    "image",
    "urlSlug",
    "tags",
    "readTime",
    "content"
  ],
  "required": [
    "title",
    "authors",
    "date",
    "description",
    "image",
    "urlSlug",
    "readTime",
    "content"
  ]
}

Below is the image content type schema used in this example:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.com/blog-image.json",
  "title": "Image",
  "description": "Image schema",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "image": {
      "title": "Image",
      "description": "insert an image",
      "type": "object",
      "anyOf": [
        {
          "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/image-link"
        }
      ]
    },
    "altText": {
      "type": "string",
      "minLength": 1,
      "maxLength": 150,
      "title": "Alt text",
      "description": "insert image alt text"
    }
  },
  "propertyOrder": ["image", "altText"],
  "required": ["image", "altText"]
}

Below is the video content type schema used in this example:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.com/blog-video.json",
  "title": "Video",
  "description": "Video schema",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "video": {
      "title": "Video",
      "type": "object",
      "anyOf": [
        {
          "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/video-link"
        }
      ]
    }
  },
  "propertyOrder": ["video"],
  "required": ["video"]
}

Below is the text content type schema used in this example:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.com/blog-text.json",
  "title": "Text",
  "description": "Text schema",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "text": {
      "type": "string",
      "format": "markdown",
      "title": "Text",
      "description": "",
      "minLength": 1,
      "maxLength": 30000
    }
  },
  "propertyOrder": ["text"],
  "required": ["text"]
}

Below is the author content type schema used in this example:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "http://example.com/blog-author.json",
  "title": "Author",
  "description": "Author schema",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "name": {
      "title": "Author name",
      "description": "Name of the author",
      "type": "string",
      "minLength": 1,
      "maxLength": 100
    },
    "avatar": {
      "title": "Avatar",
      "description": "The author's avatar",
      "type": "object",
      "allOf": [
        {
          "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content-link"
        },
        {
          "properties": {
            "contentType": {
              "title": "image",
              "enum": ["http://example.com/blog-image.json"]
            }
          }
        }
      ]
    }
  },
  "propertyOrder": ["name", "avatar"],
  "required": ["name"]
}

Finding your account name

Your account name is the endpoint for your published content. You can find out your account name by opening the content item properties pane for a published content item and viewing or copying the Content Delivery URL. The value used for "store" is your account name.

Below is an example of finding the account name, where the account name is "ampproduct":

Extra steps: Snapshot published

If you want to invoke your webhook when a content item published, in order to index content in your production environment, you can follow all the steps in this example, but you'll need to make a few changes.

• In the webhook triggers section choose "Snapshot Published". Ensure Content item- Created and Updated are not selected
• The standard payload for "Snapshot Published" is different to "Content item created" and "Content item updated". You need to include payload.rootContentItem.id in the webhook URL:

https://<yourAlgoliaAppId>.algolia.net/1/indexes/<yourIndexName>/{{payload.rootContentItem.id}}

• In your custom payload change the call to the withDeliveryContentItem helper as follows:

{{#withDeliveryContentItem contentItemId=payload.rootContentItem.id snapshotId=payload.id account="myAccountId" stagingEnvironment="vseURL"}}

Replace "myAccountId" with your Amplience account and "vseURL" with your staging domain as explained above.

• Trigger the webhook by publishing a content item

Related pages

Webhooks overview

Customizable webhooks

Webhook payloads

Example: Search integration (Algolia), Part 2

Algolia documentation

What to put in your record

Reducing object size