Part 3- Splitting large content items into multiple Algolia records
In Parts 1 and 2 of this example we showed you how to update an Algolia search index when a content item is created or updated in Dynamic Content. We also explained how to refine the custom payload to keep the size of each record below the recommended maximum size of 5KB.
Content items created using the blog content type used in this example can often exceed 5KB. You could limit the size of the record created in Algolia by indexing an excerpt of the blog, but then only the excerpt of the blog would be searchable. In the cases where the resulting Algolia records are expected to be above 5KB, the best practice is to split the output into multiple records. To do this you need to use the Algolia batch end point.
In Part 3 we walk you through the steps required to update the configuration and custom payload of the webhook you built in parts 1 and 2 to create multiple records from a blog content item and link them together in Algolia.
PrerequisitesLink copied!
This example uses the content type schemas included in Part 1 and Part 2, but the concepts will be the same whatever schemas you are using in your own projects.
Step 1: Configuring the webhook URLLink copied!
To generate multiple records from the blog content item you will send a POST request to the batch endpoint of the Algolia API.
- Update the webhook URL to use the batch endpoint. Your webhook URL should now look something like:
- Change the webhook method to POST
Your webhook settings should now look those in the image below.
Step 2: Defining the custom payloadLink copied!
The next step is to update the custom payload. The handlebars shown below will index each piece of linked content from the blog content item as a separate record in Algolia. So if the blog contains two text content items, two records will be created.
Some key points to note about the custom payload:
"action": "updateObject"
is a request to the Algolia batch API to add or replace an object with the specified objectid.- The Algolia objectid for each record must be unique and is set using the deliveryid of the item for which the record is created.
- The "id" field will be used to deduplicate the records. This is set to the deliveryid of the root content item- in this case the blog content item.
The custom payload will now look like the image shown below.
Step 3: Trigger the webhook and view the activity logLink copied!
To trigger the webhook you can create or update a blog content item or use the webhook testing feature.
To use the testing feature you need to have triggered the webhook at least once and choose a previous webhook delivery to test with the new settings. Open the testing pane, choose a recent webhook delivery (1 in the image below) and click the "Test" button (2). The webhook will be invoked in testing mode and the webhook request will be displayed, but no request will be sent to the webhook URL. The latest version of your settings will be used for testing, you don't need to save the changes before you test.
A successful webhook test is shown below. In this example, the webhook body includes the request that will be sent to the Algolia API to update records for each of the content items linked to the blog item.
The webhook body generated from the custom payload is as follows. The payload is generated from two content items, each linked to one blog content item. Note the unique objectids and the same id field, allowing for deduplication in Algolia.
Step 4: Deduplicating your Algolia indexLink copied!
You Algolia index will contain two new records (depending on the structure of the content item you updated). Records generated from the same blog item will have the same value for the id field.
In your Algolia index, you need to deduplicate your search index setting "attributeForDistinct" to the "id" field and "distinct" to true. More information can be found in the Algolia guide to indexing large documents.
Related pagesLink copied!
Example: Search integration (Algolia), Part 1
Example: Search integration (Algolia), Part 2