Automatic shoppable images

This content field extension saves you time by using the power of AI to automatically detect objects to set focal points, and point and polygon hotspots. Setting a focal point, also referred to as a point of interest (POI), ensures that the most important part of your image is always in focus when resized across devices. Hotspots enable interactive experiences, allowing users to interact with an image. For example, when conveying product details, prices, or availability.

This extension is particularly useful for generating polygon hotspots, which can be used to mark the area of complex shapes.

Labs preview

The AI assistant feature of the Automatic shoppable images extension is classified as an Amplience labs preview for you to evaluate and provide feedback. It's provided as is and is unsupported. Each time you use it to identify objects you'll use one Amplience credit.

Updated URL for the hosted extension

The automatic object detection feature was previously part of a separate extension that was hosted with the URL: https://shoppable-image-ai.extensions.content.amplience.net/. All of the functionality is now included in the original shoppable image extension and https://shoppable-image-ai.extensions.content.amplience.net/ will shortly be deprecated. We recommend updating your schemas and registered extensions to use https://shoppable-image.extensions.content.amplience.net/.

Using the extension

When you add images to content that uses this extension, you can automatically set focal points and hotspots for them.

The "Auto object detection" dialog is displayed when you add an image. You can show and hide this dialog by clicking "AI Assistant".

To generate a list of objects to use for focal points and hotspots, click "Detect objects".

In this example, we've added a focal point to the clothes in the centre of the image, hotspots to the camera and pouch bag, and polygon hotspots to the handbag, sunglasses and shirt.

tip

You can also manually define your own focal points and hotspots, using the buttons in the action bar.

Schema examples

To make it easier to use the Automatic shoppable images extension in your own schemas we've included some schema examples to help get you started.

There are two schema examples included: one that uses a registered extension and one that includes the extension URL directly in the schema. We've also included a partial, as explained below.

In most cases you will want to register the extension so that you do not have to specify the installation parameters each time you use it in a schema.

The shoppable image partial

The structure of the data returned by the extension includes arrays for the point and polygon hotspots, and an object to store the focal point (POI). We've included a partial to make it easier to include in your own schemas. The partial is used in both of the schema examples.

To use the schema examples, you must also add the partial to your hub. Partials are not registered as content types.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://schema-examples.com/automatic-shoppable-image-partial",
  "definitions": {
    "shoppableImage": {
      "type": "object",
      "properties": {
        "image": {
          "title": "Image",
          "description": "The base image that the shoppable image elements are for.",
          "allOf": [
            {
              "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/image-link"
            }
          ]
        },
        "poi": {
          "title": "Focal Point",
          "description": "The focal point defined on the image. Coordinates are normalized 0-1 based on image size.",
          "type": "object",
          "properties": {
            "x": {
              "type": "number"
            },
            "y": {
              "type": "number"
            },
            "w": {
              "type": "number"
            },
            "h": {
              "type": "number"
            }
          }
        },
        "hotspots": {
          "title": "Hotspots",
          "description": "Interactable hotspots defined on the image. Coordinates are normalized 0-1 based on image size.",
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "points": {
                "title": "Point",
                "description": "The location of this hotspot on the image.",
                "type": "object",
                "properties": {
                  "x": {
                    "type": "number"
                  },
                  "y": {
                    "type": "number"
                  }
                }
              },
              "id": {
                "title": "ID",
                "description": "Unique identifier for this hotspot.",
                "type": "string"
              },
              "selector": {
                "title": "Selector",
                "description": "The selector associated with this hotspot.",
                "type": "string"
              },
              "target": {
                "title": "Target",
                "description": "The target associated with this hotspot.",
                "type": "string"
              }
            }
          }
        },
        "polygons": {
          "title": "Polygons",
          "description": "Interactable polygonal regions defined on the image. Coordinates are normalized 0-1 based on image size.",
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "points": {
                "title": "Polygon Points",
                "description": "The points that define this polygon. The polygon edge is drawn between all points in order, and closed at the end by drawing an edge from the last to the first point.",
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "x": {
                      "type": "number"
                    },
                    "y": {
                      "type": "number"
                    }
                  }
                }
              },
              "id": {
                "title": "ID",
                "description": "Unique identifier for this polygon.",
                "type": "string"
              },
              "selector": {
                "title": "Selector",
                "description": "The selector associated with this polygon.",
                "type": "string"
              },
              "target": {
                "title": "Target",
                "description": "The target associated with this polygon.",
                "type": "string"
              }
            }
          }
        }
      }
    }
  }
}

Schema example using a registered extension

In order to use this schema you will need to register the extension using the following details:

• Category: Content field extension
• Name: automatic-shoppable-image (if you use a different name you will need to change the name in the example schema)
• URL: https://shoppable-image.extensions.content.amplience.net/
• Initial height: 800

Permissions

No additional permissions are required when using the Amplience hosted extension.

Example schema

An example schema using a registered version of the extension is shown below. The shoppableImage property includes the partial and references the registered version of the extension.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://schema-examples.com/automatic-shoppable-images",
  "$comment": "Docs: https://amplience.com/developers/docs/ai-services/automatic-shoppable-images/#schema-examples",
  "title": "Automatic shoppable images extensions example",
  "description": "Demonstrates how to use the automatic shoppable images extension when it has been registered",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "shoppableImage": {
      "title": "Shoppable image",
      "description": "An image enriched with a focal point and hotspots.",
      "allOf": [
        {
          "$ref": "https://schema-examples.com/automatic-shoppable-image-partial#/definitions/shoppableImage"
        }
      ],
      "ui:extension": {
        "name": "automatic-shoppable-image"
      }
    }
  },
  "propertyOrder": []
}

Schema example using the extension URL

This schema example shows how to include the extension in the schema using its URL.

The shoppableImage property includes the partial and references the registered version of the extension.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://schema-examples.com/automatic-shoppable-images-url",
  "$comment": "Docs: https://amplience.com/developers/docs/ai-services/automatic-shoppable-images/#schema-examples",
  "title": "Automatic shoppable images extensions example",
  "description": "Demonstrates how to use the automatic shoppable images extension when it has been registered",
  "allOf": [
    {
      "$ref": "http://bigcontent.io/cms/schema/v1/core#/definitions/content"
    }
  ],
  "type": "object",
  "properties": {
    "shoppableImage": {
      "title": "Shoppable image",
      "description": "An image enriched with a focal point and hotspots.",
      "allOf": [
        {
          "$ref": "https://schema-examples.com/automatic-shoppable-image-partial#/definitions/shoppableImage"
        }
      ],
      "ui:extension": {
        "url": "https://shoppable-image.extensions.content.amplience.net/"
      }
    }
  },
  "propertyOrder": []
}

Usage notes

• The extension supports images of up to 20MB in size.
• If you were previously using the extension hosted at https://shoppable-image-ai.extensions.content.amplience.net/, you should update your schemas and registered extensions to use https://shoppable-image.extensions.content.amplience.net/ instead.

Related pages

Shoppable images extension- GitHub

Extensions overview

Registering extensions

Schema examples

Media delivery reference- point of interest

Content Hub- point of interest