E-commerce Integration - Shopify Connector

Feature Context

What is Shopify About?

Shopify is a user-friendly e-commerce platform that helps small businesses build an online store and sell online through one streamlined dashboard. Shopify merchants can build a modern online store and sell on social media sites, seller marketplaces, other blogs and websites and via email, text and chat.

The Marigold Engage Shopify connector allows our Shopify-using customers to integrate their eCommerce data in their marketing messaging to be able to send more nuanced and better targeted messages to their contacts.

How it works

The Shopify connector supports polling-based data sync. It connects to the client’s Shopify setup and polls the entities configured for the sync. On detecting an update, it retrieves the updated data and imports it into one or more lists in Marigold Engage.
If the data sync is set up to be two-way then any updates in the Marigold Engage list(s) are also synced back to the Shopify entity.

Following synchronization directions are supported:

  • Customers - bi-directional sync for basic customer fields

Note: Address fields can’t be updated because addresses (used in orders, shipping or billing) that aren’t provided with the update are automatically deleted at Shopify.

  • Products and productVariants - bi-directional
  • Orders, OrderTransactions - one-directional sync from Shopify to Marigold Engage

Note: We presently do not have the capability to sync messages.

The connector currently supports syncing with the following Shopify entities:

  • Customers
  • Products
  • ProductVariants
  • Orders
  • Ordertransactions

 

Activation and Configuration

Prerequisites

A Shopify app and account is required. Please find more details on how to setup this account on https://help.shopify.com/en/manual/apps/app-types/custom-apps?shpxid=bc93fe60-E05B-4FDB-AF29-782857753A55#get-the-api-credentials-for-a-custom-app.

In order to set up this Shopify account, the following access scopes are required:

  • write_customers
  • read_customers
  • write_order_edits
  • read_order_edits
  • write_orders
  • read_orders
  • write_products
  • read_products
  • read_product_listings
  • read_files
  • read_publications
  • read_inventory

Once the app is created you can copy the access token you need to input in the Connector settings in Marigold Engage.

You will also need the following information ready:

Shopify name – This is the name given to your Shopify app.

 

Set up the Connector

1. Go to the Admin Configuration / Data Integration / Connectors and add a New connector:

A properties panel is displayed on the right:

2. Set the following properties:

  • Name and Description — Provide a name and description for this connector.
  • Connector* — Select Shopify connector as type of connector from the drop-down list.
  • Server* — This is the name of the server on which Marigold Engage is installed. Select one from the drop-down list. Only servers on which the connector is installed are listed.
  • Name* — Shopify name
  • API Access token* — This token can be retrieved after the Shopify app has been created.

3. Save the connector

Note: Ensure that once the connector is set up, it is added to your Organization.

 

Configure the Data Sync

1. Go to Data Exchange / Data Syncs.

2. Create a New data sync. The wizard is launched and the following needs to be defined:

3. On the Setup step, there are two tabs available: Setup and Advanced.
In this step, you can define all the settings for the Data Sync.

Setup step - tab ‘Setup’

Connector — Choose the Shopify connector configured for the current organization in the Admin Configuration. When a connector is selected, the header for the drop-down below changes from 'Connector table' to ‘Connector name’ table'.

Connector table — Select the shopify table from the drop-down by clicking and scrolling through the list or by entering part of a table name (which filters the list) before making your selection. This list contains the tables from your Data source database, sorted alphabetically.

Engage list — Select the required list that exists in the current organization.

By default, the Schedule switch is disabled, which means the Data Sync is not scheduled. In that case, a notification is shown to explain that you can manually start the data sync execution and run it once.

Note: The execution can be manually started through the ‘Execute once now’ button on the Connector Data Syncs Start page.

When the switch is enabled, the schedule settings become available:

  • Start date and End date — You can set a start date and/or end date for the sync. For example, your task can start now but needs to run indefinitely.
  • Periodicity — Indicate when the sync should run :
  • Daily — Select the times of day at which the sync should run. You can select more than one.
  • Weekly — Select the day of the week on which the sync should run. The sync can run more than once a week. You can also set the start time.
  • Monthly — Select the days of the month on which the sync should run. You can also select the time of day at which the sync should start.
  • Periodically — Set the recurrence of the sync, expressed in minutes. For example, the sync runs every 10 minutes.
  • Once — Set once if the sync needs to be executed just once. You can also change the start time.

A message can be sent when the Data Sync has been completed successfully, or if it fails or a timeout occurs. To activate a notification, simply enable the corresponding switch and enter one or more email addresses (multiple email addresses are separated by a semicolon). You can also select a notification group. These notification groups are created in the Admin Configuration.

Graphical user interface, text, application Description automatically generated

Note: Notifications are optional. When enabling one or more notifications, you need to provide at least one email address or group for each enabled notification.

As soon as you have selected all the properties, a Next button becomes available to navigate to the Sync step.

 

Setup step - tab ‘Advanced’

Chart Description automatically generated with medium confidence

On the Advanced tab, you can define the Data source batch size and the Engage batch size.

Data source batch size — Used to determine how many records are retrieved per batch from the Data source (in case of Data source to Engage sync) or updated/inserted in the Data source (in case of Engage to Data source sync).
Shopify can only sync a maximum of 200 records. However, to define this maximum, multiple elements are taken into account such as the number of fields and the complexity and number of requested records. The more fields there are, or the higher the complexity, the less records you can sync. If you run into a rate limit when syncing, come back to this configuration and lower the batch size.

Engage batch size — Used to determine how many records are retrieved per batch from Engage (in case of Engage to Data source sync) or updated/inserted in Engage (in case of Data source to Engage sync).

Click Next to go to the next step, which is the Sync step.

4. On the Sync step, there are three tabs available : Sync, Filters and History.

 

Sync step - tab ‘Sync’

The sync tab lets you define the Data source fields that need to be synchronized with the Engage fields. A mapping between the fields is defined as well as the direction of the sync. Engage fields can be selected from the main list as well as linked lists.

Technical note: Make sure that the type and field length of the mapped fields correspond (such as syncing a Data source date field only with a Engage date field).

Note: When setting up a sync between the Data source and Engage, Data source tables and lists can be used just once in a sync.

Data source field — Data source fields can be selected from the Data source table (selected in the setup step) by clicking and scrolling through the list or by entering part of a field name (which filters the list) before making your selection.

Scope — The list (selected in the setup step) can be selected as 'Master' list, and all of its 1:1 relations. By default, the scope 'Master' is selected.

Field — Fields from the scope can be selected (a combo-box appears only when a scope is selected).

 

You can also determine the direction of the data sync using the following buttons :

  • ← : Engage list to Data source table

  • ↔ : Both ways

  • → : Data source table to Engage list

Note: Computed fields (non-persisted and persisted) can be selected as field when the chosen direction is 'Engage list to Data source table'. For 'bi-directional' and 'Data source table to Engage list', computed fields can’t be selected.

It's possible to add a new field to the target list (selected as scope), using the 'Create new field' option.
Graphical user interface, text, application Description automatically generated

When this option is selected, a new text field appears underneath the drop-down in which a field name can be entered. This value is required.

  • The new field name has a default value equal to the Data source field name (in uppercase) unless that field name already exists. In this case, the field is empty (a placeholder is shown instead).

  • The new field name can’t be the name of an existing field, already configured new field or a non-permitted keyword (front-end validation).

  • Spaces in the field name are converted to underscores (for consistency).

  • The type of the new field is automatically derived from the type of the selected Data source field.

    • If the selected Data source field has data type TEXT or LINK with length=0, then the new field will have data type LONGTEXT

  • As soon as the data sync is created/updated, the new fields are created on the list selected as scope.

Syncs can be sorted by clicking on one of the column headers (Data source Field, Direction, Scope, Field). Clicking on a header sorts the results in ascending order of the column values. Active sorting is indicated by an up (ascending) or down (descending) arrow next to the column header. Clicking on a header of an already active sorting column, changes the sorting direction for that column.

A complete row can be deleted by clicking on the bin icon.
The last remaining row can’t be deleted.

 

Sync step - tab ‘Filters’

On the Filters tab, the user can define :

  • a Data source filter — This is a text-area in which you can type a filter. Only records matching the filter will be synchronized. The filter syntax depends on the connector that you are using. For more details on the syntax, please consult the corresponding manuals/support site.

  • a Engage filter — Only records matching this filter will be synchronized during the data sync run. This filter can be built using the Constraint Builder, containing the same options as for building dynamic segments.

Note: Defining filters is optional. No filters are set by default.

When a filter is set for either the Data source fields or the fields, this is indicated by a filter icon next to the corresponding header on the 'Sync' tab.


When you click on a filter icon, the Filters tab is opened.

An organization filter is automatically applied to the Data Sync when it's defined on the Audience List.

Note: This can only be the case for synchronization of Audience List data from Engage to the Data source.

If an organization filter and a regular filter are applied, the Data Sync will be executed taking into account both filters.

 

Sync step - tab ‘History’

The history of a data sync provides details on previous executions.

This includes :

  • the source and destination table of the data sync

  • the details per run :

    • the run date

    • the direction of the data synchronization

    • the total number of records included

    • the number of records that successfully synchronized, and the ones that failed

    • the starting time of the sync

    • the duration of the sync

When no synchronization has been executed yet or no data is available yet, the user is informed :
Graphical user interface, text Description automatically generated

When execution history is available, it looks like this :





Each run can be expanded by clicking on the down-arrow on the left, to show more details about failed records.
When erroneous data is available, it will be shown here.

Note: When a data sync generates an error, this error will also be visible from the notifications in the top menu bar of Marigold Engage. Clicking the notification takes the user to the ‘History’ tab of the Data Sync.

Failed records for which an error message is returned, can be exported to a CSV file. An ‘Export’ button is available to launch the export. This ‘Export’ button will NOT be available when the cause of the failed record is UNKNOWN or when there are no failed records.

When a failure happens before the individual records are captured, the system doesn't know how many records would have been included in the sync.
In that case, 'Unknown' is shown instead of the number of failed records.


Note : In case of a long list of failed records, only the first 10 failed records are shown.

When you've finished configuring the sync, press the 'Save' button (or 'Update' button if you are modifying an existing Data Sync).

Note: The 'Save' (or 'Update') button is only available when at least one field map is defined. Otherwise the button is disabled.

 

Validation errors/warnings

One or more validation errors/warnings appear if your configuration is not correct or if (an)other issue(s) occur(s).

List of possible errors :
- Data source connector instance is in use
- Data source sync configuration is already disabled
- Data source sync configuration is already enabled
- Data source sync not scheduled
- Data source table field is used multiple times
- Data source table already used in data sync
- Schedule end date should be less than start date
- Invalid Data source batch size for Data source sync
- Invalid Data source connector instance
- Invalid Data source table
- Invalid Data source table field
- Invalid data sync direction
- Invalid list
- Invalid notification email address
- Invalid notification group
- Invalid notification timeout configuration
- Invalid notification type
- Invalid relation
- Invalid schedule data configuration
- Start date should be greater than current date
- Invalid schedule type
- Invalid Engage batch size for Data source sync
- Invalid Engage list field
- Invalid start time format
- Engage list already used in data sync
- Engage list field is used in Data source data sync
- Engage list is used in Data source data sync
- Notification configuration is missing
- Notification groups are not configured
- Schedule data configuration is missing
- Schedule end date is required
- Schedule start date is required
- Schedule type is required
- Engage list field is used multiple times
- Start time is required
- Failed to save the Data source sync data configuration
- Failed to delete the Data source data sync between Data source table [Data source TableName] and Engage list [listName]
- Failed to disable Data source sync between Engage list [listName] and Data source table [Data source TableName]
- Failed to enable Data source sync between Engage list [listName] and Data source table [Data source TableName]
- Failed to mark Data source sync between Engage list [listName] and Data source table [Data source TableName] for execution

List of possible warnings :
- The data type of the Data source table field and Engage list field does not match
- Field length of the Data source field and Engage list field does not match

Note: There are placeholders [listName] and [Data source TableName] in some messages, which are substituted at run time with information based on the context.

Errors and warnings are shown in a validation pane at the bottom of the screen after clicking the 'Save' button. In that case, the Save button label changes to 'Update'.

When navigating to the ‘Sync’ tab and clicking on an error/warning in the validation pane, the row linked to the error/warning is highlighted (if applicable).
This allows issues to be easily resolved.
When done, click on the ‘Update’ button to save your changes.

Example:
When selecting a warning in the validation pane, the row that causes the problem is highlighted on the Sync tab. The label of the button shows 'Update'.

Note: Some errors will continue to be displayed as notification messages at the top-right instead of using the validation pane, such as:
- 'The selected Data source table does not have any fields' (Setup step).
- 'The Data source table is already in use' (Sync step).

When everything has been set up correctly, the Data sync will be created and visible on the Start page.