Manage connections

Last update: 2024-10-23
  • Created for:
  • Admin

Once you have created or edited one or more connections, you can manage them in Connections. Connections let you:

  • View all your connections at a glance, including the owner, the sandbox, and when the connections were created and modified.
  • Edit a connection.
  • Delete a connection.
  • Create a data view from a connection.
  • View all datasets in a connection.
  • Check the status of your connection’s datasets and the status of the ingestion process. For example, when is your data available so that you can start with reporting and analysis in Analysis Workspace.
  • Identify any data discrepancies due to misconfiguration. Are you missing any rows? If so, what rows are missing and why? Did you misconfigure connections and cause missing data in Customer Journey Analytics?
  • Get insights on the usage of ingested and reportable rows across all your connections.

Connections has two interfaces: List and Usage.

List

The List interface is the default interface for Connections. If not selected, select the List tab to access the interface.

list view

The List interface shows a table of all connections available. You can quickly search for a connection using the Search Search box.

The following columns or icons are available in the table.

Column or Icon Description
Name The connection’s friendly name. To see the details of the connection, select the hyperlinked name. See Connection details.
Information To view information about Datasets included, Sandbox, Owner, and more, select Information next to the connection name.

A popup window displays details.

View connection info

Data view To create a data view for the connection, select Data view. This icon only shows when no data view is already associated with the connection.
More Select More to:

Edit Edit a connection.

Delete Delete a connection.

Data view Create new data view. To create additional data views for the connection.

Datasets One or more links to the datasets that are part of the connection. You can select the dataset hyperlink to view the dataset in the connection. If more datasets are part of the selected connection, select +x more to show a Datasets included panel. This panel shows links to all datasets and an option to search for a specific dataset that is part of the connection.

Datassets included

Selecting a dataset name opens the dataset in the Experience Platform UI in a new tab.

Sandbox The Experience Platform sandbox from which this connection draws its datasets. This sandbox was selected when you first created the connection. It cannot be changed.
Owner The person who created the connection.
Import new data The status of importing new data for datasets:

Status green)    x On for datasets configured to import new data, and

Status gray   x Off for datasets not configured to import new data.

Date created The timestamp when the connection was created.
Last modified The timestamp when the connection is last updated.
Backfill data The status for backfill data across datasets.

Status red   x backfills failed for number of failed backfills across datasets,

Status orange   x backfills processing for number of processing backfills across datasets,

Status green   x backfills completed for number of completed backfills for datasets, and

Status gray   Off in case no backfills are defined for the datasets in the connection.

To configure which columns to display select Column settings, which shows the Customize table dialog allowing you turn columns on or off in the table.

Edit a connection

  1. Select More next to the connection name
  2. Select Edit Edit from the context menu.

Alternatively, you can:

  1. Select the connection row.

  2. Select Edit Edit from the blue bar.

When editing a connection, you can:

  • Start and stop importing new data.
  • Rename a connection.
  • Refresh the dataset/s.
  • Remove dataset/s from the connections.

See Create or edit a connection for more information.

Delete a connection

  1. Select More next to the connection name.
  2. Select Delete Delete.

Alternatively, you can:

  1. Select the connection row.

  2. Select Delete Delete from the blue bar.

When you delete a connection, a Delete connection panel indicates which data views are deleted and which workspace projects are affected.

Delete connection

Select Continue to delete the connection.

See Deletion implications for more information about deleting a connection.

Create a data view for a connection

  • If no data view is associated with the connection:

    1. Select Add data view next to the connection name.
  • If one or more data views are already created for the connection:

    1. Select More next to the connection name.
    2. Select Add data view Create new data view.

Alternatively, you can:

  1. Select the connection row.

  2. Select Add data view Create data view from the blue button bar.

See Create or edit a data view for more information.

Connection details

To go to the details for a connection, select a connection name in the connections table.

All datasets window showing the widgets and settings

The Connections details interface provides a detailed view of the status of a connection. You can:

  • Check the status of your connection’s datasets and of the ingestion process.
  • Identify configuration problems that can cause skipped or deleted records.
  • See when the data is available for reporting.
User Interface Description
Edit Edit Connection To edit the details of a connection, select Edit Edit Connection. See Create or edit a connection for more information.
Dataset selector Lets you pick one or all datasets in the connection. You cannot multi-select datasets. Defaults to All datasets.
Date range selector Edit start date, end date, or select Calendar to open the date range selector. In the date range selector, select a date range by using one of the predefined periods (for example Last 6 months) or use the calendar to select start and end date. Select Apply to apply the new date range.
Records of event data available The total number of event dataset rows available for reporting, for the entire connection. This count is independent of any calendar settings. The count changes if you select a dataset from the dataset selector or by selecting a dataset in the table. Once data is added, there is a latency of 1-2 hours to get the data to appear in reporting.
Metrics Summarize the event, lookup, profile and summary dataset records that are added, skipped, and deleted, and the number of batches added. These metrics are based on the dataset and date range you have selected.

Select Check detail to show the Check skipped detail popup. The popup lists the number of skipped records and the reason for all event datasets or selected dataset.

Select Info popup with more information. For some skipped reasons, like Empty visitor ID, the popup displays Sample PSQL for EQS (Experience Platform for Query Service) you can use in Query Service to query for the skipped records in the dataset. Select Copy Copy sample PSQL for EQS to copy the SQL.

Records added Indicates how many rows were added in the selected time period, for the dataset and date range you have selected. Updated every 10 minutes.
Records skipped Indicates how many rows were skipped in the selected time period, for the dataset and date range you have selected. Reasons for skipping records include: missing timestamps, missing or invalid person ID, and so forth. Updated every 10 minutes.

Invalid person IDs (such as undefined, or 00000000, or any combination of numbers and letters in a Person ID that appears in an event more than 1 million times in a given month) are IDs that cannot be attributed to any specific user or person. These rows cannot be ingested into the system and result in error-prone ingestion and reporting. To fix invalid person IDs, you have 3 options:

  • Use Stitching to populate the undefined or all-zero user IDs with valid user IDs.
  • Blank out the user ID, which are then kipped during ingestion (preferable to invalid or all-zero user IDs).
  • Fix any invalid user IDs in your system before ingesting the data.
Records deleted Indicates how many rows were deleted in the selected time period, for the dataset and date range you have selected. Someone might have deleted a dataset in Experience Platform, for example. Updated every 10 minutes.

In some scenarios, this value can also include records replaced, as with stitching or some lookup dataset updates. Consider this example:

  • You upload one record to an XDM Individual Profile dataset, which Customer Journey Analytics is configured to ingest as profile lookup data. In the connection details, this dataset would display 1 record added.
  • You upload a duplicate of the original record into the same AEP dataset, which now contains two records. Customer Journey Analytics ingests the additional record from the profile lookup dataset. Seeing that it has already ingested a profile record in the connection for that person ID, Customer Journey Analytics deletes its earlier version and adds the new profile data. In the connection details, this action would represent 1 record added and 1 record deleted, because Customer Journey Analytics only retains the most recent profile lookup data for any ingested person ID.
  • In total, the AEP dataset contains two records that happen to be identical. Separately, the Customer Journey Analytics connection details display the status of its ingested data: 2 records added and 1 record deleted for this profile dataset.
Search Search dataset name or ID Dataset search field. You can search the datasets table by dataset name or Dataset ID.
Datasets table The datasets that are part of the connection.
Datasets The name of the dataset that is part of the connection. You can select the hyperlink to open the dataset in the Experience Platform UI in a new tab. You can select the row or the checkbox to show details for the selected dataset only.
Dataset ID Automatically generated by Experience Platform.
Records added The number of dataset records (rows) added to a connection during the selected time interval.
Records skipped The number of dataset records (rows) skipped during data transfer for a connection during the selected time interval.
Records deleted The number of dataset records (rows) removed from a connection during the selected time interval.
Batches added The number of dataset batches have been added to a connection.
Last added The timestamp of the latest batch from the dataset that has been added to a connection.
Data source type The source type of the dataset. You define the source type when creating a connection.
Dataset type The dataset type for this dataset. Type can be Event, Profile, Lookup, or Summary. Learn more
Schema The Experience Platform schema that the dataset is based on.
Import new data The status of importing new data for the dataset:

Status green   x On if dataset is configured to import new data, and

Status gray   x Off if dataset is configured not to import new data import.

Transform data The transformation status of applicable B2B lookup datasets. See Transform datasets for B2B lookups for more information.

Status green   x On for applicable datasets enabled for transformation,

Status gray   x Off for applicable datasets not enabled for transformation, and

N/A for all other datasets, not applicable for transformation.

Backfill data The status of backfill data for the dataset.

Status red   x backfills failed for number of failed backfills,

Status red   x backfills processing for number of backfills processing,

Status green   x backfills completed for number of backfills completed, and

Status gray   Off in case backfills are not configured.

Import new data The status of importing new data for the dataset:

Status green   x On if the dataset is configured to import new data, and

Status gray   x Off if the dataset is configured not to import new data.

Backfill data The status of backfill data for the dataset.

Status red   x backfills failed for number of failed backfills,

Status red   x backfills processing for number of backfills processing,

Status green   x backfills completed for number of backfills completed, and

Status gray   Off in case no backfills are configured.

IMPORTANT

Any data ingested before August 13, 2021 is not reflected in the Connections interface.

Connection panel

When no dataset is selected in the datasets table, a panel on the right side of the Connections interface shows connection options and details.

Options Description
Refresh Refresh To refresh the connection and allow recently added records to be reflected, select Refresh Refresh.
Delete Delete Delete this connection.
Add data view Create data view Create a data view based on this connection. See Data views for more information.
Connection name The friendly name of the connection.
Connection description A more detailed description that describes the purpose of this connection.
Sandbox The Experience Platform sandbox from which this connection draws its dataset/s. This sandbox was selected when you first created the connection. It cannot be changed.
Connection ID This ID is generated in Experience Platform. You can use Copy to copy the ID.
Data views using connection Lists all the data views that use this connection.
Import new data The status of importing new data for datasets:

Status green   x On for how many datasets are configured to import new data, and

Status gray   x Off for how many datasets new data import is turned off.

Backfill data The status of backfill data for datasets.

Status red   x backfills failed for number of failed backfills across datasets,

Status red   x backfills processing for number of processing backfills across datasets,

Status green   x backfills completed for number of completed backfills for datasets, and

Status gray   Off in case no backfills are defined for the datasets in the connection.

Transform data The transformation status of applicable B2B lookup datasets. See Transform datasets for B2B lookups for more information.

Status green   x On for number of datasets enabled for transformation.

Created by The name of the person who created the connection.
Last modified The timestamp of the last change to the connection.
Last modified by The person who last modified the connection.

Dataset panel

When a dataset is selected in the datasets table, a panel on the right side of the Connections interface show details for the selected dataset.

Details Description
Person ID An identity that was defined in the dataset schema in the Experience Platform. This identity is the Person ID that you selected during the creation of the connection. If you create a connection that includes datasets with different IDs, the reporting reflects that. To merge datasets, you need to use the same Person ID across datasets.
Key The key that you have specified for a lookup dataset.
Matching Key The matching key that you have specified for a lookup dataset.
Timestamp The timestamp defined for an event dataset.
Records available The total number of rows ingested for this dataset, for the particular time period selected through the calendar. There is no latency in terms of getting the data to appear in reporting, once it is added. However, when you create a brand new connection, there is latency.
Records added How many rows were added in the selected time period.
Records deleted How many records were deleted during the selected time period.
Batches added How many data batches were added into this dataset.
Records skipped How many rows were skipped during ingestion in the selected time period.

Reasons for skipping records include: Missing timestamps, missing or invalid person ID, and so forth. Updated every 10 minutes.

Invalid person IDs (such as undefined, or 00000000, or any combination of numbers and letters in a Person ID that appears in an event more than 1 million times in a given month) are IDs that cannot be attributed to any specific user or person. These rows cannot be ingested into the system and result in error-prone ingestion and reporting. To fix invalid person IDs, you have 3 options:

  • Use Stitching to populate the undefined or all-zero user IDs with valid user IDs.
  • Blank out the user ID, which is then skipped during ingestion (preferable to invalid or all-zero user IDs).
  • Fix any invalid user IDs in your system before ingesting the data.
Last added When the last batch was added.
Import new data The status of importing new data for the dataset:

Status green   x On if the dataset is configured to import new data, and

Status gray   x Off if the dataset is configured not to import new data.

Backfill data The status of backfill data for the dataset.

Status red   x backfills failed for number of failed backfills,

Status red   x backfills processing for number of backfills processing,

Status green   x backfills completed for number of backfills completed, and

Status gray   Off in case no backfills are configured.

To show a dialog with an overview of the past backfills for the dataset, select Past backfills Past backfills.

Data source type Data source type as defined when adding the dataset to the connection.
Dataset type Either Event, Profile, Lookup, or Summary. Learn more
Schema The Experience Platform schema that this dataset is based on.
Dataset ID This dataset ID is generated in Experience Platform.

Usage

The Usage interface shows the usage of ingested and reportable rows across all connections. This interface supports you to determine whether your Customer Journey Analytics usage complies with what is contractually agreed upon. In addition to monitoring purposes, you can use the Usage UI to plan your Customer Journey Analytics license renewal.

You can select a time range (between last 6 months, year to date, or last 2 Years) and an interval (between monthly or quarterly) to monitor Customer Journey Analytics usage. The interface is divided into two sections:

  • Ingested rows: total rows ingested/sent from event datasets across all Customer Journey Analytics Connections, including records skipped during ingestion
  • Reportable rows: total reportable rows that include all events data across all Customer Journey Analytics Connections

usage-view

Select the Usage tab to access the interface.

Report on usage

  1. Select a Time range. You can select between Last 6 months, Year to date, or Last 2 Years.
  2. Select an Interval. You can select between Monthly or Quarterly.

For Ingested rows:

  • A panel displays the total ingested rows that include all the events data across all the connections updated on every 2nd day of a month. Within the panel:
    • a box displays the number of ingested rows for the last month and the change in % (indicated by ▲ or ▼) from the previous month.
    • a line graph displays the ◼︎ Monthly ingested rows.
      To see a popup that displays the number of monthly ingested rows for a month, hover over any data point in the line graph.

For Reportable rows:

  • A panel displays total reportable rows that include all the events data across all the connections updated on every 2nd day of a month. Within the panel:
    • a box displays the cumulative total number of reportable rows.
    • a box displays the total number of reportable rows for the last month and the change in % (indicated by ▲ or ▼) from the previous month.
    • a line graph displays the ◼︎ Monthly reportable rows.
      To see a popup that displays the number of cumulative reportable rows for a specific month, hover over any data point in the line graph.
    • a line graph displays the ◼︎ Cumulative reportable rows.
      To see a popup that displays the number of monthly reportable rows for a month, hover over any data point in the line graph.

On this page