Automated dataset expirations
The Data Lifecycle workspace in the Adobe Experience Platform UI allows you to schedule expirations for datasets. When a dataset reaches its expiration date, the data lake, Identity Service, and Real-Time Customer Profile begin separate processes to remove the dataset’s contents from their respective services. Once the data is deleted from all three services, the expiration is marked as complete.
If a dataset is set to expire, you must manually change any dataflows that may be ingesting data into that dataset so that your downstream workflows are not negatively affected.
This document covers how to schedule and automate dataset expirations in the Platform UI.
Dataset Expiration does not currently delete data from the Adobe Experience Platform Edge Network. However, there is no possibility that data stays inside the Edge Network after the dataset is set to expire. This is because the 15-day service license agreement for Dataset Expiration overlaps with the 14-day period where data exists inside the Edge Network before being discarded.
Advanced Data Lifecycle Management supports dataset deletions through the dataset expiration endpoint and ID deletions (row-level data) using primary identities via the workorder endpoint. You can also manage dataset expirations and record deletions through the Platform UI. See the linked documentation for more information.
Data Lifecycle does not support batch deletion.
Schedule a dataset expiration
To create a request, select Create request from the main page in the workspace.
Real-Time CDP, Adobe Journey Optimizer, and Customer Journey Analytics users have 20 pending scheduled dataset expiration work orders. Healthcare Shield and Privacy and Security Shield users have 50 pending scheduled dataset expiration work orders. This means that you can have 20 or 50 datasets scheduled to be deleted at any one time.
For example, if you have 20 scheduled dataset expirations and one dataset is due to be deleted tomorrow, you cannot set any more expirations until after that that dataset has been deleted.
The request creation workflow appears. Under the Requested Action section, select Delete Dataset to update the controls for dataset expiration scheduling.
Select a date and a dataset
Under the Requested Action section, select a date that you want the dataset to be deleted by. You can enter the date manually (in the format mm/dd/yyyy) or select the calendar icon (
) to select the date from a dialog.
Next, under Dataset Details, select the database icon (
) to open a dataset selection dialog. Choose a dataset from the list to apply the expiration to, then select Done.
Only datasets belonging to the current sandbox are shown.
Submit the request
The Dataset Details section populates to include the primary identity and schema for the selected dataset. Under Request settings, enter a name and optional description for the request, followed by Submit.
A Confirm request dialog appears. You are asked to confirm the dataset name and the date that the dataset will be deleted by. Select Submit to continue.
After the request is submitted, a work order is created and appears on the main tab of the Data Lifecycle workspace. From here, you can monitor the work order’s status as it processes the request.
Refer to the overview section on timelines and transparency for details on how dataset expirations are processed once they are executed.
Edit or cancel a dataset expiration
To edit or cancel a dataset expiration, select Dataset on the main page of the workspace, and select the dataset expiration from the list.
On the details page of the dataset expiration, the right rail shows controls to edit or cancel the scheduled deletion.
Next steps
This document covered how to schedule dataset expirations in the Experience Platform UI. For information on how to perform other data minimization tasks in the UI, refer to the data lifecycle UI overview.
To learn how to schedule dataset expirations using the data hygiene API, refer to the dataset expiration endpoint guide.
