- Sources overview
- Available source connectors
- Adobe applications
- Advertising
- Analytics
- Cloud storage
- Amazon Kinesis connector
- Amazon S3 connector
- Apache HDFS connector
- Azure Data Lake Storage Gen2 connector
- Azure Blob connector
- Azure Event Hubs connector
- Azure File Storage connector
- Data Landing Zone
- FTP connector
- Google Cloud Storage connector
- Google PubSub
- Oracle Object Storage
- SFTP connector
- Amazon S3 and Azure Blob connector
- Consent & Preferences
- CRM
- Customer success
- Databases
- Amazon Redshift connector
- Apache Hive on Azure HDInsights connector
- Apache Spark on Azure HDInsights connector
- Azure Data Explorer connector
- Azure Synapse Analytics connector
- Azure Table Storage connector
- Couchbase connector
- Google BigQuery connector
- GreenPlum connector
- HP Vertica connector
- IBM DB2 connector
- MariaDB connector
- Microsoft SQL Server connector
- MySQL connector
- Oracle connector
- Phoenix connector
- PostgreSQL connector
- Snowflake Streaming connector
- Snowflake connector
- Teradata Vantage connector
- Data & identity partner
- eCommerce
- Local system
- Marketing automation
- Payments
- Protocols
- Streaming
- API tutorials
- Create a base connection
- Explore data
- Collect data
- On-demand ingestion
- Filter data at the source level
- Monitor dataflows
- Update accounts
- Update dataflows
- Retry failed dataflow runs
- Delete accounts
- Delete dataflows
- Ingest encrypted data
- Save a dataflow as a draft
- UI tutorials
- Create a source connection
- Adobe applications
- Advertising
- Analytics
- Cloud storage
- Consent & Preferences
- CRM
- Customer Success
- Databases
- Amazon Redshift
- Apache Hive on Azure HDInsights
- Apache Spark on Azure HDInsights
- Azure Data Explorer
- Azure Synapse Analytics
- Azure Table Storage
- Couchbase
- Google Big Query
- GreenPlum
- HP Vertica
- IBM DB2
- MariaDB
- Microsoft SQL Server
- MySQL
- Oracle
- Phoenix
- PostgreSQL
- Snowflake
- Snowflake Streaming
- Teradata Vantage
- Data & identity partner
- eCommerce
- Local system
- Marketing automation
- Payments
- Protocols
- Streaming
- Configure a dataflow
- Advertising connection dataflow
- Analytics connection dataflow
- Batch cloud storage connection dataflow
- Streaming cloud storage connection dataflow
- Consent & Preferences connection dataflow
- CRM connection dataflow
- Customer success connection dataflow
- Database connection dataflow
- Ecommerce connection dataflow
- Marketing automation connection dataflow
- Payment connection dataflow
- Protocol connection dataflow
- Create a sources dataflow using templates in the UI
- Ingest encrypted data
- On-demand ingestion
- Monitor batch dataflows
- Monitor streaming dataflows
- Update accounts
- Update dataflows
- Delete accounts
- Delete dataflows
- Subscribe to sources alerts
- Save a dataflow as a draft
- Create a source connection
- Self-Serve Sources (Batch SDK)
- Overview
- Configure your connection specification
- Self-Serve Sources (Batch SDK) API guide
- Documentation guide
- Streaming SDK
- Get started with Self-Serve Sources (Streaming SDK)
- Create a connection specification for a streaming source
- Update a connection specification for a streaming source
- Update the streaming flow specification
- Test and submit your connection specification for verification
- Document your source (Streaming SDK)
- Documentation self-service API streaming template
- Documentation self-service UI streaming template
- Error messages
- Flow run notifications
- IP address allow list
- Frequently asked questions
- API reference
- Platform release notes
On-demand ingestion for sources dataflows in the UI
You can use on-demand ingestion to trigger a flow run iteration of an existing dataflow using the sources workspace in the Adobe Experience Platform user interface.
This document provides you with steps on how to create dataflows on demand for sources, as well as how to retry flow runs that have been processed or have failed.
What is a flow run?
Flow runs represent an instance of dataflow execution. For example, if a dataflow is scheduled to run hourly at 9:00 AM, 10:00 AM, and 11:00 AM, then you would have three instances of a flow run. Flow runs are specific to your particular organization.
Getting started
In order to create a flow run, you must first have the flow ID of a dataflow that is scheduled for one-time ingestion.
This document requires a working understanding of the following components of Experience Platform:
- Sources: Experience Platform allows data to be ingested from various sources while providing you with the ability to structure, label, and enhance incoming data using Platform services.
- Dataflows: A dataflow is a representation of data jobs that move data across Platform. Dataflows are configured across different services, helping move data from source connectors to target datasets, to Identity Service and Real-Time Customer Profile, and to Destinations.
- Sandboxes: Experience Platform provides virtual sandboxes that partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications.
Create a dataflow on demand
Navigate to the Dataflows tab of the sources workspace. From here, find the dataflow that you want to run on demand, and then select the ellipses (...) beside your dataflow name.
Next, select Run on-demand from the dropdown menu that appears.
Configure the schedule of your on-demand ingestion. Select the Ingestion start time, the Date range start time, and the Date range end time.
| Scheduling configuration | Description |
|---|---|
| Ingestion start time | The scheduled time of when the on-demand flow run will begin. |
| Date range start time | The earliest date and time that data will be retrieved from. |
| Date range end time | The date and time that data will be retrieved up to. |
Select Schedule and allow a few moments for your on-demand dataflow to trigger.
Select your dataflow name to view your dataflow activity. Here you will see a list of your dataflow runs that have been processed. You can re-run individual iterations of your dataflow runs regardless of whether they have failed or succeeded. For run iterations that have failed, you can use Retry to initiate the run again after diagnosing and addressing any errors that may have been encountered during the creation process.
Select Scheduled to see a list of dataflow runs that are scheduled for future ingestion.
Next steps
By reading this document, you have learned how to create flow runs on demand for existing sources dataflows. For more information on sources, read the sources overview
