The Dataflow Overview

Mathias Kolind -

The basic view of Data Manager is the Dataflow Overview. Here, all established dataflows can be viewed, filtered, searched out, edited, cloned, paused, deleted, and created. To start, a quick look at the options this view presents you with.

  • Details – Shows you a log of recent activity on this dataflow, including successful runs and errors. The activity log can be viewed in detail - or limited to only showing errors to avoid clutter. A search-field is available for filtering the information by date/time. You can further explore the errors to learn the cause and in which lines they occurred.
  • Clone – Copies the dataflow as a new draft with all details intact, for simpler and more convenient creation of closely-related dataflows.
  • Clone to sister account - In addition to creating a copy of the dataflow within the same account, it is possible to clone the dataflow into one or more sister accounts by selecting ‘Sister account’ when mousing over the Clone-command and select the required accounts in the pop-up. Press "Clone dataflow". The cloned dataflow will be in draft mode, and you should go through the dataflow to check if everything is as expected before setting the dataflow into production.   

    Note: for cloning to a sister account to work, following must be observed: 
    - the recipient account must have the same source (with same name) that the dataflow uses. If not, cloning will fail. 
    - if the recipient account does not contain the same schemas (with the same name(s)), the cloned dataflow will not contain these schemas (you can clone schemas between sister accounts before cloning the dataflow, if you need schemas to be the same - see more here)
    - if a schema on the recipient account is missing mappings or contains different mappings, these mappings will not be present in the cloned dataflow  
    After cloning a dataflow, it is necessary to download the file of the dataflow on the recipient account. An information box will guide you to the download page. The cloned dataflow will be in draft mode, and you should go through the dataflow to check if everything is as expected before setting the dataflow into production.   
  • Run Now – bypasses scheduling to manually trigger a full run of the process. Be aware that this may take some time and cannot be canceled once started.
  • Pause/Unpause – Disables or reenables the scheduling of a finished dataflow.
  • Delete – Permanently removes the dataflow. This cannot be reversed.

Status: The Status of a dataflow has a large impact on what options you have for interacting with it.

The possible states and their limitations are as follows:

  • In Progress – This Dataflow is currently running, which effectively sets it as read-only until it is finished. Only purely cosmetic information (Name, Description) can be edited, and the only available options are Clone and Delete, the later of which will stop the ongoing progress. Once finished, the Dataflow will switch to Ready or Waiting.
  • Waiting – This Dataflow is complete and active, and will run again as soon as the next file finishes downloading. This state only occurs when a Dataflow is set to run continuously, and the datasource likewise continous – namely Website Tracking or StreamFile. Identical to the Ready-state in terms of available options.
  • Ready – This Dataflow is complete and active, awaiting its next scheduled or manual activation. Most options are available, and clicking on the Dataflow allows for much of the information within it to be edited. This and Waiting are the only states from which a Dataflow can be Paused
  • Paused – This Dataflow has been paused, interrupting its scheduled or continuous runs. Options are generally identical to the Ready-state, and the Dataflow can still be run manually with the ‘Run Now’ option. This is the only state in which the ‘Unpause’ option can be used, returning the Paused Dataflow to the Ready- or Waiting-state.
  • Failed – This Dataflow has crashed due to some kind of serious issue. The ‘Details’ option may provide useful information about why and how. This can be freely edited, giving you extensive options for correcting the issue. The Dataflow will not attempt to run according to its set schedule, but the ‘Run Now’ is available, letting you give it another go.
  • Draft - The dataflow is unfinished – a draft that has yet to be completed. The only options will thus be to delete it, or else open and finish the dataflow. This is signified by most of the option-bar beneath being replaced with a checklist that shows how far along the the draft is. The dataflow will appear as a Draft when clicking the 'Save as draft' button during creation of a dataflow.
Have more questions? Submit a request


Powered by Zendesk