Computation workflow

In this section the workflows for calibration and data handling processes are displayed. The focus for end-point usage is a single command to complete each process. This is indicated in the diagrams by start and terminate, respectively. The workflows are a compound of operations. To improve debugging, scalability, and code utility, the subprocesses are designed as self-contained workflows. A nested workflow in the diagram is indicated with emphasized border. All nested workflows have a section in the appendix. Diagrams for nested workflows does not have the start/terminate blocks, as they are not intended as stand-alone operations.

Calibration

The model calibration process takes a dataset and a model configuration as input. The objective is to identify the best model parameters in the scope of the dataset. If the data quality is poor, or is significantly influenced by anomalous data, the calibrated parameters will reflect this. The hindcast process can’t be utilized without first executing the calibration. Still, the calibration is considered a separate workflow. Because of the consequences of input data choice, the calibration and hindcast will in end-point usage, be used with different datasets.

Create states from file

The workflow handles the pressure and flow state computations and concatenates the result into the format required by the surrogate model. This is relevant since most applications have both types of data. The workflow checks available parameters and prepares the dataset for calibration.

One of the assumptions for successful calibration that has special relevance for the workflow, is that pressure loss and flow for each pipe must have the same sign. If this is not the case, the pressure states can be manipulated though a bias parameter. The pressure state workflow asserts that the bias parameters are given, their default value is zero. Identifying the bias is a data problem; it can’t be derived from physical properties (unlike the sensor elevation).

This workflow does not include automatic estimation of the bias of the pressure measurements. Instead, the result of the entire calibration workflow must be inspected: If the calibration produces negative resistance parameter (α) for a pipe, this indicates that the pressure loss and the flow have opposite sign. Now, use plots of the measured pressure loss and the calculated flow square to make an educated guess on the bias, and re-run the calibration workflow.

Compute pressure states

Computing the pressure states requires preprocessed data and the network model configuration. The network model has a dictionary of the measurements that are attributed as pressure measurements. In part this is necessary because the dataset (might) lack the metadata to identify these otherwise. Also, the columns are renamed to match the state names.

Compute flow states

Computing the flow states requires preprocessed data and the network model configuration. The network model has a dictionary of the measurements that are attributed as flow measurements. The computation of the mass balance requires the incidence matrix of the system. Also, the dataset (might) be lacking the metadata to identify these otherwise.

Filtering and resampling

The preprocessing workflow resamples and filters a dataset, which is useful for eliminating/limiting sources of error in more advanced operations. Also, when investigating data, an overview is much more readily obtained with cleaner and coarser sampled data.