import numpy as np
import pandas as pd
import mikeio
from mikeio import ItemInfo, EUMType, EUMUnitWrite a dfs0
A mikeio.Dataset contains the information needed to write a dfs file. A Dataset consists of one or more mikeio.DataArrays each corresponding to an “item” in a dfs file.
nt = 10
time = pd.date_range("2000-1-1", periods=nt, freq='H')
d1 = np.zeros(nt)
item = ItemInfo("Zeros", EUMType.Water_Level)
da1 = mikeio.DataArray(d1, time=time, item=item)
d2 = np.ones(nt)
item = ItemInfo("Ones", EUMType.Discharge, EUMUnit.meter_pow_3_per_sec)
da2 = mikeio.DataArray(d2, time=time, item=item)
ds = mikeio.Dataset([da1, da2])
ds<mikeio.Dataset>
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
items:
0: Zeros <Water Level> (meter)
1: Ones <Discharge> (meter pow 3 per sec)ds.is_equidistantTrueds.to_dfs("test.dfs0", title="Zeros and ones")Read a timeseries
A dfs file is easily read with mikeio.read which returns a Dataset.
ds = mikeio.read("test.dfs0")
ds<mikeio.Dataset>
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
items:
0: Zeros <Water Level> (meter)
1: Ones <Discharge> (meter pow 3 per sec)From comma separated file
df = pd.read_csv("../tests/testdata/co2-mm-mlo.csv", parse_dates=True, index_col='Date', na_values=-99.99)
df.head()| Decimal Date | Average | Interpolated | Trend | Number of Days | |
|---|---|---|---|---|---|
| Date | |||||
| 1958-03-01 | 1958.208 | 315.71 | 315.71 | 314.62 | -1 |
| 1958-04-01 | 1958.292 | 317.45 | 317.45 | 315.29 | -1 |
| 1958-05-01 | 1958.375 | 317.50 | 317.50 | 314.71 | -1 |
| 1958-06-01 | 1958.458 | NaN | 317.10 | 314.85 | -1 |
| 1958-07-01 | 1958.542 | 315.86 | 315.86 | 314.98 | -1 |
Remove missing values
df = df.dropna()
df = df[["Average","Trend"]]
df.plot()<AxesSubplot: xlabel='Date'>
A dataframe with a datetimeindex can be used to create a dfs 0 with a non-equidistant time axis.
df.to_dfs0("mauna_loa_co2.dfs0")To get a equidistant time axis first interpolate to hourly values.
df_h = df.resample('h').interpolate()df_h.to_dfs0("mauna_loa_co2_eq_1hr.dfs0")Read a timeseries
res = mikeio.read("test.dfs0")
res<mikeio.Dataset>
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
items:
0: Zeros <Water Level> (meter)
1: Ones <Discharge> (meter pow 3 per sec)res.timeDatetimeIndex(['2000-01-01 00:00:00', '2000-01-01 01:00:00',
'2000-01-01 02:00:00', '2000-01-01 03:00:00',
'2000-01-01 04:00:00', '2000-01-01 05:00:00',
'2000-01-01 06:00:00', '2000-01-01 07:00:00',
'2000-01-01 08:00:00', '2000-01-01 09:00:00'],
dtype='datetime64[ns]', freq=None)res.to_numpy()array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]])Or as a Pandas dataframe
A mikeio.Dataset ds is converted to a pandas dataframe with ds.to_dataframe()
dfs0file = "../tests/testdata/da_diagnostic.dfs0"
df = mikeio.read(dfs0file).to_dataframe()
df.head()| State 1Sign. Wave Height | State 2Sign. Wave Height | Mean StateSign. Wave Height | MeasurementSign. Wave Height | |
|---|---|---|---|---|
| 2017-10-27 00:00:00 | 1.749465 | 1.749465 | 1.749465 | 1.72 |
| 2017-10-27 00:10:00 | 1.811340 | 1.796895 | 1.807738 | NaN |
| 2017-10-27 00:20:00 | 1.863424 | 1.842759 | 1.853422 | NaN |
| 2017-10-27 00:30:00 | 1.922261 | 1.889839 | 1.897670 | NaN |
| 2017-10-27 00:40:00 | 1.972455 | 1.934886 | 1.935281 | NaN |
dfs0file = "../tests/testdata/random.dfs0"
df = mikeio.read(dfs0file).to_dataframe()
df.head()| VarFun01 | NotFun | |
|---|---|---|
| 2017-01-01 00:00:00 | 0.843547 | 0.640486 |
| 2017-01-01 05:00:00 | 0.093729 | 0.653257 |
| 2017-01-01 10:00:00 | NaN | NaN |
| 2017-01-01 15:00:00 | 0.305065 | 0.214208 |
| 2017-01-01 20:00:00 | 0.900190 | 0.999157 |
Create a timeseries with non-equidistant data
d1 = np.random.uniform(low=0.0, high=5.0, size=5)
time = pd.DatetimeIndex(["2000-1-1", "2000-1-8", "2000-1-10", "2000-2-22", "2000-11-29"])
da = mikeio.DataArray(d1, time=time, item=ItemInfo("Random"))
da<mikeio.DataArray>
name: Random
dims: (time:5)
time: 2000-01-01 00:00:00 - 2000-11-29 00:00:00 (5 non-equidistant records)
values: [3.747, 2.771, ..., 2.366]da.is_equidistantFalseda.to_dfs("neq.dfs0", title="Non equidistant")Create a timeseries with accumulated timestep
Find correct eum units
EUMType.search("prec")[Correction of precipitation,
Precipitation correction,
Precipitation,
Specific Precipitation,
Precipitation Rate]EUMType.Precipitation_Rate.units[mm per day,
mm per hour,
cm per hour,
meter per sec,
meter per day,
feet per day,
inch per hour,
inch per min,
inch per day,
mm per year]from mikecore.DfsFile import DataValueType
n= 1000
time = pd.date_range("2017-01-01 00:00", freq='H', periods=n)
# use default name and unit based on type
item = ItemInfo(EUMType.Water_Level, data_value_type=DataValueType.Instantaneous)
da1 = mikeio.DataArray(data=np.random.random([n]), time=time, item=item)
# use a custom name
item = ItemInfo("Nedbør", EUMType.Precipitation_Rate, data_value_type=DataValueType.Accumulated)
da2 = mikeio.DataArray(data=np.random.random([n]), time=time, item=item)
ds = mikeio.Dataset([da1, da2])
ds.to_dfs('accumulated.dfs0')ds = mikeio.read("accumulated.dfs0")
ds<mikeio.Dataset>
dims: (time:1000)
time: 2017-01-01 00:00:00 - 2017-02-11 15:00:00 (1000 records)
items:
0: Water Level <Water Level> (meter)
1: Nedbør <Precipitation Rate> (mm per day) - 1Modify an existing timeseries
ds = mikeio.read("test.dfs0")
ds<mikeio.Dataset>
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
items:
0: Zeros <Water Level> (meter)
1: Ones <Discharge> (meter pow 3 per sec)ds['Ones']<mikeio.DataArray>
name: Ones
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
values: [1, 1, ..., 1]Modify the data in some way…
ds['Ones'] = ds['Ones']*np.pi
ds['Ones'].valuesarray([3.14159265, 3.14159265, 3.14159265, 3.14159265, 3.14159265,
3.14159265, 3.14159265, 3.14159265, 3.14159265, 3.14159265])ds.to_dfs("modified.dfs0")res = mikeio.read("modified.dfs0")
res['Ones']<mikeio.DataArray>
name: Ones
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
values: [3.142, 3.142, ..., 3.142]The second item is not modified.
res['Zeros']<mikeio.DataArray>
name: Zeros
dims: (time:10)
time: 2000-01-01 00:00:00 - 2000-01-01 09:00:00 (10 records)
values: [0, 0, ..., 0]Convert units
Read a file with waterlevel i meters.
filename = "../tests/testdata/waterlevel_viken.dfs0"
# filename = r"C:\Program Files (x86)\DHI\MIKE Zero\2021\Examples\MIKE_21\FlowModel_FM\HD\Oresund\Data\1993\Boundary_Conditions\waterlevel_viken.dfs0"
ds = mikeio.read(filename)
ds<mikeio.Dataset>
dims: (time:577)
time: 1993-12-02 00:00:00 - 1993-12-14 00:00:00 (577 records)
items:
0: ST 2: WL (m) <Water Level> (meter)ds.plot()
The aim is to convert this timeseries to feet (1m = 3.3 ft)
ds[0] = ds[0]*3.3Which units are acceptable?
ds.items[0].type.units[meter,
kilometer,
centimeter,
millimeter,
feet,
feet US,
inch,
inch US,
mile,
mile US,
yard,
yard US]ds[0].item = ItemInfo("Viken", ds[0].item.type, EUMUnit.feet)ds.to_dfs("wl_feet.dfs0")
Extrapolation
# filename = r"C:\Program Files (x86)\DHI\MIKE Zero\2021\Examples\MIKE_21\FlowModel_FM\HD\Oresund\Data\1993\Boundary_Conditions\waterlevel_viken.dfs0"
filename = "../tests/testdata/waterlevel_viken.dfs0"
ds = mikeio.read(filename)
df = ds.to_dataframe()
df.plot()<AxesSubplot: >
rng = pd.date_range("1993-12-1","1994-1-1",freq='30t')
ix = pd.DatetimeIndex(rng)
dfr = df.reindex(ix)
dfr.plot()<AxesSubplot: >
Replace NaN with constant extrapolation (forward fill + back fill).
dfr = dfr.ffill().bfill()
dfr.plot()<AxesSubplot: >
dfr.to_dfs0("Viken_extrapolated.dfs0",
items=ds.items,
title="Caution extrapolated data!"
)Clean up
import os
os.remove("test.dfs0")
os.remove("modified.dfs0")
os.remove("neq.dfs0")
os.remove("accumulated.dfs0")
os.remove("wl_feet.dfs0")
os.remove("mauna_loa_co2_eq_1hr.dfs0")
os.remove("mauna_loa_co2.dfs0")
os.remove("Viken_extrapolated.dfs0")