Copernicus Marine provides access to a wide range of model and in-situ data . In this example we will look at how to access the in-situ data and convert it to a MIKE IO dataset.
import pandas as pd
import xarray as xr
import mikeio
fino = xr.open_dataset("../../data/NO_TS_MO_FINO1_202209.nc" )
fino
<xarray.Dataset> Size: 3MB
Dimensions: (TIME: 2879, LATITUDE: 2879, LONGITUDE: 2879, DEPTH: 7,
POSITION: 2879)
Coordinates:
* TIME (TIME) datetime64[ns] 23kB 2022-09-01 ... 2022-09-30T23:31:5...
* LATITUDE (LATITUDE) float32 12kB 54.0 54.0 54.0 54.0 ... 54.0 54.0 54.0
* LONGITUDE (LONGITUDE) float32 12kB 6.583 6.583 6.583 ... 6.583 6.583
DEPH (TIME, DEPTH) float32 81kB ...
Dimensions without coordinates: DEPTH, POSITION
Data variables: (12/22)
DEPH_QC (TIME, DEPTH) float32 81kB ...
TIME_QC (TIME) float32 12kB ...
POSITION_QC (POSITION) float32 12kB ...
TEMP (TIME, DEPTH) float64 161kB ...
TEMP_QC (TIME, DEPTH) float32 81kB ...
VHM0 (TIME, DEPTH) float64 161kB ...
... ...
VTPK_DM (TIME, DEPTH) object 161kB ...
VTZA (TIME, DEPTH) float64 161kB ...
VTZA_QC (TIME, DEPTH) float32 81kB ...
VPED (TIME, DEPTH) float64 161kB ...
VPED_QC (TIME, DEPTH) float32 81kB ...
VPED_DM (TIME, DEPTH) object 161kB ...
Attributes: (12/46)
platform_code: FINO1
platform_name:
data_mode: M
title: NWS - NRT in situ Observations
summary: Oceanographic data from North West Shelf
naming_authority: Copernicus Marine In Situ
... ...
doi:
pi_name:
qc_manual: OceanSITES User's Manual v1.2
date_update: 2022-10-07T05:39:40Z
history: 2022-10-01T18:04:25Z : Creation; 2022-10-...
wmo_inst_type: Dimensions: TIME : 2879LATITUDE : 2879LONGITUDE : 2879DEPTH : 7POSITION : 2879
Coordinates: (4)
TIME
(TIME)
datetime64[ns]
2022-09-01 ... 2022-09-30T23:31:...
long_name : Time standard_name : time valid_min : -90000.0 valid_max : 90000.0 uncertainty : comment : axis : T ancillary_variables : TIME_QC array(['2022-09-01T00:00:00.000000000', '2022-09-01T00:02:00.000000000',
'2022-09-01T00:30:00.000000000', ..., '2022-09-30T23:02:00.000000000',
'2022-09-30T23:30:00.000000000', '2022-09-30T23:31:59.999999744'],
shape=(2879,), dtype='datetime64[ns]') LATITUDE
(LATITUDE)
float32
54.0 54.0 54.0 ... 54.0 54.0 54.0
long_name : Latitude of each location standard_name : latitude units : degree_north valid_min : -90.0 valid_max : 90.0 uncertainty : comment : axis : Y ancillary_variables : POSITION_QC array([54., 54., 54., ..., 54., 54., 54.], shape=(2879,), dtype=float32) LONGITUDE
(LONGITUDE)
float32
6.583 6.583 6.583 ... 6.583 6.583
long_name : Longitude of each location standard_name : longitude units : degree_east valid_min : -180.0 valid_max : 180.0 uncertainty : comment : axis : X ancillary_variables : POSITION_QC array([6.583333, 6.583333, 6.583333, ..., 6.583333, 6.583333, 6.583333],
shape=(2879,), dtype=float32) DEPH
(TIME, DEPTH)
float32
...
long_name : Depth standard_name : depth units : m positive : down valid_min : -12000.0 valid_max : 12000.0 uncertainty : comment : axis : Z reference : sea_level data_mode : R ancillary_variables : DEPH_QC [20153 values with dtype=float32] Data variables: (22)
DEPH_QC
(TIME, DEPTH)
float32
...
long_name : Depth quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] TIME_QC
(TIME)
float32
...
long_name : Time quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [2879 values with dtype=float32] POSITION_QC
(POSITION)
float32
...
long_name : Position quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [2879 values with dtype=float32] TEMP
(TIME, DEPTH)
float64
...
standard_name : sea_water_temperature units : degrees_C long_name : Sea temperature valid_min : -2000 valid_max : 32000 comment : uncertainty : accuracy : precision : resolution : cell_methods : sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : R ancillary_variables : TEMP_QC [20153 values with dtype=float64] TEMP_QC
(TIME, DEPTH)
float32
...
long_name : Sea temperature quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VHM0
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_significant_height units : m long_name : Spectral significant wave height (Hm0) valid_min : 1 valid_max : 25000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : spectral analysis sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : M ancillary_variables : VHM0_QC VHM0_DM [20153 values with dtype=float64] VHM0_QC
(TIME, DEPTH)
float32
...
long_name : Spectral significant wave height (Hm0) quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VHM0_DM
(TIME, DEPTH)
object
...
long_name : Spectral significant wave height (Hm0) method of data processing conventions : Copernicus Marine In Situ reference table 1 flag_values : R, A, D flag_meanings : real-time adjusted-in-real-time delayed-mode [20153 values with dtype=object] VZMX
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_maximum_height units : m long_name : Maximum zero crossing wave height (Hmax) valid_min : 0 valid_max : 40000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : zero crossing sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : M ancillary_variables : VZMX_QC VZMX_DM [20153 values with dtype=float64] VZMX_QC
(TIME, DEPTH)
float32
...
long_name : Maximum zero crossing wave height (Hmax) quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VZMX_DM
(TIME, DEPTH)
object
...
long_name : Maximum zero crossing wave height (Hmax) method of data processing conventions : Copernicus Marine In Situ reference table 1 flag_values : R, A, D flag_meanings : real-time adjusted-in-real-time delayed-mode [20153 values with dtype=object] VTM02
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_mean_period_from_variance_spectral_density_second_frequency_moment units : s long_name : Spectral moments (0,2) wave period (Tm02) valid_min : 1000 valid_max : 25000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : spectral analysis sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : M ancillary_variables : VTM02_QC VTM02_DM [20153 values with dtype=float64] VTM02_QC
(TIME, DEPTH)
float32
...
long_name : Spectral moments (0,2) wave period (Tm02) quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VTM02_DM
(TIME, DEPTH)
object
...
long_name : Spectral moments (0,2) wave period (Tm02) method of data processing conventions : Copernicus Marine In Situ reference table 1 flag_values : R, A, D flag_meanings : real-time adjusted-in-real-time delayed-mode [20153 values with dtype=object] VTPK
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_period_at_variance_spectral_density_maximum units : s long_name : Wave period at spectral peak / peak period (Tp) valid_min : 1000 valid_max : 30000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : spectral analysis sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : M ancillary_variables : VTPK_QC VTPK_DM [20153 values with dtype=float64] VTPK_QC
(TIME, DEPTH)
float32
...
long_name : Wave period at spectral peak / peak period (Tp) quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VTPK_DM
(TIME, DEPTH)
object
...
long_name : Wave period at spectral peak / peak period (Tp) method of data processing conventions : Copernicus Marine In Situ reference table 1 flag_values : R, A, D flag_meanings : real-time adjusted-in-real-time delayed-mode [20153 values with dtype=object] VTZA
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_mean_period units : s long_name : Average zero crossing wave period (Tz) valid_min : 2000 valid_max : 25000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : zero crossing sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : R ancillary_variables : VTZA_QC [20153 values with dtype=float64] VTZA_QC
(TIME, DEPTH)
float32
...
long_name : Average zero crossing wave period (Tz) quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VPED
(TIME, DEPTH)
float64
...
standard_name : sea_surface_wave_from_direction_at_variance_spectral_density_maximum units : degree long_name : Wave principal direction at spectral peak valid_min : 0 valid_max : 360000 comment : uncertainty : accuracy : precision : resolution : cell_methods : type_of_analysis : spectral analysis sensor_depth : 0.0 sensor_mount : sensor_orientation : data_mode : M ancillary_variables : VPED_QC VPED_DM [20153 values with dtype=float64] VPED_QC
(TIME, DEPTH)
float32
...
long_name : Wave principal direction at spectral peak quality flag conventions : Copernicus Marine In Situ reference table 2 valid_min : 0 valid_max : 9 flag_values : [0 1 2 3 4 5 6 7 8 9] flag_meanings : no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed value_below_detection nominal_value interpolated_value missing_value [20153 values with dtype=float32] VPED_DM
(TIME, DEPTH)
object
...
long_name : Wave principal direction at spectral peak method of data processing conventions : Copernicus Marine In Situ reference table 1 flag_values : R, A, D flag_meanings : real-time adjusted-in-real-time delayed-mode [20153 values with dtype=object] Indexes: (3)
PandasIndex
PandasIndex(DatetimeIndex([ '2022-09-01 00:00:00',
'2022-09-01 00:02:00',
'2022-09-01 00:30:00',
'2022-09-01 00:32:00',
'2022-09-01 01:00:00',
'2022-09-01 01:01:59.999999744',
'2022-09-01 01:30:00',
'2022-09-01 01:32:00',
'2022-09-01 02:00:00',
'2022-09-01 02:02:00',
...
'2022-09-30 21:30:00',
'2022-09-30 21:32:00',
'2022-09-30 22:00:00',
'2022-09-30 22:01:59.999999744',
'2022-09-30 22:30:00',
'2022-09-30 22:32:00',
'2022-09-30 23:00:00',
'2022-09-30 23:02:00',
'2022-09-30 23:30:00',
'2022-09-30 23:31:59.999999744'],
dtype='datetime64[ns]', name='TIME', length=2879, freq=None)) PandasIndex
PandasIndex(Index([54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0,
...
54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0, 54.0],
dtype='float32', name='LATITUDE', length=2879)) PandasIndex
PandasIndex(Index([6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945,
...
6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945, 6.5833330154418945, 6.5833330154418945,
6.5833330154418945],
dtype='float32', name='LONGITUDE', length=2879)) Attributes: (46)
platform_code : FINO1 platform_name : data_mode : M title : NWS - NRT in situ Observations summary : Oceanographic data from North West Shelf naming_authority : Copernicus Marine In Situ id : NO_TS_MO_FINO1_202209 wmo_platform_code : 6201065 ices_platform_code : source : mooring source_platform_category_code : 48 institution_edmo_code : 3356 institution : R and D centre Kiel University of Applied Sciences institution_references : www.fh-kiel-gmbh.de/en/ site_code : comment : contact : cmems-service@bsh.de area : North Atlantic Ocean geospatial_lat_min : 54.00000 geospatial_lat_max : 54.00000 geospatial_lon_min : 6.58333 geospatial_lon_max : 6.58333 last_date_observation : 2022-09-30T23:32:00Z last_latitude_observation : 54.00000 last_longitude_observation : 6.58333 geospatial_vertical_min : 0.00000 geospatial_vertical_max : 25.00000 time_coverage_start : 2022-09-01T00:00:00Z time_coverage_end : 2022-09-30T23:32:00Z cdm_data_type : timeSeries data_type : OceanSITES time-series data bottom_depth : format_version : 1.4 Conventions : CF-1.6 Copernicus-InSituTAC-FormatManual-1.42 Copernicus-InSituTAC-SRD-1.5 Copernicus-InSituTAC-ParametersList-3.2.1 netcdf_version : netCDF-4 classic model references : http://marine.copernicus.eu http://www.marineinsitu.eu data_assembly_center : BSH update_interval : PT1H citation : These data were collected and made freely available by the Copernicus project and the programs that contribute to it distribution_statement : These data follow Copernicus standards; they are public and free of charge. User assumes all risk for use of data. User must display citation in any publication or product using data. User must contact PI prior to any commercial use of data. doi : pi_name : qc_manual : OceanSITES User's Manual v1.2 date_update : 2022-10-07T05:39:40Z history : 2022-10-01T18:04:25Z : Creation; 2022-10-07T05:39:40Z, BSH North West Shelf PU: Monthly subset 202209 extracted from history file NO_TS_MO_FINO1.nc wmo_inst_type :
CMEMS in-situ data is provided in a standardised format .
Find out which variables we are interested in to extract:
data = [
{
"name" : fino[var].name,
"standard_name" : fino[var].standard_name,
"units" : fino[var].units,
}
for var in fino.data_vars
if hasattr (fino[var], "units" )
]
pd.DataFrame(data)
0
TEMP
sea_water_temperature
degrees_C
1
VHM0
sea_surface_wave_significant_height
m
2
VZMX
sea_surface_wave_maximum_height
m
3
VTM02
sea_surface_wave_mean_period_from_variance_spe...
s
4
VTPK
sea_surface_wave_period_at_variance_spectral_d...
s
5
VTZA
sea_surface_wave_mean_period
s
6
VPED
sea_surface_wave_from_direction_at_variance_sp...
degree
The data have a DEPTH dimension, even though variables are only measured at a single level and doesn’t vary in time although the format allows for it.
I.e. temperature (TEMP) is available at level 1 (0.5 m)
fino.DEPH.plot.line(x= "TIME" )
fino['TEMP' ].plot.line("-^" ,x= 'TIME' )
fino['VHM0' ].plot.line("-^" ,x= 'TIME' )
Wave data are only available at the surface.
fino[['VHM0' ,'VTZA' ,'VPED' ]].isel(DEPTH= 0 )
<xarray.Dataset> Size: 104kB
Dimensions: (TIME: 2879)
Coordinates:
* TIME (TIME) datetime64[ns] 23kB 2022-09-01 ... 2022-09-30T23:31:59.99...
DEPH (TIME) float32 12kB ...
Data variables:
VHM0 (TIME) float64 23kB ...
VTZA (TIME) float64 23kB ...
VPED (TIME) float64 23kB ...
Attributes: (12/46)
platform_code: FINO1
platform_name:
data_mode: M
title: NWS - NRT in situ Observations
summary: Oceanographic data from North West Shelf
naming_authority: Copernicus Marine In Situ
... ...
doi:
pi_name:
qc_manual: OceanSITES User's Manual v1.2
date_update: 2022-10-07T05:39:40Z
history: 2022-10-01T18:04:25Z : Creation; 2022-10-...
wmo_inst_type: Dimensions:
Coordinates: (2)
Data variables: (3)
Indexes: (1)
PandasIndex
PandasIndex(DatetimeIndex([ '2022-09-01 00:00:00',
'2022-09-01 00:02:00',
'2022-09-01 00:30:00',
'2022-09-01 00:32:00',
'2022-09-01 01:00:00',
'2022-09-01 01:01:59.999999744',
'2022-09-01 01:30:00',
'2022-09-01 01:32:00',
'2022-09-01 02:00:00',
'2022-09-01 02:02:00',
...
'2022-09-30 21:30:00',
'2022-09-30 21:32:00',
'2022-09-30 22:00:00',
'2022-09-30 22:01:59.999999744',
'2022-09-30 22:30:00',
'2022-09-30 22:32:00',
'2022-09-30 23:00:00',
'2022-09-30 23:02:00',
'2022-09-30 23:30:00',
'2022-09-30 23:31:59.999999744'],
dtype='datetime64[ns]', name='TIME', length=2879, freq=None)) Attributes: (46)
platform_code : FINO1 platform_name : data_mode : M title : NWS - NRT in situ Observations summary : Oceanographic data from North West Shelf naming_authority : Copernicus Marine In Situ id : NO_TS_MO_FINO1_202209 wmo_platform_code : 6201065 ices_platform_code : source : mooring source_platform_category_code : 48 institution_edmo_code : 3356 institution : R and D centre Kiel University of Applied Sciences institution_references : www.fh-kiel-gmbh.de/en/ site_code : comment : contact : cmems-service@bsh.de area : North Atlantic Ocean geospatial_lat_min : 54.00000 geospatial_lat_max : 54.00000 geospatial_lon_min : 6.58333 geospatial_lon_max : 6.58333 last_date_observation : 2022-09-30T23:32:00Z last_latitude_observation : 54.00000 last_longitude_observation : 6.58333 geospatial_vertical_min : 0.00000 geospatial_vertical_max : 25.00000 time_coverage_start : 2022-09-01T00:00:00Z time_coverage_end : 2022-09-30T23:32:00Z cdm_data_type : timeSeries data_type : OceanSITES time-series data bottom_depth : format_version : 1.4 Conventions : CF-1.6 Copernicus-InSituTAC-FormatManual-1.42 Copernicus-InSituTAC-SRD-1.5 Copernicus-InSituTAC-ParametersList-3.2.1 netcdf_version : netCDF-4 classic model references : http://marine.copernicus.eu http://www.marineinsitu.eu data_assembly_center : BSH update_interval : PT1H citation : These data were collected and made freely available by the Copernicus project and the programs that contribute to it distribution_statement : These data follow Copernicus standards; they are public and free of charge. User assumes all risk for use of data. User must display citation in any publication or product using data. User must contact PI prior to any commercial use of data. doi : pi_name : qc_manual : OceanSITES User's Manual v1.2 date_update : 2022-10-07T05:39:40Z history : 2022-10-01T18:04:25Z : Creation; 2022-10-07T05:39:40Z, BSH North West Shelf PU: Monthly subset 202209 extracted from history file NO_TS_MO_FINO1.nc wmo_inst_type :
df = fino[['VHM0' ,'VTZA' ,'VPED' ]].isel(DEPTH= 0 ).to_dataframe()
The data are stored on the concurrent timesteps.
df[['VHM0' ,'VTZA' ,'VPED' ]].head()
TIME
2022-09-01 00:00:00
NaN
4.12
NaN
2022-09-01 00:02:00
1.11
NaN
2.800000
2022-09-01 00:30:00
NaN
4.18
NaN
2022-09-01 00:32:00
1.09
NaN
353.000017
2022-09-01 01:00:00
NaN
4.00
NaN
df[['VHM0' ,'VTZA' ]].plot(style= '+' )
Convert the wave height data to a mikeio dataset.
ds = mikeio.from_pandas(
df[["VHM0" ]].dropna(), items= mikeio.ItemInfo(mikeio.EUMType.Significant_wave_height)
)
ds
<mikeio.Dataset>
dims: (time:1439)
time: 2022-09-01 00:02:00 - 2022-09-30 23:31:59.999999744 (1439 non-equidistant records)
geometry: GeometryUndefined()
items:
0: VHM0 <Significant wave height> (meter)
Store the results in Dfs0 format.
ds.to_dfs("FINO1_VHM0.dfs0" )
Read the file again to check…
ds = mikeio.read("FINO1_VHM0.dfs0" )
ds
<mikeio.Dataset>
dims: (time:1439)
time: 2022-09-01 00:02:00 - 2022-09-30 23:32:00 (1439 non-equidistant records)
geometry: GeometryUndefined()
items:
0: VHM0 <Significant wave height> (meter)