Spatial chunking strategies for unstructured mesh data

Type of project

B.Sc., M.Sc.

Contact

Henrik Andersson (jan@dhigroup.com)

Topic

Zarr is a modern specification for large N-dimensional typed arrays.

A key component to enable fast subsetting in any dimension is to use store the data in chunks.

For time or cartesian grids, it is fairly easy to design an optimal chunking strategy based on typical usage patterns.

For data from MIKE 21/3 using unstructured meshes in the horizontal plane and optionally with a structured vertical discretization, finding an optimal chunking strategy is more challenging.

Focus is on enabling fast data access for e.g. model validation comparison with observations or downscaling.

Relevance to DHI

Spatial subsetting is a constant headache with large data. Being able to efficiently extracts subsets makes data management much easier.

What is the student's contribution

Develop an algorithm for efficient storage of unstructured mesh data.