Getting started#

MIdAS (MultI-scale bias AdjuStment) is developed as a tool to run various bias-adjustment methods at different scales. Currently, it features a non-parametric quantile-mapping method. It performs bias adjustment on earth science data that can be output from climate models, NWP models, observations, etc.

In all cases, we are looking at timeseries of data that should be adjusted such that it satisfies certain characteristics of a similar dataset from a different source, which is referred to as reference. The dataset to be adjusted is here referred to as model, because it usually is the output from some numerical models.

The characteristics that we try to match are typically statistical in nature, such as the mean or a set of quantiles.

Get MIdAS source code#

The MIdAS source code can be downloaded from the GitLab repository.

You can clone the public repository:

    $ git clone git@git.smhi.se:midas/midas.git

or

    $ git clone https://git.smhi.se/midas/midas.git

Create virtuell environment#

To install MIdAS, the Conda package manager or its faster sibling Mamba can be used. If you don’t have a version of Conda or Mamba, the best way to get started is by installing Miniforge, an installer for Mamba that will pre-configure your installation.

Tip

If you prefer to use Conda instead of Mamba, for example because this has been pre-installed for you, just replace mamba with conda in the following commands.

Create a new environment, my-midas, by:

mamba env create -n my-midas -f environment.yml

To use MIdAS at any time, you need to make sure that the my-midas environment is activated. To do that, execute:

mamba activate my-midas

Install MIdAS#

To install MIdAS, you need to navigate to the root folder and have conda environment, my-midas, activated. MIdAS can then be installed in two different ways.

To install a stable version of MIdAS, run this command in your terminal:

    $ pip install .

If you are planning to make changes to the MIdAS code, you can use the -e flag to make an editable install. Run this command in your terminal:

    $ pip install -e .

The -e makes this an “editable” installation, meaning that any change in the source code immediately affects the installation, as the source code is merely linked into the python library.

If you don’t have pip installed, this Python installation guide can guide you through the process.

Try running MIdAS with the help-option:

midas --help

Run MIdAS#

Following the recommended way for bias correcting e.g., daily precipitation pr, we try to run MIdAS on single node with reference_input as referenc data and model_input as model data. reference_input and model_input are key words to search reference and model data. The reference period, 1980/2000, is specified with the -p option. The exemplary output file,corrected.nc, is given with the -o option. -s or --ssr is needed to activate singularity stochastic removal (SSR) method Vrac et al., 2016 for wet-day adjustments during precipitation bias-adjustment .

midas -t pr \
      -p 1980/2000 \
      -s \
      -r reference_input \
      -m model_input \
      -o corrected.nc

Optional arguments can be found with the help-option midas --help.

MIdAS can also be run on multiple nodes using sbatch. An examplaric runscript, midas.sh can be found at the folder,jobscripts.It cointains more detailed information about the srun input arguments.

Currently tested variables#

The following variables can be bias-adjusted, categorized by the defining distribution.

  • daily precipitation (pr)

  • daily near-surface air temperature (tas)

  • daily relative humidity (hurs)

  • inverse of daily relative humidity (hurs_inv)

  • daily wind speed (sfcWind)

  • daily maximum temperature (tasmax)

  • daily minimum temperature (tasmin)

  • daily surface downwelling shortwave radiation (rsds)

  • daily surface downwelling longwave radiation (rlds)

Standardization includes standardization of units, calendar conversion, interpolation. It is needed for all input variables. It is because MIdAS is built upon python Iris module that is embedded with the NetCDF CF Metadata Conventions.

Depending on the variable to be adjusted, Pre-process and Post-process may be required before and after running the MIdAS. Pre-process refer to preparing intermediate input data so that the meteorological variables of interest are to be bias-corrected in an indirect way and then transformed via Post-process.

Along with long name and units of meteorologial variables, the recommended way to bias-adjust the meteorological variables are summarized in Recommended processing steps. The required pre-process and post-process for specified variables are explained in Indirect bias adjustment.

Note

See Recommended processing steps