RAMI Homepage

RAdiation transfer Model Intercomparison



  1. October 2020
    RAMI V
  2. 2009/2015
  3. 2008/2009
  4. 2007
    ROMC
  5. 2005
  6. 2002
  7. 1999

This is the official site of the RAdiation transfer Model Intercomparison (RAMI) initiative. RAMI proposes a mechanism to benchmark models designed to simulate the transfer of radiation at or near the Earth's terrestrial surface, i.e., in plant canopies and over soil surfaces.

As an open-access, on-going activity, RAMI operates in successive phases each one aiming at re-assessing the capability, performance and agreement of the latest generation of radiation transfer (RT) models. This in turn, will lead to model enhancements and further developments that benefit the RT modelling community.

RAMI-V maintains the abstract and actual scene definition of RAMI-IV phase. Additionally, two actual scenes, defined through a semi-parametric (Savanna) and an empirical (Wytham Wood) approaches, were included. Measurements are substantially unchanged with a focus on bidirectional reflectance factor (BRF), albedo in direct and diffuse illumination (DHR and BHR), radiant flux Transmission and Absorption through and below the canopy, and Digital Hemispherical Photography (DHP).

With respect to previous phases the spectral bands are now defined to mimic the measurements of modern Copernicus solar spectrum missions (Sentinel-3 and Sentinel-2), as well as MODIS acquisitions. Additional solar and viewing geometry configurations are adopted from real satellite overpasses, for different seasons and geographical locations.

Experiment definitions and their naming convention, slightly different from previous RAMI phases, result from the combination of a set of "scene architecture", "illumination geometry", "spectral properties" and "virtual measurement" tags (see definitions page). Quite many experiments result from the combination of the various tag, especially for abstract scenes due to the number of geographical locations considered in RAMI-V.
The News page will be constantly updated with information on the status of the fifth phase, also to report on scene bugs, corrigendum, clarifications that may be detected.

So, let start your RAMI-V experience by clicking "The new RAMI-V" link in the menu above!

RAMI evolution

RAMI phases
The figure shows all RAMI phases from the beginning.

The first phase of RAMI (RAMI-1) was launched in 1999. Its prime objective was to document the variability that existed between canopy reflectance models when run under well controlled experimental conditions [Pinty et al.,2001, JGR]. The positive response of the various RAMI-1 participants and the subsequent improvements made to a series of radiative transfer (RT) models promoted the launching of the second phase of RAMI (RAMI-2) in 2002. Here the number of test cases was expanded to focus further on the performance of models dealing with structurally complex 3-D plant environments. RAMI-2 faced an increase in the number of participating models. A better agreement between the model simulations for the structurally simple scenes inherited from RAMI-1 was observed, while it was highlighted the need to reduce the differences between some of the 3-D RT models over complex heterogeneous scenes [Pinty et al., 2004, JGR];

During RAMI-3 (2005) a further increase in the number of participants and experiments, the self-consistency (e.g., energy conservation) together with the absolute and relative performance of RT models were evaluated in detail [Widlowski et al., 2007a, JGR]. In fact, it became possible to demonstrate, for the first time, a general convergence of the whole set of submitted RT simulations (with respect to RAMI-2), and to document an agreement between six of the participating 3-D Monte Carlo RT models, below 1%.

RAMI-IV made use of existing ISO-13528:2005 standard to formalize the evaluation of the RT models’ performances. The pre‐screening of data, the identification of reference solutions, and the choice of proficiency statistics were illustrated with simulation results from the RAMI‐IV “abstract canopy” scenarios. After a series of initial consistency checks, this procedure involved (1) the definition of a tolerance criteria suitable for the determination of proficiency in RT model simulations, (2) the definition of a sufficiently precise reference solution against which the candidate models could be compared, and (3) the selection of appropriate evaluation metrics to quantify the performance of the RT models. For surface albedo simulations in the red and NIR, not a single participant was always matching the GCOS accuracy criteria (3%) for all six of the prescribed canopy architecture types (abstract), unless the required success rate is reduced to even 90%. Some RT models submitted simulation results for less than a quarter of the canopy architectures considered in RAMI-IV. This prevented to define statistically significant conclusions about the model performances for some of the experiments. The simulation performed over the six actual realistic canopies of RAMI-IV, showed much greater variance than those analysed for the abstract canopy scenarios.

Whether the differences observed in the RAMI-IV’s model comparison effort were caused by operator errors/choices or were genuine to the RT formulation/implementation, could not be determined. As a matter of facts, in RAMI-IV some of the more deviating models had never participated in previous phases of RAMI.

RAMI-1 to RAMI-IV showed that the repetition of a given set of experiments in successive intercomparison rounds lead to a gradual improvement of most models. Developers gradually identify and remove model weaknesses and software errors and improve the way the RAMI test cases are implemented in the participating models.

Related model intercomparison and quality assurance activities

IRC-IAMAS

International Radiation Commission of Internation Association of Meteorology and Atmospheric Sciences

I3RC

I3RC project

Intercomparison of 3-D Radiation Codes (I3RC) project.
pilps

PILPS

the Intercomparison of Land-surface Parameterization Schemes

Continual Intercomparison of Radiation Code

CIRC

Continual Intercomparison of Radiation Codes
Eradiate

Eradiate

A new-generation radiative transfer model for the Earth observation community.

News and events

Changelog update | 13 October 2021
RAMI-V archive: the file RAMI5list.txt containing the list of experiments has been fixed for HET51_WWO_TLS scene as it contained wrong illumination tags.
(md5 sum)
Changelog update | 30 September 2021
A new version of HET51_WWO_TLS scene
A new version (v2) of HET51_WWO_TLS scene has been obtained from the Wytham Woods 3D model: you can find more information here. This archive contains all the files needed to built HET51 version 2 (v2) scene. They can also be accessed individually on the presentation page.
  1. 19 October 2020
    Fifth RAMI phase
  2. 30 November 2021
    End of submission period
  3. January 2022
    RAMI V results will be available

All news and events  


Guidance

Definition

This page contains valuable information regarding the generation of the various scene representations, as well as the way the user simulated results are evaluated.

Formats

This page aims to facilitate the intercomparison of the submitted results files. The RAMI file-naming and formatting conventions outlined on this webpage must be adhered to.

Organization

This particular page provides information on the rationale behind the RAMI exercise, its functional organisation and data-usage policy.