RAMI3 phase
For the purpose of this RAMI exercise, an environment is deemed homogeneous if its properties can be considered independent of geographical location (horizontal spatial coordinates). Two types of spectral scenarios for the radiation transfer simulations are to be considered:
The third phase of RAMI asked for a re-run of the homogeneous experiments of the first two phases. These included simulations with spectral properties typical of the red, near-infrared (NIR) and conservative scattering conditions.
In absence of any absolute reference truth the performance of RT models can be expressed in terms of their deviation from the mean of simulation results from all other models (Nc) that performed the same experiment. For any spectral (lambda), structural (zeta), illumination(Omega_i) and viewing (Omega_v) conditions one can thus compute the model-to-ensemble deviation of some model m (in [%]):
Averaging the above (delta_m) model-to-ensemble indicator over appropriate sets (N bar) of spectral, structural, viewing and illumination conditions thus yields an integral estimator of how close a model was
with respect to all other simulations for a given set of modelling conditions (delta bar):
The following Figure provides information as to the typical model-to-ensemble deviation of the various models participating in simulations the for structurally homogeneous canopies with finite sized foliage (HOM Discrete) and turbid medium foliage (HOM Turbid) representations. Green (red) values indicate models that are in good (poor) agreement with all other models, grey values indicate missing or incomplete data. Ideally there should be no grey colour in the graphs below.
Models that do not participate in all of the prescribed test cases can only be evaluated in a partial manner at best.
Rather than integrating all individual delta_m values, one may bin them instead to generate a histogram of typical differences from the ensemble mean. The following Figure displays such histogramms for each of the participating models in the solar domain homogeneous test cases with discrete scatterers. Indicated are histogrammes based on delta_m's for the total BRFs (top left), the single uncollided (bottom left), the single collided (bottom right) and the multiple collided (top right) BRF components:
The next figure displays the enveloppe of all the above histogrammes for the total BRF as well as each of its three BRF componentns:
