The third phase of RAMI asked for a re-run of the heterogeneous experiments of the first two phases (Floating spheres, Real Zoom-in and Conifer Forest with topography). In addition a conifer forest experiments without topography, a birch stand experiment and a Floating spheres run under conservative scattering conditions were requested. Access to the results from these experiments is provided via the links below:

In absence of any absolute reference truth the performance of RT models can be expressed in terms of their deviation from the mean of simulation results from all other models (Nc) that performed the same experiment. For any spectral (lambda), structural (zeta), illumination(Omega_i) and viewing (Omega_v) conditions one can thus compute the model-to-ensemble deviation of some model m (in [%]):

Averaging the above (delta_m) model-to-ensemble indicator over appropriate sets (N bar) of spectral, structural, viewing and illumination conditions thus yields an integral estimator of how close a model was with respect to all other simulations for a given set of modelling conditions (delta bar):

The following Figure provides information as to the typical model-to-ensemble deviation of the various models that participated in simulations for 1) the birch stand, 2) the true zoom, 3) the coniferous forest without topography (Conifer-Topo) and with topography (Conifer+Topo), and the floating spheres scenes with discrete foliage (DIS Spheres) and turbid medium foliage (TUR Spheres) experiments. Green (red) values indicate models that are in good (poor) agreement with all other models, grey values indicate missing or incomplete data. Ideally there should be no grey colour in the graphs below. Models that do not participate in all of the prescribed test cases can only be evaluated in a partial manner at best.