RAMI1 phase
For the purpose of this RAMI exercise, an environment is deemed homogeneous if its properties can be considered independent of geographical location (horizontal spatial coordinates). Two types of spectral scenarios for the radiation transfer simulations are to be considered:
Prior to the performing of any RT model simulations, please refer to the 'definitions' pages for detailed instructions regarding the angular sign conventions for BRF simulations, as well as precise definitions of the required leaf normal distributions and other RT model technicalities. Also read the relevant file naming and formatting conventions that must be adhered to by all participants.
A well-designed BRF model intercomparison exercise should ideally yield practical indications on the ability of the modeling community to effectively simulate the reflectance fields of a variety of geophysical scenes. This can be done with different tools, provided they represent the same "reality" in numerically acceptable ways. A common issue arising with inverse problems is the occurrence of multiple solutions, i.e., in the present case, the possibility of defining multiple geophysical scenarios which would be able to explain the observations. This situation arises because either there are not enough measurements, or the observations are not accurate enough to constrain the numerical inversion process, or because the models themselves are incorrect or incomplete. In any case, this issue becomes one of distinguishing between models which generate different results on the basis of limited samples of imperfect data.
To address this issue, it is necessary to develop an approach where the errors of measurements are somehow compared to the variability exhibited by the different models in their representation of reality. A statistical measure of the joint behavior of these models in terms of their capability of representing a sample of data is thus proposed, and it will be seen that, at a given level of accuracy, some models cannot be distinguished while others can be declared to behave differently. One consequence of this approach is that, as measurements improve in accuracy, the differences between models become more noticeable. In a pragmatic sense, differences between RT models matter only to the extent that they exceed the level of uncertainty associated with the measured BRF fields.
The absence of any absolute "truth" renders the exercise more tricky but, nevertheless, the model discernability issue can be addressed by assessing the "most credible solutions".
As a matter of fact, it can reasonably be admitted that the latter correspond to the actual values that could be measured from an instrument with its intrinsic limited accuracy. We attempted to examine the issue of model discernability taking advantage of the fact that, at least in the case of homogeneous scenes, both one and three-dimensional models could be applied. Accordingly, we established for all scenarios, what could be considered as the "most credible solutions" by estimating the arithmetic mean of every BRF value calculated within a subset of the three-dimensional model results.
The model discernability can then be analyzed by comparing the values computed with a normalized Chi-square metric that provides an estimate of the average of the BRF values taken over a subset of three-dimensional RAMI models and the associated value of the variance of the BRF distribution of these latter models, normalized by the number of available cases. In the present analysis of model discernability, we excluded two of the five three-dimensional models that participated in RAMI: the DART and Flight models which were shown to deviate the most for some of the proposed simulation scenarios.
Finally, on the basis of the BRF values generated by the RGM, RAYTRAN and Sprint models, we obtained spectral variances values of 1.6E-03 and 1.0E-02 at the red and near-infrared wavelengths, respectively.
These values correspond approximately to 5% and 2% of the typical BRF values that can be measured over a plant canopy system at the red and near-infrared wavelengths, respectively.
Such values are within the expected range of those to be estimated from the up-coming multi-angular data to be soon gathered in space.
These values were doubled for the purpose of computing the Chi-square metric to take into account the uncertainty associated with the measurement process itself.
The issue of model validation is hotly debated in the professional literature. The following references may be useful as entry points to this question: