How to participate

Information for participants in RAMI V phase


This page contains valuable information regarding the participation of the fifth phase of RAMI.

What is a RAMI phase?

The expression 'RAMI phase' refers to a time period during which the radiation transfer within an ensemble of predefined test cases (i.e., scenarios where both the illumination conditions and the structural and spectral properties of the illuminated geophysical environment are prescribed) has to be simulated by the qualified RAMI participants (using their models), and the results of these simulations must be submitted for subsequent analysis. A RAMI phase is declared closed when all results have been made public, initially and in a more concise form via presentations on conferences as well as a publication in the refereed scientific literature, and ultimately and more comprehensively by using this web site. Conceptually a 'RAMI phase' can thus be divided into:

  • The launch period, during which the RAMI coordinators decide upon what test cases and measurements to incorporate into the next phase of RAMI. For each of these test cases, the proper documentation will then be placed on the RAMI web site. The final step being the announcement of the launch of a new phase of RAMI via email broadcast to the radiation transfer modeling community.
  • The active submission period, during which the RAMI participants visit the RAMI web site in order to obtain the necessary specification for the test cases to be performed during that phase. This information then leads to the implementation of the test cases within the models of the participants, as well as the subsequent generation of the required measurements and their submission to a location specified on the RAMI web site. The earlier the submission of a participant's results, the greater the chance that obvious mistakes can be identified and he/she be able to re-submit a new set of results prior to the closing date of the active submission period.
  • The results analysis and publication period, during which the RAMI coordinators analyse the submitted results and report files from all participants, and produce a panoply of graphical results and statistics files, some of which will be placed onto the RAMI web site. In difficult or controversial cases, the RAMI coordinators may seek a decision from the RAMI advisory body, which among others, sets and maintains the overall work ethics of the RAMI initiative. Last but not least, a paper with the most important results may also be submitted to a scientific journal.
placeholder image
A RAMI phase.

How can I participate with my model(s) in a RAMI phase?

Participation in the RAMI initiative is open to everyone subject to the criteria listed in the role, rights and responsibilities of the RAMI qualified participants section of the Guidance section. Please note, however that only models with traceable publications in the refereed scientific literature may feature in any resulting RAMI publication. Other models may still use RAMI as a testbed to evaluate their performance through comparison with graphical results files where available. Note that our privacy and data-usage policy does not allow the distribution of data submitted by individual RAMI participants. Upon decision to participate in a RAMI phase, it would be courteous to inform the RAMI coordinator of your intent.

All information regarding the structural and spectral properties of the various geophysical environments (scenes), their illumination conditions and associated measurements can be accessed via the various links available at the main level of that phase. When implementing the structural scenario of a given 3-D experiment, you should aim to stick as close as possible to the information provided. Obviously different models have different RT formulations and also scene representation capabilities. This is why - in the discrete cases - we provided large text files containing the location and orientation of every leaf in the scene. Obviously such information would be of no relevance for some geometric optical models that may need a description of the envelope of the foliage volumes instead. In any case, we hope that you will find sufficient information on the various experiment description pages that allow your model to generate geophysical environments that are quite similar to those proposed for a given phase. If you feel, however, that some information is missing please do not hesitate in sending an email to us. We also suggest that you note any approximations (or better still a short description of the functioning of your model) in the report file that you will submit with your results files later on. Please make also sure that you utilize the proper definitions regarding angular sign conventions, leaf normal distributions, and other RT model technicalities prior to starting your simulations (in particular the reference level to which the RT model simulations refer to). Also read the relevant file naming and formatting conventions that must be adhered to by all participants. Again, in case of doubts or problems please do not hesitate in contacting us.

How to create an account in to the EU Login system?

Go to ECAS: create an account webpage to create a new account.

Enter all information needed and click on .

You should receive an e-mail allowing you to confirm the registration: follow instructions to complete the registration process. You are now registered to ECAS authentication website.

Consult the help for external users here.

How to sign in to the EU Login system?

Go to ECAS authentication website and open the EU Login - Sign in page.
Enter your email address and click 'Next', enter your EU Login password and click 'Sign in'.
You are now connected.

How can I check a result?

Please, go to RAMI On-Line Format Checker webpage to check one sample output file of your model for each of the recommended RAMI measurements for a given phase.

RAMI On-line format checker example page.
RAMI On-line format checker example page.

'Drag and Drop' or select a valid .mes file to check it.

Once you have checked the correctness of the filenames and formats of your model outputs (in the sense of the RAMI file-naming and formatting specifications), you are now ready to submit your results.

How can I submit my model?

Upon completion of your model simulations, you are strongly encouraged to perform an on-line format check, to see if your model has produced correctly named and formatted result-files (ideally you need to check only one sample output file of your model for each of the recommended RAMI measurements for a given phase).

Please, go to Model Submission webpage to submit your model and your results.

RAMI On-line Model Subsmission example page.
RAMI On-line Model Subsmission example page.

Notice that this link will only be visible during the active submission period of a given RAMI phase (i.e., the period of time for which the submission of simulation results is valid).

After login to the EU authentication webpage, please go to the RAMI On-line Model Submission and follow instruction to submit your results.

Participants who do not perform the mandatory format on typical output files risk that their submitted results files will be rejected for reasons of incorrect naming and / or formatting. In this case, the participant will have to engage in the correction and re-submission of files (sometimes several hundred data) if he/she wishes to involve his/her model. To avoid this last minute stress, we encourage to use the online format checker to ensure that the results follow both RAMI formatting and file naming conventions before formally submitting all of the results. The participants will be contacted by the RAMI coordinators to acknowledge receipt of their data files.

How many of the prescribed scenes do I need to generate?

The simple answer to this question is: all of those that your model is capable of simulating. Within RAMI, the performance of each model is described by comparing its output to those of all (mutually different) models that completed the same experiment i.e., the prescribed set of measurements on a given scene structure. Exhaustive testing of your model in all feasible scene structures is thus strongly encouraged, as this effort will provide you with the best diagnostics of the performance of your model. However, given the various degrees of complexity that exist between different radiation transfer models, RAMI experiments were subdivided into two sets of test cases: structurally homogeneous environments that can be represented by 1-D and 3-D RT models, and structurally heterogeneous environments that can only be represented with 3-D RT models. One-dimensional RT models thus need to perform only those test cases prescribed under homogeneous environments, whereas three-dimensional RT models need to perform both the experiments prescribed under structurally homogeneous and heterogeneous environments. In fact it is strongly recommended that 3-D models perform the proposed set of homogeneous experiments. This will allow to ascertain their mutual consistency prior to the launching of RT simulations in environments with increased structural complexity. Similarly before addressing highly specialized simulations within heterogeneous domains, it may be advantageous to first perform the various baseline scenarios in the heterogeneous category, that is the ''floating spheres'' experiments. In general, RAMI participants should aim at performing all of the prescribed test cases (within the applicability of their model), since the behaviour of any given RT model may depend (sometimes very strongly) on the kind of structural and also spectral conditions that are encountered. In this context particular attention should also be given to the various purist corner scenarios.

How many of the prescribed measurements do I need to perform?

Again, the simple answer to this question is: all of those that your model is capable of simulating. As was already mentioned previously, within RAMI, the performance of a model is described with respect to as comprehensive as possible an ensemble of structural and spectral scenarios, and this over multiple types of measurements. Participation in as large as possible a number of measurements is thus a necessity for the comprehensive evaluation of a particular model. Spectral albedo, for example, is of interest since it may give an idea of the absolute accuracy of a model simulation under conservative scattering conditions (purist corner). Similarly the collided, uncollided and multiple collided BRF components can help identify issues in the mathematical formulation within computer codes that otherwise may remain untraceable from total BRF simulations alone. Simulations in the principal plane may point to issues regarding the hotspot formulations. In general, the more simulations are available from any given model the more confidence can be placed in its overall evaluation.

What is the privacy and data usage policy of RAMI?

All data and information submitted by a RAMI participant belongs to the RAMI coordinators for the specific and exclusive purpose of running this benchmarking exercise. The RAMI coordinators will not distribute results or report files that were submitted by its participants. Further information about our privacy and data-usage policy can be found here.

Can I participate with multiple RT models?

The participation with multiple models is generally permissible. This, however, brings up the question as to when a model is in fact a new model, rather than a replacement of an older version of the same model (for example due to enhanced computing powers, etc). If multiple models are at the disposition of a given participant, and if these models can all be traced to some common ancestor model, then the interested participant should decide (for the benefit of the scientific community) which one of these is the most suited one to satisfactorily complete all the prescribed test cases and measurements within RAMI. Failure to do so may prompt the RAMI coordinators to exclude some of the submitted RT models from participation in any subsequent results analysis - in particular if the output of these models are very similar, and their differences can be traced to slight modifications of some common ancestor model that do not impact significantly on a better representation the physics of radiative transfer contained within them. It is thus recommended to participate only with one RT model - unless significant methodological differences exist between the models that are to be submitted by the same participant.

What if other RAMI participants use a similar version of a publicly available RT model than myself?

Each model can participate only once to each exercise of a given phase. In other words, multiple participants cannot simultaneously submit results obtained from the same or similar models. Prospective participants should contact the RAMI coordinators in case of doubt. For more details on the role, rights and responsibilities of the RAMI qualified participants, click here.

How can I contact the RAMI coordinators?

The RAMI coordinator can be contacted with the following e-mail.