Information for participants in RAMI phases
This page contains valuable information regarding the participation of RAMI phases.
The expression 'RAMI phase' refers to a time period during which the radiation transfer within an ensemble of predefined test cases (i.e., scenarios where both the illumination conditions and the structural and spectral properties of the illuminated geophysical environment are prescribed) has to be simulated by the qualified RAMI participants (using their models), and the results of these simulations must be submitted for subsequent analysis. A RAMI phase is declared closed when all results have been made public, initially and in a more concise form via presentations on conferences as well as a publication in the refereed scientific literature, and ultimately and more comprehensively by using this web site. Conceptually a 'RAMI phase' can thus be divided into:
Participation in the RAMI initiative is open to everyone subject to the criteria listed in the role, rights and responsibilities of the RAMI qualified participants section of the Guidance section. Please note, however that only models with traceable publications in the refereed scientific literature may feature in any resulting RAMI publication. Other models may still use RAMI as a testbed to evaluate their performance through comparison with graphical results files where available. Note that our privacy and data-usage policy does not allow the distribution of data submitted by individual RAMI participants. Upon decision to participate in a RAMI phase, it would be courteous to inform the RAMI coordinator of your intent.
All information regarding the structural and spectral properties of the various geophysical environments (scenes), their illumination conditions and associated measurements can be accessed via the various links available at the main level of that phase. When implementing the structural scenario of a given 3-D experiment, you should aim to stick as close as possible to the information provided. Obviously different models have different RT formulations and also scene representation capabilities. This is why - in the discrete cases - we provided large text files containing the location and orientation of every leaf in the scene. Obviously such information would be of no relevance for some geometric optical models that may need a description of the envelope of the foliage volumes instead. In any case, we hope that you will find sufficient information on the various experiment description pages that allow your model to generate geophysical environments that are quite similar to those proposed for a given phase. If you feel, however, that some information is missing please do not hesitate in sending an email to us. We also suggest that you note any approximations (or better still a short description of the functioning of your model) in the report file that you will submit with your results files later on. Please make also sure that you utilize the proper definitions regarding angular sign conventions, leaf normal distributions, and other RT model technicalities prior to starting your simulations (in particular the reference level to which the RT model simulations refer to). Also read the relevant File format that must be adhered to by all participants. Again, in case of doubts or problems please do not hesitate in contacting us.
Go to ECAS: create an account webpage to create a new account.
Enter all information needed and click on 
.
You should receive an e-mail allowing you to confirm the registration: follow instructions to complete the registration process. You are now registered to ECAS authentication website.
Consult the help for external users here.
Go to ECAS authentication website and open the EU Login - Sign in page.
Enter your email address and click 'Next', enter your EU Login password and click 'Sign in'.
You are now connected.
Once the ECAS authentication is completed, please go to the RAMI On-line Model Submission link under the selected phase menu and follow instructions to register a model:
After clicking on Register a Model button, an email will be sent to the RAMI team in order to verify and validate your model request: you will receive in your mailbox a confirmation message when the model has been successfully created or rejected.
The model’s name must follow the rules defined in the specific phase. In RAMI4ATM, any model name should be tagged with the name of the user institution and, in case of different parametrization/configuration of the same model, with an increasing numbering (e.g. model[-institute-[01,02,…]]). The name of the institution is optional if the participant is the model owner/developer.
A different Model name could be suggested to avoid duplication and agree on the better name selection.
Please contact the RAMI coordinators in case of doubts.
Please, go to RAMI On-Line Format Checker link under the selected phase menu to check one sample output file of your model for each of the recommended RAMI measurements for a given phase.
'Drag and Drop' or select a valid .mes file to check it.
Once you have checked the correctness of the filenames and formats of your model outputs (in the sense of the RAMI file-naming and formatting specifications), you are now ready to submit your results.
Upon completion of your model simulations, you are strongly encouraged to perform an on-line format check (ROFC), to see if you reformatted your model outputs accordingly to the filename and format defined for a specific RAMI phase (ideally you need to check only one sample output file of your model for each of the recommended RAMI measurements for a given phase).
Notice that this link will only be visible during the active submission period of a given RAMI phase (i.e., the period of time for which the submission of simulation results is valid).
Please go to the RAMI On-line Model Submission link under the selected phase menu to submit your results:
On the right column, select one or more files using the Browse button or the "drag and drop" functionality: confirm your results just by clicking on Update Model results button.
Participants who do not perform the mandatory format on typical output files risk that their submitted results files will be rejected for reasons of incorrect naming and / or formatting. In this case, the participant will have to engage in the correction and re-submission of files (sometimes several hundred data) if he/she wishes to involve his/her model. To avoid this last minute stress, we encourage to use the RAMI On-Line Format Checker to ensure that the results follow both RAMI formatting and file naming conventions before formally submitting all of the results. The participants will be contacted by the RAMI coordinators to acknowledge receipt of their data files.
The simple answer to this question is: all of those that your model is capable of simulating. Within RAMI, the performance of each model is described by comparing its output to those of all (mutually different) models that completed the same experiment i.e., the prescribed set of measurements on a given scene structure. Exhaustive testing of your model in all feasible scene structures is thus strongly encouraged, as this effort will provide you with the best diagnostics of the performance of your model. However, given the various degrees of complexity that exist between different radiation transfer models, RAMI experiments were subdivided into two sets of test cases: structurally homogeneous environments that can be represented by 1-D and 3-D RT models, and structurally heterogeneous environments that can only be represented with 3-D RT models. One-dimensional RT models thus need to perform only those test cases prescribed under homogeneous environments, whereas three-dimensional RT models need to perform both the experiments prescribed under structurally homogeneous and heterogeneous environments. In fact it is strongly recommended that 3-D models perform the proposed set of homogeneous experiments. This will allow to ascertain their mutual consistency prior to the launching of RT simulations in environments with increased structural complexity. Similarly before addressing highly specialized simulations within heterogeneous domains, it may be advantageous to first perform the various baseline scenarios in the heterogeneous category, that is the ''floating spheres'' experiments. In general, RAMI participants should aim at performing all of the prescribed test cases (within the applicability of their model), since the behaviour of any given RT model may depend (sometimes very strongly) on the kind of structural and also spectral conditions that are encountered. In this context particular attention should also be given to the various purist corner scenarios.
Again, the simple answer to this question is: all of those that your model is capable of simulating. As was already mentioned previously, within RAMI, the performance of a model is described with respect to as comprehensive as possible an ensemble of structural and spectral scenarios, and this over multiple types of measurements. Participation in as large as possible a number of measurements is thus a necessity for the comprehensive evaluation of a particular model. Spectral albedo, for example, is of interest since it may give an idea of the absolute accuracy of a model simulation under conservative scattering conditions (purist corner). Similarly the collided, uncollided and multiple collided BRF components can help identify issues in the mathematical formulation within computer codes that otherwise may remain untraceable from total BRF simulations alone. Simulations in the principal plane may point to issues regarding the hotspot formulations. In general, the more simulations are available from any given model the more confidence can be placed in its overall evaluation.
All data and information submitted by a RAMI participant belongs to the RAMI coordinators for the specific and exclusive purpose of running this benchmarking exercise. The RAMI coordinators will not distribute results or report files that were submitted by its participants. Further information about our privacy and data-usage policy can be found here.
The participation with multiple models is generally permissible.
If multiple models are at the disposition of a given participant, and if these models can all be traced to some common ancestor model, then the interested participant should decide (for the benefit of the scientific community) which one of these is the most suited one to satisfactorily complete the prescribed experiments of RAMI. Failure to do so may prompt the RAMI coordinators to exclude some of the submitted RT models from participation in any subsequent results analysis - in particular if the output of these models are very similar, and their differences can be traced to slight modifications of some common ancestor model that do not impact significantly on a better representation the physics of radiative transfer contained within them. It is thus recommended to participate only with one RT model - unless significant methodological differences exist between the models that are to be submitted by the same participant.
Each model can participate only once to each exercise of a given phase, except for RAMI4ATM. In RAMI4ATM phase in fact RAMI would investigate also any differences between results obtained by different groups using the same model, and even further different parametrization of the same model by the same group (for example: using a built-in aerosol model or an ad-hoc aerosol model in simulations). For this reason the naming convetion of the model in RAMI4ATM is extended to include the institute name and optional flags to identify a specific parametrization (see How can I register a model?) Prospective participants should contact the RAMI coordinators in case of doubt. For more details on the role, rights and responsibilities of the RAMI qualified participants, click here.
The RAMI coordinator can be contacted with the following e-mail.
