From our paper on the Application Usability Level Framework in the Journal Space Weather Space Climate.
- Application – A specific use for a project, such as a data product from a mission, a service such as satellite hardware anomaly assessments, or a forecast of a specific quantity from a numerical model. Each application has its own unique requirements and metrics for validation. For instance, an application may be forecasts of surface charging events that have an 85% success rate and less than a 5% rate of false alarms.
- Applied Research – Research pursued with a focus on providing a practical application or new technology.
- AUL – The Application Usability Levels (AULs) are the scale that tracks the progress of work on a given project for a specific application, as summarized in Figure 1 and Table 1. More details about the three phases and nine levels of the AUL framework are found in Section 3.
- Feasibility – The ability to achieve success with the available resources.
- Metric – A quantitative measure of project or application performance. When applied to project progress, this constitutes measures that define whether the project meets its goals and milestones. When applied to applications, metrics consist of quantities appropriate for measuring performance, such as accuracy, bias, or skill score.
- Operations to Research (O2R) – Through the process of transitioning targeted or applied research to operations, new phenomena or discoveries can be found and inform subsequent research projects. This part of the feedback process is referred to as Operations to Research (O2R).
- Operational Environment – The conditions in which the application will be used. For example, a geophysical research model that will be delivered to the Community Coordinated Modeling Center (CCMC) for on-demand runs will define their operation environment as the computer system used by CCMC.
- Phases – AULs are grouped into three phases: 1) discovery and viability; 2) development, testing, and validation; and 3) usability, final validation, and implementation. Each of these phases is discussed in more detail in the following sections and summarized in Figure 1 and Table 1.
- Product – A project outcome that is routinely used to enhance the decision-making process of a user or provide input to another research project or application.
- Project – A research or development initiative designed to make progress towards a single application. Examples of projects include the development or modification of models, new uses for available data, using a data product to improve decision making, and using current knowledge to develop future projects.
- Research to Research (R2R) – A targeted research project where both the “researcher” and “user” are researchers who may be in the same sub-field, or in completely different disciplines.
- Research to Operation (R2O) – A targeted or applied research project which takes a research application and transitions it into the operational environment.
- Requirements – The set of necessary conditions outlined by the user, which may include metrics, time frames, and operational environments that the project must meet for the resulting application to be considered successful.
- Relevant context – The environment in which the project must be validated and verified (e.g., during geomagnetic storm periods or in interplanetary space).
- Targeted Research – Investigations pursued with a specific objective.
- Transition – The process or set of activities that take a product or service from a testing environment and move it to an operational environment.
- User – The anticipated person or group who will make use of or operate the project’s application. This may be another researcher, broker, or industry partner. Other common terms for user, appropriate for different fields, include “end user”, “forecaster”, “customer”, or even “another collaborator”.
- Validation – The determination of the skill of the project’s outputs, quantified by identified metrics for the defined operating environment and relevant context.
- Verification – The determination that the product conforms with the project requirements, as described in the relevant design documents.
- Viability – The project’s value or level of return for the researchers and users.