# data[].attributes.headline
Is a “Certification” Framework really needed? Why is it needed?

# data[].attributes.subject
Is a “Certification” Framework really needed? Why is it needed?

# data[].attributes.attachment_link.readable_text
The EDAP project has defined a set of 'best practice' guidelines that define a framework for the quality assessments it performs, aligned with the principles of QA4EO. These guidelines are intended to provide a description of the high-level principles and activities that are required in quality assessments for all types of Earth Observation missions, since these are largely common between different domains.

At the top level, these guidelines have been drafted to reflect a near 'ideal' scenario, which is hoped to serve as an aspiration to new space providers as well as space agencies. It is understood that many of the missions the project will assess will only partly comply with these requirements (to different degrees). This is acknowledged in the assessment grading system which is intended to primarily test whether aspects of a given mission are 'fit for purpose' within the context of the mission's stated performance and application.

The product evaluation activity they undertake is divided into six sections:

Details

Generation

Quality Flags

Uncertainty Characterisation

Validation

Inter-comparison

These sections are themselves divided into sub-sections, which constitute each of the different aspects of the data product that should be assessed and graded, either as Basic, Intermediate, Good or Excellent.

The information is gathered in a Quality Assurance Report (QAR) and summarised in a product quality evaluation matrix.

EDAP Product Quality Evaluation Matrix - example

Product Details
Certain basic descriptive information should be provided with any EO data product, some of which is absolutely necessary for the data to be at all meaningful. Additionally, some of this information, such as claimed measurement quality or resolution, can provide both the data product quality assessor and the reader of their report with a frame of reference from which to set expectations for a given product.

It is therefore required by the EDAP assessment that any EO product provides the following:

Product name

Sensor Name

Sensor Type

Product version number

Product ID

Processing level of product

Measured quantity name

Measured quantity units

Stated measurement quality

Spatial Resolution

Spatial Coverage

Temporal Resolution

Temporal Coverage

Mission coverage

Also recommended is the following (based on INSPIRE metadata):

Point of contact (Responsible organisation, including email address)

Product locator (e.g. URL, DOI if applicable)

Conditions for access and use

Limitation on public access

Product abstract (summary of resource)

Not Assessed

Assessment outside of the scope of study

Not Assessable

Relevant information not made available

Basic

Many pieces of important information missing

Good

Some pieces of important information missing

Excellent

Almost all required information available

Ideal

All required information available

Uncertainty Characterisation
Uncertainty values should be provided in EO data products per-pixel, in a manner that describes the pixel error-covariance. Since it is not practical to provide a full error-covariance matrix for an EO data product due to their data volume various approaches have been developed to approximate this. For example, the FIDUCEO project FCDRs contain three components of uncertainty - independent, structured and common - to describe the three typical scales of error correlation.

It is still typical however for uncertainty values to be provided, if at all, on a per-product or, more often, a per-mission basis - losing a great deal of information significant to users.

Not Assessed

Assessment outside the scope of study

Not Assessable

No uncertainty information provided

Basic

Uncertainty established by limited comparison to measurements by other sensor/s

Good

Limited use of GUM approach, and/or, an expanded comparison to measurements by other sensors. Most important sources of uncertainty are included

Excellent

Full GUM approach is used to estimate measurement uncertainty, all important sources of uncertainty are included. Uncertainty per pixel provided

Ideal

Full GUM approach is used to estimate measurement uncertainty, including a treatment of error-covariance. Per pixel uncertainties in components, e.g., random systematic - as appropriate for the error-correlation structure of the data

Download the Best Practice Guidelines with detailed focus on Optical Missions and a template for the Mission Quality Assessment Report:

For information purposes only, previous versions used by old assessment may also be downloaded:

Version 2.0 of the Best Practice Guidelines provides an updated framework, where the reporting divides the assessment results between two Cal/Val maturity matrices, as follows:

Summary Cal/Val Maturity Matrix

Validation Cal/Val Maturity Matrix

A useful overview of the QA Framework produced described above, which has been further developed alongside NASA, may be found in the below presentation.
Framework for EO Data Quality Assessment
A generalised framework for assessing the fitness of purpose of EO data is difficult to obtain as this would clearly depend on the users intent; however, there are key characteristics that give confidence to a user and allow them to decide if the data is fit for their individual purpose.

In particular, users wish to understand data coverage, data provenance, data characterisation, data availability and also data correctness. Each of these factors, given a different weighting by each user, can ultimately affect a user's opinion of how good the data is.

EDAP aims to address both data integrity and data credibility for the various missions identified, which may go on to become TPMs within ESA's Earthnet Programme.

QA4EO
The Quality Assurance Framework for Earth Observation (QA4EO) was created and endorsed by CEOS explicitly for the purpose of providing guidance on EO quality assurance so that data users are able to trust the data / information they are provided with is fit for purpose and fully interoperable with other datasets from both satellite and in-situ providers.

As there is potential for some commercial missions to become TPM candidates, for the most efficient exploitation of EO data, assessment of data quality, calibration and validation are indispensable tasks and also form the basis for reliable scientific conclusions. It is for this reason that one of the core functions provided by EDAP is focused on data quality, with an emphasis on the adoption of QA4EO guidelines.

# data[].attributes.attachment_link.description
The ESA Earthnet Data Assessment Pilot (EDAP) project will perform assessments for various missions to ensure the delivered data is fit for purpose.

# data[].attributes.body
You are invited to gather online to participate and contribute by sharing ideas, asking questions, providing answers, and exchanging thoughts. This space is designed to foster active engagement and collaboration in support of VH-RODA and the scientific community. Seed questions will be shared by the committee to trigger the discussion and ideas exchanges.

# data[].attributes.headline
Is the term “Certification” appropriate?

# data[].attributes.subject
Is the term “Certification” appropriate?

# data[].attributes.body
Do you have a better alternative?

# data[].attributes.headline
Would you like to see more training sessions at VH-RODA?

# data[].attributes.subject
Would you like to see more training sessions at VH-RODA?

# data[].attributes.body
What kind of training sessions are you interested in?

# data[].attributes.body
The 2024 VH-RODA workshop provides an open forum (for the new space, commercial and institutional space sectors) for presenting and discussing the current status and future developments related to Earth Observation data quality, calibration and validation of space-borne very high-resolution SAR and Optical sensors and data products.

# data[].attributes.attachment_link.readable_text
The Very High-resolution Radar & Optical Data Assessment (VH-RODA) 2024 Workshop will be held in ESA-ESRIN, Frascati (Italy), from 2-6 December 2024.

The 5th Edition of the VH-RODA Workshop will provide an open forum (for the new space, commercial and institutional space sectors) to present and discuss about the current status and future developments related to Earth Observation (EO) data quality, calibration and validation of space-borne very high-resolution SAR, Optical, and, for the first time, atmospheric constellations, sensors and data products, with a dedicated focus on commercial EO data providers and related Cal/Val activities, synergies between optical, SAR and atmospheric communities, presentation of standards and best practices for data quality.

Calibration Techniques (requirements, definitions, database, methodologies)

Calibration Sites (cross-cal/val, intercalibration, field campaigns)

Fiducial Reference Measurements

Analysis Ready Data, Digital Elevation Models

Quality Control, Best Practices, Product Validation

Processing and Algorithms (including Artificial Intelligence for Cal/Val)

Cal/Val and Data quality for Constellations and Big Data

Calibration of Future Missions (Innovative instrument concepts)

Atmospheric composition and reference sites

Details

Reference Date(s)

Registration and Abstract submission openings

3 June 2024

Vendors Exhibit submission opening

3 June 2024

Vendors Exhibit & Abstract submission deadline

25 September 2024

Acceptance notification

18 October 2024

Registration deadline

14 November 2024

VH-RODA 2024 Workshop

2-6 December 2024

ESA's Centre for Earth Observation (ESRIN) will host the conference. It is located in Frascati, a small town 20km south-east of Rome in Italy.

Please click here for information about how to travel to ESRIN.

Location Address:

ESA-ESRIN
Largo Galileo Galilei, 1
00044 Frascati (RM), Italy

Please find here a list of hotels in Frascati centre recommended by ESA:

Hotel Antica Colonia
Hotel Bellavista
Hotel Cacciani
Hotel Colonna
Hotel Flora
Hotel Villa Mercede
Hotel Villa Tuscolana
Workshop Organising Committee

Philippe Goryl (ESA)

Leonardo De Laurentiis (ESA)

Clément Albinet (ESA)

Sabrina Pinori (Serco)

Valentina Boccia (ESA)

Peggy Fischer (ESA)

Muriel Pinheiro (ESA)

Antonio Valentino (RHEA)

Rosalia Fonseca (ESA)

Montserrat Del Riego Gelabert (SOLENIX)

Chloe Helenè Martella (Serco)

For any information about the VH-RODA Workshop, please email events.organisation@esa.int
VH-RODA 2023 VH-RODA 2022 VH-RODA 2021 VH-RODA and CEOS SAR workshop 2019

# data[].attributes.attachment_link.description
The 2024 VH-RODA workshop provides an open forum (for the new space, commercial and institutional space sectors) for presenting and discussing the current status and future developments related to Earth Observation data quality, calibration and validation of space-borne very high-resolution SAR and Optical sensors and data products.

# data[].attributes.body
Use the hashtag #VHRODA

qa4eoCalVal on X

CEOS WGCV Cal/Val porta l on X

BAQUNIN Supersite on X

# data[].attributes.headline
What and how can the EDAP+/ESA-NASA framework and collaboration improve to further support Commercial Data Providers?

# data[].attributes.subject
What and how can the EDAP+/ESA-NASA framework and collaboration improve to further support Commercial Data Providers?

# data[].attributes.headline
How to improve harmonization between Commercial and Institutional Missions?

# data[].attributes.subject
How to improve harmonization between Commercial and Institutional Missions?

# data[].attributes.headline
What improvements would a potential (Product/Performance) Certification framework bring? 

# data[].attributes.subject
What improvements would a potential (Product/Performance) Certification framework bring? 

# data[].attributes.body
Does New Space agree with the certification approach?

# data[].attributes.headline
In what ways can International Cooperation bring innovation and improvements to New Space? Specifically, what for Data Quality and Cal/Val?

# data[].attributes.subject
In what ways can International Cooperation bring innovation and improvements to New Space? Specifically, what for Data Quality and Cal/Val?

# data[].attributes.headline
Would New Space Missions be trusted without gold standard/reference missions from Institutional Space Agencies?

# data[].attributes.subject
Would New Space Missions be trusted without gold standard/reference missions from Institutional Space Agencies?

# data[].attributes.headline
What are the main challenges to Data Quality and Cal/Val of ATM/HYP missions?

# data[].attributes.subject
What are the main challenges to Data Quality and Cal/Val of ATM/HYP missions?

# data[].attributes.headline
How to improve the EDAP+/ESA-NASA Atmospheric guidelines?

# data[].attributes.subject
How to improve the EDAP+/ESA-NASA Atmospheric guidelines?

# data[].attributes.headline
What improvements can be made to the use of reference missions for ATM/HYP New Space Cal/Val?

# data[].attributes.subject
What improvements can be made to the use of reference missions for ATM/HYP New Space Cal/Val?

# data[].attributes.headline
In what ways can the Fiducial Reference Measurements (FRM) (e.g., for GHG) further support Commercial Missions Cal/Val?

# data[].attributes.subject
In what ways can the Fiducial Reference Measurements (FRM) (e.g., for GHG) further support Commercial Missions Cal/Val?

# data[].attributes.headline
What will be the role of CHIME and other Institutional Missions for ATM/HYP New Space Cal/Val?

# data[].attributes.subject
What will be the role of CHIME and other Institutional Missions for ATM/HYP New Space Cal/Val?

# data[].attributes.headline
 What are some innovative solutions to support ATM/HYP Cal/Val activities?

# data[].attributes.subject
 What are some innovative solutions to support ATM/HYP Cal/Val activities?

# data[].attributes.headline
Is the UAV support for Cal/Val of EO Satellite missions considered as mature? Is it trusted and how can we fill any gaps?

# data[].attributes.subject
Is the UAV support for Cal/Val of EO Satellite missions considered as mature? Is it trusted and how can we fill any gaps?

# data[].attributes.headline
What is needed to support the assessment of the Image Quality Parameters?

# data[].attributes.subject
What is needed to support the assessment of the Image Quality Parameters?

# data[].attributes.headline
How do commercial data providers view uncertainty and its importance for end users? (Towards Certification)

# data[].attributes.subject
How do commercial data providers view uncertainty and its importance for end users? (Towards Certification)

# data[].attributes.headline
What are the biggest challenges in the use of SAR data from different platforms? (e.g., from the same constellation or different missions)

# data[].attributes.subject
What are the biggest challenges in the use of SAR data from different platforms? (e.g., from the same constellation or different missions)

# data[].attributes.headline
What are some innovative solutions to support SAR Cal/Val activities?

# data[].attributes.subject
What are some innovative solutions to support SAR Cal/Val activities?

# data[].attributes.headline
In what ways can the data quality of specific SAR products be improved? (e.g. InSAR) How can the current infrastructures be improved in this regard?

# data[].attributes.subject
In what ways can the data quality of specific SAR products be improved? (e.g. InSAR) How can the current infrastructures be improved in this regard?

# data[].attributes.headline
How can CEOS agencies improve existing Cal/Val Sites and Reference Networks?

# data[].attributes.subject
How can CEOS agencies improve existing Cal/Val Sites and Reference Networks?

# data[].attributes.headline
What are the current barriers to perform vicarious Cal/Val for TIR?

# data[].attributes.subject
What are the current barriers to perform vicarious Cal/Val for TIR?

# data[].attributes.headline
Would you propose innovative solutions to improve existing SAR and OPT Cal/Val sites?

# data[].attributes.subject
Would you propose innovative solutions to improve existing SAR and OPT Cal/Val sites?

# data[].attributes.headline
How can the ESA Cal/Val Park further support New Space Missions? Is the current design good enough to accommodate your needs?

# data[].attributes.subject
How can the ESA Cal/Val Park further support New Space Missions? Is the current design good enough to accommodate your needs?

# data[].attributes.headline
What are the main limitations of existing vicarious Cal/Val Networks for ATM?

# data[].attributes.subject
What are the main limitations of existing vicarious Cal/Val Networks for ATM?

# data[].attributes.headline
How would commercial data providers provide free and open data over a limited and defined set of Cal/Val sites?

# data[].attributes.subject
How would commercial data providers provide free and open data over a limited and defined set of Cal/Val sites?

# data[].attributes.headline
What innovative solutions would you suggest to improve the use of reference missions e.g., Sentinels, Landsat , for New Space Cal/Val and Data Quality?

# data[].attributes.subject
What innovative solutions would you suggest to improve the use of reference missions e.g., Sentinels, Landsat , for New Space Cal/Val and Data Quality?

# data[].attributes.headline
What are the main difficulties to clearly define accurate Target Mission Requirements?

# data[].attributes.subject
What are the main difficulties to clearly define accurate Target Mission Requirements?

# data[].attributes.headline
What are the the biggest barriers to perform more thorough pre-launch calibration activities?

# data[].attributes.subject
What are the the biggest barriers to perform more thorough pre-launch calibration activities?

# data[].attributes.headline
What are the biggest challenges in performing Cal/Val for constellations?

# data[].attributes.subject
What are the biggest challenges in performing Cal/Val for constellations?

# data[].attributes.headline
Should further studies be supported on TIR characteristic effects? (e.g., Multitemporal-Multiangular, polarization,..)

# data[].attributes.subject
Should further studies be supported on TIR characteristic effects? (e.g., Multitemporal-Multiangular, polarization,..)

# data[].attributes.headline
What can be improved within the EDAP+/ESA-NASA framework to better address TIR missions?

# data[].attributes.subject
What can be improved within the EDAP+/ESA-NASA framework to better address TIR missions?

# data[].attributes.headline
How can the existing and upcoming TIR Institutional Missions further support New Space TIR Cal/Val?

# data[].attributes.subject
How can the existing and upcoming TIR Institutional Missions further support New Space TIR Cal/Val?

# data[].attributes.headline
What is missing in terms of open-source solutions to further support New Space Cal/Val?

# data[].attributes.subject
What is missing in terms of open-source solutions to further support New Space Cal/Val?

# data[].attributes.headline
How can the support to Atmospheric Correction algorithms be improved?  

# data[].attributes.subject
How can the support to Atmospheric Correction algorithms be improved?  

# data[].attributes.headline
Is AI trusted at a sufficient level for operational processors? What’s missing?

# data[].attributes.subject
Is AI trusted at a sufficient level for operational processors? What’s missing?

# data[].attributes.headline
Is AI trusted at a sufficient level for operational quality control and Cal/Val activities? What’s missing?

# data[].attributes.subject
Is AI trusted at a sufficient level for operational quality control and Cal/Val activities? What’s missing?

# data[].attributes.headline
What are the biggest challenges to cloud detection/cloud masking?

# data[].attributes.subject
What are the biggest challenges to cloud detection/cloud masking?

# data[].attributes.headline
Is the current scenario in terms of open dataset considered as adequate? 

# data[].attributes.subject
Is the current scenario in terms of open dataset considered as adequate? 

# data[].attributes.body
How can Institutional Agencies help improving the current scenario?

# data[].attributes.headline
What is GCPIX missing to be useful for New Space Cal/Val?

# data[].attributes.subject
What is GCPIX missing to be useful for New Space Cal/Val?

# data[].attributes.headline
Is the current scenario in terms of open DEM dataset considered as adequate? 

# data[].attributes.subject
Is the current scenario in terms of open DEM dataset considered as adequate? 

# data[].attributes.body
How can Institutional Agencies help improving the current scenario?

# data[].attributes.headline
What can be improved in terms of DEM accuracy and availability?

# data[].attributes.subject
What can be improved in terms of DEM accuracy and availability?

# data[].attributes.headline
Would you propose innovative solutions to improve existing DEM tools?

# data[].attributes.subject
Would you propose innovative solutions to improve existing DEM tools?

# data[].attributes.headline
How to harmonize different DEM sources?

# data[].attributes.subject
How to harmonize different DEM sources?

# data[].attributes.headline
What are the biggest barriers to start using new products or to declare products as Application-Ready Data vs Analysis-Ready Data? 

# data[].attributes.subject
What are the biggest barriers to start using new products or to declare products as Application-Ready Data vs Analysis-Ready Data? 

# data[].attributes.headline
What is ARD missing with respect to “New Space integration”? How should It be modified?

# data[].attributes.subject
What is ARD missing with respect to “New Space integration”? How should It be modified?

# data[].attributes.headline
Are the current gold standard/reference missions enough and adequate to support Cross-Calibration activities for all domains (Multi/Hyper, TIR, SAR, ATM,..)?

# data[].attributes.subject
Are the current gold standard/reference missions enough and adequate to support Cross-Calibration activities for all domains (Multi/Hyper, TIR, SAR, ATM,..)?

# data[].attributes.headline
Should the Certification topic be addressed at CEOS-level?

# data[].attributes.subject
Should the Certification topic be addressed at CEOS-level?

# data[].attributes.headline
Is Moon Calibration considered within the New Space Cal/Val plans? What are the current gaps?

# data[].attributes.subject
Is Moon Calibration considered within the New Space Cal/Val plans? What are the current gaps?

# data[].attributes.headline
How to improve coordinated (independent!) Cal/Val of New Space missions?

# data[].attributes.subject
How to improve coordinated (independent!) Cal/Val of New Space missions?

# data[].attributes.headline
Would it be useful to have a free service that offers extracts of DEM, surface models (DSM) or terrain models (DTM) at different ground sampling distances (GSD)?

# data[].attributes.subject
Would it be useful to have a free service that offers extracts of DEM, surface models (DSM) or terrain models (DTM) at different ground sampling distances (GSD)?

# data[].attributes.headline
Since land use/land cover (LU/LC) is included in many models, would it be useful to launch an intercomparison project of LU/LC (aka LUCIX)?

# data[].attributes.subject
Since land use/land cover (LU/LC) is included in many models, would it be useful to launch an intercomparison project of LU/LC (aka LUCIX)?

# data[].attributes.headline
How can international cooperation further improve the coordination of existing evaluation and assessment frameworks? (EDAP, JACIE, etc.)

# data[].attributes.subject
How can international cooperation further improve the coordination of existing evaluation and assessment frameworks? (EDAP, JACIE, etc.)

# data[].attributes.headline
What additional resources are needed by commercial data providers to improve data quality? (in-situ networks, field measurements, reference missions, etc.)

# data[].attributes.subject
What additional resources are needed by commercial data providers to improve data quality? (in-situ networks, field measurements, reference missions, etc.)

# data[].attributes.headline
“What do commercial data providers need from government agencies for references? (GCPs, radiometry, etc.)

# data[].attributes.subject
“What do commercial data providers need from government agencies for references? (GCPs, radiometry, etc.)

# data[].attributes.body
How can the Global Reference Image (GRI) be further improved to increasingly support New Space Cal/Val activities?

# data[].attributes.headline
What is the level of your satisfaction with government support for commercial SAR?

# data[].attributes.subject
What is the level of your satisfaction with government support for commercial SAR?

# data[].attributes.body
How can it be improved?

# data[].attributes.headline
Would you like to see more SAR data quality training sessions at VH-RODA/JACIE?

# data[].attributes.subject
Would you like to see more SAR data quality training sessions at VH-RODA/JACIE?

# data[].attributes.headline
What are the main difficulties when cross calibrating a new mission with an existing reference mission?

# data[].attributes.subject
What are the main difficulties when cross calibrating a new mission with an existing reference mission?

# data[].attributes.headline
What is needed to be considered a “gold standard” reference mission?

# data[].attributes.subject
What is needed to be considered a “gold standard” reference mission?

# data[].attributes.headline
How important is the reflective spectrum to TIR missions?

# data[].attributes.subject
How important is the reflective spectrum to TIR missions?

# data[].attributes.body
Water exercise 1: https://shorturl.at/M2Cjq

Water exercise 2: https://shorturl.at/J1QFP

Land exercise: https://shorturl.at/8HtX1

# data[].attributes.headline
What role will the Emerging Commercial Data Providers play in the Earth Observation ecosystem in the next 5 years?

# data[].attributes.subject
What role will the Emerging Commercial Data Providers play in the Earth Observation ecosystem in the next 5 years?