1  Introduction

1.1 Document overview

This is the official “Readme document” for the Level 2 (L2) data products from the Pandonia Global Network (PGN). It is also deliverable 8 (D8) of the European Space Agency (ESA) project “Fiducial Reference Measurements for Air Quality 2 (FRM4AQ-2)” (Fiducial Reference Measurements for Air Quality [Proposal], LuftBlick Proposal 201805DEV, Issue 1 2018; Fiducial Reference Measurements for Air Quality [Contract and Statement of Work], ESA Contract No. 4000125841/18/I-NS 2018) and gives guidelines on the correct use of the PGN data. Key questions answered include:

• What do the data represent?

• Which part of the atmosphere is sampled?

• How should the data be used?

• What shortcomings might the data have?

Each retrieval version is identified by the “retrieval setup code” (or “Retrieval Setup Code (r-code)”) and the Blick Processing Software (BlickP) processor version (BlickP processor version (ProcVers)), which were used for the data processing. r-code and ProcVers can be found in this way:

• Native PGN data format: r-code and ProcVers appear in the L2 file name . X is the instrument number, Y the spectrometer number, LLL the location name, RRRR the r-code and P the ProcVers.

• Generic Earth Observation Metadata Standard (GEOMS) data format: r-code and ProcVers appear in the GEOMS file attribute “DATA_PROCESSOR” . RRRR is the r-code and P the ProcVers.

The r-code defines the detailed settings for the spectral fitting (e.g. the length of the fitting window) and the algorithm-type used for the retrieval. Different r-codes in general cause the values of L2 data to change.

When changing the ProcVers, there are two effects on the L2 data:

Data format changes

The data format of each ProcVers can be found in the respective Blick Software Suite Manual on the PGN-webpage under /Downloads/Manuals. Within the same ProcVers the format does not change, while newer ProcVers typically have more output columns than previous versions. E.g., from ProcVers 1.7 to 1.8 more columns describing the data uncertainty have been added.

Retrieval algorithm may change

It is possible that retrieval algorithm of a newer ProcVers is modified compared to the previous one. E.g., a newer ProcVers might use a different temperature climatology or different formula for the Rayleigh correction in the sky algorithms. Therefore the values of data from different ProcVers might vary even if the same r-code was used.

A PGN data product can have one of the following “Product status”:

• Product status “official”: Data are currently an official product of the PGN.

• Product status “disused”: Data are a former official product of the PGN, but are not an official product anymore.

• Product status “unvalidated”: Data are not yet officially validated.

• Product status “develop”: Data are in research stage and only produced for testing purposes.

• Product status “unusable”: Data should not be used, since they are typically only for an additional output gas in the spectral fitting, for which the wavelength range is not optimized.

Section Chapter 2 gives the so-called “Readme pages” for those r-codes with product status “official” or “disused” or also “unvalidated” in case it is expected that this r-code will soon become an official one. The r-codes are sorted in alphabetical order (numbers are after letters, i.e. “nvs0” would be after “nvsa”). Hence the user of a certain PGN data product can just look up the r-code of the data file, search for it in the content section of this document, and then jump directly to the respective readme page.

The sections after section Chapter 2 give additional information with respect to the understanding of the data and are referenced in the readme pages.

Since this document is intended to give a concise overview for each data product, the user is referred to the Algorithm Theoretical Basis Document (ATBD) for more details. The ATBD is given as section “Algorithms Theoretical Basis” in the Blick Software Suite Manual (Cede 2020).

1.2 PGN data taxonomy

Table 1.1: Overview of the hierarchical data naming taxonomy used for PGN data, from fundamental elements (species, product types) to higher-level groupings (data sets, collections). Each row lists the term, an illustrative example, and a brief explanation of its meaning.

Term	Example	Meaning
Species	NO2	Short name (often the chemical formula) of the atmospheric species considered.
Data product type	TotCol	The type of the retrieved value (e.g., total column (TotCol), surface number density (SurfDens)).
Data product	NO2 TotCol	Combination of the Species and the Data product type. Note that data products can have multiple versions.
Retrieval setup	rnvs3	Summarizes the settings used for the retrieval (r-code), following the nomenclature in Cede (2020).
Processor version	p1-8	Blick processing software version (“processor”).
Retrieval version	rnvs3p1-8	A specific combination of Retrieval setup and Processor version.
Versioned data product	NO2 TotCol rnvs3p1-8	A Data product referencing a specific Retrieval version.

Table 1.2: Higher-level PGN data groupings

Term	Meaning
Data set	A time series of a Versioned data product for a specific deployment (location + period).
Data set group	A group of Data sets for a single deployment (e.g., combining multiple species, time ranges, or measurement types).
Data collection	A larger collection of Data set groups (e.g., from multiple deployments or sites) gathered for a campaign or project.

1.3 PGN processor qualification

Table 1.3: Qualification of official PGN processors. After a demonstration period, a new processor is released in a staged way, reflecting the ramp-up phase. Concurrently the current processor is phased-out. After the new processor covers all instruments and products, the old one is retired.

Qualifier	Instruments	Products	Meaning
demonstration	selected	selected	Processor is applicable to a selected number of instruments for a subgroup of products. This release might serve as demonstration of new features.
ramp-up	selected	all	Processor roll out is staged towards full instrument and data product coverage.
network	all	all	Processor is applied to the entire network, and capable to process all operational data products.
phase-out	selected	selected	Processing for instruments is continued until replacement from newer version is available.
retired	none	none	Processor is retired and no processing is possible anymore.

%%{init: {"gantt": {"barHeight": 26, "barGap": 9, "topPadding": 58, "leftPadding": 92, "gridLineStartPadding": 28, "fontSize": 12, "sectionFontSize": 14, "numberSectionStyles": 4}} }%%
gantt
    title Lifecycle timeline of PGN processor versions
    dateFormat YYYY-MM-DD
    axisFormat %Y-%m
    tickInterval 6month
    todayMarker on

    section v1.7
    network   :2019-12-01, 2021-05-31
    phase-out :crit, active, 2021-06-01, 2023-12-31
    retired   :milestone, 2024-06-01, 0d

    section v1.8
    demonstration :crit, 2020-11-01, 2021-05-31
    ramp-up       :active, 2021-06-01, 2023-03-31
    network       :2023-04-01, 2026-06-30

    section v1.9
    demonstration :crit, 2025-06-01, 2026-05-31
    ramp-up       :active, 2026-06-01, 2026-06-30

Figure 1.1: Lifecycle timeline of PGN processor versions.

demonstration ramp-up network phase-out retired

Cede, A. 2020. Manual for Blick Software Suite 1.8, Version 1-8-1. https://www.pandonia-global-network.org/wp-content/uploads/2020/12/BlickSoftwareSuite_Manual_v1-8-1.pdf.

Fiducial Reference Measurements for Air Quality [Contract and Statement of Work], ESA Contract No. 4000125841/18/I-NS. 2018.

Fiducial Reference Measurements for Air Quality [Proposal], LuftBlick Proposal 201805DEV, Issue 1. 2018.