Using pre-computed results¶
Results computed by the CMIP7 Assessment Fast Track Rapid Evaluation Framework are available from the website: https://dashboard.climate-ref.org and the associated API: https://api.climate-ref.org. This API provides an OpenAPI schema that documents what queries are available. The API documentation can be viewed at: https://api.climate-ref.org/docs.
This Jupyter notebook shows how to use this API to download pre-computed results and use those to do your own analyses.
!uvx --quiet --from openapi-python-client openapi-python-client generate --url https://api.climate-ref.org/api/v1/openapi.json --meta setup --output-path climate_ref_client --overwrite
Generating climate_ref_client
!pip install --quiet ./climate_ref_client
from pathlib import Path
import cartopy.crs
import matplotlib.pyplot as plt
import pandas as pd
import requests
import seaborn as sns
import xarray as xr
from climate_rapid_evaluation_framework_client import Client
from climate_rapid_evaluation_framework_client.api.diagnostics import (
diagnostics_list,
diagnostics_list_metric_values,
)
from climate_rapid_evaluation_framework_client.api.executions import executions_get
from climate_rapid_evaluation_framework_client.models.metric_value_type import (
MetricValueType,
)
from IPython.display import Markdown
from pandas_indexing import formatlevel
%load_ext rich
View the available diagnostics¶
We start by setting up a client for interacting with the server:
client = Client("https://api.climate-ref.org")
Retrieve the available diagnostics from the server, and inspect the first one:
diagnostics = diagnostics_list.sync(client=client).data
diagnostics[0]
DiagnosticSummary( id=25, provider=ProviderSummary(slug='esmvaltool', name='ESMValTool', additional_properties={}), slug='climate-at-global-warming-levels', name='Climate at Global Warming Levels', description='\nCalculate climate variables at global warming levels.\n', execution_groups=[6507, 6508, 6509, 6510], has_metric_values=False, has_scalar_values=False, has_series_values=False, execution_count=16, successful_execution_count=4, execution_group_count=4, successful_execution_group_count=4, group_by=[GroupBy(source_type='cmip6', group_by=['experiment_id'], additional_properties={})], aft_link=AFTDiagnosticDetail( id='5.3', name='Evaluation of key climate variables at global warming levels', theme='Impacts and Adaptation', version_control='version 1 - 24-11-04 REF launch', reference_dataset='GPCP-SG-2-3, HadCRUT5-0-2-0', endorser='CMIP Model Benchmarking Task Team', provider_link='https://docs.esmvaltool.org/en/latest/recipes/recipe_calculate_gwl_exceedance_stats.html', description='nan', short_description='This diagnostic calculates years of Global Warming Level (GWL) exceedances in CMIP models as described in Swaminathan et al (2022). Time series of the anomalies in annual global mean surface air temperature (GSAT) are calculated with respect to the 1850-1900 time-mean of each individual time series. To limit the influence of short-term variability, a 21-year centered running mean is applied to the time series. The year at which the time series exceeds warming levels or temperatures such as 1.5C is then recorded for the specific model ensemble member and future scenario. Once the years of exceedance are calculated, the time averaged global mean and standard deviation for the multimodel ensemble over the 21-year period around the year of exceedance are plotted.', diagnostics=[ RefDiagnosticLink( provider_slug='esmvaltool', diagnostic_slug='climate-at-global-warming-levels', additional_properties={} ) ], additional_properties={} ), reference_datasets=[], tags=['scenarios', 'warming-levels'], additional_properties={} )
To get an idea of what is available, we create a list of all diagnostics with short descriptions (a full overview is available in Appendix C of Hoffman et al., 2025):
txt = ""
for diagnostic in sorted(diagnostics, key=lambda diagnostic: diagnostic.name):
title = f"### {diagnostic.name}"
description = diagnostic.description.strip()
if not description.endswith("."):
description += "."
if diagnostic.aft_link:
description += f" {diagnostic.aft_link.short_description.strip()}"
if not description.endswith("."):
description += "."
if (aft_description := diagnostic.aft_link.description.strip()) != "nan":
description += f" {aft_description}"
if not description.endswith("."):
description += "."
txt += f"{title}\n{description}\n\n"
Markdown(txt)
Annual Cycle Analysis¶
Calculate the annual cycle for a dataset. Maps of seasonal and annual climatology are generated for the reference datasets and model output on a target grid (2.5x2.5 deg), then calculate diverse metrics including bias, RMSE, spatial pattern correlation, and standard deviation.
Atlantic Meridional Overturning Circulation (RAPID)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Comparison of the model Atlantic meridional ocean circulation (AMOC) circulation strength with refernce data from RAPID-v2023-1. Provides a key indicator of the strength of ocean circulation, which redistributes freshwater, heat and carbon across the Atlantic Basin (Le Bras et al., 2023). The AMOC is a key component of the global ocean conveyor belt and plays an important role in transporting heat poleward and ocean biogeochemical tracers from the surface into the ocean interior. The strength of the AMOC at 26.5◦N is commonly used for evaluation of model fidelity since it can be compared with the long-term RAPID-MOCHA (Rapid Climate Change - Meridional Overturning Circulation and Heatflux Array) observational dataset (Moat et al., 2025).
Burnt Fraction (GFED)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. The diagnostic relies on the processing of fire climate drivers through the ConFire model and is based on Jones et al. (2024). The diagnostic computes the burnt fraction for each grid cell based on a number of drivers. Additionally, the respective controls due to fire weather and fuel load/continuity are computed. The stochastic control corresponds to the unmodelled processed influencing to fire occurrence.
Climate Drivers for Fire¶
Calculate diagnostics regarding climate drivers for fire. The diagnostic relies on the processing of fire climate drivers through the ConFire model and is based on Jones et al. (2024). The diagnostic computes the burnt fraction for each grid cell based on a number of drivers. Additionally, the respective controls due to fire weather and fuel load/continuity are computed. The stochastic control corresponds to the unmodelled processed influencing to fire occurrence.
Climate at Global Warming Levels¶
Calculate climate variables at global warming levels. This diagnostic calculates years of Global Warming Level (GWL) exceedances in CMIP models as described in Swaminathan et al (2022). Time series of the anomalies in annual global mean surface air temperature (GSAT) are calculated with respect to the 1850-1900 time-mean of each individual time series. To limit the influence of short-term variability, a 21-year centered running mean is applied to the time series. The year at which the time series exceeds warming levels or temperatures such as 1.5C is then recorded for the specific model ensemble member and future scenario. Once the years of exceedance are calculated, the time averaged global mean and standard deviation for the multimodel ensemble over the 21-year period around the year of exceedance are plotted.
Cloud Radiative Effects¶
Plot climatologies and zonal mean profiles of cloud radiative effects (sw + lw) for a dataset. Maps and zonal means of longwave and shortwave cloud radiative effect.
Cloud Scatterplots for Reference dataset¶
Reference scatterplots of two cloud-relevant variables. 2D histograms with focus on clouds.
Cloud-Precipitation Scatterplots (clwvi vs pr)¶
Scatterplot of clwvi vs pr. 2D histograms with focus on clouds.
Cloud-Radiation Scatterplots (clivi vs lwcre)¶
Scatterplot of clivi vs lwcre. 2D histograms with focus on clouds.
Cloud-Radiation Scatterplots (clt vs swcre)¶
Scatterplot of clt vs swcre. 2D histograms with focus on clouds.
Cloud-Temperature Scatterplots (cli vs ta)¶
Scatterplot of cli vs ta. 2D histograms with focus on clouds.
ENSO Basic Climatology¶
Calculate the ENSO CLIVAR metrics - background climatology. ENSO CLIVAR metrics - reproducing background climatology and ENSO characteristics. The El Niño Southern Oscillation (ENSO) is the primary mode of the global interannual climate variability, mainly reflected by the variations in surface wind stress and ocean temperature in the tropical Pacific Ocean. The ENSO variability can be calculated from both sea surface temperature and atmospheric pressure differences 680 between different tropical Pacific areas. The Southern Oscillation Index (SOI) uses pressure differences between the Tahiti and Darwin regions. The Oceanic Niño Index (ONI) summarizes SST anomalies in the Niño 3.4 region.Given its implications for regional climate variability, capturing the observed ENSO spatial and temporal characteristics would increase the fidelity and robustness in a model’s climate projections.
ENSO Characteristics¶
Calculate the ENSO CLIVAR metrics - basic ENSO characteristics. ENSO CLIVAR metrics - reproducing background climatology and ENSO characteristics. The El Niño Southern Oscillation (ENSO) is the primary mode of the global interannual climate variability, mainly reflected by the variations in surface wind stress and ocean temperature in the tropical Pacific Ocean. The ENSO variability can be calculated from both sea surface temperature and atmospheric pressure differences 680 between different tropical Pacific areas. The Southern Oscillation Index (SOI) uses pressure differences between the Tahiti and Darwin regions. The Oceanic Niño Index (ONI) summarizes SST anomalies in the Niño 3.4 region.Given its implications for regional climate variability, capturing the observed ENSO spatial and temporal characteristics would increase the fidelity and robustness in a model’s climate projections.
ENSO Processes¶
Calculate the ENSO performance metrics for a dataset. ENSO CLIVAR metrics - reproducing background climatology and ENSO characteristics. The El Niño Southern Oscillation (ENSO) is the primary mode of the global interannual climate variability, mainly reflected by the variations in surface wind stress and ocean temperature in the tropical Pacific Ocean. The ENSO variability can be calculated from both sea surface temperature and atmospheric pressure differences 680 between different tropical Pacific areas. The Southern Oscillation Index (SOI) uses pressure differences between the Tahiti and Darwin regions. The Oceanic Niño Index (ONI) summarizes SST anomalies in the Niño 3.4 region.Given its implications for regional climate variability, capturing the observed ENSO spatial and temporal characteristics would increase the fidelity and robustness in a model’s climate projections.
ENSO Teleconnections¶
Calculate the ENSO performance metrics for a dataset. ENSO CLIVAR metrics - reproducing background climatology and ENSO characteristics. The El Niño Southern Oscillation (ENSO) is the primary mode of the global interannual climate variability, mainly reflected by the variations in surface wind stress and ocean temperature in the tropical Pacific Ocean. The ENSO variability can be calculated from both sea surface temperature and atmospheric pressure differences 680 between different tropical Pacific areas. The Southern Oscillation Index (SOI) uses pressure differences between the Tahiti and Darwin regions. The Oceanic Niño Index (ONI) summarizes SST anomalies in the Niño 3.4 region.Given its implications for regional climate variability, capturing the observed ENSO spatial and temporal characteristics would increase the fidelity and robustness in a model’s climate projections.
Equilibrium Climate Sensitivity (ECS)¶
Calculate the global mean equilibrium climate sensitivity for a dataset. Equilibrium climate sensitivity is defined as the change in global mean temperature as a result of a doubling of the atmospheric CO2 concentration compared to pre-industrial times after the climate system has reached a new equilibrium. This diagnostic uses a regression method based on Gregory et al. (2004).
Global Mean Timeseries¶
Calculate the annual mean global mean timeseries for a dataset.
Gross Primary Productivity (FLUXNET2015)¶
Apply the standard ILAMB analysis with respect to a given reference dataset.
Gross Primary Productivity (WECANN)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model gross primary production (GPP) with reference data from FLUXNET2015-1-0 and WECANN-1-0. Gross primary production is the process by which plants “fix” atmospheric or aqueous carbon dioxide through photosynthetic reduction into organic compounds, and it is affected by increases in atmospheric carbon dioxide (CO2) levels and warming (Anav et al., 2015). A fraction of gross primary productivity supports plant respiration and the rest is stored as biomass in stems, leaves, roots, or other plant parts. Land use change, heat and drought stress due to anthropogenic warming, and rising atmospheric CO2 will differentially influence gross primary production in ecosystems and alter the global carbon cycle. Thus, models must be evaluated to ensure they capture the observed responses to these changes.
Leaf Area Index (AVH15C1)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model leaf area index (LAI) with reference data from NOAA-NCEI-LAI-5-0, and LAI4g-1-2.
Net Biome Productivity (Hoffman)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Comparison of the model integrated land net ecosystem carbon balance with reference data from HOFFMAN-1-0.
North Atlantic Oscillation (NAO)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
North Pacific Gyre Oscillation (NPGO)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
North Pacific Oscillation (NPO)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
Northern Annular Mode (NAM)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
Ocean Heat Content (NOAA)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model ocean heat content (OHC) with reference data from MGOHCTA-WOA09. The Ocean Heat Content (OHC) may provide one of the most reliable signals about the long-term climate change and decadal to multidecadal variablity, including their temporal variation and spatial patterns. It is compared, between models and observations, on a gridded basis (1◦ × 1◦), based on almost all available in situ ocean observations (e.g., Argo, conductivity–temperature–depth (CTD) profilers, Mechanical BathyThermographs, bottles, moorings, gliders, and animal-borne ocean sensors; Cheng et al., 2024). Before use, the data are carefully bias corrected, vertically and horizontally interpolated and mapped onto a grid for comparison with models.
Pacific Decadal Oscillation (PDO)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
Pacific North American Pattern (PNA)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
Regional Historical Annual Cycle¶
Plot regional historical annual cycle of climate variables. Time series, linear trend, and annual cycle for IPCC regions.
Regional Historical Timeseries¶
Plot regional historical mean and anomaly of climate variables. Time series, linear trend, and annual cycle for IPCC regions.
Regional Historical Trends¶
Plot regional historical trend of climate variables. Time series, linear trend, and annual cycle for IPCC regions.
Runoff (LORA)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model surface runoff with reference data from LORA-1-1. Surface water runoff plays an important role in the hydrological cycle by returning excess precipitation to the oceans and controlling how much water flows into water systems (Trenberth et al., 2007; Trenberth and Caron, 2001). Changes in atmospheric circulation and distributions of precipitation have a direct effect on changes in runoff from land. Models must be evaluated to ensure they exhibit the observed responses to precipitation and soil moisture processes that lead to runoff and transport of freshwater into rivers and oceans.
Sea Ice Area Basic Metrics¶
Calculate seasonal cycle and time series of NH and SH sea ice area. Seasonal cycle of Arctic (NH) and Antarctic (SH) sea ice area, time series of Arctic September (NH) and Antarctic February (SH) sea ice area. The sea ice area, calculated as the sum over the Northern (Arctic) and Southern (Antarctic) Hemisphere grid cell areas multiplied by the sea ice fraction within each cell, exhibits a distinct seasonal cycle. Arctic sea ice area typically has minimum values in September, while Antarctic sea ice area is lowest in February. The seasonal cycle is driven by the seasonal cycle of the insolation, sea ice processes, as well as the exchange with the atmosphere and ocean and can be seen as an overview metric for the general state of the sea ice in a model. In addition to the multi-year average seasonal cycle of Arctic and Antarctic sea ice area, the diagnostic produces time series of the September (Arctic) and February (Antarctic) sea ice area.
Sea Ice Sensitivity¶
Calculate sea ice sensitivity. Rate of sea ice area loss per degree of warming, as in plot 1d of Notz et al. and figure 3e of Roach et al. This metric evaluates the rate of sea ice loss per degree of global warming, following the approach used for sea ice benchmarking within the Sea Ice Model Intercomparison Project analysis (Notz and SIMIP Community, 2020; Roach et al., 2020). The metric is calculated by regressing the time-series of sea ice area on global mean temperature. Sea ice responds strongly to climate forcing and warming.
Sea Water Salinity (WOA2023 Surface)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model sea surface temperature (SST) and sea surface salinity (SSS) with reference data from WOA2023. Th Sea surface temperature (SST) bias, Sea surface salinity (SSS) distributions provide large scale patterns of surface ocean circulation as well as reflecting dynamical air-sea interactions and ocean-sea ice interactions in the polar regions. SST and SSS biases have a significant impact on the coupling of ESM’s two majors components, the atmosphere and the ocean. Satellite data products and localized moored sensors are used to produce measurements that are incorporated into reference data to calculate SST and SSS biases in models.
Sea Water Temperature (WOA2023 Surface)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model sea surface temperature (SST) and sea surface salinity (SSS) with reference data from WOA2023. Th Sea surface temperature (SST) bias, Sea surface salinity (SSS) distributions provide large scale patterns of surface ocean circulation as well as reflecting dynamical air-sea interactions and ocean-sea ice interactions in the polar regions. SST and SSS biases have a significant impact on the coupling of ESM’s two majors components, the atmosphere and the ocean. Satellite data products and localized moored sensors are used to produce measurements that are incorporated into reference data to calculate SST and SSS biases in models.
Snow Cover (ESA CCI)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model snow cover with reference data from CCI-CryoClim-FSC-1.
Soil Carbon (HWSD2)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model soil carbon with reference data from HWSD-2-0 and NCSCD-2-2. Soil carbon is the organic matter and inorganic carbon in global soils. It is an important component of the global carbon cycle and affects soil moisture retention and saturation. Analyzing stored soil carbon helps track quantify the dynamics of the terrestrial carbon cycle within models and the movement of carbon through the Earth system.
Southern Annular Mode (SAM)¶
Calculate the extratropical modes of variability for a given area. For extratropical modes of variability, maps of variability mode pattern and thier principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their stand deviations.
Surface Soil Moisture (WangMao)¶
Apply the standard ILAMB analysis with respect to a given reference dataset. Apply the ILAMB Methodology* to compare the model surface soil moisture with reference data from OLC-ORS-V0.
Transient Climate Response (TCR)¶
Calculate the global mean transient climate response for a dataset. The transient climate response (TCR) is defined as the global and annual mean surface air temperature anomaly in the 1pctCO2 scenario (1% CO2 increase per year) for a 20 year period centered at the time of CO2 doubling, i.e. using the years 61 to 80 after the start of the simulation. We calculate the temperature anomaly by subtracting a linear fit of the piControl run for all 140 years of the 1pctCO2 experiment prior to the TCR calculation (see Gregory and Forster, 2008).
Transient Climate Response to Emissions (TCRE)¶
Calculate the global mean Transient Climate Response to Cumulative CO2 Emissions. The idea that global temperature rise is directly proportional to the total amount of carbon dioxide (CO2) released into the atmosphere is fundamental to climate policy. The concept stems from research showing a clear linear relationship between cumulative CO2 emissions and global temperature change in climate models (Allen et al. 2009; Matthews et al. 2009; Zickfeld et al. 2009). This relationship is called the Transient Climate Response to Cumulative CO2 Emissions (TCRE), which represents the amount of global warming caused by each trillion tonnes of carbon emitted. This simple yet powerful tool allows policymakers to directly link emission budgets to specific temperature targets and compare the long-term effects of different emissions scenarios.
Zero Emission Commitment (ZEC)¶
Calculate the global mean Zero Emission Commitment (ZEC) temperature. The Zero Emissions Commitment (ZEC) quantifies the change in global mean temperature expected to occur after net carbon dioxide (CO2) emissions cease. ZEC is therefore important to consider when estimating the remaining carbon budget. Calculation of ZEC requires dedicated simulations with emissions set to zero, branching off a base simulation with emissions. In CMIP6 the simulations were part of ZECMIP, with the simulations called esm-1pct-brch-xPgC branching off the 1pctCO2 simulation when emissions reach x PgC. The default x was 1000PgC, with additional simulations for 750PgC and 2000PgC. In CMIP7, ZEC simulations (esm-flat10-zec) are part of the fast track and branch off (esm-flat10) with constant emissions of 10GtC/yr at year 100 (Sanderson 2024).
Metrics¶
Many of the diagnostics provide "metric" values, single values that describe some property of a model. Here we show how to access these values and create a plot.
# Select the "Atlantic Meridional Overturning Circulation (RAPID)"
# diagnostic as an example
diagnostic_name = "Atlantic Meridional Overturning Circulation (RAPID)"
diagnostic = next(d for d in diagnostics if d.name == diagnostic_name)
# Inspect an example value.
diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SCALAR,
client=client,
).data[0]
ScalarValue( dimensions=ScalarValueDimensions( additional_properties={ 'source_id': 'CanESM5', 'experiment_id': 'historical', 'grid_label': 'gn', 'member_id': 'r10i1p2f1', 'metric': 'timeseries', 'region': 'None', 'statistic': 'Bias Score' } ), value=0.7856985712687937, id=12801, execution_group_id=825, execution_id=825, attributes=None, is_outlier=False, verification_status=None, additional_properties={} )
Read the metric values into a Pandas DataFrame:
df = (
pd.DataFrame(
metric.dimensions.additional_properties | {"value": metric.value}
for metric in diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SCALAR,
client=client,
).data
)
.replace("None", pd.NA)
.drop_duplicates()
)
# Drop a few columns that appear to be the same for all entries of
# particular diagnostic.
df.drop(columns=["experiment_id", "metric", "region"], inplace=True)
# Use the columns that do not contain the metric value for indexing
df.set_index([c for c in df.columns if c != "value"], inplace=True)
df
| value | ||||
|---|---|---|---|---|
| source_id | grid_label | member_id | statistic | |
| CanESM5 | gn | r10i1p2f1 | Bias Score | 0.785699 |
| CanESM5-CanOE | gn | r1i1p2f1 | Bias Score | 0.770994 |
| CanESM5 | gn | r18i1p2f1 | Bias Score | 0.749779 |
| r19i1p1f1 | Bias Score | 0.764098 | ||
| r19i1p2f1 | Bias Score | 0.771855 | ||
| ... | ... | ... | ... | ... |
| MPI-ESM1-2-LR | gn | r1i1p1f1 | Taylor Score | 0.356818 |
| MRI-ESM2-0 | gr2z | r1i1p1f1 | Taylor Score | 0.505872 |
| NorCPM1 | grz | r1i1p1f1 | Taylor Score | 0.430062 |
| NorESM2-MM | grz | r1i1p1f1 | Taylor Score | 0.570105 |
| SAM0-UNICON | gn | r1i1p1f1 | Taylor Score | 0.535892 |
330 rows × 1 columns
and create a portrait diagram:
# Use the median metric value for models with multiple ensemble
# members to keep the figure readable.
df = df.groupby(level=["source_id", "grid_label", "statistic"]).median()
# Convert df to a "2D" dataframe for use with the seaborn heatmap plot
df_2D = (
formatlevel(df, model="{source_id}.{grid_label}", drop=True)
.reset_index()
.pivot(columns="statistic", index="model", values="value")
)
figure, ax = plt.subplots(figsize=(5, 8))
_ = sns.heatmap(
df_2D / df_2D.median(),
annot=df_2D,
cmap="viridis",
linewidths=0.5,
ax=ax,
cbar_kws={"label": "Color indicates value relative to the median"},
)
Series¶
Many of the diagnostics provide "series" values, a range of values along with an index that describe some property of a model. Here we show how to access these values and create a plot.
# Select the "Sea Ice Area Basic Metrics" diagnostic as an example
diagnostic_name = "Sea Ice Area Basic Metrics"
diagnostic = next(d for d in diagnostics if d.name == diagnostic_name)
# Inspect an example series value:
diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SERIES,
client=client,
).data[0]
SeriesValue( id=568190, dimensions=SeriesValueDimensions( additional_properties={ 'source_id': 'ACCESS-ESM1-5', 'grid_label': 'gn', 'member_id': 'r10i1p1f1', 'region': 'Northern Hemisphere', 'statistic': '20-year average seasonal cycle of the sea ice area' } ), values=[ 12.625557899475098, 13.885064125061035, 14.218955993652344, 13.654378890991211, 11.968358993530273, 9.601765632629395, 7.028141498565674, 4.875391006469727, 4.349080562591553, 5.6265997886657715, 7.671941757202148, 10.290356636047363 ], execution_group_id=7678, execution_id=74747, index=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0], index_name='month_number', attributes=SeriesValueAttributesType0( additional_properties={ 'caption': '20-year average seasonal cycle of the sea ice area in million km2 from ACCESS-ESM1-5.r10i1p1f1.gn compared with OSI-450.', 'values_name': 'siconc', 'index_name': 'month_number', 'value_long_name': 'Sea-Ice Area Percentage (Ocean Grid)', 'index_long_name': 'month_number', 'value_units': '1e6 km2', 'index_units': '1' } ), additional_properties={} )
Read the metric values into a Pandas DataFrame:
statistic_name = "20-year average seasonal cycle"
value_name = "sea ice area (1e6 km2)"
df = pd.DataFrame(
metric.dimensions.additional_properties | {value_name: value, "month": int(month)}
for metric in diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SERIES,
client=client,
).data
if metric.dimensions.additional_properties["statistic"].startswith(statistic_name)
for value, month in zip(metric.values, metric.index)
if value < 1e10 # Ignore some invalid values.
)
df
| source_id | grid_label | member_id | region | statistic | sea ice area (1e6 km2) | month | |
|---|---|---|---|---|---|---|---|
| 0 | ACCESS-ESM1-5 | gn | r10i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 12.625558 | 1 |
| 1 | ACCESS-ESM1-5 | gn | r10i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 13.885064 | 2 |
| 2 | ACCESS-ESM1-5 | gn | r10i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 14.218956 | 3 |
| 3 | ACCESS-ESM1-5 | gn | r10i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 13.654379 | 4 |
| 4 | ACCESS-ESM1-5 | gn | r10i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 11.968359 | 5 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 2395 | EC-Earth3 | gn | r1i1p1f1 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 8.590763 | 8 |
| 2396 | EC-Earth3 | gn | r1i1p1f1 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 9.544237 | 9 |
| 2397 | EC-Earth3 | gn | r1i1p1f1 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 9.049510 | 10 |
| 2398 | EC-Earth3 | gn | r1i1p1f1 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 6.577136 | 11 |
| 2399 | EC-Earth3 | gn | r1i1p1f1 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 2.754174 | 12 |
2400 rows × 7 columns
and create a plot:
_ = sns.relplot(
data=df.sort_values("source_id"),
x="month",
y=value_name,
col="region",
hue="source_id",
kind="line",
)
Files¶
Many of the diagnostics produce NetCDF files that can be used for further analysis or custom plotting. We will look at the global warming levels diagnostic and create our own figure using the available data.
Each diagnostic can be run (executed) multiple times with different input data. The global warmings levels diagnostic has been executed several times, leading to multiple "execution groups":
diagnostic_name = "Climate at Global Warming Levels"
diagnostic = next(d for d in diagnostics if d.name == diagnostic_name)
[executions_get.sync(g, client=client).key for g in diagnostic.execution_groups]
['cmip6_ssp126', 'cmip6_ssp245', 'cmip6_ssp370', 'cmip6_ssp585']
Let's select the "ssp585" scenario and look at the output files that were produced:
for group in diagnostic.execution_groups:
execution = executions_get.sync(group, client=client)
if execution.key.endswith("ssp585"):
ssp585_outputs = execution.latest_execution.outputs
break
else:
msg = "Failed to find the ssp585 execution group"
raise ValueError(msg)
[o.filename for o in ssp585_outputs]
[ 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_1.5.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_3.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_4.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_5.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_1.5.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_2.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_3.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_4.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_5.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_1.5.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_2.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_3.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_4.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_5.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_1.5.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_2.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_3.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_4.0.png', 'executions/recipe_20251002_035030/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_5.0.png', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_1.5.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_3.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_4.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_5.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_1.5.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_2.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_3.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_4.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_5.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_1.5.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_3.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_4.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_5.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_1.5.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_2.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_3.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_4.0.nc', 'executions/recipe_20251002_035030/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_5.0.nc', 'executions/recipe_20251002_035030/work/calculate_gwl_exceedance_years/gwl_exceedance_calculation/GWL_exceedance_years.csv', 'executions/recipe_20251002_035030/index.html' ]
Select one of the output files and inspect it:
filename = "tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc"
file = next(f for f in ssp585_outputs if f.filename.endswith(filename))
file
ExecutionOutput( id=115129, execution_id=58273, output_type=<ResultOutputType.DATA: 'data'>, filename='executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', short_name='executions/recipe_20251002_035030/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', long_name='Multimodel mean of Temperature at 2.0 $^\\circ$ C', description='', created_at=datetime.datetime(2025, 10, 2, 4, 7, 28), updated_at=datetime.datetime(2025, 10, 2, 4, 7, 28), url='https://api.climate-ref.org//api/v1/results/115129', additional_properties={} )
Download the file and open it with xarray:
local_file = Path(Path(file.filename).name)
local_file.write_bytes(requests.get(file.url, timeout=120).content)
ds = xr.open_dataset(local_file).drop_vars("cube_label")
ds
<xarray.Dataset> Size: 272kB
Dimensions: (lat: 180, lon: 360, bnds: 2)
Coordinates:
* lat (lat) float64 1kB -89.5 -88.5 -87.5 -86.5 ... 86.5 87.5 88.5 89.5
* lon (lon) float64 3kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5
time datetime64[ns] 8B ...
Dimensions without coordinates: bnds
Data variables:
tas (lat, lon) float32 259kB ...
lat_bnds (lat, bnds) float64 3kB ...
lon_bnds (lon, bnds) float64 6kB ...
time_bnds (bnds) datetime64[ns] 16B ...
Attributes:
Conventions: CF-1.7
software: Created with ESMValTool v2.12.0
caption: Multimodel mean of Temperature at 2.0 $^\circ$ CCreate our own plot:
plot = ds.tas.plot.contourf(
cmap="viridis",
vmin=-30,
vmax=30,
levels=11,
figsize=(12, 5),
transform=cartopy.crs.PlateCarree(),
subplot_kws={
"projection": cartopy.crs.Orthographic(
central_longitude=-100,
central_latitude=40,
),
},
)
_ = plot.axes.coastlines()
/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/466/.venv/lib/python3.11/site-packages/cartopy/io/__init__.py:241: DownloadWarning: Downloading: https://naturalearth.s3.amazonaws.com/110m_physical/ne_110m_coastline.zip
warnings.warn(f'Downloading: {url}', DownloadWarning)
