Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recreate NCL diagnostic perfmetrics in python #3550

Open
13 of 20 tasks
lukruh opened this issue Mar 20, 2024 · 1 comment · May be fixed by #3551
Open
13 of 20 tasks

Recreate NCL diagnostic perfmetrics in python #3550

lukruh opened this issue Mar 20, 2024 · 1 comment · May be fixed by #3551
Assignees

Comments

@lukruh
Copy link
Contributor

lukruh commented Mar 20, 2024

Short description of the diagnostic
As discussed at the recent hackathon, we are aiming for a replacement of diag_scripts/perfmetrics in python. We might be able to use preprocessor functions to calculate different performance metrics in the future. Alternative diagnostics to plot timeseries and hovmöller diagrams already exist. Therefore, we focus on developing a flexible plotting diagnostic that can produce figures similar to the permetrics overview figure 5 and gleckler plots produced by the ETCCDI recipe.
This would be one step towards recreating one figure from the set of perfmetric diagnostics.

  • discuss translation of ncl code -> recreate portrait plot from scratch in python
  • calculate rmse and pearson corelation using preprocessor
  • create example plots for some models and variables
  • custom y and x axis (metadata keys)
    • example IPCC reference regions
  • group into seperate blocks (gaps) (grouby, metadata keys)
  • split rectangle into parts (splitby, metadata keys)
    • metadata keys or extra facets for custom labels
    • 2 to 4 triangles per cell. Shape selected based on number of splits
  • improve colorbar: layout with figure, customize ticks, labels etc..
  • use same scale of colormap for each group plot (does not consider splits)
  • customize plot (title, axis labels, cmap etc..)
  • custom color for nan values and option to hide nan triangles
  • documentation
    • API documentation for diagnostic
    • Recipe documentation
  • example recipe, test recipe
  • test with icon data
  • normalization of metrics (mean, median, centered_mean, centered_median)
  • add column for mean/median model?
  • recreate perfmetrics example and validate results
  • recreate ETCCDI example (depends on xclim diagnostic)

Branch and pull request
PR: #3551
Branch: perfmetric_python
Recipe: recipe_perfmetrics_python.yml

Produced figures

Image
image

@lukruh lukruh linked a pull request Mar 20, 2024 that will close this issue
16 tasks
@lukruh
Copy link
Contributor Author

lukruh commented Apr 2, 2024

We made some progress:

  • It's possible to add up to 3 additional (4 in total) reference datasets and show them as triangular overlay. They don't need to be complete and the drawing of non existing (white) data is optional.
  • A legend is added and can be customized in position and size, to manually fix layout issues in very wide/narrow figures.
  • Optional normalization for metrics is added (supported "mean", "median", "centered_mean", "centered_median") similar to shared/statistics.ncl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

2 participants