diff --git a/15-advanced-testing-techniques.Rmd b/15-advanced-testing-techniques.Rmd index fd80bfc..23b8d64 100644 --- a/15-advanced-testing-techniques.Rmd +++ b/15-advanced-testing-techniques.Rmd @@ -2,12 +2,374 @@ **Learning objectives:** -- THESE ARE NICE TO HAVE BUT NOT ABSOLUTELY NECESSARY +- Test Fixture Strategies +- Effective Helper Function Creation +- Side Effect Management +- Fixture Loading Techniques +- Custom Expectation Development +- Testing Optimization and Skip Handling -## SLIDE 1 -- ADD SLIDES AS SECTIONS (`##`). -- TRY TO KEEP THEM RELATIVELY SLIDE-LIKE; THESE ARE NOTES, NOT THE BOOK ITSELF. +## Test Fixtures {-} + +When it's not practical to make your test entirely self-sufficient. + +- Put repeated code in a constructor-type helper function. + - If the repeated code has **side effects** make sure to **clean up** afterwards by writing a custom local_*(). + +- Otherwise save the result as a **static file** and load it. + +## Creating Helper Function {-} + +Creating a helper function make sense if it meet the following conditions: + +- Create it takes **several lines of code**. +- The creating process doesn't requires much **time or memory**. + +It can be stored under any folder loaded by calling `devtools::load_all()`: + +- Under the `R/` folder as not exported function. +- Under a R script starting with "helper" under the `tests/testthat/`folder. + +```r +new_useful_thing <- function() { + # your fiddly code to create a useful_thing goes here +} +``` + + +## Creating Helper Function {-} + +Then you will be able to call it each time needed. + +```r +test_that("foofy() does this", { + useful_thing1 <- new_useful_thing() + expect_equal(foofy(useful_thing1, x = "this"), EXPECTED_FOOFY_OUTPUT) +}) + +test_that("foofy() does that", { + useful_thing2 <- new_useful_thing() + expect_equal(foofy(useful_thing2, x = "that"), EXPECTED_FOOFY_OUTPUT) +}) +``` + +> In many cases you might found out that the function can be **useful** even for creating **vignettes** and you might end **exporting the function**. + +## Avoiding Side Effects {-} + +If your functions needs to do any of the following actions you need to **clean up afterwards**: + +- Create a file or directory +- Create a resource on an external system +- Set an R option +- Set an environment variable +- Change working directory +- Change an aspect of the tested package's state + +## Using `on.exit()` to clean up {-} + +```r +neat <- function(x, sig_digits) { + op <- options(digits = sig_digits) + on.exit(options(op), add = TRUE, after = FALSE) + print(x) +} + +pi +#> [1] 3.141593 +neat(pi, 2) +#> [1] 3.1 +pi +#> [1] 3.141593 +``` + +## `on.exit()` Limitations {-} + +- You should always call it with `add = TRUE` to ensure that the call is **added to the list** of deferred tasks (instead of replaces). + +- You should always call it with `after = FALSE` to ensure that the **cleanup occurs in reverse** order to setup. + +- It doesn't work in the global environment for interactive checking. + +- You can't wrap up `on.exit()` in a **helper function**. + + +## Using `withr::defer()` to Clean Up {-} + +```r +neat <- function(x, sig_digits) { + op <- options(digits = sig_digits) + withr::defer(options(op)) + print(x) +} +``` + +## `withr::defer()` benefits {-} + +- It has the behaviour we want by default; no extra arguments needed. + +- It works when called in the global environment thanks to `deferred_run()` and `deferred_clear()` as the global environment isn't perishable. + +```r +withr::defer(print("hi")) +#> Setting deferred event(s) on global environment. +#> * Execute (and clear) with `deferred_run()`. +#> * Clear (without executing) with `deferred_clear()`. + +withr::deferred_run() +#> [1] "hi" +``` + +- It lets you pick **which function to clean up**. This makes it possible to create helper functions. + +```r +local_digits <- function(sig_digits, env = parent.frame()) { + op <- options(digits = sig_digits) + withr::defer(options(op), env) +} + +neater <- function(x, sig_digits) { + local_digits(1) + print(x) +} + +neater(pi) +#> [1] 3 +``` + + +## Storing as a Static Test Fixture {-} + +If a `useful_thing` is **costly to create**, in terms of time or memory. It can be saved in the `tests/testthat/fixtures/` folder, but with its corresponding companion **R script**. + +``` +. +├── ... +└── tests + ├── testthat + │ ├── fixtures + │ │ ├── make-useful-things.R + │ │ ├── useful_thing1.rds + │ │ └── useful_thing2.rds + │ ├── helper.R + │ ├── setup.R + │ └── (all the test files) + └── testthat.R +``` + +## Loading a Static Test Fixture {-} + +```r +test_that("foofy() does this", { + useful_thing1 <- readRDS(test_path("fixtures", "useful_thing1.rds")) + expect_equal(foofy(useful_thing1, x = "this"), EXPECTED_FOOFY_OUTPUT) +}) + +test_that("foofy() does that", { + useful_thing2 <- readRDS(test_path("fixtures", "useful_thing2.rds")) + expect_equal(foofy(useful_thing2, x = "that"), EXPECTED_FOOFY_OUTPUT) +}) +``` + + +## Helper Defined Inside a Test {-} + +A hyper-local helper like `trunc()` is particularly useful when it allows you to fit all the important business for each expectation on one line. + +```r +# from stringr (actually) +test_that("truncations work for all sides", { + + trunc <- function(direction) str_trunc( + "This string is moderately long", + direction, + width = 20 + ) + + expect_equal(trunc("right"), "This string is mo...") + expect_equal(trunc("left"), "...s moderately long") + expect_equal(trunc("center"), "This stri...ely long") +}) +``` + +> A helper like trunc() is yet another place where you can **introduce a bug**. + +## Custom expectations {-} + +- `expect_usethis_error()` checks that an error has the "usethis_error" class. + +```r +expect_usethis_error <- function(...) { + expect_error(..., class = "usethis_error") +} +``` + +- `expect_proj_file()` is a simple wrapper around file_exists() that searches for the file in the current project. + +```r +expect_proj_file <- function(...) { + expect_true(file_exists(proj_path(...))) +} +``` + +## Skipping a Test {-} + +Sometimes it's impossible to perform a test : + +- You may not have an internet connection. +- You may not have access to the necessary credentials. + +The only **drawback** is that skipping is completely invisible in CI, so you will need to **dig into the `R CMD check` results**. + +## Creating your Own Skip Functions {-} + +We can create the helper function. + +```r +skip_if_no_api() <- function() { + if (api_unavailable()) { + skip("API not available") + } +} +``` + +And later used each time needed. + +```r +test_that("foo api returns bar when given baz", { + skip_if_no_api() + ... +}) + +test_that("foo api returns an errors when given qux", { + skip_if_no_api() + ... +}) + +``` + +## Built-in skip() functions {-} + +There is a family of skip() functions that anticipate some common situations. + +```r +test_that("foo api returns bar when given baz", { + skip_if(api_unavailable(), "API not available") + ... +}) +test_that("foo api returns bar when given baz", { + skip_if_not(api_available(), "API not available") + ... +}) + +skip_if_not_installed("sp") +skip_if_not_installed("stringi", "1.2.2") + +skip_if_offline() +skip_on_cran() +skip_on_os("windows") +``` + +## Dangers of Skipping {-} + +if you automatically skip too many tests, it's easy to fool yourself that all your tests are passing when in fact **they're just being skipped!** + +## `skip()` in Summary {-} + +```r +devtools::test() +#> ℹ Loading abcde +#> ℹ Testing abcde +#> ✔ | F W S OK | Context +#> ✔ | 2 | blarg +#> ✔ | 1 2 | foofy +#> ──────────────────────────────────────────────────────────────────────────────── +#> Skip (test-foofy.R:6:3): foo api returns bar when given baz +#> Reason: API not available +#> ──────────────────────────────────────────────────────────────────────────────── +#> ✔ | 0 | yo +#> ══ Results ═════════════════════════════════════════════════════════════════════ +#> ── Skipped tests ────────────────────────────────────────────────────────────── +#> • API not available (1) +#> +#> [ FAIL 0 | WARN 0 | SKIP 1 | PASS 4 ] +#> +#> 🥳 +``` + +## Mocking {-} + +It is the action of **replacing** something that's: + +- _Complicated_ +- _Unreliable_ +- _Out of our control_ + +With something that's **fully within our control**. + +That is really useful if your package wraps an external API that **requires authentication** or **has occasional downtime**. + +## Tools for Mocking {-} + +**From `testthat`** + +Provide tools for "mocking", temporarily redefining a function so that it behaves differently during tests. + +- `with_mocked_bindings()` +- `local_mocked_bindings()` + +Other alternatives: + +- `mockery`: https://github.com/r-lib/mockery +- `mockr`: https://krlmlr.github.io/mockr/ +- `httptest`: https://enpiar.com/r/httptest/ +- `httptest2`: https://enpiar.com/httptest2/ +- `webfakes`: https://webfakes.r-lib.org + +## Manage credentials {-} + +It is likely that you will need to provide a set of test credentials to **fully test your package**. + +In that case you can use `httr2 ` which offers substantial support for **secret management**. + +```r +usethis::edit_r_environ() +``` + +Then you can add your key to your user-level `.Renviron` file. + +``` +YOURPACKAGE_KEY=key_you_generated_with_secret_make_key +``` + +Now you can restart R + +```r +rstudioapi::restartSession() +``` + +You also can add your key to your workflows. + +```yml + env: + YOURPACKAGE_KEY: ${{ secrets.YOURPACKAGE_KEY }} +``` + +## Special considerations for CRAN packages {-} + +Situations to use the helper `skip_on_cran()`: + +- For long-running tests you might want to skip it as CRAN packages need to run ideally in **less than a minute** in total. + +- For valuable, well-written tests that are prone to **occasional nuisance failure** like the accesses a website or web API. + +- For snapshot tests (default) used to monitor how various informational messages look. + +For CRAN packages you can only write into the session **temp directory** and to **clean up after** yourself. + +It is best to **turn off any clipboard** functionality in your tests. + ## Meeting Videos