Ending summary

15-advanced-testing-techniques.Rmd

@@ -2,12 +2,374 @@
 
 **Learning objectives:**
 
-- THESE ARE NICE TO HAVE BUT NOT ABSOLUTELY NECESSARY
+- Test Fixture Strategies
+- Effective Helper Function Creation
+- Side Effect Management
+- Fixture Loading Techniques
+- Custom Expectation Development
+- Testing Optimization and Skip Handling
 
-## SLIDE 1
 
-- ADD SLIDES AS SECTIONS (`##`).
-- TRY TO KEEP THEM RELATIVELY SLIDE-LIKE; THESE ARE NOTES, NOT THE BOOK ITSELF.
+## Test Fixtures {-}
+
+When it's not practical to make your test entirely self-sufficient.
+
+- Put repeated code in a constructor-type helper function.
+  - If the repeated code has **side effects** make sure to **clean up** afterwards by writing a custom local_*().
+
+- Otherwise save the result as a **static file** and load it.
+
+## Creating Helper Function {-}
+
+Creating a helper function make sense if it meet the following conditions:
+
+- Create it takes **several lines of code**.
+- The creating process doesn't requires much **time or memory**.
+
+It can be stored under any folder loaded by calling `devtools::load_all()`: 
+
+- Under the `R/` folder as not exported function.
+- Under a R script starting with "helper" under the `tests/testthat/`folder.
+
+```r
+new_useful_thing <- function() {
+  # your fiddly code to create a useful_thing goes here
+}
+```
+
+
+## Creating Helper Function {-}
+
+Then you will be able to call it each time needed.
+
+```r
+test_that("foofy() does this", {
+  useful_thing1 <- new_useful_thing()
+  expect_equal(foofy(useful_thing1, x = "this"), EXPECTED_FOOFY_OUTPUT)
+})
+
+test_that("foofy() does that", {
+  useful_thing2 <- new_useful_thing()
+  expect_equal(foofy(useful_thing2, x = "that"), EXPECTED_FOOFY_OUTPUT)
+})
+```
+
+> In many cases you might found out that the function can be **useful** even for creating **vignettes** and you might end **exporting the function**.
+
+## Avoiding Side Effects {-}
+
+If your functions needs to do any of the following actions you need to **clean up afterwards**:
+
+- Create a file or directory
+- Create a resource on an external system
+- Set an R option
+- Set an environment variable
+- Change working directory
+- Change an aspect of the tested package's state
+
+## Using `on.exit()` to clean up {-}
+
+```r
+neat <- function(x, sig_digits) {
+  op <- options(digits = sig_digits)
+  on.exit(options(op), add = TRUE, after = FALSE)
+  print(x)
+}
+
+pi
+#> [1] 3.141593
+neat(pi, 2)
+#> [1] 3.1
+pi
+#> [1] 3.141593
+```
+
+## `on.exit()` Limitations {-}
+
+- You should always call it with `add = TRUE` to ensure that the call is **added to the list** of deferred tasks (instead of replaces).
+
+- You should always call it with `after = FALSE` to ensure that the **cleanup occurs in reverse** order to setup.
+
+- It doesn't work in the global environment for interactive checking.
+
+- You can't wrap up `on.exit()` in a **helper function**.
+
+
+## Using `withr::defer()` to Clean Up {-}
+
+```r
+neat <- function(x, sig_digits) {
+  op <- options(digits = sig_digits)
+  withr::defer(options(op))
+  print(x)
+}
+```
+
+## `withr::defer()` benefits {-}
+
+- It has the behaviour we want by default; no extra arguments needed.
+
+- It works when called in the global environment thanks to `deferred_run()` and `deferred_clear()` as the global environment isn't perishable.
+
+```r
+withr::defer(print("hi"))
+#> Setting deferred event(s) on global environment.
+#>   * Execute (and clear) with `deferred_run()`.
+#>   * Clear (without executing) with `deferred_clear()`.
+
+withr::deferred_run()
+#> [1] "hi"
+```
+
+- It lets you pick **which function to clean up**. This makes it possible to create helper functions.
+
+```r
+local_digits <- function(sig_digits, env = parent.frame()) {
+  op <- options(digits = sig_digits)
+  withr::defer(options(op), env)
+}
+
+neater <- function(x, sig_digits) {
+  local_digits(1)
+  print(x)
+}
+
+neater(pi)
+#> [1] 3
+```
+
+
+## Storing as a Static Test Fixture {-}
+
+If a `useful_thing` is **costly to create**, in terms of time or memory. It can be saved in the `tests/testthat/fixtures/` folder, but with its corresponding companion **R script**.
+
+```
+.
+├── ...
+└── tests
+    ├── testthat
+    │   ├── fixtures
+    │   │   ├── make-useful-things.R
+    │   │   ├── useful_thing1.rds
+    │   │   └── useful_thing2.rds
+    │   ├── helper.R
+    │   ├── setup.R
+    │   └── (all the test files)
+    └── testthat.R
+```
+
+## Loading a Static Test Fixture {-}
+
+```r
+test_that("foofy() does this", {
+  useful_thing1 <- readRDS(test_path("fixtures", "useful_thing1.rds"))
+  expect_equal(foofy(useful_thing1, x = "this"), EXPECTED_FOOFY_OUTPUT)
+})
+
+test_that("foofy() does that", {
+  useful_thing2 <- readRDS(test_path("fixtures", "useful_thing2.rds"))
+  expect_equal(foofy(useful_thing2, x = "that"), EXPECTED_FOOFY_OUTPUT)
+})
+```
+
+
+## Helper Defined Inside a Test {-}
+
+A hyper-local helper like `trunc()` is particularly useful when it allows you to fit all the important business for each expectation on one line. 
+
+```r
+# from stringr (actually)
+test_that("truncations work for all sides", {
+
+  trunc <- function(direction) str_trunc(
+    "This string is moderately long",
+    direction,
+    width = 20
+  )
+
+  expect_equal(trunc("right"),   "This string is mo...")
+  expect_equal(trunc("left"),    "...s moderately long")
+  expect_equal(trunc("center"),  "This stri...ely long")
+})
+```
+
+> A helper like trunc() is yet another place where you can **introduce a bug**.
+
+## Custom expectations {-}
+
+- `expect_usethis_error()` checks that an error has the "usethis_error" class.
+
+```r
+expect_usethis_error <- function(...) {
+  expect_error(..., class = "usethis_error")
+}
+```
+
+- `expect_proj_file()` is a simple wrapper around file_exists() that searches for the file in the current project.
+
+```r
+expect_proj_file <- function(...) {
+  expect_true(file_exists(proj_path(...)))
+}
+```
+
+## Skipping a Test {-}
+
+Sometimes it's impossible to perform a test :
+
+- You may not have an internet connection.
+- You may not have access to the necessary credentials.
+
+The only **drawback** is that skipping is completely invisible in CI, so you will need to **dig into the `R CMD check` results**.
+
+## Creating your Own Skip Functions {-}
+
+We can create the helper function.
+
+```r
+skip_if_no_api() <- function() {
+  if (api_unavailable()) {
+    skip("API not available")
+  }
+}
+```
+
+And later used each time needed.
+
+```r
+test_that("foo api returns bar when given baz", {
+  skip_if_no_api()
+  ...
+})
+
+test_that("foo api returns an errors when given qux", {
+  skip_if_no_api()
+  ...
+})
+
+```
+
+## Built-in skip() functions {-}
+
+There is a family of skip() functions that anticipate some common situations.
+
+```r
+test_that("foo api returns bar when given baz", {
+  skip_if(api_unavailable(), "API not available")
+  ...
+})
+test_that("foo api returns bar when given baz", {
+  skip_if_not(api_available(), "API not available")
+  ...
+})
+
+skip_if_not_installed("sp")
+skip_if_not_installed("stringi", "1.2.2")
+
+skip_if_offline()
+skip_on_cran()
+skip_on_os("windows")
+```
+
+## Dangers of Skipping {-}
+
+if you automatically skip too many tests, it's easy to fool yourself that all your tests are passing when in fact **they're just being skipped!**
+
+## `skip()` in Summary {-}
+
+```r
+devtools::test()
+#> ℹ Loading abcde
+#> ℹ Testing abcde
+#> ✔ | F W S  OK | Context
+#> ✔ |         2 | blarg
+#> ✔ |     1   2 | foofy
+#> ────────────────────────────────────────────────────────────────────────────────
+#> Skip (test-foofy.R:6:3): foo api returns bar when given baz
+#> Reason: API not available
+#> ────────────────────────────────────────────────────────────────────────────────
+#> ✔ |         0 | yo                                                              
+#> ══ Results ═════════════════════════════════════════════════════════════════════
+#> ── Skipped tests  ──────────────────────────────────────────────────────────────
+#> • API not available (1)
+#> 
+#> [ FAIL 0 | WARN 0 | SKIP 1 | PASS 4 ]
+#> 
+#> 🥳
+```
+
+## Mocking {-}
+
+It is the action of **replacing** something that's:
+
+- _Complicated_
+- _Unreliable_
+- _Out of our control_
+
+With something that's **fully within our control**.
+
+That is really useful if your package wraps an external API that **requires authentication** or **has occasional downtime**.
+
+## Tools for Mocking {-}
+
+**From `testthat`**
+
+Provide tools for "mocking", temporarily redefining a function so that it behaves differently during tests.
+
+- `with_mocked_bindings()` 
+- `local_mocked_bindings()`
+
+Other alternatives:
+
+- `mockery`: https://github.com/r-lib/mockery
+- `mockr`: https://krlmlr.github.io/mockr/
+- `httptest`: https://enpiar.com/r/httptest/
+- `httptest2`: https://enpiar.com/httptest2/
+- `webfakes`: https://webfakes.r-lib.org
+
+## Manage credentials {-}
+
+It is likely that you will need to provide a set of test credentials to **fully test your package**.
+
+In that case you can use `httr2 ` which offers substantial support for **secret management**.
+
+```r
+usethis::edit_r_environ()
+```
+
+Then you can add your key to your user-level `.Renviron` file.
+
+```
+YOURPACKAGE_KEY=key_you_generated_with_secret_make_key
+```
+
+Now you can restart R
+
+```r
+rstudioapi::restartSession()
+```
+
+You also can add your key to your workflows.
+
+```yml
+    env:
+      YOURPACKAGE_KEY: ${{ secrets.YOURPACKAGE_KEY }}
+```
+
+## Special considerations for CRAN packages {-}
+
+Situations to use the helper `skip_on_cran()`:
+
+- For long-running tests you might want to skip it as CRAN packages need to run ideally in **less than a minute** in total.
+
+- For valuable, well-written tests that are prone to **occasional nuisance failure** like the accesses a website or web API.
+
+- For snapshot tests (default) used to monitor how various informational messages look.
+
+For CRAN packages you can only write into the session **temp directory** and to **clean up after** yourself.
+
+It is best to **turn off any clipboard** functionality in your tests. 
+
 
 ## Meeting Videos