Interactive Exercises (2024)

Source: vignettes/articles/exercises.Rmd

exercises.Rmd

Overview

Exercises are interactive R code chunks that allow readers todirectly execute R code and see its results.

Exercise Options

There are many options associated with tutorial exercises (all ofwhich are described in more detail below):

Option	Description
`exercise.cap`	Caption for exercise chunk (defaults to “Code”)
`exercise.eval`	Whether to pre-evaluate the exercise so the reader can see somedefault output (defaults to `FALSE`).
`exercise.lines`	Lines of code for exercise editor (default to size of codechunk).
`exercise.pipe`	The code to insert when the user presses`Ctrl/Cmd + Shift + M` (defaults to `\|>` for R>= 4.1.0, otherwise `%>%`).
`exercise.timelimit`	Number of seconds to limit execution time to (defaults to 30).
`exercise.checker`	Function used to check exercise answers (e.g.,`gradethis::grade_learnr()`).
`exercise.blanks`	Regular expression to find blanks requiring replacement in theexercise code. See Checking Blanksbelow.
`exercise.error.check.code`	A string containing R code to use for checking code when an exerciseevaluation error occurs (e.g.,`"gradethis::grade_code()"`).
`exercise.completion`	Whether to enable code completion in the exercise editor.
`exercise.diagnostics`	Whether to enable code diagnostics in the exercise editor.
`exercise.startover`	Whether to include a “Start Over” button for the exercise.
`exercise.warn_invisible`	Whether to display an invisible result warning if the last valuereturned is invisible.
`exercise.reveal_solution`	Whether or not the solution should be revealed to the user (defaultsto `TRUE`). See Hiding Solutionsbelow.

Note that these options can all be specified either globally orper-chunk. For example, the following code sets global default optionsusing the setup chunk and also sets some local options onthe addition chunk:

```{r setup, include=FALSE}library(learnr)tutorial_options(exercise.timelimit = 60)``````{r addition, exercise=TRUE, exercise.timelimit = 60}1 + 1```

Exercise Support Chunks

There are also some other specialized chunks that can be used with anexercise chunk. These chunks a linked together

Exercise -setup chunksenable you to execute code to setup the environment immediately prior toexecuting submitted code.
Exercise -solutionchunks enable you to provide a solution to the exercise that can beoptionally viewed by users of the tutorial.
Exercise -hint chunksare used to provide a series of hints to help the user progress throughthe exercise.
Exercise -check,-code-check, and -error-check chunks areused to provide logic for checking the user’s attempt to solve theexercise.
Finally, exercise -tests chunks can store testcases or testing code related to the exercise.

The use of these special chunks is also described in detailbelow.

Exercise Evaluation

By default, exercise code chunks are NOT pre-evaluated (i.e there isno initial output displayed for them). However, in some cases you maywant to show initial exercise output (especially for exercises like theones above where the user is asked to modify code rather than write newcode from scratch).

You can arrange for an exercise to be pre-evaluated (and its outputshown) using the exercise.eval chunk option. This optioncan also be set either globally or per-chunk:

```{r setup, include=FALSE}library(learnr)tutorial_options(exercise.eval = TRUE)``````{r filter, exercise=TRUE, exercise.eval=FALSE}# Change the filter to select February rather than Januaryfilter(nycflights, month == 1)```

Exercise Setup

Code chunks with exercise=TRUE are evaluated withinstandalone environments. This means that they don’t have access toprevious computations from within the document. This constraint isimposed so that users can execute exercises in any order (i.e.correctexecution of one exercise never depends on completion of a priorexercise).

Exercise Setup Chunks

You can arrange for setup code to be run before evaluation of anexercise to ensure that the environment is primed correctly. There arethree ways to provide setup code for an exercise:

Add code to a global setupchunk
Create a shared setup chunk
Create an exercise-specific setupchunk

Each of these is described in more detail below. Note that you mayalso chain setup chunks, which isparticularly helpful when exercises build upon each other.

Add code to the global `setup` chunk

Code in the global setup chunk is run once at thestartup of the tutorial and is shared by all exercises within thetutorial. For example:

```{r setup, include=FALSE}nycflights

If you don’t want to rely on global setup but would rather createsetup code that’s used by only a handful of exercises you can use theexercise.setup chunk attribute to provide the label ofanother chunk that will perform setup tasks. To illustrate, we’llre-write the previous example to use a shared setup chunk namedprepare-flights:

Create an exercise-specific `-setup` chunk

learnr will automatically associate any chunk with alabel in the format <exercise>-setup as a setup chunkspecifically associated with <exercise>, where<exercise> is replaced with the label of the exercisechunk. For example, the filter-setup chunk will be used asthe setup chunk for the filter exercise:

Chained setup chunks

If may also chain setup chunks where each setup chunk inherits itsparent setup chunk using the exercise.setup chunk option.(Note: You must use exercise.setup forchaining. You cannot rely on the -setup suffix labelingscheme.) learnr will keep following the trail ofexercise.setup chunks until there are no more chunks to befound. To demonstrate, we can convert the first exercise in the aboveexamples to be another setup chunk called filtered-flightswith its exercise.setup=prepare-flights. This will nowfilter the data and store it and can be referenced inside thearrange exercise:

```{r prepare-flights}nycflights

You can chain use exercise chunks in the setup chunk chain, but keepin mind that the code as it appears in the R Markdown source is used toserve as the setup code, not the user input. For example, if you turnedfiltered-flights back to an exercise, the pre-filled codeis used as setup for the arrange exercise that use it asits setup:

```{r prepare-flights}nycflights

Using Files in Exercises

Occasionally, you may write an exercise that requires users tointeract with files on disk, such as data files in .csv or.rds format. learnr will look for a foldernamed data/ in the same directory as the tutorial sourceand, if present, will make this directory available to users in allexercises in the tutorial.

To ensure consistency between each evaluation of the user’s code,users interact with a temporary copy of the original directory. Thisway, users cannot overwrite or delete the original files. Additionally,in the exercise, the directory is always called data/.

There are three ways authors can include files for use inexercises:

Store the files in a data/ directory in the samedirectory as the tutorial.
Use the setup chunk to download or write the files to adata/ directory.
Use the tutorial.data_dir global option or theTUTORIAL_DATA_DIR environment variables to specify a pathto a data directory.

In the example below, the global setup chunk is used to writedata/flights_jan.csv and users are asked to load this filewith read_csv().

```{r setup}library(tidyverse)flights_jan % filter(month == 2) dir.create("data", showWarnings = FALSE)write_csv(flights_jan, "data/flights_jan.csv")``````{r read-flights, exercise=TRUE}read_csv("data/flights_jan.csv")```

Hints and Solutions

You can optionally provide a hint or solution for each exercise thatcan be optionally displayed by users. Hints can be based on either Rcode snippets or on custom markdown/HTML content.

R Code Hints

To create a hint or solution based on R code simply create a new codechunk with “-hint” or “-solution” chunk label suffix. For example:

```{r filter, exercise=TRUE}# Change the filter to select February rather than Januarynycflights

A “Hint” or “Solution” button is added to the left side of theexercise header region:

Markdown Hints

To create a hint based on custom markdown content simply add a<div> tag with an id attribute thatmarks it as hint for your exercise (e.g.“filter-hint”). Forexample:

```{r filter, exercise=TRUE}# filter the flights table to include only United and American flightsflights```**Hint:** You may want to use the dplyr `filter` function.

The content within the <div> will be displayedunderneath the R code editor for the exercise whenever the user pressesthe “Hint” button.

Multiple Hints

For R code hints you can provide a sequence of hints that revealprogressively more of the solution as desired by the user. To do thiscreate a sequence of indexed hint chunks (e.g.“-hint-1”,“-hint-2,”-hint-3”, etc.) for your exercise chunk. For example:

```{r filter, exercise=TRUE}# filter the flights table to include only United and American flightsflights``````{r filter-hint-1}filter(flights, ...)``````{r filter-hint-2}filter(flights, UniqueCarrier=="AA")``````{r filter-hint-3}filter(flights, UniqueCarrier=="AA" | UniqueCarrier=="UA")```

Hiding Solutions

By default, the exercise solution is made available to the user withthe “Solution” or “Hint” button (if there are hints those will appearfirst). If you would prefer not to reveal the solution to an exercise,you can disable revealing the solution by addingexercise.reveal_solution = FALSE to the chunk options ofeither the exercise or its corresponding *-solutionchunk.

```{r filter, exercise=TRUE}# filter the flights table to include only United and American flightsflights``````{r filter-hint-1}filter(flights, ...)``````{r filter-hint-2}filter(flights, UniqueCarrier=="AA")``````{r filter-solution, exercise.reveal_solution = FALSE}filter(flights, UniqueCarrier=="AA" | UniqueCarrier=="UA")```

You can also set this option globally in the globalsetup chunk with tutorial_options(). When setthis way, the chunk-level option will take precedence over the globaloption so that you can choose to always reveal or hide the solution to aparticular exercise. The current default is to reveal exercisesolutions, but in a future version of learnr the default behavior willchange to hide solutions.

Progressive Reveal

You might want users of your tutorials to see only one sub-topic at atime as they work through the material (this can be helpful to reducedistractions and maintain focus on the current task). If you specify theprogressive option then all Level 3 headings(###) will be revealed progressively. For example:

---title: "Hello, Tutorial!"output: learnr::tutorial: progressive: trueruntime: shiny_prerendered---

You can also specify this option on a per topic basis using thedata-progressive attribute. For example, the following codeenables progressive rendering for a single topic:

## Topic 1 {data-progressive=TRUE}

You can also use data-progressive=FALSE to disableprogressive rendering for an individual topic if the globalprogressive option is TRUE.

Progressive reveal provides an easy way to cue exercises one at atime: place each exercise under its own Level 3 header(###). This can be useful when a second exercises builds onthe first, giving away the answer to the first.

Note that this feature is only available if you are using the learnr::tutorial RMarkdown format (other custom formats may have their own internalmechanisms for progressive reveal).

Exercise Skipping

When the progressive option is set to true, the tutorialwill require students to complete any exercises in a sub-section beforeadvancing to the next sub-section.

You may want to allow users of your tutorials to skip throughexercises that they can’t quite figure out. This might especially betrue if you want users to be able to optionally see the next exerciseeven if they haven’t completed the prior one.

If you specify the allow_skip option then students willbe able to advance through a sub-section without completing theexercises. For example:

---title: "Hello, Tutorial!"output: learnr::tutorial: progressive: true allow_skip: trueruntime: shiny_prerendered---

You can also specify this option on a per sub-topic basis using thedata-allow-skip attribute. For example, the following codeenables exercise skipping within a single sub-topic:

### Exercise 2 {data-allow-skip=TRUE}

You can also use data-allow-skip=FALSE to disableexercise skipping rendering for an individual sub-topic if the globalallow-skip option is TRUE.

Exercise Checking

learnr provides allows full control over feedbackprovided to exercise submissions via exercise.checker intutorial_options(). We’ll eventually cover how to implementa custom exercise.checker, but for sake of demonstration,this section uses gradethis’sapproach to exercise checking, which doesn’t require knowledge ofexercise.checker (NOTE:gradethis is still a work-in-progress. You may want toconsider alternatives such as checkr).To use gradethis’s approach to exercise checking insideof learnr, just call gradethis_setup() ina setup chunk, which will settutorial_options(exercise.checker = gradethis::grade_learnr)(among other things).

Checking Results

Checking of exercise results may be done through a*-check chunk. With a gradethis setup,results can be graded with grade_result(), like so:

```{r setup, include=FALSE}library(learnr)gradethis::gradethis_setup()```* Submit `1+1` to receive a correct grade.```{r exercise1, exercise = TRUE}``` ```{r exercise1-check}grade_result( pass_if(~identical(.result, 2)))```

When an exercise *-check chunk is provided,learnr provides an additional “Submit Answer” button,which allows users to experiment with various answers before formallysubmitting an answer:

Checking Code

Checking of exercise code may be done through a*-code-check chunk. With a gradethissetup, if you supply a *-solution chunk and callgrade_code() inside *-code-check, then you getdetection of differences in the R syntax between the submitted exercisecode and the solution code.

```{r setup, include=FALSE}library(learnr)gradethis::gradethis_setup()```* Submit `1+1` to receive a correct grade.```{r exercise1, exercise = TRUE}``````{r exercise1-solution}1+1``` ```{r exercise1-code-check}grade_code()```

It’s worth noting that, when a *-code-check chunk issupplied, the check is done prior to evaluation of the exercisesubmission, meaning that if the *-code-check chunk returnsfeedback, then that feedback is displayed, no exercise code isevaluated, and no result check is performed.

Checking Blanks

Occasionally, you may include blanks in the pre-filled code in yourexercise prompts — sections of the code in the exercise prompt thatstudents should fill in. By default, learnr will detectsequences of three or more underscores, e.g.____ asblanks, regardless of where they appear in the user’s submittedcode.

Fill in the blank to create an expression that adds up to **4**.```{r blank, exercise = TRUE, exercise.blanks = "___+"}1 + ____``````{r blank-solution}1 + 3```

You can choose your own blank pattern by setting theexercise.blanks chunk option to a regular expression thatidentifies your blanks. You may also setexercise.blanks = TRUE to use the defaultlearnr blank pattern, orexercise.blanks = FALSE to skip blank checking altogether.Globally tutorial_options() can be used to set the value ofthis argument for all exercises.

Submitted code with blanks will still be evaluated, but the otherexercise checks will not be performed. This lets the student see theoutput of their code—which may produce a valid result—but still drawsthe student’s attention to the code that needs to be completed.

Checking Errors

In the event that an exercise submission generates an error, checkingof the code (or its result, which is an error condition) may be donethrough either a *-error-check chunk or through the globalexercise.error.check.code option. If an*-error-check chunk is provided, you must also include a*-check chunk, typically to provide feedback in case thesubmission doesn’t generate an error.

With a gradethis setup,exercise.error.check.code is set tograde_code(). This means that, by default, users willreceive intelligent feedback for a submission that generates an errorusing the *-solution chunk, if one is provided.

To learn more about grading exercises withgradethis, see its grading demo (gradethis::gradethis_demo()).

Custom Checking

If you need custom exercise checking logic that isn’t alreadyprovided grading packages like gradethis,then you may want to write your own exercise.checkerfunction. This function is called on exercise submission whenever*-check or *-code-check chunks exist. Whencalled, this function receives all the information thatlearnr knows about the exercise at the time of thechecking, including the user_code,solution_code, check_code, exerciseenvironments, the last_value (if any), and thestage of checking. Checking can be performed at threedifferent time points, so the values supplied can differ depending onthe time point:

Before exercise evaluation, at this stage:
- stage is "code_check".
- check_code contains *-code-check chunkcode.
- envir_result, evaluate_result, andlast_value are all NULL.
During evaluation, when an error occurs:
- stage is "error_check".
- check_code contains *-error-check chunkcode.
- last_value contains the error condition object.
After exercise evaluation, at this stage:
- stage is "check".
- check_code contains *-check chunkcode.
- last_value contains the last printed value.

If, at any one of these stages, feedback should be provided, thenexercise.checker should return an R list with, at the veryleast, a correct flag and feedbackmessage:

Field	Description
`message`	Feedback message. Can be a plain character vector or can HTMLproduced via the htmltoolspackage.
`correct`	TRUE/FALSE logical value indicating whether the submitted answer wascorrect.
`type`	Feedback type (visual presentation style). Can be “auto”, “success”,“info”, “warning”, “error”, or “custom”. Note that “custom” implies thatthe “message” field is custom HTML rather than a character vector.
`location`	Location for feedback (“append”, “prepend”, or “replace”).

Below is a rough sketch of how one might implement anexercise.checker function.

```rcustom_checker

See the table below for a full description of all the argumentssupplied to exercise.checker.

Argument	Description
`label`	Label for exercise chunk.
`user_code`	R code submitted by the user.
`solution_code`	Code provided within the “-solution” chunk for the exercise.
`check_code`	Code provided within the “-check” chunk for the exercise.
`envir_result`	The R environment after the execution of thechunk.
`evaluate_result`	The return value from the `evaluate::evaluate`function.
`envir_prep`	A copy of the R environment before the execution ofthe chunk.
`last_value`	The last value of the evaluated chunk.
`engine`	The `engine` value of the evaluated chunk.
`stage`	The current checking `stage`.
`...`	Unused (include for compatibility with parameters to be added in thefuture)

Test Code or Cases

In some cases, you may want to record test cases — examples ofalternative solutions, common mistakes made by students, edge cases thatare missed by your checking code, etc.

You can place this test code in a *-tests chunk. It willbe preserved in the tutorial’s R Markdown source and stored in learnr’sinternal data for the exercise, but it won’t appear in the tutorialtext. learnr doesn’t currently provide a mechanism for testingexercises, but support for exercise testing is on the roadmap for futuredevelopment.

```{r addition, exercise = TRUE}``````{r addition-solution}1 + 1``````{r addition-tests}1 + 1# one plus two ----1 + 2# one plus three ----1 + 3# one equals three ----1 = 3# 2 minus one ----2 - 1```

Exercise Caption

By default exercises are displayed with caption of “Code”. However,in some cases you may want either a custom per-chunk caption or ageneric caption with a different connotation (e.g.“Exercise” or“Sandbox”). For example:

```{r setup, include=FALSE}library(learnr)tutorial_options(exercise.cap = "Sandbox")```

Code Assistance

By default, code completions are automatically provided as users typewithin the exercise editor:

You can optionally disable code completion (either globally or on aper-chunk basis) using the exercise.completion option. Forexample, the following illustrates turning off completions globally thenenabling them for an individual chunk:

```{r setup, include=FALSE}library(learnr)tutorial_options(exercise.completion = FALSE)``````{r histogram-plot, exercise=TRUE, exercise.completion=TRUE}```

Similarly, simple code diagnostics can also be provided, to informusers of errors in the R code written in exercise editors. Diagnosticscan be controlled (either globally or on a per-chunk basis) using theexercise.diagnostics option.

Editor Size

By default, the size of the exercise editor provided to users willmatch the number of lines in your code chunk (with a minimum of 2lines). If the user adds additional lines in the course of editing theeditor will grow vertically up to 15 lines, after which it will displaya scrollbar.

You can also specify a number of lines explicitly using theexercise.lines chunk option (this can be done on aper-chunk or global basis). For example, the following chunk specifiesthat the exercise code editor should be 15 lines high:

```{r add-function, exercise=TRUE, exercise.lines=15}# Write a function to add two numbers togetheradd_numbers

Time Limits

To mediate the problem of code which takes longer than expected torun you can specify the exercise.timelimit chunk option oralternatively the global tutorial.exercise.timelimitoption.

The following demonstrates setting a 10 second time limit as a globaloption, document level option, and chunk level option:

options(tutorial.exercise.timelimit = 10)```{r setup, include=FALSE}tutorial_options(exercise.timelimit = 10)``````{r exercise1, exercise=TRUE, exercise.timelimit=10}```

Since tutorials are a highly interactive format you should in generalbe designing exercises that take no longer than 5 or 10 seconds toexecute. Correspondingly, the default value fortutorial.exercise.timelimit if not otherwise specified is30 seconds.