diff --git a/lessons/R-complex-time-series-models-1/r_tutorial.qmd b/lessons/R-complex-time-series-models-1/r_tutorial.qmd index 94ddaae..e5872c2 100644 --- a/lessons/R-complex-time-series-models-1/r_tutorial.qmd +++ b/lessons/R-complex-time-series-models-1/r_tutorial.qmd @@ -142,6 +142,8 @@ mcmc_plot(baseline_model, type = "trace", variable = c("mintemp", "ar1[1]", "sig * But part of what's going on here is that we're using a poorly specified model given the data because we've assumed Gaussian errors * So let's look at the model & forecast and then keep going + + * We can look at the model structure using `summary()` ```r @@ -196,10 +198,14 @@ hist(rpois(n = 1000, lambda = 4.5)) * If the mean is 4.5 sometimes you'll draw a 4, sometimes a 5, sometimes a 0 or a 10 + + * $\lambda_t$ can be a decimal * We could expect an average of 4.5 rodents based on the environment even though we can only observe an integer number * But $\lambda_t$ it does have to be positive because we can't reasonably expect to see negative rodents + + * To handle this we use a log link function to give us only positive values of $\lambda_t$ * The log link means that instead of modeling $\lambda_t$ directly we model $log(\lambda_t)$ @@ -226,6 +232,8 @@ plot(poisson_model) * Model structure looks OK, be we still have some residual seasonal lag we haven't captured + + * Now let's look at the forecast ```r @@ -296,6 +304,8 @@ plot_predictions(poisson_gam_model, condition = "mintemp") * Abundance then increases with temperature * But starts to asymptote at higher temperatures + + * How does this influence the forecasts ```r diff --git a/lessons/R-complex-time-series-models-2/r_tutorial.qmd b/lessons/R-complex-time-series-models-2/r_tutorial.qmd index b76e541..7d6e9d8 100644 --- a/lessons/R-complex-time-series-models-2/r_tutorial.qmd +++ b/lessons/R-complex-time-series-models-2/r_tutorial.qmd @@ -142,6 +142,8 @@ mcmc_plot(baseline_model, type = "trace", variable = c("mintemp", "ar1[1]", "sig * But part of what's going on here is that we're using a poorly specified model given the data because we've assumed Gaussian errors * So let's look at the model & forecast and then keep going + + * We can look at the model structure using `summary()` ```r @@ -196,10 +198,14 @@ hist(rpois(n = 1000, lambda = 4.5)) * If the mean is 4.5 sometimes you'll draw a 4, sometimes a 5, sometimes a 0 or a 10 + + * $\lambda_t$ can be a decimal * We could expect an average of 4.5 rodents based on the environment even though we can only observe an integer number * But $\lambda_t$ it does have to be positive because we can't reasonably expect to see negative rodents + + * To handle this we use a log link function to give us only positive values of $\lambda_t$ * The log link means that instead of modeling $\lambda_t$ directly we model $log(\lambda_t)$ @@ -226,6 +232,8 @@ plot(poisson_model) * Model structure looks OK, be we still have some residual seasonal lag we haven't captured + + * Now let's look at the forecast ```r @@ -296,6 +304,8 @@ plot_predictions(poisson_gam_model, condition = "mintemp") * Abundance then increases with temperature * But starts to asymptote at higher temperatures + + * How does this influence the forecasts ```r diff --git a/lessons/R-evaluating-forecasts-2/r_tutorial.qmd b/lessons/R-evaluating-forecasts-2/r_tutorial.qmd index 71a967e..0f0912b 100644 --- a/lessons/R-evaluating-forecasts-2/r_tutorial.qmd +++ b/lessons/R-evaluating-forecasts-2/r_tutorial.qmd @@ -112,6 +112,8 @@ length(in_interval[in_interval == TRUE]) / length(in_interval) * Instead of evaluating the mean prediction (point forecast) evaluate the prediction interval * _Add two sets of intervals on the axes_ + + * Winkler Score * Width of the prediction interval + a penalty for points outside the interval @@ -142,6 +144,8 @@ accuracy(forecasts, test, list(winkler = winkler_score), level = 80) * _Add a distribution to the axes with the mode matching $\hat{y}$_ * The best models most closely match the empirical distribution + + * Doing this is technically complicated * Continuous Rank Probability Score * Scores each value relative to the predicted cumulative distribution function diff --git a/lessons/R-evaluating-forecasts/r_tutorial.qmd b/lessons/R-evaluating-forecasts/r_tutorial.qmd index c37147b..20ce46f 100644 --- a/lessons/R-evaluating-forecasts/r_tutorial.qmd +++ b/lessons/R-evaluating-forecasts/r_tutorial.qmd @@ -68,6 +68,8 @@ glance(portal_models) * We know we don't have this for TSLM, so it's likelihood and IC values are invalid * These comparisons also typically tell us about 1 step ahead forecasts and we may care about more steps + + * So it's best to evaluate forecasts themselves * Ideally we make a forecast for the future, collect new data, and then see how well the forecast performed * This is the gold standard, but it requires waiting for time to pass @@ -171,6 +173,8 @@ $$RMSE = \sqrt{mean[(y_{t+h} - \hat{y}_{t+h})^2]}$$ * Which gives us a measure of the average error in the units of the response variable + + * `accuracy` function shows a number of common measures of forecast accuracy * 1st argument: forecast object from model on training data * 2nd argument: test data time-series diff --git a/lessons/R-intro-to-forecasting/r_tutorial.qmd b/lessons/R-intro-to-forecasting/r_tutorial.qmd index 2a36ebc..2664635 100644 --- a/lessons/R-intro-to-forecasting/r_tutorial.qmd +++ b/lessons/R-intro-to-forecasting/r_tutorial.qmd @@ -92,6 +92,8 @@ ar1_forecast * The expected value, or point forecast, is in `$.mean` * And information about the error term in `abundance ` + + * Change the number of time-steps in the forecast using h (default to 2 seasonal cycles) ```r @@ -140,6 +142,9 @@ autoplot(ar1_forecast, pp_data) * This lets us run a single forecast including a randomly chosen value for $epsilon_t$ at each time step * Let's run it a few times + + + * Quantify variability of forecast outcomes * If we set `times` to `1000` @@ -152,6 +157,9 @@ autoplot(ar1_forecast, pp_data) * Shaded areas provides uncertainty * These are the "prediction intervals" * The region within which some percentage of our 1000 forecast values occurred + + + * For many models the values can be calculated without making separate forecasts * This is the default behavior that we first saw @@ -186,6 +194,9 @@ $$y_t = c + \beta_1 x_{1,t} + \epsilon_t$$ * So we need values of $x_{1,t}$ to make predictions for time step $t$ * To forecast with covariates we need forecasts for those covariates + + + * Since our time-series ended in 2020 we'll use the observed values for the next two years ```r diff --git a/lessons/R-species-distribution-models/r_tutorial.qmd b/lessons/R-species-distribution-models/r_tutorial.qmd index 4bfd2b1..0f416d7 100644 --- a/lessons/R-species-distribution-models/r_tutorial.qmd +++ b/lessons/R-species-distribution-models/r_tutorial.qmd @@ -41,6 +41,8 @@ library(terra) * Fore forecasts of environmental data we'll use CMIP5 50 year forecasts (CMIP = coupled model intercomparison project) * Download data from [https://course.naturecast.org/data/sdm_data.zip](https://course.naturecast.org/data/sdm_data.zip) and unzip it into your working directory + + * The hooded warbler data is in a csv file * Load it using `read.csv()` @@ -51,6 +53,8 @@ head(hooded_warb_data) * We can see that this file has information on locations as latitude & longitude plus whether or not the species is present at that location + + * The environmental data is stored in raster stacks in `.grd` files * Load those using the `rast()`, which is part of the `terra` package @@ -153,6 +157,8 @@ points(present_loc, pch='+', cex = 0.5) * The 1:1 line on this graph is what we would get if we randomly assigned each point to be present or absent * So we definitely want a model that is above that line + + * Let's make our own ROC curve * Split the data in into presences and absences diff --git a/lessons/R-state-space-models-1/r_tutorial.qmd b/lessons/R-state-space-models-1/r_tutorial.qmd index bac5625..46eca4f 100644 --- a/lessons/R-state-space-models-1/r_tutorial.qmd +++ b/lessons/R-state-space-models-1/r_tutorial.qmd @@ -189,6 +189,8 @@ plot(time, predictions, type = "l") * And this looks very similar to the observed dynamics of y + + * Add prediction intervals as range containing 95% of MCMC samples ``` diff --git a/lessons/R-state-space-models-1/r_tutorial_mvgam.qmd b/lessons/R-state-space-models-1/r_tutorial_mvgam.qmd index b289e6f..904d489 100644 --- a/lessons/R-state-space-models-1/r_tutorial_mvgam.qmd +++ b/lessons/R-state-space-models-1/r_tutorial_mvgam.qmd @@ -310,6 +310,8 @@ plot(time, predictions, type = "l") * And this looks very similar to the observed dynamics of y + + * Add prediction intervals as range containing 95% of MCMC samples ``` diff --git a/lessons/R-state-space-models-2/r_tutorial.qmd b/lessons/R-state-space-models-2/r_tutorial.qmd index 8b1e131..6695f15 100644 --- a/lessons/R-state-space-models-2/r_tutorial.qmd +++ b/lessons/R-state-space-models-2/r_tutorial.qmd @@ -27,6 +27,8 @@ excellent * This means that this type of modeling approach also works naturally with missing data * This is different from many types of time-series models that require continuously sampled time-series + + * To illustrate this we'll add `NA`'s to a portion of our time series as an example of missing data * Go back to the line where we replaced the end of the time-series with `NA`'s and add some `NA`'s inside the training portion of the data * We'll use two examples of missing data diff --git a/lessons/R-time-series-data/material.qmd b/lessons/R-time-series-data/material.qmd index 3892092..7ae5064 100644 --- a/lessons/R-time-series-data/material.qmd +++ b/lessons/R-time-series-data/material.qmd @@ -10,4 +10,6 @@ order: 1 * [NEON_HarvardForest_date.csv](/data/NEON_Harvardforest_date_2001_2006.csv) * [NEON_HarvardForest_datetime.csv](/data/NEON_Harvardforest_datetime.csv) + + * R_tutorial videos and written notes available through R tutorial link. diff --git a/lessons/R-time-series-modeling-2/r_tutorial.qmd b/lessons/R-time-series-modeling-2/r_tutorial.qmd index 1d3eda8..3aacb01 100644 --- a/lessons/R-time-series-modeling-2/r_tutorial.qmd +++ b/lessons/R-time-series-modeling-2/r_tutorial.qmd @@ -46,6 +46,8 @@ data_ts * Greenness isn't really driven by greenness in the same way a populations abundance is * But if it's greener than normal this month it's probably a good year so it's likely to be greener than normal next month + + * Both AR and MA structures can be present in time-series * We can combine them in an ARMA model @@ -63,6 +65,8 @@ data_ts * So, if we difference the data we are now modeling the change from time-step to time-step, not the actual values + + * Adding AR, MA, and differencing together gives us an ARIMA model * AR: Autoregressive - Uses past values to predict future values * I: Integrated - Uses differencing to handle trends @@ -93,6 +97,8 @@ report(arima_model) * So, if NDVI was above average over the last 3 months we expect it to be above average now * This lines up with our idea that MA might be better for NDVI since it indicates a good year + + * But what is the `sar1` coefficient? * That's our season signal * A seasonal autoregressive term of order 1 diff --git a/lessons/R-time-series-modeling-3/r_tutorial.qmd b/lessons/R-time-series-modeling-3/r_tutorial.qmd index a0661dc..68e4002 100644 --- a/lessons/R-time-series-modeling-3/r_tutorial.qmd +++ b/lessons/R-time-series-modeling-3/r_tutorial.qmd @@ -91,6 +91,8 @@ gg_tsresiduals(tslm_model) * This ramp suggests that there might be a trend * It's possible we have a decrease over time + + * We can try to model the trend explicitly to remove the autocorrelation * We can do this by adding a fitted trend to our model using `trend()` @@ -186,6 +188,8 @@ gg_tsresiduals(arima_exog_model) * Look at the report, is this model better than the model with just mintemp? * What does that tell us? + + * But did you notice anything else weird? * Negative abundances don't make sense * Why do we have them? diff --git a/lessons/R-time-series-modeling/r_tutorial.qmd b/lessons/R-time-series-modeling/r_tutorial.qmd index a4bdb22..052acd3 100644 --- a/lessons/R-time-series-modeling/r_tutorial.qmd +++ b/lessons/R-time-series-modeling/r_tutorial.qmd @@ -163,11 +163,15 @@ gg_tsdisplay(data_ts, NDVI) * b_1 is a coefficient determining how y_t is related to y at a 1 time-step lag, i.e., the previous time step * e_t is normally distributed error + + * Does this model remind you of a biological model? * This model is basically a Gompertz population model if y is log(N) * The idea is that the current value influences the future values * Makes a lot of sense for things like population dynamics + + * Since we have also have lag 2 autocorrelation we add a term for two time steps back > `y_t = c + b1 * y_t-1 + b2 * y_t-2 + e_t, where e_t ~ N(0, sigma)` diff --git a/lessons/election-forecasts/discussion_questions.qmd b/lessons/election-forecasts/discussion_questions.qmd index 7d0dd47..0b5dc35 100644 --- a/lessons/election-forecasts/discussion_questions.qmd +++ b/lessons/election-forecasts/discussion_questions.qmd @@ -8,6 +8,8 @@ order: 2 * What dot he simulations tell us? * How does this approach compare to that used for hurricane forecasting? What are the similarities and differences? + + * What is the main difference between 538's approach and MRP? * What data does it use and what are the sources of uncertainty related to it? * Why does the author argue that this method pointing in the same direction as other methods should strengthen the overall strength of our predictions? diff --git a/lessons/intro-time-series-data/instructor_notes.qmd b/lessons/intro-time-series-data/instructor_notes.qmd index 87621f8..75ee312 100644 --- a/lessons/intro-time-series-data/instructor_notes.qmd +++ b/lessons/intro-time-series-data/instructor_notes.qmd @@ -19,10 +19,14 @@ order: 4 * Resurvey -> Idiosyncratic -> Continuous w/Gaps -> Continuous * Place purely spatial data off on one end the side with a "spatial only" label. + + * Ask what types of data are easier/harder to collect * Ask what types are more common for going further back in time vs. more recent * (optionally) Draw these gradients on board + + * How long do you think most ecological studies/time-series are? * Show: Recent study by Estes et al. 2018 diff --git a/lessons/paleo-dynamics/material.qmd b/lessons/paleo-dynamics/material.qmd index a49a786..db33214 100644 --- a/lessons/paleo-dynamics/material.qmd +++ b/lessons/paleo-dynamics/material.qmd @@ -5,4 +5,6 @@ order: 1 * Reading: Just the Introduction of [Ecological Change, Range Fluctuations and Population Dynamics during the Pleistocene](https://doi.org/10.1016/j.cub.2009.06.030) (i.e. stop when you reach the section titled "The Glacial Refugium Theory") + + * Reading: Read all of [Novel climates, no-analog communities, and ecological surprises](https://doi.org/10.1890/070037) diff --git a/lessons/uncertainty-dietze-ch-2/instructor_notes.qmd b/lessons/uncertainty-dietze-ch-2/instructor_notes.qmd index 48d47ee..3a72a8c 100644 --- a/lessons/uncertainty-dietze-ch-2/instructor_notes.qmd +++ b/lessons/uncertainty-dietze-ch-2/instructor_notes.qmd @@ -46,6 +46,8 @@ order: 4 * How does disturbance make forecasting difficult? * What could be done to ameliorate the influence of disturbance on forecasts? + + * Which components of uncertainty (e.g., in eq. 2.1) have the potential to grow through time? * What implications does this have for our ability to forecast? @@ -57,6 +59,8 @@ order: 4 * What are the implications for forecasting? * How can we tell them apart? + + * What kinds of uncertainty/variability do you think are most important in ecological systems? * If a forecast model doesn't include all of these sources of variance can it be