Merge pull request #145 from UA-Libraries-Research-Data-Services/update-r-recipes

Update Wiley and ScienceDirect in R recipes

Author: MoenMi

requirements.txt

@@ -1,3 +1,3 @@
-jupyter-book
+jupyter-book<2
 matplotlib
 numpy

src/python/usa-spending.ipynb

@@ -17,7 +17,7 @@
     "- Terms\n",
     "    - <a href=\"https://github.com/fedspendingtransparency/usaspending-api?tab=CC0-1.0-1-ov-file\" target=\"_blank\">USAspending API License</a>: <a href=\"\" target=\"_blank\">CC0 1.0 Univeral</a>\n",
     "- Data Reuse\n",
-    "    - <a href=\"https://www.usaspending.gov/about#about-licensing\" target=\"_blank\">USspending Data Reuse</a>\n",
+    "    - <a href=\"https://www.usaspending.gov/about#about-licensing\" target=\"_blank\">USAspending Data Reuse</a>\n",
     "\n",
     "*These recipe examples were tested on May 5, 2025.*"
    ]

src/r/arxiv.md

@@ -30,9 +30,10 @@ Please see the following resources for more information on API usage:
 
 The following packages libraries need to be installed into your environment to run the code examples in this tutorial. These packages can be installed with `install.packages()`.
 
-- <a href="https://cran.r-project.org/web/packages/aRxiv/index.html" target="_blank">arXiv: Interface to the arXiv API</a>
+- <a href="https://cran.r-project.org/web/packages/aRxiv/index.html" target="_blank">aRxiv: Interface to the arXiv API</a>
 - <a href="https://cran.r-project.org/web/packages/ggplot2/index.html" target="_blank">ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics</a>
 
+We load the libraries used in this tutorial below:
 
 ``` r
 library(aRxiv)

src/r/sdirect.md

@@ -1,40 +1,72 @@
+---
+title: "ScienceDirect API in R"
+output: 
+  html_document:
+    keep_md: true
+---
+
 # ScienceDirect API in R
 
 by Michael T. Moen
 
-These recipe examples demonstrate how to use Elsevier’s [ScienceDirect API](https://dev.elsevier.com/) to retrieve full-text articles in various formats (XML, text).
+These recipe examples demonstrate how to use Elsevier’s <a href="https://dev.elsevier.com/" target="_blank">ScienceDirect API</a> to retrieve full-text articles in various formats (XML, text).
 
 *This tutorial content is intended to help facilitate academic research. Please check your institution for their Text and Data Mining or related License Agreement with Elsevier.*
 
-- **Documentation**
-  - [ScienceDirect API](https://dev.elsevier.com/)
-  - [ScienceDirect API Documentation](https://dev.elsevier.com/sd_api_spec.html)
-
-- **Terms**
-  - [ScienceDirect API Terms of Use](https://dev.elsevier.com/api_key_settings.html)
+Please see the following resources for more information on API usage:
 
-- **Data Reuse**
-  - [Elsevier Text & Data Mining](https://dev.elsevier.com/tecdoc_text_mining.html)
+- Documentation
+    - <a href="https://dev.elsevier.com/" target="_blank">ScienceDirect API</a>
+    - <a href="https://dev.elsevier.com/sd_api_spec.html" target="_blank">ScienceDirect API Documentation</a>
+- Terms
+    - <a href="https://dev.elsevier.com/api_key_settings.html" target="_blank">ScienceDirect API Terms of Use</a>
+- Data Reuse
+    - <a href="https://dev.elsevier.com/tecdoc_text_mining.html" target="_blank">Elsevier Text & Data Mining</a>
 
-> **Note:** See your institution's rate limit in the [ScienceDirect API Terms of Use](https://dev.elsevier.com/api_key_settings.html).
+_**NOTE:**_ See your institution's rate limit with <a href="https://dev.elsevier.com/api_key_settings.html" target="_blank">ScienceDirect API Terms of Use</a>.
 
+*If you have copyright or other related text and data mining questions, please contact The University of Alabama Libraries or your respective library/institution.*
 
-*These recipe examples were tested on February 7, 2025.*
+*These recipe examples were tested on October 27, 2025.*
 
 ## Setup
 
 ### Import Libraries
 
-```r
+The following packages need to be installed into your environment to run the code examples in this tutorial. These packages can be installed with `install.packages()`.
+
+- <a href="https://cran.r-project.org/web/packages/httr/index.html" target="_blank">httr: Tools for Working with URLs and HTTP</a>
+
+We load the libraries used in this tutorial below:
+
+
+``` r
 library(httr)
 ```
 
 ### Import API Key
 
-An API key is required to access the ScienceDirect API. Registration is available on the [Elsevier developer portal](https://dev.elsevier.com/). The key is imported from an environment variable below:
+An API key is required for to access the ScienceDirect API. You can sign up for one at <a href="https://dev.elsevier.com/" target="_blank">Elsevier developer portal</a>.
+
+We keep our token in a `.Renviron` file that is stored in the working directory and use `Sys.getenv()` to access it. The `.Renviron` should have an entry like the one below.
 
-```r
-myAPIKey <- Sys.getenv("sciencedirect_key")
+```text
+SCIENCE_DIRECT_API_KEY="PUT_YOUR_API_KEY_HERE"
+```
+
+Below, we can test to whether the key was successfully imported.
+
+
+``` r
+if (nzchar(Sys.getenv("SCIENCE_DIRECT_API_KEY"))) {
+  print("API key successfully loaded.")
+} else {
+  warning("API key not found or is empty.")
+}
+```
+
+```
+## [1] "API key successfully loaded."
 ```
 
 ### Identifier Note
@@ -43,41 +75,51 @@ We will use DOIs as the article identifiers. See our Crossref and Scopus API tut
 
 ## 1. Retrieve full-text XML of an article
 
-```r
+
+``` r
 # For XML download
 elsevier_url <- "https://api.elsevier.com/content/article/doi/"
 doi1 <- '10.1016/j.tetlet.2017.07.080' # Example Tetrahedron Letters article
-fulltext1 <- GET(paste0(elsevier_url, doi1, "?APIKey=", myAPIKey, "&httpAccept=text/xml"))
+fulltext1 <- GET(paste0(
+  elsevier_url, doi1,
+  "?APIKey=", Sys.getenv("SCIENCE_DIRECT_API_KEY"),
+  "&httpAccept=text/xml"))
 
 # Save to file
 writeLines(content(fulltext1, "text"), "fulltext1.xml")
 ```
 
 ## 2. Retrieve plain text of an article
 
-```r
+
+``` r
 # For simplified text download
 doi2 <- '10.1016/j.tetlet.2022.153680' # Example Tetrahedron Letters article
-fulltext2 <- GET(paste0(elsevier_url, doi2, "?APIKey=", myAPIKey, "&httpAccept=text/plain"))
+fulltext2 <- GET(paste0(
+  elsevier_url, doi2,
+  "?APIKey=", Sys.getenv("SCIENCE_DIRECT_API_KEY"),
+  "&httpAccept=text/plain"))
 
 # Save to file
 writeLines(content(fulltext2, "text"), "fulltext2.txt")
 ```
 
 ## 3. Retrieve full-text in a loop
 
-```r
+
+``` r
 # Make a list of 5 DOIs for testing
 dois <- c('10.1016/j.tetlet.2018.10.031',
           '10.1016/j.tetlet.2018.10.033',
           '10.1016/j.tetlet.2018.10.034',
           '10.1016/j.tetlet.2018.10.038',
           '10.1016/j.tetlet.2018.10.041')
-```
 
-```r
 for (doi in dois) {
-  article <- GET(paste0(elsevier_url, doi, "?APIKey=", myAPIKey, "&httpAccept=text/plain"))
+  article <- GET(paste0(
+    elsevier_url, doi,
+    "?APIKey=", Sys.getenv("SCIENCE_DIRECT_API_KEY"),
+    "&httpAccept=text/plain"))
   doi_name <- gsub("/", "_", doi)
   writeLines(content(article, "text"), paste0(doi_name, "_plain_text.txt"))
   Sys.sleep(1)

src/r/wiley-tdm.md

@@ -1,5 +1,5 @@
 ---
-title: "wiley-tdm"
+title: "Wiley Text and Data Mining (TDM) in R"
 output: 
   html_document:
     keep_md: true
@@ -9,26 +9,32 @@ output:
 
 by Michael T. Moen
 
-This tutorial is designed to support academic research. Please consult your institution’s library or legal office regarding its Text and Data Mining license agreement with Wiley.
+The Wiley Text and Data Mining (TDM) API allows users to retrieve the full-text articles of subscribed Wiley content in PDF form. TDM use is for non-commercial scholarly research, see terms and restrictions in below links.
 
-### Documentation
-- [Wiley Text and Data Mining](https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining)
+*This tutorial content is intended to help facilitate academic research. Please check your institution for their Text and Data Mining or related License Agreement with Wiley.*
 
-### Terms of Use
-- [Wiley Text and Data Mining Agreement](https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining#accordionHeader-3)
+Please see the following resources for more information on API usage:
 
-### Data Reuse
-- [Service Name] Data Reuse *(link to be provided by the service)*
+- Documentation
+    - <a href="https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining" target="_blank">Wiley Text and Data Mining</a>
+- Terms
+    - <a href="https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining#accordionHeader-3" target="_blank">Wiley Text and Data Mining Agreement</a>
+- Data Reuse
+    - <a href="https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining#accordionHeader-3" target="_blank">Wiley TDM Data Reuse</a> (see sections 4 and 5 of Text and Data Mining Agreement)
 
-*These recipe examples were tested on February 12, 2025.*
+*These recipe examples were tested on October 27, 2025.*
 
 **_NOTE:_** The Wiley TDM API limits requests to a maximum of 3 requests per second.
 
 ## Setup
 
 ### Import Libraries
 
-This tutorial uses the following libraries:
+The following packages need to be installed into your environment to run the code examples in this tutorial. These packages can be installed with `install.packages()`.
+
+- <a href="https://cran.r-project.org/web/packages/httr/index.html" target="_blank">httr: Tools for Working with URLs and HTTP</a>
+
+We load the libraries used in this tutorial below:
 
 
 ``` r
@@ -37,14 +43,27 @@ library(httr)
 
 ### Text and Data Mining Token
 
-A token is required to access the Wiley TDM API. Sign up can be found [here](https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining#accordionHeader-2). Import your token below:
+A token is required for text and data mining with Wiley. You can sign up for one on the <a href="https://onlinelibrary.wiley.com/library-info/resources/text-and-datamining#accordionHeader-2" target="_blank">Wiley Text and Data Mining page</a>.
+
+We keep our token in a `.Renviron` file that is stored in the working directory and use `Sys.getenv()` to access it. The `.Renviron` should have an entry like the one below.
+
+```text
+WILEY_TDM_TOKEN="PUT_YOUR_TOKEN_HERE"
+```
+
+Below, we can test to whether the key was successfully imported.
 
 
 ``` r
-wiley_token <- Sys.getenv("wiley_token")
+if (nzchar(Sys.getenv("WILEY_TDM_TOKEN"))) {
+  print("API key successfully loaded.")
+} else {
+  warning("API key not found or is empty.")
+}
+```
 
-# The token will be sent as a header in the API calls
-headers <- add_headers("Wiley-TDM-Client-Token" = wiley_token)
+```
+## [1] "API key successfully loaded."
 ```
 
 ## 1. Retrieve full-text of an article
@@ -59,14 +78,13 @@ In the first example, we download the full-text of the article with the DOI "10.
 doi <- "10.1002/net.22207"
 url <- paste0("https://api.wiley.com/onlinelibrary/tdm/v1/articles/", doi)
 
-response <- GET(url, headers)
+response <- GET(url, add_headers("Wiley-TDM-Client-Token" = Sys.getenv("WILEY_TDM_TOKEN")))
 
 if (status_code(response) == 200) {
   # Download if status code indicates success
   filename <- paste0(gsub("/", "_", doi), ".pdf")
   writeBin(content(response, "raw"), filename)
   cat(paste0(filename, " downloaded successfully\n"))
-  
 } else {
   # Print status code if unsuccessful
   cat(paste0("Failed to download PDF. Status code: ", status_code(response), "\n"))
@@ -96,14 +114,13 @@ dois <- c(
 # Loop through DOIs and download each article
 for (doi in dois) {
   url <- paste0("https://api.wiley.com/onlinelibrary/tdm/v1/articles/", doi)
-  response <- GET(url, headers)
+  response <- GET(url, add_headers("Wiley-TDM-Client-Token" = Sys.getenv("WILEY_TDM_TOKEN")))
 
   if (status_code(response) == 200) {
     # Download if status code indicates success
     filename <- paste0(gsub("/", "_", doi), ".pdf")
     writeBin(content(response, "raw"), filename)
     cat(paste0(filename, " downloaded successfully\n"))
-    
   } else {
     # Print status code if unsuccessful
     cat(paste0("Failed to download PDF. Status code: ", status_code(response), "\n"))