diff --git a/doc/guide/author/topics.xml b/doc/guide/author/topics.xml index 92350435b4..b3e338220d 100644 --- a/doc/guide/author/topics.xml +++ b/doc/guide/author/topics.xml @@ -4075,10 +4075,17 @@

For server based programs, the other elements will be copied into files based on their filename attributes.

This allows you to split code into multiple files and thus create more complex interconnections than when using include.

-
  • - compile-also -

    (C/C++ only) A comma separated string identifying one or more other programs by xml:id that are also to be compiled when this program is compiled. They will be assumed to also be a part of the add-files collection. Each of those programs should have an filename.

    -
    • +
  • + compile-also +

    A comma separated string identifying one or more other programs or files by xml:id that are to be added to the program's build. They will be assumed to also be a part of the add-files collection, so they are made available in the program's working directory as well. Each of those elements should have a filename.

    +

    What it means for a file to be added to the build depends on the program's language and on whether the file is text or binary (a binary file with format set to binary):

    + +
  • interpreter-args

    (Python2/3/Java/octave only. Only applies to Python that is run on the server. See .) A comma separated list of strings to be passed to the compiler. Ex: -Xrs, -Xss8m, -Xmx200m. A book-level default can be specified - see .

    @@ -4311,40 +4318,50 @@
    Data Files -

    In concert with interactive programs (see ) you can define a file of data that may be employed by those programs. The necessary element is datafile. It requires a label. To allow programs to link to the datafile, it will also need an xml:id (which can be the same as the label). A filename is also required and is the name the file is known by in an ActiveCode program. Do not try to impose any sort of directory structure on this name. Just a filename. In the case of a text file (see below), the editable attribute is optional. The value no is the default, with yes as the other option. The attributes rows and cols are optional for text files, and default to 20 and 60 respectively. Finally, a non-editable text file (only) may have its contents hidden by setting the hide attribute to yes, rather than the default value of no.

    +

    In concert with interactive programs (see ) you can define a file of data that may be employed by those programs. The necessary element is file. It requires a label. To allow programs to link to the file, it will also need an xml:id (which can be the same as the label). A filename is also required and is the name the file is known by in an ActiveCode program. Do not try to impose any sort of directory structure on this name. Just a filename.

    + +

    A file serves one of three kinds of data, set with the format attribute: pre for text files (the default), image for images, and binary for any other kind of file. A file is invisible to the reader by default, but may be made visible to the reader with the user-interaction attribute, described below.

    + +

    The datafile element was the original version of file. It is deprecated, and an automatic conversion is made when processes your source: a datafile with editable set to yes becomes a file with user-interaction set to edit, a text file with hide set to yes becomes none, and any other datafile becomes view. Please update your source to use file at your first convenience, as the automatic conversion may not respect your intent in all cases.

    -

    Where might you place a datafile? Lots of places are possible, such as in an example or a programming exercise, close to an ActiveCode program. So, in expository material or in activities for readers to work through. The purpose-built COMPUTATION-LIKE block, data, which will get you a heading, number, title, cross-reference target, etc. (see ), is an option if the file itself needs more prominence or dedicated explanation. Notice that this feature is very powerful, and thus requires a bit of machinery to support. If you just want to point your reader to a file (and leave them to work with it outside of your project), either globally or locally, the read about the dataurl element at .

    +

    Where might you place a file? Lots of places are possible, such as in an example or a programming exercise, close to an ActiveCode program. So, in expository material or in activities for readers to work through. The purpose-built COMPUTATION-LIKE block, data, which will get you a heading, number, title, cross-reference target, etc. (see ), is an option if the file itself needs more prominence or dedicated explanation. Notice that this feature is very powerful, and thus requires a bit of machinery to support. If you just want to point your reader to a file (and leave them to work with it outside of your project), read about the dataurl element at .

    Text as Data Files -

    Inside of a datafile place an pre element. There are then two options: provide the contents of the text file right in your source file, as you might for other preformatted text, or supply a source attribute whose value is the name of an external text file you provide. The former is appropriate for toy examples, while the latter may be used for serious files with many lines, or with long lines. Note that if you provide the file as the content of the pre element, it can be indented to match your source file indentation, and will undergo some manipulation, such as removing leading whitespace, and ensuring a final newline, but preserving any relative indentation. If provided via a source attribute, there is no manipulation.

    +

    Inside of a file place a pre element. There are then two options: provide the contents of the text file right in your source file, as you might for other preformatted text, or, with format="pre", supply a source attribute whose value is the name of an external text file you provide. The former is appropriate for toy examples, while the latter may be used for serious files with many lines, or with long lines. Note that if you provide the file as the content of the pre element, it can be indented to match your source file indentation, and will undergo some manipulation, such as removing leading whitespace, and ensuring a final newline, but preserving any relative indentation; consider keeping the pre tag on the same line as the included content should you use xi:include there (see ). If provided via a source attribute, there is no such manipulation.

    -

    Such a text file may be declared editable by the reader, presumably to allow them to witness the resulting behavior of a some employing program. The rows and cols attributes describe the viewport into the file provided in the HTML output. Typically scrollbars will allow the reader to survey all of a large file. In static outputs, the first few lines are shown, given by the value of rows, and lines are truncated according to the value of cols.

    +

    Such a text file may be declared editable by the reader by setting the user-interaction attribute, presumably to allow them to witness the resulting behavior of an employing program. The attribute may be none (the default), view, or edit: a file with none stays hidden from the reader, while with view or edit the reader will be shown the file's text. The rows and cols attributes describe the viewport into the text shown in the HTML output, and default to 20 and 60 respectively. Typically scrollbars will allow the reader to survey all of a large file. In static outputs, the first few lines are shown, given by the value of rows, and lines are truncated according to the value of cols.

     Images as Data Files -

    Inside of a datafile place an image element with a source attribute. As usual, this attribute should be the name of an external file you provide. Most common formats are supported, but it is important to use standard extensions, so the format can be discerned. Now this file may be explored programmatically by opening the file using the name provided in filename.

    +

    For an image file, set format="image", and supply the name of an external file with a source attribute. As usual, this attribute names an external file you provide. Most common formats are supported, but it is important to use standard extensions, so the format can be discerned. The image may then be examined programmatically by opening the file with the name given in filename. If you want the image to be seen by the reader, use a separate image element with the same source.

    Keep the size of the image small, say 300400 pixels in each direction. You may also supply the usual layout controls, such as width, and these will be consulted in the formation of output formats. Ideally, you should use a width that scales the image to look something like its native resolution, since part of an image-processing exercise may depend on this aspect of the input. HTML output uses a 600 pixel overall width, so a percentage can be computed based on this parameter.

     + + Binary Files as Data Files + +

    Any other kind of file is called binary: set format="binary" and provide the file with a source attribute. Binary files may be used for such things as compiled programs or archives. In the HTML output they receive the same treatment as image files, available to any employing program, but they are never shown to the reader directly.

    +     + Notes on Data Files -

    Some notes that apply to each type of data file.

      +

      Some notes that apply to each format of data file.

      • Note that the name of the data file in a source attribute need not have any resemblance to the new name given to the file via the filename attribute. In other words, the reader will never know (or care) what source was.
        • -
      • Whenever the source attribute is used, there needs to be an advance step performed by the CLI - or the pretext/pretext script () to generate an auxillary file (yes, a third file!) to aid the transistion from an external file to a file that can be used by the reader in programs.
        • +
      • Whenever the source attribute is used, there needs to be an advance step performed by the CLI + or the pretext/pretext script () to generate an auxiliary file (yes, a third file!) to aid the transition from an external file to a file that can be used by the reader in programs.
        • -
      • For a program to use a data file, it must be able to find the appropriate file. This is accomplished by specifying an add-files attribute on the program element that is a list (separated by commas or space) of the xml:id's for files that are available for use in the program. For programs that run strictly in the browser as opposed to on a Runestone server (see for what languages run in the browser), a program may be able to open a file by name without specifying its xml:id in the add-files. However, for all programs, best practice is to make sure that the xml:id of any datafile a program should have access to is correctly listed in that program's add-files
        • +
      • For a program to use a data file, it must be able to find the appropriate file. This is accomplished by specifying an add-files attribute on the program element that is a list (separated by commas or space) of the xml:id's of files that should be available to the program. For programs that run strictly in the browser as opposed to on a Runestone server (see for what languages run in the browser), a program may be able to open a file by name without specifying its xml:id in the add-files. However, for all programs, best practice is to make sure that the xml:id of any file a program should have access to is correctly listed in that program's add-files.
      • -

        In all cases, for an HTML build the contents of the data file live within an HTML page, as text for a text file, and as a base-64 encoding for an image file. Hence for a non-Runestone build, any employing program must be on the same page, and an author should think ahead about the granularity of how a project might be chunked into pages ().

        +

        In all cases, for an HTML build the contents of the data file live within an HTML page, as text for a text file, and as a base-64 encoding for an image file or a binary file. Hence for a non-Runestone build, any employing program must be on the same page, and an author should think ahead about the granularity of how a project might be chunked into pages ().

        -

        In a build for use on Runestone Academy (), the file will be in the Runestone database and usuable throughout.

        +

        In a build for use on Runestone Academy (), the file will be in the Runestone database and usable throughout.

      diff --git a/examples/sample-book/backmatter.xml b/examples/sample-book/backmatter.xml index e673e56aef..9e83280423 100644 --- a/examples/sample-book/backmatter.xml +++ b/examples/sample-book/backmatter.xml @@ -84,11 +84,11 @@ along with PreTeXt. If not, see . Some Runestone Elements

      These elements should end up in the Runestone manifest.

      - + 
                   Ipsum lorem...
      -       + print("Hello from the appendix!") diff --git a/examples/sample-book/ext/datafiles/Helper.java b/examples/sample-book/ext/datafiles/Helper.java new file mode 100644 index 0000000000..f701095bdb --- /dev/null +++ b/examples/sample-book/ext/datafiles/Helper.java @@ -0,0 +1,24 @@ +/* + * This source file and its compiled companion (helper.jar) form a tiny, + * self-contained example that exercises PreTeXt's "file" element with + * @format = "binary" and @source pointing at an actual compiled Java + * archive. In the sample book (examples/sample-book/rune.xml) a program + * with @add-files referencing this file makes the class available to + * server-side (Jobe) execution. + * + * To rebuild the archive after editing this source: + * + * javac helper.java + * jar cfe helper.jar Helper helper.class + * + * PreTeXt stores a base64 (text) representation of the binary file in the + * generated "gen/datafile/" directory; the Runestone renderer registers it + * with the browser as a hidden payload flagged with "data-isbinary" so a + * future Runestone can hand it to a server verbatim. + */ +public class Helper { + /** The square of n. */ + public static int square(int n) { + return n * n; + } +} diff --git a/examples/sample-book/ext/datafiles/climate.gz b/examples/sample-book/ext/datafiles/climate.gz new file mode 100644 index 0000000000..0c1bf1b42e Binary files /dev/null and b/examples/sample-book/ext/datafiles/climate.gz differ diff --git a/examples/sample-book/ext/datafiles/helper.jar b/examples/sample-book/ext/datafiles/helper.jar new file mode 100644 index 0000000000..ed25339262 Binary files /dev/null and b/examples/sample-book/ext/datafiles/helper.jar differ diff --git a/examples/sample-book/gen/datafile/climate-gz.xml b/examples/sample-book/gen/datafile/climate-gz.xml new file mode 100644 index 0000000000..b631c39de0 --- /dev/null +++ b/examples/sample-book/gen/datafile/climate-gz.xml @@ -0,0 +1,2 @@ + + \ No newline at end of file diff --git a/examples/sample-book/gen/datafile/helper-jar.xml b/examples/sample-book/gen/datafile/helper-jar.xml new file mode 100644 index 0000000000..e977b1f9a0 --- /dev/null +++ b/examples/sample-book/gen/datafile/helper-jar.xml @@ -0,0 +1,2 @@ + + \ No newline at end of file diff --git a/examples/sample-book/rune.xml b/examples/sample-book/rune.xml index 38725b750e..50fda95b55 100644 --- a/examples/sample-book/rune.xml +++ b/examples/sample-book/rune.xml @@ -1185,10 +1185,10 @@ TEST_CASE( "Test the add function" ) {
      Data Files -

      In the following file of climate data, the first column is Year, second column is Global Average Temperature (Celcius), and the third column is Global Emmisions C02 (Giga-tons). [Normally you might place this inside a block with the datafile.]

      +

      In the following file of climate data, the first column is Year, second column is Global Average Temperature (Celcius), and the third column is Global Emmisions C02 (Giga-tons). [Normally you might place this inside a block with the file.]

      - + 
                   1850    -0.37     2.24E-7
             1860    -0.34     3.94E-7
@@ -1209,7 +1209,7 @@ TEST_CASE( "Test the add function" ) {
             2010     0.56     32.7
             2019     0.74     33.3
      -       +

      Browser based programs are automatically given access to data files by their filename.

      @@ -1247,15 +1247,12 @@ TEST_CASE( "Test the add function" ) {
    • LanguageWorkedWith

      • - - 
      -            
+            
         
 
         
-        
-            
-        
+        
+        
 
         
             
@@ -1270,6 +1267,15 @@ TEST_CASE( "Test the add function" ) {
             
         
 
+        
+        
+        
+        
+            
      +10 20 30
+            
      
+              
+
         
             Golden Gate Bridge Image processing
 
@@ -1278,9 +1284,8 @@ TEST_CASE( "Test the add function" ) {
             
      [Now a data file and a program to process it, all bundled up inside a computation, since an example gets knowled and the ActiveCode does not fill.]
      
 
             
-            
-                
-            
+            
+            
 
             
      This program changes every pixel of the image.
      
 
@@ -1314,13 +1319,13 @@ TEST_CASE( "Test the add function" ) {
             
               
 
-        
      The examples above all use Python, which will run in your browser.   Other languages will only run when a project is hosted on Runestone Academy servers.  And in this case there is a small twist.  You need to indicate which existing datafile your program needs, even if that seems obvious by reading the code.  To do this, make sure each datafile has an xml:id and then use the add-files attribute on program to specify a list of xml:id's that you want to be available.  These datafiles will be written to the program's working directory using the filename specified by the filename attribute of the datafile element, and are the names you use in your program's code.  As before, no path information is needed, nor allowed.
      
+        
      The examples above all use Python, which will run in your browser.   Other languages will only run when a project is hosted on Runestone Academy servers.  And in this case there is a small twist.  You need to indicate which existing file your program needs, even if that seems obvious by reading the code.  To do this, make sure each file has an xml:id and then use the add-files attribute on program to specify a list of xml:id's that you want to be available.  These files will be written to the program's working directory using the filename specified by the filename attribute of the file element, and are the names you use in your program's code.  As before, no path information is needed, nor allowed.
      
 
         
      When the language attribute of a program is set to python3 that means in-browser Python is not good enough, and you want the greater power and flexibility of having your code run on a Runestone Academy server.  So this is our first example of using the datafile attribute.
      
 
         
      The data file is an abbreviated version of the example above, just to be different.  And is not editable.
      
 
-        
+        
             
                   1900    -0.2      2.38
             1910    -0.49     3.34
@@ -1336,7 +1341,7 @@ TEST_CASE( "Test the add function" ) {
             2010     0.56     32.7
             2019     0.74     33.3
             
      
-              
+              
 
         
      The program is identical to the above, but we specify python3 as the language, and use the smaller file.  So this example is only active when this content is hosted on a Runestone Academy server.
      
 
@@ -1357,17 +1362,15 @@ TEST_CASE( "Test the add function" ) {
         
             Flowers
 
-            
      Two flower images as datafile for use in upcoming Java program.
      
+            
      Two flower images as file for use in upcoming Java program.
      
 
             
 
-            
-                
-            
+            
+            
 
-            
-                
-            
+            
+            
               
 
         
@@ -1560,11 +1563,10 @@ TEST_CASE( "Test the add function" ) {
         
         
 
-        
-            
-        
+        
+        
 
-        
+        
             
                   
             
      
             
-              
+              
 
         
             
@@ -2854,6 +2856,52 @@ TEST_CASE( "Test the add function" ) {
                 ]]>
             
         
+
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+        
+
+        
+            
+            public class Main {
+                public static void main(String[] args) {
+                    System.out.println(Helper.square(5));
+                    System.out.println(Helper.square(-4));
+                }
+            }
+            
+        
+
+        
+        
+        
+        
+        
+        
+        
+
+        
+            
+            import gzip
+
+            with gzip.open("climate.txt.gz", "rt") as f:
+                for line in f:
+                    print(line, end="")
+            
+
      @@ -3481,9 +3529,8 @@ TEST_CASE( "Test the add function" ) {

      After you check a correct answer you will be able to Run the code you created - it will be used to modify the image shown below.

      - - - + + diff --git a/pretext/lib/pretext.py b/pretext/lib/pretext.py index 7f27d3b984..b743c5038f 100644 --- a/pretext/lib/pretext.py +++ b/pretext/lib/pretext.py @@ -898,6 +898,23 @@ def datafiles_to_xml(xml_source, pub_file, stringparams, xmlid_root, dest_dir): log.info("PTX:WARNING : MIME type of image {} not determined".format(data_file)) mime_type = "unknown" + # Open binary file and encode in base64 with standard module + with open(data_file, "rb") as f: + base64version = base64.b64encode(f.read()).decode("utf8") + xml_representation = image_info.format(mime_type, base64version) + elif file_type == "binary": + # best guess of the MIME type, based on the filename extension + _, extension = os.path.splitext(data_file) + # normalize, drop leading period + # in rough popularity order + lcext = extension[1:].lower() + if lcext == "zip": + mime_type = "application/zip" + elif lcext == "jar": + mime_type = "application/java-archive" + else: + mime_type = "application/octet-stream" + # Open binary file and encode in base64 with standard module with open(data_file, "rb") as f: base64version = base64.b64encode(f.read()).decode("utf8") diff --git a/schema/pretext.xml b/schema/pretext.xml index 9dbc73b1ea..704b7cf765 100644 --- a/schema/pretext.xml +++ b/schema/pretext.xml @@ -2917,6 +2917,27 @@ } BlockStatement |= Datafile +# File element used to provide support files for programs. + # A "pre" child names a text file, so @format is optional + # (and necessarily "pre") when it is present, defaulting + # accordingly. An external @source file is otherwise + # required to carry an explicit @format. + File = + element file { + UniqueID?, + LabelID?, + Component?, + attribute filename {text}, + attribute rows {xsd:integer}?, + attribute cols {xsd:integer}?, + attribute user-interaction {"none" | "view" | "edit"}?, + ( + (attribute format {"pre"}?, element pre {text}) | + (attribute format {"binary" | "image" | "pre"}, attribute source {text}) + ) + } + BlockStatement |= File + # Query (poll/survey) element; results are private to the # instructor unless @visibility says otherwise Query = diff --git a/xsl/extract-datafile.xsl b/xsl/extract-datafile.xsl index a127ea3a1c..8c4a9fc289 100644 --- a/xsl/extract-datafile.xsl +++ b/xsl/extract-datafile.xsl @@ -68,4 +68,27 @@ along with PreTeXt. If not, see . + + + + + + + + + + + + + + + + pre + + + + + + + diff --git a/xsl/pretext-assembly.xsl b/xsl/pretext-assembly.xsl index 09c9ee98c3..3cb81636c7 100644 --- a/xsl/pretext-assembly.xsl +++ b/xsl/pretext-assembly.xsl @@ -2496,7 +2496,7 @@ along with PreTeXt. If not, see . - + @@ -2949,6 +2949,65 @@ along with PreTeXt. If not, see . + + + + + + + + + + + + + + + + + + + + + + edit + + + none + + + view + + + + + + + + + image + + + + + + + + + pre + + + + + + + + + + + + + @@ -4860,7 +4919,7 @@ along with PreTeXt. If not, see . + datafile/ @@ -5110,4 +5169,25 @@ along with PreTeXt. If not, see . + + + + + + + + + + + + + + + + + + + + + diff --git a/xsl/pretext-common.xsl b/xsl/pretext-common.xsl index 92064be1cd..6c9bce424b 100644 --- a/xsl/pretext-common.xsl +++ b/xsl/pretext-common.xsl @@ -3799,7 +3799,7 @@ Book (with parts), "section" at level 3 - + @@ -11721,6 +11721,13 @@ http://andrewmccarthy.ie/2014/11/06/swung-dash-in-latex/ + + + + + + + diff --git a/xsl/pretext-html.xsl b/xsl/pretext-html.xsl index f8a17ebf2c..99e608b3a9 100644 --- a/xsl/pretext-html.xsl +++ b/xsl/pretext-html.xsl @@ -9864,7 +9864,7 @@ along with PreTeXt. If not, see . - + diff --git a/xsl/pretext-runestone-static.xsl b/xsl/pretext-runestone-static.xsl index 203c1f3d71..95e3ba8d2b 100644 --- a/xsl/pretext-runestone-static.xsl +++ b/xsl/pretext-runestone-static.xsl @@ -1499,4 +1499,65 @@ along with PreTeXt. If not, see . + + + + + + + + + + + + + + + +

      + + data + + : + + + +

      + 
      +                    
+                    
+                        
+                            
+                                
+                            
+                            
+                                
+                            
+                        
+                    
+                    
+                        
+                            
+                                
+                            
+                            
+                                
+                            
+                        
+                    
+                    
+                    
+                        
+                        
+                        
+                            
+                            
+                        
+                    
+
      +       +       +       +       + diff --git a/xsl/pretext-runestone.xsl b/xsl/pretext-runestone.xsl index e97f299a0f..63641655ef 100644 --- a/xsl/pretext-runestone.xsl +++ b/xsl/pretext-runestone.xsl @@ -329,7 +329,7 @@ along with PreTeXt. If not, see . - + @@ -613,7 +613,7 @@ along with PreTeXt. If not, see . - + @@ -671,7 +671,7 @@ along with PreTeXt. If not, see . - + @@ -912,7 +912,7 @@ along with PreTeXt. If not, see . - + @@ -933,7 +933,7 @@ along with PreTeXt. If not, see . - + @@ -959,12 +959,12 @@ along with PreTeXt. If not, see . - + - + @@ -2890,6 +2890,163 @@ along with PreTeXt. If not, see . + + + + + + + + + + + + + + + + + + + + + + + + + + + + + pre + + + + + + textarea + + + pre + + + + + + + + +
      +
      + + Data: + + +
      + + + + + + + + + + + + + + + + + + + + + data: + + ;base64, + + + + + + + + + + + + + + + + + + + datafile + + + + + true + + + + + + + + + + + + + + + + datafile + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      +       +