Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default print method for SQL-chunks #307

Closed
thohan88 opened this issue Apr 22, 2021 · 9 comments
Closed

Default print method for SQL-chunks #307

thohan88 opened this issue Apr 22, 2021 · 9 comments
Labels

Comments

@thohan88
Copy link

Thanks you so much for this awesome package. It's a fantastic format for presentations. I am currently making some training material for SQL and discovered that the standard formatting of SQL-results appear to be using kable without line breaks for formatting results.

Given the following Rmd-file:

---
  output:
    xaringan::moon_reader
---

```{r include=FALSE}
library(DBI)
con <- dbConnect(RSQLite::SQLite(), dbname = ":memory:")
foo <- dbExecute(con, "DROP TABLE IF EXISTS packages")
foo <- dbExecute(con, "CREATE TABLE packages (id INTEGER, name TEXT)")
foo <- dbExecute(con, "INSERT INTO packages VALUES (1, 'readr'), (2, 'readxl'), (3, 'haven')")
```

```{sql connection=con}
SELECT * FROM packages
```

The result looks like this:

image

I have spent some time investigating how this could be overridden, but have come up short. I suspect that there may be some chunk options I have missed.

Session info
sessioninfo::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 4.0.2 (2020-06-22)
#>  os       Windows 10 x64              
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  Norwegian Bokmål_Norway.1252
#>  ctype    Norwegian Bokmål_Norway.1252
#>  tz       Europe/Paris                
#>  date     2021-04-22                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package     * version date       lib source        
#>  backports     1.1.7   2020-05-13 [1] CRAN (R 4.0.0)
#>  bit           4.0.4   2020-08-04 [1] CRAN (R 4.0.2)
#>  bit64         4.0.2   2020-07-30 [1] CRAN (R 4.0.2)
#>  blob          1.2.1   2020-01-20 [1] CRAN (R 4.0.2)
#>  cli           2.4.0   2021-04-05 [1] CRAN (R 4.0.5)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.2)
#>  DBI         * 1.1.0   2019-12-15 [1] CRAN (R 4.0.2)
#>  digest        0.6.25  2020-02-23 [1] CRAN (R 4.0.2)
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 4.0.2)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.2)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.0.2)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.4)
#>  highr         0.8     2019-03-20 [1] CRAN (R 4.0.2)
#>  htmltools     0.5.1.1 2021-01-22 [1] CRAN (R 4.0.3)
#>  knitr         1.32    2021-04-14 [1] CRAN (R 4.0.5)
#>  lifecycle     0.2.0   2020-03-06 [1] CRAN (R 4.0.2)
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 4.0.2)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 4.0.2)
#>  pillar        1.4.6   2020-07-10 [1] CRAN (R 4.0.2)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.0.2)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.0.2)
#>  Rcpp          1.0.5   2020-07-06 [1] CRAN (R 4.0.2)
#>  reprex        2.0.0   2021-04-02 [1] CRAN (R 4.0.5)
#>  rlang         0.4.10  2020-12-30 [1] CRAN (R 4.0.3)
#>  rmarkdown     2.7     2021-02-19 [1] CRAN (R 4.0.5)
#>  RSQLite       2.2.0   2020-01-07 [1] CRAN (R 4.0.2)
#>  rstudioapi    0.11    2020-02-07 [1] CRAN (R 4.0.2)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.2)
#>  stringi       1.4.6   2020-02-17 [1] CRAN (R 4.0.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.2)
#>  styler        1.3.2   2020-02-23 [1] CRAN (R 4.0.3)
#>  tibble        3.0.3   2020-07-10 [1] CRAN (R 4.0.2)
#>  vctrs         0.3.2   2020-07-15 [1] CRAN (R 4.0.2)
#>  withr         2.4.1   2021-01-26 [1] CRAN (R 4.0.3)
#>  xfun          0.22    2021-03-11 [1] CRAN (R 4.0.5)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.2)
@gadenbuie
Copy link
Contributor

The problem partially lies in knitr. The SQL chunk engine, for HTML output, wraps the output in <div class="knitsql-table">. So the raw output you see in the slides is

<div class="knitsql-table">


Table: 3 records

|id |name   |
|:--|:------|
|1  |readr  |
|2  |readxl |
|3  |haven  |

</div>

(I unescaped some HTML characters to make it easier to read.)

The markdown parser used by remarkjs ignores markdown that appears inside <div> elements, so the markdown version of the table isn't converted to HTML in the slides.

There's also no (easy?) way around this; the wrapping <div> is always added for SQL chunk output in HTML documents: https://github.com/yihui/knitr/blob/db7345ba1414c75ac0d050598cba965b40a9dce7/R/engine.R#L596

      # wrap html output in a div so special styling can be applied
      if (is_html_output()) cat('<div class="knitsql-table">\n')

@cderv could the sql engine be updated to turn off this feature? Or to possibly use the html.tag chunk option?

@cderv
Copy link
Collaborator

cderv commented Apr 22, 2021

Oh yes we certainly need to do a special handling for using this engine in a xaringan format.
Thanks for digging into it!

By html.tag chunk option, what do you mean ? I know 2 special engines use this option but not the SQL engine. It is not really a generic chunk option in knitr currently I believe. But I may have miss something.

Currently the workaround is to define a custom method that will print correctly for xaringan format.

---
output:
  xaringan::moon_reader: default
---

```{r include=FALSE}
library(DBI)
con <- dbConnect(RSQLite::SQLite(), dbname = ":memory:")
foo <- dbExecute(con, "DROP TABLE IF EXISTS packages")
foo <- dbExecute(con, "CREATE TABLE packages (id INTEGER, name TEXT)")
foo <- dbExecute(con, "INSERT INTO packages VALUES (1, 'readr'), (2, 'readxl'), (3, 'haven')")

# custom print method
custom_print <- function(x) {
  paste(knitr::kable(x, format = "markdown"), collapse = "\n")
}
knitr::opts_knit$set(sql.print = custom_print)
```

```{sql connection=con}
SELECT * FROM packages
```

So maybe this should be fixed in xaringan directly by defining a setting sql.print to a custom function ?

@gadenbuie
Copy link
Contributor

Ah a custom sql.print function, that's a great approach @cderv! I was thinking of the options$html.tag used in eng_block() here: https://github.com/yihui/knitr/blob/db7345ba1414c75ac0d050598cba965b40a9dce7/R/engine.R#L418

@cderv
Copy link
Collaborator

cderv commented Apr 22, 2021

The markdown parser used by remarkjs ignores markdown that appears inside

elements, so the markdown version of the table isn't converted to HTML in the slides.

Is there a way around that ? Not sure.

the wrapping

is always added for SQL chunk output in HTML documents:

If we have a way to detect this is a xaringan format from within knitr we could not add this for this format also. That could be another solution : teach knitr about xaringan

a custom sql.print function

maybe that is the cleanest way: modify knitr default behavior from within xaringan

@thohan88
Copy link
Author

Thanks - you guys are awesome! The workaround provided by @cderv works perfectly. Is it possible to extend on this to get datatable (DT ) formatting of the result as well?

@gadenbuie
Copy link
Contributor

Is it possible to extend on this to get datatable (DT ) formatting of the result as well?

Yes, almost certainly! Try removing the paste() and calling DT::datatable() instead of knitr::kable(). (And if that doesn't work let us know and we'll take a look.)

@cderv
Copy link
Collaborator

cderv commented Apr 22, 2021

Not sure it will work as is @gadenbuie .
Currently knitr calls cat()

sql.print = opts_knit$get("sql.print")
if (!is.null(sql.print)) {
    options$results = "asis"
    cat(sql.print(data))
}

So DT::datatable output is a list, it won't work that way - this knit option is not made for that I believe.

Best way to output in any table format for now is to save into a variable and print in another chunk.

---
output:
  xaringan::moon_reader: default
---

```{r include=FALSE}
library(DBI)
con <- dbConnect(RSQLite::SQLite(), dbname = ":memory:")
foo <- dbExecute(con, "DROP TABLE IF EXISTS packages")
foo <- dbExecute(con, "CREATE TABLE packages (id INTEGER, name TEXT)")
foo <- dbExecute(con, "INSERT INTO packages VALUES (1, 'readr'), (2, 'readxl'), (3, 'haven')")

# custom print method
custom_print <- function(x) {
  paste(knitr::kable(x, format = "markdown"), collapse = "\n")
}
```

```{sql connection=con, output.var = "tab"}
SELECT * FROM packages
```

```{r, echo = FALSE}
DT::datatable(tab)
```

@thohan88
Copy link
Author

Is it possible to extend on this to get datatable (DT ) formatting of the result as well?

Yes, almost certainly! Try removing the paste() and calling DT::datatable() instead of knitr::kable(). (And if that doesn't work let us know and we'll take a look.)

I instinctively reached for that as my first attempt as well, but it throws an error as pointed out by @cderv.

Best way to output in any table format for now is to save into a variable and print in another chunk.

That's the solution I was using before creating the issue, but it felt kind of clunky. Datatable was more of a "nice to have", so I'm more than happy with the custom print-function you provided for the time being.

Thank you both again. Your quick response solved what I had been spending a hours searching and debugging. I was about
to ditch xaringan for the project, which would have been a real shame.

@yihui
Copy link
Owner

yihui commented Mar 18, 2022

Should be fixed with the development versions of xaringan and knitr now:

remotes::install_github(c('yihui/knitr', 'yihui/xaringan'))

The <div> will no longer be added to the Markdown table, and the table should be correctly generated now. Thanks!

@yihui yihui added the bug label Mar 18, 2022
clrpackages pushed a commit to clearlinux-pkgs/R-knitr that referenced this issue Mar 29, 2022
Christof Schötz (2):
      bug fix in group_indices() (#2097)
      fix bug in Sweave2knitr when removing unnecessary chunk ends (#2106)

Christophe Dervieux (2):
      Add a newline on empty string in `comment_out(newline = TRUE)` (#2096)
      Update GHA workflow (#2110)

Kenneth C. Arnold (1):
      Give a warning if include_graphics() is given a pathname starting a tilde (#2063)

Konrad Siek (1):
      Engine for ditaa text diagrams (#2092)

Lionel Henry (1):
      Pass user-supplied calling handlers to `evaluate()` (#2079)

Xianying Tan (1):
      add rnw2pdf(), an opinionated wrapper to convert Rnw to PDF (#2115)

Yihui Xie (27):
      use xfun::read_all() to read more than one file for the chunk option `file`
      just use format() directly and let S3 dispatch it
      fix #2086: apply partition_chunk() after the code has been processed to remove the possible indentation and >
      Add a generic engine for executing a command (#2073)
      simplify the code that determines the plot file extension from the dev and fig.ext
      for the `exec` engine, allow the full path of the command to be specified in the chunk option engine.path, just like other command-based engines
      make the chunk option `lang` available to all engines
      language -> lang
      no longer modify the `engine` option for syntax highlighting, but set the `lang` option instead
      lowercase Rscript
      add a engine eng_plot() as a helper function to generate a plot from a command, and apply this function to the dot/asy engines
      update language name inside eng2lang() instead
      close #2100: bump the minimal required version of R from 3.2.3 (2015-12-10) to 3.3.0 (2016-05-03)
      lowercase engine name
      set the language early in eng_dot() for the asy engine; otherwise eng_exec() will set the language to `asy` later
      set the correct language name in eng_exec(), e.g., when the command is `Rscript`, `lang` should be `r`
      merge eng_dot() into eng_plot()
      close #2105: clarify the description of the knitr package written in 2011, without assuming that users know Sweave
      forgot to add () in tests in #2099
      factor out create_fence() so that it can be reused for both ``` and ::: blocks (#2099)
      fix yihui/xaringan#307: don't add a div around the sql table output
      use a more general option knitr.sql.html_div instead of detecting a specific xaringan output format, so that users can set this option to FALSE to disable the div
      let the $set() method return old values instead of NULL
      factor out a function in_input_dir()
      fix #2081: use the input directory (or root.dir, if specified) as the working directory to evaluate chunk options
      close #2061: add fig.scap to opts_knit$get('eval.after')
      CRAN release v1.38

Zhian N. Kamvar (1):
      document and export partition_chunk() (#2083)

atusy (1):
      Implement class.chunk and attr.chunk option to wrap entire chunk in Pandoc's fenced div (closes #2000) (#2099)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants