Error: Can't find the following sampler diagnostic(s) in the output: treedepth__, divergent__

[wlandau-lilly]

Describe the bug

$fit() errors out when I try to run it in the Mac OS terminal. Oddly enough, the same example runs without errors in the RStudio IDE.

To Reproduce

library(cmdstanr)
#> This is cmdstanr version 0.2.0
#> - Online documentation and vignettes at mc-stan.org/cmdstanr
#> - CmdStan path set to: /Users/c240390/.cmdstanr/cmdstan-2.25.0
#> - Use set_cmdstan_path() to change the path
data <- list(
  n = 100,
  x = rnorm(100),
  y = rnorm(100)
)
writeLines(
  c(
    "data {",
    "  int <lower = 1> n;",
    "  vector[n] x;",
    "  vector[n] y;",
    "}",
    "parameters {",
    "  real alpha;",
    "  real beta;",
    "  real <lower = 0> sigma;",
    "}",
    "model {",
    "  y ~ normal(alpha + x * beta, sigma);",
    "  alpha ~ normal(0, 1);",
    "  beta ~ normal(0, 1);",
    "  sigma ~ cauchy(0, 1);",
    "}"
  ),
  "example.stan"
)
model <- cmdstan_model("example.stan")
#> Compiling Stan program...
fit <- model$sample(data = data, refresh = 0)
#> Running MCMC with 4 sequential chains...
#> 
#> Chain 1 finished in 0.0 seconds.
#> Chain 2 finished in 0.1 seconds.
#> Chain 3 finished in 0.1 seconds.
#> Chain 4 finished in 0.0 seconds.
#> 
#> All 4 chains finished successfully.
#> Mean chain execution time: 0.1 seconds.
#> Total execution time: 0.7 seconds.
#> Error: Can't find the following sampler diagnostic(s) in the output: treedepth__, divergent__

^{Created on 2020-11-22 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.3 (2020-10-10)
#>  os       macOS Catalina 10.15.7      
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2020-11-22                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date       lib source                            
#>  assertthat    0.2.1   2019-03-21 [1] CRAN (R 4.0.0)                    
#>  backports     1.2.0   2020-11-02 [1] CRAN (R 4.0.2)                    
#>  callr         3.5.1   2020-10-13 [1] CRAN (R 4.0.2)                    
#>  checkmate     2.0.0   2020-02-06 [1] CRAN (R 4.0.0)                    
#>  cli           2.2.0   2020-11-20 [1] CRAN (R 4.0.2)                    
#>  cmdstanr    * 0.2.0   2020-11-22 [1] Github (stan-dev/cmdstanr@3b7fbe5)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 4.0.0)                    
#>  data.table    1.13.2  2020-10-19 [1] CRAN (R 4.0.2)                    
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 4.0.0)                    
#>  devtools      2.3.2   2020-09-18 [1] CRAN (R 4.0.2)                    
#>  digest        0.6.27  2020-10-24 [1] CRAN (R 4.0.2)                    
#>  ellipsis      0.3.1   2020-05-15 [1] CRAN (R 4.0.0)                    
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.0.0)                    
#>  fansi         0.4.1   2020-01-08 [1] CRAN (R 4.0.0)                    
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.0.2)                    
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.0.2)                    
#>  highr         0.8     2019-03-20 [1] CRAN (R 4.0.0)                    
#>  htmltools     0.5.0   2020-06-16 [1] CRAN (R 4.0.2)                    
#>  jsonlite      1.7.1   2020-09-07 [1] CRAN (R 4.0.2)                    
#>  knitr         1.30    2020-09-22 [1] CRAN (R 4.0.2)                    
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 4.0.2)                    
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 4.0.0)                    
#>  pkgbuild      1.1.0   2020-07-13 [1] CRAN (R 4.0.2)                    
#>  pkgload       1.1.0   2020-05-29 [1] CRAN (R 4.0.0)                    
#>  prettyunits   1.1.1   2020-01-24 [1] CRAN (R 4.0.0)                    
#>  processx      3.4.4   2020-09-03 [1] CRAN (R 4.0.2)                    
#>  ps            1.4.0   2020-10-07 [1] CRAN (R 4.0.2)                    
#>  R6            2.5.0   2020-10-28 [1] CRAN (R 4.0.2)                    
#>  remotes       2.2.0   2020-07-21 [1] CRAN (R 4.0.2)                    
#>  rlang         0.4.8   2020-10-08 [1] CRAN (R 4.0.2)                    
#>  rmarkdown     2.5     2020-10-21 [1] CRAN (R 4.0.2)                    
#>  rprojroot     2.0.2   2020-11-15 [1] CRAN (R 4.0.2)                    
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.0.0)                    
#>  stringi       1.5.3   2020-09-09 [1] CRAN (R 4.0.2)                    
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.0.0)                    
#>  testthat      3.0.0   2020-10-31 [1] CRAN (R 4.0.2)                    
#>  usethis       1.6.3   2020-09-17 [1] CRAN (R 4.0.2)                    
#>  withr         2.3.0   2020-09-22 [1] CRAN (R 4.0.2)                    
#>  xfun          0.19    2020-10-30 [1] CRAN (R 4.0.2)                    
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.0.0)                    
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Expected behavior

No error.

Operating system

Mac OS 10.15.7

CmdStanR version number

0.2.0

Additional context

[rok-cesnovar]

Thanks for the report.

I dont have access to a Mac machine so will ask @jgabry if he can reproduce this and maybe debug this some more, when he has time.

[jgabry]

I'm actually not able to reproduce this. It works fine for me in the Mac terminal. Only difference is that I still have Mac OS 10.15.6 instead of 10.15.7.

[jgabry]

Also, @wlandau-lilly totally unrelated, but in case it helps we have a function write_stan_file() that I think is a little simpler than passing a character vector to writeLines(), although it doesn't really matter.

[jgabry]

If anyone else who has a Mac sees this and can reproduce it please let us know. Otherwise since I'm not able to reproduce this I'm not sure what we can do for now.

[wlandau]

I was afraid that would happen.

For what it's worth, here is an example CSV file from that reprex:
example.csv.zip

When I run cmdstanr::read_cmdstan_csv("example.csv") in the RStudio IDE, it returns the desired array. But when I open R in the Terminal and run it, I get another error:

Error in if (metadata$method == "generate_quantities") { : 
  argument is of length zero

The traceback is unfortunately not very informative.

1: cmdstanr::read_cmdstan_csv("example.csv")

[wlandau]

I put a browser() statement right above here and noticed that when I run cmdstanr::read_cmdstan_csv("example.csv") in the terminal, the metadata names include special Terminal characters. This does not happen in the RStudio IDE.

Browse[1]> names(metadata)
 [1] "sampler_diagnostics"                                    
 [2] "model_params"                                           
 [3] "\033[1;35;40m\033[K\033[m\033[K stan_version_minor"     
 [4] "\033[1;35;40m\033[K\033[m\033[K stan_version_patch"     
 [5] "\033[1;35;40m\033[K\033[m\033[K model"                  
 [6] "\033[1;35;40m\033[K\033[m\033[K method"                 
 [7] "\033[1;35;40m\033[K\033[m\033[K     num_samples"        
 [8] "\033[1;35;40m\033[K\033[m\033[K     num_warmup"         
 [9] "\033[1;35;40m\033[K\033[m\033[K     save_warmup"        
[10] "\033[1;35;40m\033[K\033[m\033[K     thin"               
[11] "\033[1;35;40m\033[K\033[m\033[K       engaged"          
[12] "\033[1;35;40m\033[K\033[m\033[K       gamma"            
[13] "\033[1;35;40m\033[K\033[m\033[K       delta"            
[14] "\033[1;35;40m\033[K\033[m\033[K       kappa"            
[15] "\033[1;35;40m\033[K\033[m\033[K       t0"               
[16] "\033[1;35;40m\033[K\033[m\033[K       init_buffer"      
[17] "\033[1;35;40m\033[K\033[m\033[K       term_buffer"      
[18] "\033[1;35;40m\033[K\033[m\033[K       window"           
[19] "\033[1;35;40m\033[K\033[m\033[K     algorithm"          
[20] "\033[1;35;40m\033[K\033[m\033[K         engine"         
[21] "\033[1;35;40m\033[K\033[m\033[K             max_depth"  
[22] "\033[1;35;40m\033[K\033[m\033[K         metric"         
[23] "\033[1;35;40m\033[K\033[m\033[K         stepsize"       
[24] "\033[1;35;40m\033[K\033[m\033[K         stepsize_jitter"
[25] "\033[1;35;40m\033[K\033[m\033[K id"                     
[26] "\033[1;35;40m\033[K\033[m\033[K   file"                 
[27] "\033[1;35;40m\033[K\033[m\033[K init"                   
[28] "\033[1;35;40m\033[K\033[m\033[K   seed"                 
[29] "\033[1;35;40m\033[K\033[m\033[K   refresh"              
[30] "\033[1;35;40m\033[K\033[m\033[K   sig_figs"             
[31] "\033[1;35;40m\033[K\033[m\033[K Step size"              
[32] "model_name" 

[wlandau]

So I guess I should be looking at cmdstanr:::read_csv_metadata("example.csv").

[wlandau]

I think the problem is here:

cmdstanr/R/read_csv.R

Line 397 in 3b7fbe5

fread_cmd <- paste0("grep '^[#a-zA-Z]' ", csv_file)

My grep automatically prints with colors because my default .bashrc file has alias grep='grep --color=auto'. Others may have this same issue.

[wlandau]

Would you consider grep '^[#a-zA-Z]' --color=never or use a safer method to get the metadata lines?

[jgabry]

Awesome, thanks a lot for tracking down the problem, much appreciated!

Would you consider grep '^[#a-zA-Z]' --color=never or use a safer method to get the metadata lines?

That sounds fine to me, but I will defer to @rok-cesnovar on this as he knows much more than I do about this sort of thing.

[rok-cesnovar]

Would you consider grep '^[#a-zA-Z]' --color=never or use a safer method to get the metadata lines?

Seems fine to me as well. Thanks for tracking this one down! Not sure if there are other cross platform solutions* as opposed to using grep.

*grep is in Rtools utils on Windows