CmdStanMCMC object - add function `get_var` and `get_vars`

[mitzimorris]

For container program variables, i.e., vectors, matrices, and arrays, users need to be able to get the draws for that variable structured accordingly. For scalar variables, this is easy; it is just one column's worth of data and the variable name matches the column name.
For structured variables, we need to assemble all element of the container.

In RStan this is done by the extract function.

The name extract is a poor choice, extract is associated with SQL and there is an extract function in the tidyverse, also in corresponding Python libraries.

Possible names are get_param, get_var or get_variable.

There is a parallel issue on CmdStanPy - stan-dev/cmdstanpy#245.

Let's use this issue to discuss names.

[mitzimorris]

stan-dev/cmdstanpy#252

[rok-cesnovar]

@jgabry the changes in stan-dev/posterior#106 are going to allow us to implement this right? Or am I misunderstanding that?

[mitzimorris]

just to be clear, names we ended up going with are: stan_variable, stan_variables and sampler_diagnostics (for the sampler parameters lp__ et al).

[jgabry]

@jgabry the changes in stan-dev/posterior#106 are going to allow us to implement this right? Or am I misunderstanding that?

Yeah, I'm pretty sure it should let us do something similar to what @mitzimorris did.

[mitzimorris]

PR stan-dev/cmdstanpy#346 updated functionality of stan_variable et al, also added properties num_draws_warmup, num_draws_sampling, thin, stan_vars_dims, stan_vars_cols, and sampler_vars_cols which are used by downstream processing to access rows, columns of the draws array.

[rok-cesnovar]

Just curious, why is it num_draws_warmup/sampling? Doesnt that differ from the argument naming style (iter_*)?

cmdstanr will solve this one with rvar from posterior which will be available soon.

[mitzimorris]

why is it num_draws_warmup/sampling? Doesnt that differ from the argument naming style (iter_*)

big difference: https://cmdstanpy.readthedocs.io/en/v0.9.68/sample.html

Note that the terms iterations and draws are not synonymous. The HMC sampler is configured to run a specified number of iterations. By default, at the end of each iteration, the values of all sampler and parameter variables are written to the Stan CSV output file. Each reported set of estimates constitutes one row’s worth of data, each row of data is called a “draw”. The sampler argument thin controls the rate at which iterations are recorded as draws. By default, thin is 1, so every iteration is recorded. Increasing the thinning rate will reduce the frequency with which the iterations are recorded, e.g., thin = 5 will record every 5th iteration.

[mitzimorris]

cmdstanr will solve this one with rvar from posterior which will be available soon.

will it be exposed with similar function names, or are interfaces diverging along R-thonic / Py-thonic lines?

[jgabry]

PR stan-dev/cmdstanpy#346 updated functionality of stan_variable et al, also added properties num_draws_warmup, num_draws_sampling, thin, stan_vars_dims, stan_vars_cols, and sampler_vars_cols which are used by downstream processing to access rows, columns of the draws array.

cmdstanr will solve this one with rvar from posterior which will be available soon.

will it be exposed with similar function names, or are interfaces diverging along R-thonic / Py-thonic lines?

I'm now thinking that this is one place where we'll diverge because in R we're just outsourcing this to the posterior package. I think it makes sense for CmdStanPy to have these methods and additional properties because there's no posterior package in python, but I think adding all this to CmdStanR would just be duplicating a lot of stuff already provided by posterior, which seems like overkill / extra code to maintain with no great benefit.

So instead of needing to address this issue for v1.0, we just need to fix #582 so it's clear how the user can get the correct rvars objects.

We can definitely reopen this if we decide to reconsider.