Expose diagnostic check functions

[rok-cesnovar]

check_sampler_transitions_treedepth and check_divergences should be accessible to the user in some way.

Either separately or as part of fit$diagnose() or something with possibly other checks.

Cmdstan's diagnostics checks E-BMFI which is simple (only requires energy__ I think) and two that require reading all data (ESS and RHAT checks).

[jgabry]

Yeah I agree we should expose these. I think we should also change the name of check_sampler_transitions_treedepth to just check_treedepth.

Maybe a method fit$diagnose(diagnostic = NULL), where if diagnostic=NULL it checks all of them but the user can also specify which ones, e.g. fit$diagnose(c("divergence", "ebfmi"))?

Also I think it would be useful if these methods returned the value of diagnostic to the user instead of only printing messages (it can also print messages).

[rok-cesnovar]

If stan-dev/posterior will have these checks (see stan-dev/posterior#77), we should just use those.

The divergences, treedepth and ebfmi use sampler diagnostics so those are Stan specific and will be done by our helper functions.

cc @paul-buerkner

[jgabry]

Yeah I agree

[rok-cesnovar]

What should we do here for now:

• add fit$diagnose() that runs treedepth, divergence and ebmfi without rhat/ess
• add fit$diagnose() that runs treedepth, divergence and ebmfi and also implement warnings for rhat/ess (at least the latter would get replaced by posterior)

This one should be simple.

[jgabry]

I think posterior is going to end up including warnings also for HMC/NUTS, so I think all of them could end up being replace by posterior. Do you think we should just wait for posterior or implement them and then replace them with posterior's versions?

[rok-cesnovar]

Lets just wait for posterior then and move the milestone if you agree.

[jgabry]

Sounds good

[rok-cesnovar]

So what I think the API for this should be:

• add a diagnostics = c("divergences", "treedepth", "bfmi") argument to $sample()
• deprecate validate_csv. I think the name was my idea. In hindsight its a bad name, not informative at all. validate_csv = FALSE is the same as diagnostics = NULL or ""
• add fit$diagnose(diagnostics = c("divergences", "treedepth", "bfmi"))

I think we should just go with it now that we have the three sampler diagnostics checks. The rhat/eff checks can be added later.

[jgabry]

I like this idea. We should think carefully about what object fit$diagnose() should return (aside from what it prints/warns about). When posterior eventually implements these hmc diagnostics we should use them internally in fit$diagnose() but ideally not change what it returns so we don't have to deprecate it in favor of a new method that uses the posterior stuff.