ggSprings package: Help wanted

[friendly]

I've long been interested in data visualization to illustrate statistical ideas by their physical analogs, and a prime case is least squares regression as a case of points connected to a movable line by springs.

A blog post by Joshua Loftus, Least squares by springs illustrates this, and the code derives from the ggplot2 book, https://ggplot2-book.org/ext-springs

I've taken that code, and created a ggSprings package. It works, in so far as the examples in the ggplot2 book, but is not yet usable for my intended purposes, described in the README.

I'm asking if anyone in the ggplot-extension-club is willing to help or, hopefully become a collaborator on this project. I'm happy to add anyone interested as a package author, or even give it up to someone who wants to take it over as the maintainer.

Here's my current TODO list

TODO

• Finish documenting the package. I don't quite know how to document a ggproto or to use @inheritParams for ggplot2 extensions. Add some more examples illustrating spring aesthetics and
  features.

• Use the package to re-create the gapminder example.

• Try to use gganimate for an animated example.

• Make a hex logo

• Write a vignette explaining the connection between least squares and springs better. ...

[teunbrand]

I don't quite know how to document a ggproto

They are usually accompanied by @export, @format NULL and @usage NULL roxygen tags and refer with @rdname to a pretty generic piece of documentation stating that these are ggproto classes used for extending ggplot2 and are not intended to be used by users directly.
An example of that from one of my extensions can be found here: https://github.com/teunbrand/ggh4x/blob/main/R/ggh4x_extensions.R

to use @inheritParams for ggplot2 extensions

If you're going for a geom_spring(), you can use something like @inheritParams ggplot2::geom_path or other geom that maximises overlap between arguments.

[corybrunson]

I don't have the background to address some of the task list, but i may be interested to contribute in the near future. I'm also curious how you envision the package developing from here:

1. Is the idea, or potential idea, to expand this to a larger collection of graphical constructions that illustrate physical analogs? Are there others you have in mind? That seems like a reasonable scope for a single {ggplot2} extension, i.e. this package could expand to include them rather than them all being put in different packages. It also seems like a neat community project.
2. Would it also make sense to have some sort of convenience function that would deploy the string geom in a ggplot of an lm or prcomp object? That would make the package a kind of hybrid between two poles—a direct extension, i.e. providing new ggprotos and their shortcuts; and a "helper" package that generates plots that happen to use {ggplot2}.

[friendly]

First, I want to say a great big THANKS to the ggplot-extention-club. I'm glad I found you! But, I'm mainly a ggplot user, rather than a developer, so I hope I can learn from you here.

@corybrunson Thanks for these comments. On (1), yes -- there is another domain in which there is an ideal physical domain -- that for categorical data and loglinear models, where the physical analog of observations in cells of a frequency table is that of gas molecules that exert pressure, and fitting a LLM is just the balancing of forces. This is described in my American Statistician paper

You'll see in the paper graphical realizations of this idea for two way tables, and a pseudo-animation of the process of iterative proportional fitting. Hey, I did this all in SAS!

I'll throw this back to the group: If you accept that springs are a visual realization of the forces of least squares, what kind of stats/geoms would represent the pressure forces of observations in cells of a frequency table?

Clearly, it's just random points, whose number is proportional to the observed frequency in the cells, but the key thing is to make the sizes of the tiles proportional to those under an independence model

[corybrunson]

I had come across the spring–tension analogy before, but the gas–pressure analogy is new to me. Very cool.

It also seems (to me) to beg for a similar coupling of a new layer, say geom_gas(), parameterized by the numbers of molecules and the dimensions of the cells, and a convenience function that generates a plot directly from a fitted log-linear model. (I'm not great at animations, but naturally there could also be convenience functions to generate the animations you describe from the fitted models.)

I'll be interested to contribute when i have the bandwidth, maybe unexpectedly!

[friendly]

Thanks @teunbrand for tips on documenting a proto; that solved a few problems. Before going further, I'm stuck on documenting the two aesthetics, diameter and tension used in drawing the springs. My limited test examples seem to work with these aesthetics.

set.seed(4234)
df <- tibble(
  x = runif(5, max = 10),
  y = runif(5, max = 10),
  xend = runif(5, max = 10),
  yend = runif(5, max = 10),
  class = sample(letters[1:2], 5, replace = TRUE),
  tension = runif(5),
  diameter = runif(5, 0.5, 1.5)
)

ggplot(df, aes(x, y, xend = xend, yend = yend)) +
  geom_spring(aes(tension = tension,
                  diameter = diameter,
                  color = class),
              linewidth = 1.2)

But R cmd check complains,

N  checking R code for possible problems (4.3s)
   stat_spring: no visible binding for global variable 'diameter'
   stat_spring: no visible binding for global variable 'tension'
   Undefined global functions or variables:
     diameter tension
✔  checking Rd files ... 

and

W  checking Rd \usage sections (1s)
   Documented arguments not in \usage in Rd file 'geom_spring.Rd':
     'diameter' 'tension'
   
   Functions with \usage entries need to have the appropriate \alias
   entries, and all their arguments documented.

[teunbrand]

Ah right, this just means that you're documenting parameters that are not arguments to the constructor. For example, you document diameter but geom_spring() has no diameter argument. If diameter is an aesthetic (rather than a fixed value), you could make a small @section Aesthetics: to document these manually. See for example ggforce::geom_circle():

https://github.com/thomasp85/ggforce/blob/92928223e9d8de7d7f1a9f02d75909f0e492e0c6/R/circle.R#L17-L30

[friendly]

I've added those changes, but I still get the same warnings as above. The @section Aesthetics: sections have always been a mystery to me, because they don't actually link to any description anywhere. They just seem like text, except for bolding required ones.

[friendly]

I've invited @teunbrand and @corybrunson to the repo, in case you want to access / mess with the code. Any help is very welcomed!

[friendly]

I wasn't sure of the protocol for enlisting help from a gg-club member, and certainly don't want to create any burden or obligation. That is the furthest from my intention.

That said, I'd be extremely happy for you to take a look and prepare a PR.

[corybrunson]

I hadn't thought about that; @friendly feel free to drop me as a collaborator if you'd prefer that for now—though i would be interested in collaborating more directly, either some time in the near future or just at a slow pace.

[friendly]

I only issued invitations. I guess I rescind them

[corybrunson]

I already accepted, so just tell me and i'll bow out. : )

[EvaMaeRey]

Hi @friendly. Thank you for joining us and creating ggsprings and bringing it to our attention! 🙂 I think there's a lot that can be learned by looking at packaging the springs case study. I'm taking notes on your documentation questions and @teunbrand's answers! Plus, the vizzes are so attention-getting (the static ones and imagined animated) - very motivating. A concise syntax to express them is compelling.

I do think a little departure from the book, of having a default behavior for tension proportional to spring length (or would it be squared?), will make the Geom easier to use for the applications that are anticipated. I played with this here and hopefully will think through it a little more and revisit in a fork of your package.

Down the road I can see it working nicely w/ ggsmoothfit's residual drawing user-facing function, in which GeomSegment could be substituted with GeomSpring! https://github.com/EvaMaeRey/ggsmoothfit 📈🤩

[friendly]

@corybrunson It is already possible to illustrate the something of the gas pressure analog using ggmosaic::geom_mosaic_jitter().
Except that in my versions, the sizes of the tiles are proportional to expected frequency, rather than observed frequency. This allows one
to see deviations from independence as variation in the density of points in the cells.

ggmosaic actually isn't as flexible as vcd::mosaic. I'll see if I can make this work.

library(ggmosaic)
ggplot(data = titanic) +
  geom_mosaic(aes(x = product(Class), fill = Survived), alpha = 0.3) +
  geom_mosaic_jitter(aes(x = product(Class), color = Survived))

[corybrunson]

Oh, that's right—the internal code might be a good place from which to adapt. Though it makes sense to me to have a simpler geom_gas() that would require x and y limits from the user as well as the point count, maybe n. (Maybe position_jitter() would work, if one dimension were controlled via width rather than by limits of the other of x,y.) A higher-level stat, analogous to stat_mosaic(), would compute the rectangle positions, and it could be paired with the rect geom to construct the mosaic and with the jitter or gas geom to construct the points. Though i'm more fond of having multiple stat–geom pairings than a lot of extenders; many prefer to have natural pairings like the mosaic stat and the mosaic geom in {ggmosaic}.

[friendly]

Hi @EvaMaeRey Thanks so much for your interest in this topic. I looked at your ggsmoothfit stuff and this ggSprings stuff is closely related. I love the examples you used there, particularly how you used geom_smooth(formula=y~1) to illustrate the intercept/mean.

On springs, you are right that the tension should be proportional to spring length (not squared). I've begun to work on a vignette that will describe all the physics and the implementation.

Here is what I did to illustrate finding the centroid of a bivariate sample by springs. I'm setting tension explicitly in the data.frame to the distance between (x,y) and the mean.

set.seed(1234)
N <- 10
df <- tibble(
  x = runif(N, 0, 10),
  y = runif(N, 0, 10)
)

means <- colMeans(df)
xbar <- means[1]; ybar <- means[2]

df <- df |>
  mutate(tension = sqrt((x - xbar)^2 + (y - ybar)^2),
         diameter = 0.4)

When I call geom_spring ...

ggplot(df, aes(x=x, y=y)) +
  geom_point(size = 5, color = "red") +
  geom_spring(aes(x = x, xend = xbar,
                  y = y, yend = ybar,
                  tension = tension / 5,
                  diameter = diameter),
              color = "blue",
              linewidth = 1.2) +
  geom_point(x = xbar, y = ybar,
             size = 7,
             shape = 15,
             color = "black") +
  theme_minimal(base_size = 15)

I get:

I take your point: linear tension this should be the default, but I don't quite know how to implement this. I think your code removed tension as an aesthetic, in favor of calculating it internally. Is there a way to still allow it to be specified as an aes(tension =) expression?

Finally: the spring constant,$k$ was set in my code to 0.2 = tension/5 for convenience. But then:

• adding a weight aesthetic, with default weight=1 would make it more general
• allowing aes(weight = var) would implement weighted least squares!

[EvaMaeRey]

I was also wondering if you thought it would be worth getting the Least squares by springs author involved? I'm less versed on go-to routines for PCA visualization w ggplot2 for example, so would be curious what tools he reached for in the PCA example.

[friendly]

I contacted Joshua Loftus recently and have had no reply.

[EvaMaeRey]

@aphalo directed me to his work in ggpmisc that also shows residuals in this issue... might be useful to consider the GeomSpring stat_fit_deviations() interplay! I think Pedro could speak to any details about compatibility. EvaMaeRey/ggsmoothfit#1

[EvaMaeRey]

adding a weight aesthetic, with default weight=1 would make it more general

The weighting stuff sounds like a cool move!

I finally did something w/ the fork I made - mostly cosmetic - pulling .R files into the readme using chunk option 'code' 🙃. https://github.com/evamaerey/ggsprings I feel like this helps me get the big picture but it's kinda weird. I see the tension and diameter default adds!! 🤩

I tend to use a lot of ggplot2:: syntax in my packaging, but see that you don't. Can you point me to what you are doing? There's some way to globally say 'use the ggplot2 function if you don't find the function'. This feel like a question I should know the answer to, but somehow don't! 🤷‍♀️

[EvaMaeRey]

I reviewed this 'Best practices for programming with ggplot2' https://youtu.be/nqI5CmdUYRQ?si=mvnQnAz6-UMZLtx0&t=761 and kind of expected an importsFrom ggplot2 aes ggproto move, but don't see that:
https://github.com/evamaerey/ggsprings?tab=readme-ov-file#geomspringr

If I haven't linked to it already, #53 has some ggplot2 packaging considerations discussion.

[friendly]

@EvaMaeRey I incorporated your PR#1. Thx.

I'm still getting a Note and Warning I don't understand and don't know how to fix when I check the package:

❯ checking R code for possible problems ... NOTE
  stat_spring: no visible binding for global variable 'diameter'
  stat_spring: no visible binding for global variable 'tension'
  Undefined global functions or variables:
    diameter tension

I got rid of that by restoring diameter and tension as arguments to stat_spring:

stat_spring <- function(mapping = NULL,
                        data = NULL,
                        geom = "path",
                        position = "identity",
                        ...,
                        diameter = 1,
                        tension = 0.75,
                        n = 50,
                        na.rm = FALSE,
                        show.legend = NA,
                        inherit.aes = TRUE) {
  layer(
    data = data,
    mapping = mapping,
    stat = StatSpring,
    geom = geom,
    position = position,
    show.legend = show.legend,
    inherit.aes = inherit.aes,
    params = list(
      diameter = diameter,
      tension = tension,
      n = n,
      na.rm = na.rm,
      ...
    )
  )
}

The Warning says:

  Documented arguments not in \usage in Rd file 'geom_spring.Rd':
    'diameter' 'tension'
  
  Functions with \usage entries need to have the appropriate \alias
  entries, and all their arguments documented.

But, they are documented in geom_spring.R (lines 32-33)

#' @param diameter Diameter of the spring, i.e., the diameter of a circle that is stretched into a spring shape.
#' @param tension  Spring tension constant. This is calibrated as the total distance moved from the start point to the end point, divided by the size of the generating circle.

I'm still a bit at sea trying to understand how to document ggplot2 stats / protos, when defining new aesthetics.

[EvaMaeRey]

I think maybe the construction and documention should look more like geom_springs(). But are you intending to keep the stat_spring() function? I guess I think that the narrative laid out in the case study is move away from the Stat in favor of having the computation performed within the Geom - I think it would be reasonable to focus just on GeomSpring.

[EvaMaeRey]

... but, thinking about your animation ambition (fun!), experimenting with some Stats to complement the GeomSpring might be the way to go!

Something like the following could be a start point, but is probably not as helpful as it could be...

compute_fit_custom_linear <- function(data, scales, slope = 0, intercept = NULL){
  
  intercept <- intercept %||% mean(data$y)
  
  data |>
    dplyr::mutate(yend = y,
                  y = slope * .data$x + intercept) |> 
    dplyr::mutate(xend = .data$x)
    
  
}

StatFitCustomLinear <- ggproto("StatFitCustomLinear", Stat, compute_group = compute_fit_custom_linear)

anscombe |>
  ggplot() + 
  aes(x2 - mean(x2), y2- mean(y2)) + 
  geom_point() + 
  ggsprings::geom_spring(stat = StatFitCustomLinear) + 
  geom_line(stat = StatFitCustomLinear)

[friendly]

No more R cmd check warnings thx to @teunbrand !
Version now:: 0.1.0

I completed a vignette in which I describe, more than I've seen before, the physics of least squares by springs, some of the history and relation to geometric / algebraic solutions for least squares.

What's missing

For regression problems, it is inconvenient to have to fit the same model that has been constructed by geom_smooth() and add the fitted value to the dataset. The ggsmoothfit repo provides convienient functions, geom_smooth_fit() and geom_smooth_residuals() that provide some useful methods, and the README file defines a GeomSmoothSpring that has this capability. However, this is not a package that can be installed from GitHub, and the implementation there depends on another package statexpress

I copied @EvaMaeRey 's code to R/GeomSmoothSpring.R.

Smoothing methods & families

It is also important to be able to construct spring plots for any model method that can be represented
by geom_smooth(method = AA, formula = y ~ BB), where:

• method can be: lm, glm, loess or nearly any other fitting function, such as MASS::rlm for robust fits
  or mgcv::gam for generalized additive models,
  (TODO: Another reason to implement an observation weight aesthetic param.)

• formula can be any model formula involving only y and x, such as y ~ x or y ~ poly(x, 2) or even y ~ splines::ns(x, df =)

Any ideas / help in implementing these in the package would be very welcome.

[friendly]

Is there some way I can create a link to a readable version of my vignette? The GitHub link: https://github.com/friendly/ggsprings/blob/master/doc/least-squares.html just shows the raw HTML. It's still to early to go to pkgdown or gh-pages for this.

[EvaMaeRey]

Another idea: pulled the vignette into the readme using child = "vignettes/.." in the chunk options for now?

[EvaMaeRey]

https://github.com/EvaMaeRey/ggsprings/blob/6de60aecf9f3a1b8043cfe434fa8a7992a6d40aa/README.Rmd#L242

[EvaMaeRey]

I did a PR to add geom_smooth_fit, geom_residuals, and geom_residual_springs. But, I do think the residuals_squared probably is needed to to speak to the minimized potential energy of the system - like maybe physics gives us intuition about stats and stats gives us intuition about hooks law!

When the system of the line and springs has stopped moving and settled into an equilibrium, this equilibrium position minimizes the energy of the system, which in this case is just the potential energy. The potential energy stored in a spring that is stretched a certain distance is the integral of the force over that distance, and since the force scales with the distance this means the energy scales with the squared distance. Hence, the equilibrium of this physical system minimizes the total potential energy...

Theres a little bit of messiness in getting from geom_residuals_squared to show something that looks square, however, (I do 3-parter - 1) + aes(x = standardize(x)) 2) + aes(y = standardize(y)) 3) + coord_equal(), which feels like a little much - but maybe there is a different, (one step) way to handle this... or perhaps not.