Improve benchmarking

[jhlegarreta]

What would you like to see added in this software?

PR #55 added capabilities for benchmarking NiFreeze. As noted in #55 (comment), it was only an initial attempt that only intended to open the path towards more sophisticated and insightful benchmarking.

The benchmarks/README file provides further context on the use of the benchmarking tool.

Going forward, a number of items may need to be addressed:

• All relevant methods be benchmarked.
• Reasonable size testing/benchmarking data.
• Provide more abstraction and sophistication in benchmark code.
  The aim would be to avoid boilerplate code and make writing benchmarks easier. Given that part of the core of NiFreeze revolves around the fit/predict philosophy and scikit-learn-derived capabilities, its benchmarking approach may be helpful to consolidate benchmarking in NiFreeze, e.g.
  https://github.com/scikit-learn/scikit-learn/blob/30603841f43533d5ba09e434757a035c80f3ac63/asv_benchmarks/benchmarks/common.py
• Compare a PR's benchmarks to the main's.
  To avoid introducing code that is detrimental to the performance of NiFreeze, and thus a CI that compares the measures of a standing PR against the main measures would be required. The DataLad folks do this already:
  https://github.com/datalad/datalad/blob/a37b51d95d84d681c081d3f140edfafa84deecbc/.github/workflows/benchmarks.yml#L53-L59
• Think about the need to pin dependencies.
  Pinning dependencies (at least for benchmarking purposes) would make comparisons reliable; however, it will also require maintenance, since at some point we will want to measure the performance with modern, faster versions of packages.

Do you have any interest in helping implement the feature?

Yes

Additional information / screenshots

No response