Develop E2E Testing Method with Testscript.

[trulede]

Would there be any interest for a development of an E2E testing method using Testscript? Testscript is a different (but complementary) approach to the current unit tests, which makes it a easier to write tests which capture the user perspective (i.e. E2E tests). The Testscript package can also be extended, adding commands for specific cases (e.g. xpath like queries for yaml, or graph database queries).

An example testcase might look like this:

# Templating SpecialVariables Dirs
task default
stdout 'ROOTDIR:: $WORK $WORK/env/root $WORK/gvar/root $WORK/lvar/root'
stdout 'TASKFILE_DIR:: $WORK $WORK/env/taskfile $WORK/gvar/taskfile $WORK/lvar/taskfile'
stdout 'TASK_DIR:: $WORK $WORK/env/task $WORK/gvar/task $WORK/lvar/task'
stdout 'USER_WORKING_DIR:: $WORK $WORK/env/user_working $WORK/gvar/user_working $WORK/lvar/user_working'

-- Taskfile.yaml --
version: '3'

env:
  env_ROOT_DIR: '{{.ROOTDIR}}/env/root'
  env_TASKFILE_DIR: '{{.TASKFILE_DIR}}/env/taskfile'
  env_TASK_DIR: '{{.TASK_DIR}}/env/task'
  env_USER_WORKING_DIR: '{{.USER_WORKING_DIR}}/env/user_working'

vars:
  gvar_ROOT_DIR: '{{.ROOTDIR}}/gvar/root'
  gvar_TASKFILE_DIR: '{{.TASKFILE_DIR}}/gvar/taskfile'
  gvar_TASK_DIR: '{{.TASK_DIR}}/gvar/task'
  gvar_USER_WORKING_DIR: '{{.USER_WORKING_DIR}}/gvar/user_working'

tasks:
  default:
    vars:
      lvar_ROOT_DIR: '{{.ROOTDIR}}/lvar/root'
      lvar_TASKFILE_DIR: '{{.TASKFILE_DIR}}/lvar/taskfile'
      lvar_TASK_DIR: '{{.TASK_DIR}}/lvar/task'
      lvar_USER_WORKING_DIR: '{{.USER_WORKING_DIR}}/lvar/user_working'
    cmds:
      - 'echo "ROOTDIR:: {{.ROOTDIR}} {{.env_ROOT_DIR}} {{.gvar_ROOT_DIR}} {{.lvar_ROOT_DIR}}"'
      - 'echo "TASKFILE_DIR:: {{.TASKFILE_DIR}} {{.env_TASKFILE_DIR}} {{.gvar_TASKFILE_DIR}} {{.lvar_TASKFILE_DIR}}"'
      - 'echo "TASK_DIR:: {{.TASK_DIR}} {{.env_TASK_DIR}} {{.gvar_TASK_DIR}} {{.lvar_TASK_DIR}}"'
      - 'echo "USER_WORKING_DIR:: {{.USER_WORKING_DIR}} {{.env_USER_WORKING_DIR}} {{.gvar_USER_WORKING_DIR}} {{.lvar_USER_WORKING_DIR}}"'

An E2E test suite is simply a collection of txtar files. They can be operated with go test. Adding support for Testscript based testcases in the Task repo would not be particularly difficult (it would slot in alongside cmd/task/task.go (as cmd/task/task_test.go).

[joeblew999]

I am def interested is a good way to test task files are working. I am hitting little bugs in task files that are hard to assert that they are not breaking.

How best to do that I guess is an open debate.

I like the idea of file based testing. The reason I say this is because I would like to be able to write tests for my own Taskfiles too. like here: https://github.com/joeblew999/pb-stack/tree/main/mod/_template/source-remote. Then I know the base task files are working for all the Developers and Users that use my base task files. And so do they :)