User test requests fail too silently

[DanielRosenwasser]

A couple of days ago, we requested a test run on the user test suite in a PR for the awaited type. Today, @mattbierner opened up #37556, uncovering a series of breaks in the Visual Studio Code codebase. While we're lucky that we're only shipping beta at this point, we really should have been able to make an informed call last week on whether these breaks were good or not.

Looking at the logs, it appears that the error is that an existing PR was already out but not accepted.

Branch 'rwc-pr-35998' set up to track remote branch 'rwc-pr-35998' from 'origin'.
Error: TF401179: An active pull request for the source and target branch already exists.
    at RestClient.<anonymous> (/home/vsts/work/1/s/internal/scripts/tsscripts/node_modules/typed-rest-client/RestClient.js:173:28)
    at Generator.next (<anonymous>)
    at fulfilled (/home/vsts/work/1/s/internal/scripts/tsscripts/node_modules/typed-rest-client/RestClient.js:6:58)
    at process._tickCallback (internal/process/next_tick.js:68:7)

There are actually a couple of PRs like this: https://typescript.visualstudio.com/TypeScript/_pipeline/analytics/stageawareoutcome?definitionId=11

I think there are a few things that need to be fixed here.

1. If some infrastructure part from @typescript-bot fails, we need to see some sort of comment stating that something went wrong. Here we saw no follow-up comment, so nobody proactively sought a new task.
2. It's questionable whether this task should fail when another PR is outstanding. Is there a way to just "do the right thing" here?

[weswigham]

It's questionable whether this task should fail when another PR is outstanding. Is there a way to just "do the right thing" here?

Is it doing the right thing, insofar as failing the job and updating the existing PR. What you might want is another comment to tell you the status when it's done, since we no longer have build status indicators for these builds since we started to use the merge commit. (Can we ping the devops people about this oversight?)