Always use the failures-only reporter for testing #160
Conversation
The default reporter has verbose output which is not useful but eats up a lot of input tokens once it's in the history. Always run tests with `--reporter=failures-only` to reduce the output size. This output format still includes a count of passing tests.
PR HealthChangelog Entry ✔️
Changes to files need to be accounted for in their respective changelogs. |
|
I have not had much success so far in prodding clients into following specific instructions for using any of the tools. I also can't consistently get clients to run the actual CLI interfaces following specific instructions provided by the MCP server. It's all pretty easy to handle with user specified config, but difficult to make a responsibility of the MCP tooling. I think we'll have better luck getting the clients to interact with the MCP tools in the ways we want than the direct CLI so we should try to make each of them as useful for model input as possible. An easy win is to reduce the output size for tests, so land that first. |
There's no audience for this, and the version doesn't matter for packages not published to pub... Satisfy the health check.
pkgs/dart_mcp_server/CHANGELOG.md
Outdated
| @@ -1,6 +1,7 @@ | |||
| # 0.1.1-wip | |||
|
|
|||
| * Instruct clients to prefer MCP tools over running tools in the shell. | |||
| * Reduce output size of `run_tests` tool to save on input tokens. | |||
There was a problem hiding this comment.
Same comment as other PR, I would move this into the section below as we will do more rolls before the last beta release for 3.8.
pkgs/dart_mcp_server/CHANGELOG.md
Outdated
| * Reduce output size of `run_tests` tool to save on input tokens. | ||
|
|
||
| # 0.1.0 (Dart SDK 3.8.0) | ||
| # 0.1.0-wip (Dart SDK 3.8.0) |
There was a problem hiding this comment.
The -wip is a bit odd, I do get that it is more clear that new entries should be added to it. But, it is already "released" in the sense that it is rolled into the SDK. I don't really want to push a last minute merge into the SDK just before the final beta cut just to change the version. So, that is why I marked it as not -wip, even though it is wip 🤣 .
Anyways, I don't feel strongly here, we could just remove the -wip at some point later after the beta cut happens, and just accept that in the SDK the version will always be wip.
There was a problem hiding this comment.
We could do several version bumps, versioning it each time we want to merge into the SDK, but that will add extra process to doing those SDK merges (an extra PR each time) and for little value.
We could also just remove the version number here entirely.
There was a problem hiding this comment.
As discussed separately - I dropped the package level version number and made the header of the changelog just the SDK version reference with a WIP marker to prioritize easy edits with an (optional) step to remove it for the stable SDK publish.
Title the changelog based on the intended SDK version. Leave a WIP marker for clarity on future edits which may be cleaned up before published but is not a risk if retained.
Revisions updated by `dart tools/rev_sdk_deps.dart`. ai (https://github.com/dart-lang/ai/compare/1d9d60c..f2b48c6): f2b48c6 2025-06-09 Nate Bosch Retain compatibility with 3.7 SDKs (dart-lang/ai#163) 52adf08 2025-06-06 Jacob MacDonald add homepage, repository, and documentation links to the pub result (dart-lang/ai#155) fa1c2be 2025-06-06 Nate Bosch Always use the failures-only reporter for testing (dart-lang/ai#160) 66a152f 2025-06-06 Nate Bosch Instruct clients to prefer MCP (dart-lang/ai#161) 55ad850 2025-06-05 Jacob MacDonald Add a retroactive changelog (dart-lang/ai#157) b08a610 2025-06-05 Jacob MacDonald Update instructions, add cursor install link (dart-lang/ai#159) web (https://github.com/dart-lang/web/compare/f1becf0..de6b3e4): de6b3e4 2025-06-06 Srujan Gaddam Add missing copyrights and delete empty files (dart-lang/web#371) 74a33ba 2025-06-06 Kevin Moore Add in a missing library directive, missing new line (dart-lang/web#370) 4d24eb5 2025-06-06 nikeokoronkwo [web_generator] Setting up `web_generator` for Dart JS Interop Gen (dart-lang/web#368) webdev (https://github.com/dart-lang/webdev/compare/55941b0..661dafd): 661dafd4 2025-06-08 Srujan Gaddam Defer execution of main until resume for hot restart with DDC library bundle format (dart-lang/webdev#2623) 01a3b9d7 2025-06-06 Nicholas Shahan Remove skip from chrome proxy service test Change-Id: If3d4326d0bacf47a4d95520dbd0aac5fb58de439 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/433363 Commit-Queue: Devon Carew <[email protected]> Reviewed-by: Konstantin Shcheglov <[email protected]>
The default reporter has verbose output which is not useful but eats up
a lot of input tokens once it's in the history. Always run tests with
--reporter=failures-onlyto reduce the output size. This output formatstill includes a count of passing tests.