Errors for queries with imbalanced parentheses are reported only for too many left parentheses

[isker]

Here are some input query strings and the results of parsing them, including the error, if any. Some of them should result in parse errors but do not.

( -> <nil> (query: missing close paren, got token <nil>)
(( -> <nil> (error parsing regexp: missing closing ): `((`)
((( -> <nil> (error parsing regexp: missing closing ): `(((`)
() -> TRUE (<nil>)
) -> TRUE (<nil>) # should error
)) -> TRUE (<nil>) # should error
))) -> TRUE (<nil>) # should error
foo -> substr:"foo" (<nil>)
foo) -> substr:"foo" (<nil>) # should error
foo)) -> substr:"foo" (<nil>) # should error
foo))) -> substr:"foo" (<nil>) # should error
(foo -> <nil> (error parsing regexp: missing closing ): `(foo`)
((foo -> <nil> (error parsing regexp: missing closing ): `((foo`)
(((foo -> <nil> (error parsing regexp: missing closing ): `(((foo`)
(foo) -> substr:"foo" (<nil>)
(foo)) -> substr:"foo" (<nil>) # should error
((foo)) -> substr:"foo" (<nil>)
(((foo)) -> <nil> (error parsing regexp: missing closing ): `(((foo))`)

[keegancsmith]

I took a quick look at this and have a potential solution at #569. However, I don't fully understand the nitty gritty of the parse functions so needs more testing/review before taking further.

[BlackBuck]

Hi [@keegancsmith ],
Thanks for your previous work on this issue. While exploring the bug with unbalanced parentheses, I noticed that the core problem might not be fully handled in the parseExprList and nextToken functions.

I saw your commit (#569). While the solution you provided should work, I’d like to share an additional direction that could help:

Proposal: Track Parentheses Balance in parseExprList, Not nextToken

Right now, nextToken uses a local parenCount to guess whether a ) is unmatched. But since nextToken is stateless across calls, it can’t reliably detect global imbalance (e.g., "foo))" silently loses the extra )).

Instead, we could handle parentheses like a stack in the parser itself (parseExprList) and validate:

1. Every time we hit a tokParenOpen, increment a counter (e.g., openParens++).
2. Every time we hit a tokParenClose, decrement it (openParens--).
3. If openParens < 0 at any point → report unmatched ) at that location.
4. At the end of parseExprList, if openParens > 0 → report a missing closing ).

This avoids overloading nextToken and places structural correctness where it belongs — in the parser logic.

Why This Helps

• Tokenization stays simple — ) is just a token.
• Structural validation is centralized.
• Better error messages are possible: "Unmatched closing parenthesis at position N" or "Missing closing ')' for '(' at byte M".

Happy to collaborate or PR this if it helps!
Let me know if this direction seems viable.