Description
I'm opening the issue here, specifically, because I'm not seeing such drastic slowdown with other grammars (i.e. they apparently have similar query performance regardless of the version).
I have this huge and really repetitive query which I use for querying identifiers inside specific constructs. When generating the grammar with tree-sitter-cli > 0.16.9
, the timings get much worse. I've observed that it's not related to the file size, nor due to how complex the file is, nor due to recent commits in this repository... After experimenting a bit, my gut impression is that "query optimization" got worse > 0.16.9
- the timings stay pretty much the same between different versions if I remove the deeply-nested s-expressions.
ts_query
(function_declaration
parameters: (formal_parameters
(required_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(function_declaration
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(shorthand_property_identifier) @identifier))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))))
(function_declaration
parameters: (formal_parameters
(required_parameter ((array_pattern
(identifier) @identifier)))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* ((identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter ((array_pattern
(identifier) @identifier)))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(function_declaration
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter ((array_pattern
(identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* ((identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter ((array_pattern
(identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter ((array_pattern
(identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* ((identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(required_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter ((array_pattern
(identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* ((identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* ((identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* ((identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters
(optional_parameter (array_pattern (* (* (* (* ((identifier) @identifier)))))))))
(function_declaration
parameters: (formal_parameters
(required_parameter (identifier) @identifier)))
(function_declaration
parameters: (formal_parameters
(optional_parameter (identifier) @identifier)))
(arrow_function
parameters: (formal_parameters
(required_parameter (identifier) @identifier)))
(arrow_function
parameters: (formal_parameters
(optional_parameter (identifier) @identifier)))
(arrow_function
parameters: (formal_parameters
(required_parameter (identifier) @identifier)))
(arrow_function
parameters: (formal_parameters
(optional_parameter (identifier) @identifier)))
(for_in_statement
left: (object_pattern
(shorthand_property_identifier) @identifier))
(for_in_statement
left: (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))
(for_in_statement
left: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))
(for_in_statement
left: (identifier) @identifier)
(assignment_expression
left: (identifier) @identifier)
(import_statement
(import_clause
(named_imports
(import_specifier
name: (identifier) @identifier))))
(import_statement
(import_clause
(named_imports
(import_specifier
alias: (identifier) @identifier))))
(import_statement
(import_clause
(identifier) @identifier))
(variable_declarator
name: (object_pattern
(shorthand_property_identifier) @identifier))
(variable_declarator
name: (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))
(variable_declarator
name: (object_pattern
(pair
value: (identifier) @identifier)))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (identifier) @identifier)))))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (identifier) @identifier)))))))
(variable_declarator
name: ((array_pattern
(identifier) @identifier)))
(variable_declarator
name: (array_pattern (* ((shorthand_property_identifier) @identifier))))
(variable_declarator
name: (array_pattern (* ((identifier) @identifier))))
(variable_declarator
name: (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))
(variable_declarator
name: (array_pattern (* (* ((identifier) @identifier)))))
(variable_declarator
name: (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))
(variable_declarator
name: (array_pattern (* (* (* ((identifier) @identifier))))))
(variable_declarator
name: (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))
(variable_declarator
name: (array_pattern (* (* (* (* ((identifier) @identifier)))))))
(variable_declarator
name: (identifier) @identifier)
(function_declaration name: (identifier) @identifier)
(namespace_import (identifier) @identifier)
((type_identifier) @identifier)
(object_pattern
(pair
value: (identifier) @identifier))
(labeled_statement
label: (statement_identifier) @identifier)
For the following timings, I'm running the above query against the React Typings (source) which has 3169 lines.
If it's generated with tree-sitter 0.16.9 (12341dbbc03075e0b3bdcbf05191efbac78731fe)
time ../node_modules/.bin/tree-sitter query ts_query react.d.ts
real 0m0.068s
user 0m0.053s
sys 0m0.014s
If it's generated with tree-sitter 0.17.0 (b6fba7ca4c32207fa9b387b594a8da2ff66ee4be)
(and above)
time ../node_modules/.bin/tree-sitter query ts_query react.d.ts
real 0m0.492s
user 0m0.473s
sys 0m0.017s
Earlier it was mentioned
I'm not seeing such drastic slowdown with other grammars (i.e. they apparently behave up to par regardless of the version).
On that note, I here's a query very similar to the one posted above, but for JavaScript instead
js_query
(function_declaration
parameters: (formal_parameters (object_pattern
(shorthand_property_identifier) @identifier)))
(function_declaration
parameters: (formal_parameters (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier)))))
(function_declaration
parameters: (formal_parameters (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier)))))))))
(function
parameters: (formal_parameters (object_pattern
(shorthand_property_identifier) @identifier)))
(function
parameters: (formal_parameters (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier)))))
(function
parameters: (formal_parameters (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier)))))))))
(arrow_function
parameters: (formal_parameters (object_pattern
(shorthand_property_identifier) @identifier)))
(arrow_function
parameters: (formal_parameters (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier)))))))))
(function_declaration
parameters: (formal_parameters ((array_pattern
(identifier) @identifier))))
(function_declaration
parameters: (formal_parameters (array_pattern (* ((shorthand_property_identifier) @identifier)))))
(function_declaration
parameters: (formal_parameters (array_pattern (* ((identifier) @identifier)))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* ((shorthand_property_identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* ((identifier) @identifier))))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* (* ((shorthand_property_identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* (* ((identifier) @identifier)))))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters (array_pattern (* (* (* (* ((identifier) @identifier))))))))
(function
parameters: (formal_parameters ((array_pattern
(identifier) @identifier))))
(function
parameters: (formal_parameters (array_pattern (* ((shorthand_property_identifier) @identifier)))))
(function
parameters: (formal_parameters (array_pattern (* ((identifier) @identifier)))))
(function
parameters: (formal_parameters (array_pattern (* (* ((shorthand_property_identifier) @identifier))))))
(function
parameters: (formal_parameters (array_pattern (* (* ((identifier) @identifier))))))
(function
parameters: (formal_parameters (array_pattern (* (* (* ((shorthand_property_identifier) @identifier)))))))
(function
parameters: (formal_parameters (array_pattern (* (* (* ((identifier) @identifier)))))))
(function
parameters: (formal_parameters (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier))))))))
(function
parameters: (formal_parameters (array_pattern (* (* (* (* ((identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters ((array_pattern
(identifier) @identifier))))
(arrow_function
parameters: (formal_parameters (array_pattern (* ((shorthand_property_identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters (array_pattern (* ((identifier) @identifier)))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* ((shorthand_property_identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* ((identifier) @identifier))))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* (* ((shorthand_property_identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* (* ((identifier) @identifier)))))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier))))))))
(arrow_function
parameters: (formal_parameters (array_pattern (* (* (* (* ((identifier) @identifier))))))))
(function_declaration
parameters: (formal_parameters (identifier) @identifier))
(function
parameters: (formal_parameters (identifier) @identifier))
(arrow_function
parameters: (formal_parameters (identifier) @identifier))
(function_declaration
parameters: (formal_parameters (rest_parameter
(identifier) @identifier)))
(function
parameters: (formal_parameters (rest_parameter
(identifier) @identifier)))
(arrow_function
parameters: (formal_parameters (rest_parameter
(identifier) @identifier)))
(for_in_statement
left: (object_pattern
(shorthand_property_identifier) @identifier))
(for_in_statement
left: (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))
(for_in_statement
left: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))
(for_in_statement
left: (identifier) @identifier)
(assignment_expression
left: (identifier) @identifier)
(import_statement
(import_clause
(named_imports
(import_specifier
name: (identifier) @identifier))))
(import_statement
(import_clause
(named_imports
(import_specifier
alias: (identifier) @identifier))))
(import_statement
(import_clause
(identifier) @identifier))
(variable_declarator
name: (object_pattern
(shorthand_property_identifier) @identifier))
(variable_declarator
name: (object_pattern
(pair
value: (object
(shorthand_property_identifier) @identifier))))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (object
(shorthand_property_identifier) @identifier))))))))
(variable_declarator
name: (object_pattern
(pair
value: (identifier) @identifier)))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (identifier) @identifier)))))
(variable_declarator
name: (object_pattern
(pair
value: (object
(pair
value: (object
(pair
value: (identifier) @identifier)))))))
(variable_declarator
name: ((array_pattern
(identifier) @identifier)))
(variable_declarator
name: (array_pattern (* ((shorthand_property_identifier) @identifier))))
(variable_declarator
name: (array_pattern (* ((identifier) @identifier))))
(variable_declarator
name: (array_pattern (* (* ((shorthand_property_identifier) @identifier)))))
(variable_declarator
name: (array_pattern (* (* ((identifier) @identifier)))))
(variable_declarator
name: (array_pattern (* (* (* ((shorthand_property_identifier) @identifier))))))
(variable_declarator
name: (array_pattern (* (* (* ((identifier) @identifier))))))
(variable_declarator
name: (array_pattern (* (* (* (* ((shorthand_property_identifier) @identifier)))))))
(variable_declarator
name: (array_pattern (* (* (* (* ((identifier) @identifier)))))))
(variable_declarator
name: (identifier) @identifier)
(class_declaration name: (identifier) @identifier)
(function_declaration name: (identifier) @identifier)
(namespace_import (identifier) @identifier)
(object_pattern
(pair
value: (identifier) @identifier))
(labeled_statement
label: (statement_identifier) @identifier)
I did the same experiment running js_query against d3.js' source code (source) which has 19540 lines. In this case, regardless of the tree-sitter-cli
version used, the timings are pretty much the same.
time ./node_modules/.bin/tree-sitter query js_query d3.js
real 0m0.252s
user 0m0.211s
sys 0m0.040s
To summarize, I'm noticing a strange behavior where (apparently only) this grammar, specifically, runs queries much slower if it's generated with tree-sitter-cli > 0.16.9
.