Skip to content

Commit dd7fed0

Browse files
authored
Fork the globset strategy for turbopack (#79333)
## What After discussion with [upstream](BurntSushi/ripgrep#3049) it was determined that upstreaming a function like `can_skip_directory` wasn't aligned. It turns out the needs of ripgrep and ours are somewhat divergent. Instead they encouraged me to just fork and modify which is what we do here. This copies the glob->regex translation logic and drops much of the other functionality that is optimized for large sets of globs. Then I have added our `can_skip_directory` function following a similar pattern. ## Why This approach allows us to easily support for character ranges (e.g. `[A-Za-z0-9_-]`) and improves performance. I also suspect that this will be easier to maintain as debugging the generated regular expressions should be quite straightforward. ## Alternatives We could just adopt globset directly as proposed in #79116 however we would lose the ability to skip directories during traversal which can reduce the cost of setting up turbotasks dependencies and directory watchers. The other alternative that i pursued was a direct NFA implementation (see #78976). This would be more memory efficient than regular expressions since both queries can share the same NFA, but the regex crate is much faster and better maintained. If memory overhead becomes a concern we can configure the regexes to use less, however, i suspect many of our generated regexes will be quite simple and thus generate small automata. Closes PACK-4401 Fixes #72196
1 parent e06d2d8 commit dd7fed0

File tree

11 files changed

+778
-403
lines changed

11 files changed

+778
-403
lines changed

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/next-core/src/next_server/resolve.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -119,8 +119,8 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
119119
request_glob,
120120
}) = *exception_glob
121121
{
122-
let path_match = path_glob.await?.execute(&raw_fs_path.path);
123-
let request_match = request_glob.await?.execute(&request_str);
122+
let path_match = path_glob.await?.matches(&raw_fs_path.path);
123+
let request_match = request_glob.await?.matches(&request_str);
124124
if path_match || request_match {
125125
return Ok(ResolveResultOption::none());
126126
}
@@ -135,8 +135,8 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
135135
request_glob,
136136
}) = *external_glob
137137
{
138-
let path_match = path_glob.await?.execute(&raw_fs_path.path);
139-
let request_match = request_glob.await?.execute(&request_str);
138+
let path_match = path_glob.await?.matches(&raw_fs_path.path);
139+
let request_match = request_glob.await?.matches(&request_str);
140140

141141
if !path_match && !request_match {
142142
return Ok(ResolveResultOption::none());

turbopack/crates/turbo-tasks-fs/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ mime = { workspace = true }
4141
notify = { workspace = true }
4242
parking_lot = { workspace = true }
4343
rayon = { workspace = true }
44+
regex = { workspace = true }
4445
rustc-hash = { workspace = true }
4546
serde = { workspace = true, features = ["rc"] }
4647
serde_bytes = { workspace = true }

0 commit comments

Comments
 (0)