Skip to content

Fix PackageGraph traversal performance regressions #7248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 17, 2024

Conversation

MaxDesiatov
Copy link
Contributor

@MaxDesiatov MaxDesiatov commented Jan 12, 2024

Depends on #7258

This restores performance of traversal of deep package graphs previously available when ResolvedTarget, ResolvedProduct, and ResolvedPackage were reference types. Now instead of recursively applying Hashable and Equatable conformances to all properties in graph's node, we only hash and compare for equality their identities defined by Identifiable.

Uses of Set on these types are now replaced with the new IdentifiableSet type, which uses a dictionary of [Element.ID: Element] type as its storage.

The main benefit of this approach is that build triple of ResolvedProduct and ResolvedTarget is now a part of their identity. Thus we can still use value types, while the same Target from PackageModel instantiated as two different ResolvedTargets with different BuildTriples will have different identities.

Resolves rdar://120861833
Resolves rdar://120711211

@MaxDesiatov MaxDesiatov added bug performance Performance optimizations and improvements modules graph Modules dependency resolution labels Jan 12, 2024
@MaxDesiatov MaxDesiatov self-assigned this Jan 12, 2024
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov MaxDesiatov enabled auto-merge (squash) January 12, 2024 23:15
@MaxDesiatov
Copy link
Contributor Author

MaxDesiatov commented Jan 12, 2024

The actual fix is also verified by PackageGraphPerfTests, which with the regression takes minutes to complete, but now with the fix passes in seconds. Not sure though why the regression doesn't trigger baseline checks on CI, maybe this test is somehow skipped? I previously did see it catching a more severe performance regression on CI (on Linux!), but only once. 🤔

@MaxDesiatov
Copy link
Contributor Author

Currently blocked due to #7249

This restores the performance of traversal of deep package graphs previously available when `ResolvedTarget`, `ResolvedProduct`, and `ResolvedPackage` were value types. Now instead of recursively applying `Hashable` conformance to all properties in graph's node, we only hash and compare for equality their identities.

The main benefit of this approach is that build triple of `ResolvedProduct` and `ResolvedTarget` is now a part of their identity. Thus we can still use value types, while the same `Target` from `PackageModel` instantiated as two different `ResolvedTarget`s with different `BuildTriples` will have different identities.

Resolves rdar://120861833
Resolves rdar://120711211
@MaxDesiatov MaxDesiatov force-pushed the maxd/nonshable-packagegraph branch from e9ede20 to 98fd47f Compare January 16, 2024 15:45
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

MaxDesiatov added a commit to swiftlang/sourcekit-lsp that referenced this pull request Jan 17, 2024
`ResolvedTarget` is no longer `Hashable` in SwiftPM, SourceKit-LSP should use `ResolvedTarget.ID` instead. Depends on swiftlang/swift-package-manager#7248.
@MaxDesiatov
Copy link
Contributor Author

@MaxDesiatov
Copy link
Contributor Author

swiftlang/sourcekit-lsp#1025

@swift-ci test windows

1 similar comment
@MaxDesiatov
Copy link
Contributor Author

swiftlang/sourcekit-lsp#1025

@swift-ci test windows

@MaxDesiatov MaxDesiatov merged commit 3baeee5 into main Jan 17, 2024
@MaxDesiatov MaxDesiatov deleted the maxd/nonshable-packagegraph branch January 17, 2024 17:19
MaxDesiatov added a commit to swiftlang/sourcekit-lsp that referenced this pull request Jan 17, 2024
`ResolvedTarget` is no longer `Hashable` in SwiftPM, SourceKit-LSP should use `ResolvedTarget.ID` instead. Depends on swiftlang/swift-package-manager#7248.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug modules graph Modules dependency resolution performance Performance optimizations and improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants