[llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes #144399

jalopezg-git · 2025-06-16T17:32:37Z

This small changelist reduces memory footprint of instances of the Core classes. Specifically,

For LVProperties, use underlying type of uint32_t if there are at most 32 properties to keep track of. Otherwise, fallback to the generic std::bitset<N>.
The use of llvm::SmallBitVector is disregarded in this case, as the upper bound on the size of the bitset can be determined statically (no heap alloc ever needed).
Reorder members in LVElement s.t. padding between members is reduced.
LVScopeCompileUnit: fix a couple of members which should be static constexpr instead.

This effectively reduces the sizes of key classes to

	`sizeof(T)` before this patch	`sizeof(T)` after this patch	% reduction
`T = LVObject`	48	40	20%
`T = LVElement`	104	80	30%
`T = LVScope`	176	144	22.22%
`T = LVSymbol`	176	144	22.22%
`T = LVScopeCompileUnit`	1016	960	5.83%

This, in turn, should have a very visible effect for parsing large debug information files.

llvmbot · 2025-06-16T17:32:56Z

@llvm/pr-subscribers-debuginfo

Author: Javier Lopez-Gomez (jalopezg-git)

Changes

Use underlying type of uint32_t if there are at most 32 properties
to keep track of. Otherwise, switch to std::bitset<N>.
This effectively reduces the size of LVObject from 48 to 40 bytes.

Full diff: https://github.com/llvm/llvm-project/pull/144399.diff

1 Files Affected:

(modified) llvm/include/llvm/DebugInfo/LogicalView/Core/LVSupport.h (+22-4)

diff --git a/llvm/include/llvm/DebugInfo/LogicalView/Core/LVSupport.h b/llvm/include/llvm/DebugInfo/LogicalView/Core/LVSupport.h
index 01137f80c0f8b..4fb7d1d13e513 100644
--- a/llvm/include/llvm/DebugInfo/LogicalView/Core/LVSupport.h
+++ b/llvm/include/llvm/DebugInfo/LogicalView/Core/LVSupport.h
@@ -21,9 +21,11 @@
 #include "llvm/Support/Format.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
+#include <bitset>
 #include <cctype>
 #include <map>
 #include <sstream>
+#include <type_traits>
 
 namespace llvm {
 namespace logicalview {
@@ -38,14 +40,30 @@ using LVLexicalIndex =
 
 // Used to record specific characteristics about the objects.
 template <typename T> class LVProperties {
-  SmallBitVector Bits = SmallBitVector(static_cast<unsigned>(T::LastEntry) + 1);
+  static constexpr unsigned N_PROPS = static_cast<unsigned>(T::LastEntry);
+  std::conditional_t<(N_PROPS > 32), std::bitset<N_PROPS>, uint32_t> Bits{};
 
 public:
   LVProperties() = default;
 
-  void set(T Idx) { Bits[static_cast<unsigned>(Idx)] = 1; }
-  void reset(T Idx) { Bits[static_cast<unsigned>(Idx)] = 0; }
-  bool get(T Idx) const { return Bits[static_cast<unsigned>(Idx)]; }
+  void set(T Idx) {
+    if constexpr (std::is_same_v<decltype(Bits), uint32_t>)
+      Bits |= 1 << static_cast<unsigned>(Idx);
+    else
+      Bits.set(static_cast<unsigned>(Idx));
+  }
+  void reset(T Idx) {
+    if constexpr (std::is_same_v<decltype(Bits), uint32_t>)
+      Bits &= ~(1 << static_cast<unsigned>(Idx));
+    else
+      Bits.reset(static_cast<unsigned>(Idx));
+  }
+  bool get(T Idx) const {
+    if constexpr (std::is_same_v<decltype(Bits), uint32_t>)
+      return Bits & (1 << static_cast<unsigned>(Idx));
+    else
+      return Bits[static_cast<unsigned>(Idx)];
+  }
 };
 
 // Generate get, set and reset 'bool' functions for LVProperties instances.

jalopezg-git · 2025-06-16T21:18:37Z

@CarlosAlbertoEnciso FYI 🙂. Feel free to review / leave comments.

Also, could you measure peak memory usage (both, before and after this change) for the Chrome debug info case?

Use underlying type of `uint32_t` if there are at most 32 properties to keep track of. Otherwise, switch to `std::bitset<N>`. This effectively reduces the size of `LVObject` from 48 to 40 bytes.

…ize padding

…opeCompileUnit

jalopezg-git · 2025-06-17T16:08:49Z

I have also made a couple of (very trivial) changes to LVScopeCompileUnit definition leading to some additional memory savings.

@CarlosAlbertoEnciso, @OCHyams, if you don't have further comments, I think this can be merged too 👍.

CarlosAlbertoEnciso · 2025-06-18T13:58:11Z

@CarlosAlbertoEnciso FYI 🙂. Feel free to review / leave comments.

Also, could you measure peak memory usage (both, before and after this change) for the Chrome debug info case?

I am preparing some memory usage data before/after for Chrome.

jalopezg-git · 2025-06-18T15:05:09Z

@CarlosAlbertoEnciso FYI 🙂. Feel free to review / leave comments.
Also, could you measure peak memory usage (both, before and after this change) for the Chrome debug info case?

I am preparing some memory usage data before/after for Chrome.

Thank you 👍!

CarlosAlbertoEnciso · 2025-06-18T16:10:38Z

Before

llvm-debuginfo-analyzer --print=all --attribute=all --internal=id,memory --output=split chrome.elf

Split View Location: '[...]chrome.elf_cus/'

Logical View:
[0x0000000001][0x0000000000][000]            {File} 'chrome.elf' -> elf64-x86-64

GID: 0x0007cd5909 GLevel: 0x0000000011

-------------------------------------------
Memory at: compareReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         77,308,243,968
Physical Memory Used:        63,048,773,632
Virtual Memory Used (me):    50,344,706,048
Physical Memory Used (me):   47,515,443,200

-------------------------------------------
Memory at: createReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         77,290,975,232
Physical Memory Used:        62,005,583,872
Virtual Memory Used (me):    50,343,739,392
Physical Memory Used (me):   47,427,366,912

-------------------------------------------
Memory at: printReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         77,308,243,968
Physical Memory Used:        63,048,773,632
Virtual Memory Used (me):    50,344,706,048
Physical Memory Used (me):   47,515,443,200

-------------------------------------------
Memory at: process
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         28,671,459,328
Physical Memory Used:        22,771,384,320
Virtual Memory Used (me):         4,685,824
Physical Memory Used (me):       16,748,544

CarlosAlbertoEnciso · 2025-06-18T16:12:30Z

After

llvm-debuginfo-analyzer --print=all --attribute=all --internal=id,memory --output=split chrome.elf

Split View Location: '[...]chrome.elf_cus/'

Logical View:
[0x0000000001][0x0000000000][000]            {File} 'chrome.elf' -> elf64-x86-64

GID: 0x0007cd5909 GLevel: 0x0000000011

-------------------------------------------
Memory at: compareReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         73,589,620,736
Physical Memory Used:        60,444,229,632
Virtual Memory Used (me):    47,460,720,640
Physical Memory Used (me):   46,571,905,024

-------------------------------------------
Memory at: createReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         74,473,222,144
Physical Memory Used:        61,511,487,488
Virtual Memory Used (me):    47,460,651,008
Physical Memory Used (me):   46,572,023,808

-------------------------------------------
Memory at: printReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         73,589,620,736
Physical Memory Used:        60,444,229,632
Virtual Memory Used (me):    47,460,720,640
Physical Memory Used (me):   46,571,905,024

-------------------------------------------
Memory at: process
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         27,781,230,592
Physical Memory Used:        17,211,813,888
Virtual Memory Used (me):         4,689,920
Physical Memory Used (me):       16,736,256

CarlosAlbertoEnciso · 2025-06-18T16:21:43Z

@jalopezg-git I added some memory data collection:

Error LVReaderHandler::process() {
  captureMemoryData("process");

  if (Error Err = createReaders())
    captureMemoryData("createReaders");
    return Err;
  if (Error Err = printReaders())
    captureMemoryData("printReaders");
    return Err;
  if (Error Err = compareReaders())
    captureMemoryData("compareReaders");
    return Err;

  return Error::success();
}

jalopezg-git · 2025-06-18T16:44:06Z

After

llvm-debuginfo-analyzer --print=all --attribute=all --internal=id,memory --output=split chrome.elf

Split View Location: '[...]chrome.elf_cus/'

Logical View:
[0x0000000001][0x0000000000][000]            {File} 'chrome.elf' -> elf64-x86-64

GID: 0x0007cd5909 GLevel: 0x0000000011

-------------------------------------------
Memory at: compareReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         73,589,620,736
Physical Memory Used:        60,444,229,632
Virtual Memory Used (me):    47,460,720,640
Physical Memory Used (me):   46,571,905,024

-------------------------------------------
Memory at: createReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         74,473,222,144
Physical Memory Used:        61,511,487,488
Virtual Memory Used (me):    47,460,651,008
Physical Memory Used (me):   46,572,023,808

-------------------------------------------
Memory at: printReaders
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         73,589,620,736
Physical Memory Used:        60,444,229,632
Virtual Memory Used (me):    47,460,720,640
Physical Memory Used (me):   46,571,905,024

-------------------------------------------
Memory at: process
-------------------------------------------
Total Virtual Memory:       137,112,543,232
Total Physical Memory:       68,393,066,496
Virtual Memory Used:         27,781,230,592
Physical Memory Used:        17,211,813,888
Virtual Memory Used (me):         4,689,920
Physical Memory Used (me):       16,736,256

Awesome 🚀! So this already improves the situation a bit! I guess we should then merge; what do you think?

jalopezg-git · 2025-06-18T16:56:08Z

A possible follow-up here: #144750 (that addresses suggested improvement in #69160).

CarlosAlbertoEnciso · 2025-06-18T17:14:29Z

@jalopezg-git Thanks for this improvement.

jalopezg-git requested review from compnerd, OCHyams and CarlosAlbertoEnciso June 16, 2025 17:32

jalopezg-git self-assigned this Jun 16, 2025

jalopezg-git added debuginfo tools:llvm-debuginfo-analyzer labels Jun 16, 2025

jalopezg-git force-pushed the jalopezg-uint32_t-LVProperties branch 2 times, most recently from 30b6aeb to ed5fcb9 Compare June 16, 2025 20:53

jalopezg-git changed the title ~~[llvm-debuginfo-analyzer] Reduce size of LVProperties where possible~~ [llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes Jun 16, 2025

compnerd approved these changes Jun 16, 2025

View reviewed changes

jalopezg-git added 3 commits June 17, 2025 17:58

[llvm-debuginfo-analyzer] Reduce size of LVProperties where possible

4363916

Use underlying type of `uint32_t` if there are at most 32 properties to keep track of. Otherwise, switch to `std::bitset<N>`. This effectively reduces the size of `LVObject` from 48 to 40 bytes.

[llvm-debuginfo-analyzer] Reorder LVElement members in order to minim…

09dbffd

…ize padding

[llvm-debuginfo-analyzer] Fix a couple of member declarations in LVSc…

909a38e

…opeCompileUnit

jalopezg-git force-pushed the jalopezg-uint32_t-LVProperties branch from ed5fcb9 to 909a38e Compare June 17, 2025 16:00

jalopezg-git mentioned this pull request Jun 18, 2025

[llvm-debuginfo-analyzer] Remove LVScope::Children container #144750

Draft

CarlosAlbertoEnciso approved these changes Jun 18, 2025

View reviewed changes

jalopezg-git merged commit c4d7ea8 into llvm:main Jun 18, 2025
7 checks passed

jalopezg-git deleted the jalopezg-uint32_t-LVProperties branch June 18, 2025 21:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes #144399

[llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes #144399

Uh oh!

jalopezg-git commented Jun 16, 2025 •

edited

Loading

Uh oh!

llvmbot commented Jun 16, 2025

Uh oh!

jalopezg-git commented Jun 16, 2025

Uh oh!

jalopezg-git commented Jun 17, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025 •

edited

Loading

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025 •

edited

Loading

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

[llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes #144399

[llvm-debuginfo-analyzer] Apply various memory savings in Core/LVxxx base classes #144399

Uh oh!

Conversation

jalopezg-git commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Jun 16, 2025

Uh oh!

jalopezg-git commented Jun 16, 2025

Uh oh!

jalopezg-git commented Jun 17, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

jalopezg-git commented Jun 18, 2025

Uh oh!

CarlosAlbertoEnciso commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

jalopezg-git commented Jun 16, 2025 •

edited

Loading

CarlosAlbertoEnciso commented Jun 18, 2025 •

edited

Loading

CarlosAlbertoEnciso commented Jun 18, 2025 •

edited

Loading