directly output LLVM bitcode rather than using LLVM's IRBuilder API

[andrewrk]

Zig can be built with or without -Denable-llvm. Currently, Zig is not very useful without enabling LLVM extensions. However, as we move into the future, Zig is intending to compete directly with LLVM, making builds of Zig without LLVM a compelling option for the backends directly supported by Zig.

There are a few reasons why one might want an LLVM-less binary:

• The executable is 4.4 MiB instead of 169 MiB.
• Bootstrapping it only requires a C compiler instead of requiring a modern C++ compiler, Python 3, bash, and CMake (also C++).
  • This would make it much easier to obtain a Zig compiler on a new operating system or a limited environment such as a calculator.

This proposal is to treat LLVM bitcode files (.bc) as the target output format, rather than going through the C++ IRBuilder API. This would make it possible for even non-LLVM-enabled builds of Zig to still output LLVM IR that could be consumed by Clang, other LLVM tools, or integrated with other software.

One example user story comes from Roc. I'd like to get @rtfeldman's take on this - I know that you're using Zig to output .bc files, but then what happens? Does a different tool compile that together with other code, or do you use Zig for the final link step too? I'm guessing that Roc would be able to use the non-LLVM-enabled Zig binaries for their use case.

There is a second major reason for this proposal, which is perhaps even the better argument in favor of it, which is to make incremental compilation work more robustly. As the Zig project moves forward, we want to make CacheMode.incremental the default for all backends including LLVM (caddbbc). This means we would want to save the LLVM IR module (.bc) with every compilation and restore it for subsequent compilations, using the IRBuilder API to add and remove declarations as necessary from the LLVM IR module, keeping the .bc file on disk in sync for future incremental compilations.

However... the API lacks functionality. For example, aliases cannot be deleted:

zig/src/codegen/llvm.zig

Lines 1330 to 1348 in e67c756

	// TODO LLVM C API does not support deleting aliases. We need to
	// patch it to support this or figure out how to wrap the C++ API ourselves.
	// Until then we iterate over existing aliases and make them point
	// to the correct decl, or otherwise add a new alias. Old aliases are leaked.
	for (exports[1..]) |exp| {
	const exp_name_z = try module.gpa.dupeZ(u8, exp.options.name);
	defer module.gpa.free(exp_name_z);
	if (self.llvm_module.getNamedGlobalAlias(exp_name_z.ptr, exp_name_z.len)) |alias| {
	alias.setAliasee(llvm_global);
	} else {
	_ = self.llvm_module.addAlias(
	llvm_global.globalGetValueType(),
	0,
	llvm_global,
	exp_name_z,
	);
	}
	}

If Zig were in control of outputting the .bc file instead, then Zig could simply not emit aliases that are not supposed to exist. We no longer are limited by what the IRBuilder API can do. This would make the LLVM backend very similar to the WebAssembly backend in the sense that it gains a linking component and directly outputs the module.

Finally, in the incremental compilation sense, Zig would already be trying to keep a .bc file on disk up-to-date via the IRBuilder API. Doing it directly instead of via a limited API is a more direct way to solve the problem, and the performance would be in our hands rather than in the hands of the LLVM project.

I think these two reasons combined make this proposal worth seriously considering, despite the downsides of taking on additional maintenance with LLVM upgrades, and introducing an entirely new class of bugs from generating malformed .bc files.

[andrewrk]

Two relevant links:

• LLVM Bitcode File Format
• IR Backwards Compatibility

[andrewrk]

Another motivation for this would be to reduce memory usage - it seems the main culprit of using a lot of memory when using the LLVM backend is LLVM itself:

[lin72h]

I'm so exciting about this proposal.

One of the future direction can benefit from this is the new GlobalIsel pipeline, which I think is production ready for arm64.

By decoupling the llvm ir generation from the zig pipeline, we can target the GMIR directly, and take advantage of the years huge effort by Apple

[rtfeldman]

I'd like to get @rtfeldman's take on this - I know that you're using Zig to output .bc files, but then what happens? Does a different tool compile that together with other code, or do you use Zig for the final link step too? I'm guessing that Roc would be able to use the non-LLVM-enabled Zig binaries for their use case.

Yep, I think we should be fine. Basically what we do for our optimized build is:

• Our zig standard library gets compiled into .bc files
• We load those up in LLVM to get our initial modules
• We compile the user's .roc code directly into those LLVM modules
• We tell LLVM to convert the modules to binary

So as long as Zig still supports emitting .bc files, we should be fine! 👍

[Snektron]

I just want to highlight that LLVM bitcode is not stable, so this will add friction for the user. Is the plan to still have integration with clang, but via shelling out rather than linking? Adding that either in the compiler or in build.zig would provide a smoother experience as it would ensure that the user does not need to deal with this versioning stuff.

[RealNeGate]

I just want to highlight that LLVM bitcode is not stable, so this will add friction for the user. Is the plan to still have integration with clang, but via shelling out rather than linking? Adding that either in the compiler or in build.zig would provide a smoother experience as it would ensure that the user does not need to deal with this versioning stuff.

https://llvm.org/docs/DeveloperPolicy.html#ir-backwards-compatibility

you can emit outdated LLVM bitcode and get away with that, it's relatively stable in that way. "The current LLVM version supports loading any bitcode since version 3.0"

[andrewrk]

@Snektron you seem to be getting this issue confused with #16270. This issue, when implemented, will mean that Zig outputs .bc files compatible with the same version of LLVM that Zig links against, in memory, and then uses LLVMParseBitcodeInContext2 to convert that to an LLVMModuleRef rather than using the LLVMIRBuilder API. There will be no visible difference for users, except that it will be faster, and incremental compilation will work better. It means targeting the bitcode format as an ABI rather than targeting the LLVMIRBuilder API, greatly reducing our LLVM API surface area.