Provide `arm64` build of `tsc-dyn.dylib` for Apple Silicon (M1) Macs

[3c1u]

Currently, tsc-dyn.dylib has only x86_64 binary.

✗ ~ file tsc-dyn.dylib 
tsc-dyn.dylib: Mach-O 64-bit dynamically linked shared library x86_64

tsc-dyn.dylib can be cross-compiled using the latest Rust toolchain.

[ubolonton]

This will likely need system toolchain for arm64 as well, in addition to Rust toolchain.
Then we will need someone with an M1 Mac to test it.

Have you been able to cross-compile it? Or do you have a link to a working guide for targeting aarch64-apple-darwin on x86_64-apple-darwin?

[3c1u]

Tested on x86_64 macOS 11.1.

Use master branch of tree-sitter (arm64 is now supported), and emacs crate with updated bindgen (libclang fails to load somehow otherwise)

emacs = { path = "../../emacs-module-rs" }
tree-sitter = { git = "https://github.com/tree-sitter/tree-sitter.git" }

and run this command:

env SDKROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk" cargo build --all --release --target aarch64-apple-darwin

[ubolonton]

Did you install XCode Beta?

My MacOS SDK doesn't support arm64. I'm getting error architecture not supported.
I don't have access to XCode Beta, so I cannot try this out at the moment.

https://stackoverflow.com/questions/64313634/xcode-12-compiling-for-macos-arm64-arch

[3c1u]

Nope, I'm using the latest stable Xcode 12.3 (12C33) on macOS Big Sur 11.1.

[ubolonton]

Ah ok, I misread as 10.11. Yeah, Big Sur's SDK supports arm64 arch.

I'll check this again when I have access to an SDK supporting arm64.

[ubolonton]

and emacs crate with updated bindgen (libclang fails to load somehow otherwise)

Does it work if you delete Cargo.lock instead?

[3c1u]

Deleting Cargo.lock worked on x86_64 cross-compilation. I also tried on an Apple Silicon Mac and libclang fails to load, which cannot be solved by deleting Cargo.lock.

[theothornhill]

I'm struggling with how to build this on my M1 mac. I seem to be stuck on resolving dependencies, the earliest error being:

   Compiling emacs_module v0.12.0
The following warnings were emitted during compilation:

warning: couldn't execute `llvm-config --prefix` (error: No such file or directory (os error 2))
warning: set the LLVM_CONFIG_PATH environment variable to a valid `llvm-config` executable

error: failed to run custom build command for `emacs_module v0.12.0`

Caused by:
  process didn't exit successfully: `/Users/theodor/Git/emacs-tree-sitter/target/debug/build/emacs_module-e1ade98717ceb887/build-script-build` (exit code: 101)
  --- stdout
  cargo:warning=couldn't execute `llvm-config --prefix` (error: No such file or directory (os error 2))
  cargo:warning=set the LLVM_CONFIG_PATH environment variable to a valid `llvm-config` executable

  --- stderr
  thread 'main' panicked at 'libclang error; possible causes include:
  - Invalid flag syntax
  - Unrecognized flags
  - Invalid flag arguments
  - File I/O errors
  If you encounter an error missing from this list, please file an issue or a PR!', /Users/theodor/.cargo/registry/src/proxy-git.cwkhome.fun-1ecc6299db9ec823/bindgen-0.51.1/src/ir/context.rs:571:15
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

With some help from someone more knowledgeable in rust I can try to build this

[ubolonton]

@theothornhill Seems like you need to install llvm. It's needed to generate the Rust binding for emacs-module.h

[theothornhill]

Yeah, you're right. emacs_module fails on bindgen-0.51, as mentioned other places.

[theothornhill]

I think I managed to compile it to M1 now.

running file on the created dylib returns

tsc-dyn.dylib: Mach-O 64-bit dynamically linked shared library arm64

However, now I'm stuck at compiling the rust.dylib.
Running make ensure/rust it fails with:

Debugger entered--Lisp error: (tsc-lang-load-failed "dlopen(/Users/theodor/Git/emacs-tree-sitter/langs/...")
  tsc--load-language("/Users/theodor/Git/emacs-tree-sitter/langs/bin/rus..." "tree_sitter_rust" rust)
  tree-sitter-load(rust nil nil)
  tree-sitter-require(rust)
  (condition-case nil (tree-sitter-require lang-symbol) (error (display-warning 'tree-sitter-langs (format "Could not load grammar for `%s', trying to compile..." lang-symbol)) (tree-sitter-langs-compile lang-symbol) (tree-sitter-require lang-symbol)))
  (unwind-protect (condition-case nil (tree-sitter-require lang-symbol) (error (display-warning 'tree-sitter-langs (format "Could not load grammar for `%s', trying to compile..." lang-symbol)) (tree-sitter-langs-compile lang-symbol) (tree-sitter-require lang-symbol))) (tree-sitter-langs--copy-query lang-symbol))
  tree-sitter-langs-ensure(rust)
  (progn (require 'tree-sitter-langs) (tree-sitter-langs-ensure 'rust))
  eval((progn (require 'tree-sitter-langs) (tree-sitter-langs-ensure 'rust)) t)
  command-line-1(("--directory" "/Users/theodor/Git/emacs-tree-sitter/core" "--directory" "/Users/theodor/Git/emacs-tree-sitter/lisp" "--directory" "/Users/theodor/Git/emacs-tree-sitter/langs" "--eval" "(progn (require 'tree-sitter-langs) (tree-sitter-l..."))
  command-line()
  normal-top-level()

after it has run the tree-sitter test. I've tried it both from normal terminal and shell inside emacs with load-path explicitly set before running the command. Any tips for how to proceed?

[douglasdavis]

I'm seeing something similar. I built tsc-dyn.dylib but I see the following when running make ensure/python:

$ EMACS=/Applications/Emacs.app/Contents/MacOS/Emacs make ensure/python
[...]
Debugger entered--Lisp error: (tsc-lang-abi-too-new 13 (9 . 12) "/Users/ddavis/software/repos/emacs-tree-sitter/lan...")
  tsc--load-language("/Users/ddavis/software/repos/emacs-tree-sitter/lan..." "tree_sitter_python" python)
  (let ((language (tsc--load-language full-path native-symbol-name lang-symbol))) (let* ((key lang-symbol)) (condition-case nil (with-no-warnings (map-put! tree-sitter-languages key language nil)) (map-not-inplace (setq tree-sitter-languages (map-insert tree-sitter-languages key language))))) language)
  (let* ((lang-name (symbol-name lang-symbol)) (fallback-name (replace-regexp-in-string "-" "_" lang-name)) (native-symbol-name (or native-symbol-name (format "tree_sitter_%s" fallback-name))) (files (if file (list file) (cons lang-name (if (string= lang-name fallback-name) nil (list fallback-name))))) (full-path (seq-some #'(lambda (base-name) (locate-file base-name tree-sitter-load-path tree-sitter-load-suffixes)) files))) (if full-path nil (error "Cannot find shared library for language: %S" lang-symbol)) (let ((language (tsc--load-language full-path native-symbol-name lang-symbol))) (let* ((key lang-symbol)) (condition-case nil (with-no-warnings (map-put! tree-sitter-languages key language nil)) (map-not-inplace (setq tree-sitter-languages (map-insert tree-sitter-languages key language))))) language))
  tree-sitter-load(python nil nil)
  (or (alist-get lang-symbol tree-sitter-languages) (tree-sitter-load lang-symbol file native-symbol-name))
  tree-sitter-require(python)
  (condition-case nil (tree-sitter-require lang-symbol) (error (display-warning 'tree-sitter-langs (format "Could not load grammar for `%s', trying to compile..." lang-symbol)) (tree-sitter-langs-compile lang-symbol) (tree-sitter-require lang-symbol)))
  (unwind-protect (condition-case nil (tree-sitter-require lang-symbol) (error (display-warning 'tree-sitter-langs (format "Could not load grammar for `%s', trying to compile..." lang-symbol)) (tree-sitter-langs-compile lang-symbol) (tree-sitter-require lang-symbol))) (tree-sitter-langs--copy-query lang-symbol))
  tree-sitter-langs-ensure(python)
  (progn (require 'tree-sitter-langs) (tree-sitter-langs-ensure 'python))
  eval((progn (require 'tree-sitter-langs) (tree-sitter-langs-ensure 'python)) t)
  command-line-1(("--directory" "/Users/ddavis/software/repos/emacs-tree-sitter/cor..." "--directory" "/Users/ddavis/software/repos/emacs-tree-sitter/lis..." "--directory" "/Users/ddavis/software/repos/emacs-tree-sitter/lan..." "--eval" "(progn (require 'tree-sitter-langs) (tree-sitter-l..."))
  command-line()
  normal-top-level()

make: *** [ensure/python] Error 255