Cannot start language server on MacOS "Invalid Byte Sequence"

[felixlinker]

Your environment

Which OS do you use: MacOS

Steps to reproduce

1. Clone this repository: https://github.com/tamarin-prover/tamarin-prover
2. Open a .hs file in lib
3. Attempt to start the language server

Expected behaviour

The language server should start

Actual behaviour

The language server does not start, but crashes five times with this specific error message:

023-12-07 12:45:53.9670000 [client] INFO Executing 'haskell-language-server-wrapper --project-ghc-version' in cwd '/Users/felixlinker/git/tamarin-prover-test'
2023-12-07 12:45:54.0040000 [client] ERROR Error executing 'haskell-language-server-wrapper --project-ghc-version' with error code 1
2023-12-07 12:45:54.0040000 [client] ERROR stderr: No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover-test/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)

2023-12-07 12:45:54.0040000 [client] ERROR Error: haskell-language-server --project-ghc-version exited with exit code 1:

No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover-test/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)

Weirdly enough, I cannot reproduce this in my shell. If I execute haskell-language-server-wrapper --project-ghc-version in zsh, it outputs:

felixlinker@... tamarin-prover % haskell-language-server-wrapper --project-ghc-version
No 'hie.yaml' found. Try to discover the project type!
2023-12-07T12:48:20.418580Z | Debug | executing command: stack setup --silent
2023-12-07T12:48:20.891226Z | Debug | executing command: stack exec ghc -- --numeric-version
9.2.8

Maybe this is an encoding issue? I cannot find how I could set the encoding used in the shell that is launched by the extension. According to vscode, the cabal file in accountability is UTF-8 encoded. I do not think that this issue is specific to that cabal file, though. Probably, accountability just gets loaded first (starts with "a").

Include debug information

Debug output:

Attached as: debug.txt

Extension log:

2023-12-07 12:44:50.7230000 [client] DEBUG Environment variables:
2023-12-07 12:44:50.7230000 [client] DEBUG   PATH: /Users/felixlinker/.nvm/versions/node/v21.1.0/bin:/Users/felixlinker/.cabal/bin:/Users/felixlinker/.ghcup/bin:/Users/felixlinker/bin/maude:/Users/felixlinker/git/diff-so-fancy:/Users/felixlinker/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/TeX/texbin:/usr/local/go/bin
2023-12-07 12:44:50.7230000 [client] INFO Finding haskell-language-server
2023-12-07 12:44:50.7230000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:50.7260000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:50.7260000 [client] INFO Executing 'ghcup --no-verbose upgrade' in cwd '/'
2023-12-07 12:44:50.8520000 [client] DEBUG Execution of 'ghcup --no-verbose upgrade' terminated with code 0
2023-12-07 12:44:50.8520000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:50.8540000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:50.8540000 [client] INFO Executing 'ghcup --no-verbose list -t hls -c installed -r' in cwd '/'
2023-12-07 12:44:50.9150000 [client] DEBUG Execution of 'ghcup --no-verbose list -t hls -c installed -r' terminated with code 0
2023-12-07 12:44:50.9150000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:50.9160000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:50.9160000 [client] INFO Executing 'ghcup --no-verbose list -t cabal -c installed -r' in cwd '/'
2023-12-07 12:44:50.9910000 [client] DEBUG Execution of 'ghcup --no-verbose list -t cabal -c installed -r' terminated with code 0
2023-12-07 12:44:50.9910000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:50.9930000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:50.9930000 [client] INFO Executing 'ghcup --no-verbose list -t stack -c installed -r' in cwd '/'
2023-12-07 12:44:51.0500000 [client] DEBUG Execution of 'ghcup --no-verbose list -t stack -c installed -r' terminated with code 0
2023-12-07 12:44:51.0520000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.0530000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.0530000 [client] INFO Executing 'ghcup --no-verbose whereis hls 2.4.0.0' in cwd '/'
2023-12-07 12:44:51.0730000 [client] DEBUG Execution of 'ghcup --no-verbose whereis hls 2.4.0.0' terminated with code 0
2023-12-07 12:44:51.0730000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.0750000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.0750000 [client] INFO Executing 'ghcup --no-verbose whereis cabal 3.6.2.0' in cwd '/'
2023-12-07 12:44:51.0920000 [client] DEBUG Execution of 'ghcup --no-verbose whereis cabal 3.6.2.0' terminated with code 0
2023-12-07 12:44:51.0920000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.0940000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.0940000 [client] INFO Executing 'ghcup --no-verbose whereis stack 2.11.1' in cwd '/'
2023-12-07 12:44:51.1130000 [client] DEBUG Execution of 'ghcup --no-verbose whereis stack 2.11.1' terminated with code 0
2023-12-07 12:44:51.1150000 [client] INFO Executing 'ghc --numeric-version' in cwd '/'
2023-12-07 12:44:51.1590000 [client] DEBUG Execution of 'ghc --numeric-version' terminated with code 0
2023-12-07 12:44:51.1590000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.1600000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.1610000 [client] INFO Executing 'ghcup --no-verbose run --hls 2.4.0.0 --cabal 3.6.2.0 --stack 2.11.1 --install' in cwd '/'
2023-12-07 12:44:51.2170000 [client] DEBUG Execution of 'ghcup --no-verbose run --hls 2.4.0.0 --cabal 3.6.2.0 --stack 2.11.1 --install' terminated with code 0
2023-12-07 12:44:51.2170000 [client] INFO Working out the project GHC version. This might take a while...
2023-12-07 12:44:51.2170000 [client] INFO Executing 'haskell-language-server-wrapper --project-ghc-version' in cwd '/Users/felixlinker/git/tamarin-prover'
2023-12-07 12:44:51.2420000 [client] DEBUG Execution of 'haskell-language-server-wrapper --project-ghc-version' terminated with code 1
2023-12-07 12:44:51.2430000 [client] ERROR Error executing 'haskell-language-server-wrapper --project-ghc-version' with error code 1
2023-12-07 12:44:51.2430000 [client] ERROR stderr: No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)

2023-12-07 12:44:51.2430000 [client] ERROR Error: haskell-language-server --project-ghc-version exited with exit code 1:

No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)

2023-12-07 12:44:51.2430000 [client] INFO Executing 'ghc --numeric-version' in cwd '/'
2023-12-07 12:44:51.2880000 [client] DEBUG Execution of 'ghc --numeric-version' terminated with code 0
2023-12-07 12:44:51.3470000 [client] INFO Platform constants: Darwin, A_ARM64
2023-12-07 12:44:51.3470000 [client] DEBUG HLS 1.4.0 compatible with GHC Versions: 8.10.7
2023-12-07 12:44:51.3470000 [client] DEBUG HLS 1.5.0 compatible with GHC Versions: 8.10.7
2023-12-07 12:44:51.3470000 [client] DEBUG HLS 1.5.1 compatible with GHC Versions: 8.10.7
2023-12-07 12:44:51.3470000 [client] DEBUG HLS 1.6.0.0 compatible with GHC Versions: 8.10.7
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.6.1.0 compatible with GHC Versions: 8.10.7
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.7.0.0 compatible with GHC Versions: 8.10.7,9.0.2,9.2.1,9.2.2
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.8.0.0 compatible with GHC Versions: 8.10.7,9.0.2,9.2.3,9.2.4,9.4.1,9.4.2
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.9.0.0 compatible with GHC Versions: 8.10.7,9.0.2,9.2.5,9.4.3,9.4.4
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.9.1.0 compatible with GHC Versions: 8.10.7,9.2.5,9.2.6,9.4.4
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 1.10.0.0 compatible with GHC Versions: 8.10.7,9.2.7,9.4.4,9.6.1
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.0.0.0 compatible with GHC Versions: 8.10.7,9.2.7,9.4.5,9.6.1
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.0.0.1 compatible with GHC Versions: 8.10.7,9.2.7,9.2.8,9.4.5,9.6.2
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.1.0.0 compatible with GHC Versions: 8.10.7,9.2.8,9.4.5,9.4.6,9.6.2
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.2.0.0 compatible with GHC Versions: 8.10.7,9.2.8,9.4.5,9.4.6,9.4.7,9.6.2
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.3.0.0 compatible with GHC Versions: 9.2.8,9.4.7,9.6.2,9.6.3
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.4.0.0 compatible with GHC Versions: 9.2.8,9.4.7,9.6.3,9.8.1
2023-12-07 12:44:51.3480000 [client] DEBUG HLS 2.5.0.0 compatible with GHC Versions: 9.2.8,9.4.8,9.6.3,9.8.1
2023-12-07 12:44:51.3480000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.3500000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.3500000 [client] INFO Executing 'ghcup --no-verbose list -t hls -c installed -r' in cwd '/'
2023-12-07 12:44:51.4070000 [client] DEBUG Execution of 'ghcup --no-verbose list -t hls -c installed -r' terminated with code 0
2023-12-07 12:44:51.4070000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.4090000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.4090000 [client] INFO Executing 'ghcup --no-verbose whereis bindir' in cwd '/'
2023-12-07 12:44:51.4280000 [client] DEBUG Execution of 'ghcup --no-verbose whereis bindir' terminated with code 0
2023-12-07 12:44:51.4280000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.4300000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.4300000 [client] INFO Executing 'ghcup --no-verbose whereis hls 2.4.0.0' in cwd '/'
2023-12-07 12:44:51.4480000 [client] DEBUG Execution of 'ghcup --no-verbose whereis hls 2.4.0.0' terminated with code 0
2023-12-07 12:44:51.4480000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.4500000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.4500000 [client] INFO Executing 'ghcup --no-verbose whereis ghc 9.4.7' in cwd '/'
2023-12-07 12:44:51.4690000 [client] DEBUG Execution of 'ghcup --no-verbose whereis ghc 9.4.7' terminated with code 0
2023-12-07 12:44:51.4690000 [client] INFO Checking for ghcup installation
2023-12-07 12:44:51.4700000 [client] INFO found ghcup at ghcup
2023-12-07 12:44:51.4700000 [client] INFO Executing 'ghcup --no-verbose run --hls 2.4.0.0 --cabal 3.6.2.0 --stack 2.11.1 --ghc 9.4.7 --install' in cwd '/'
2023-12-07 12:44:51.5370000 [client] DEBUG Execution of 'ghcup --no-verbose run --hls 2.4.0.0 --cabal 3.6.2.0 --stack 2.11.1 --ghc 9.4.7 --install' terminated with code 0
2023-12-07 12:44:51.5380000 [client] INFO Activating the language server in working dir: /Users/felixlinker/git/tamarin-prover (the workspace folder)
2023-12-07 12:44:51.5380000 [client] INFO run command: /Users/felixlinker/.ghcup/tmp/ghcup-ghc-9.4.7_cabal-3.6.2.0_hls-2.4.0.0_stack-2.11.1/haskell-language-server-wrapper --lsp -d
2023-12-07 12:44:51.5380000 [client] INFO debug command: /Users/felixlinker/.ghcup/tmp/ghcup-ghc-9.4.7_cabal-3.6.2.0_hls-2.4.0.0_stack-2.11.1/haskell-language-server-wrapper --lsp -d
2023-12-07 12:44:51.5380000 [client] INFO server environment variables:
2023-12-07 12:44:51.5380000 [client] INFO   PATH=/Users/felixlinker/.ghcup/tmp/ghcup-ghc-9.4.7_cabal-3.6.2.0_hls-2.4.0.0_stack-2.11.1:/Users/felixlinker/.nvm/versions/node/v21.1.0/bin:/Users/felixlinker/.cabal/bin:/Users/felixlinker/.ghcup/bin:/Users/felixlinker/bin/maude:/Users/felixlinker/git/diff-so-fancy:/Users/felixlinker/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/TeX/texbin:/usr/local/go/bin
2023-12-07 12:44:51.5380000 [client] DEBUG document selector patten: /Users/felixlinker/git/tamarin-prover/**/*
2023-12-07 12:44:51.5380000 [client] INFO Starting language server
[Trace - 13:44:51] Sending request 'initialize - (0)'.
No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)
[Info  - 13:44:51] Connection to server got closed. Server will restart.
[Trace - 13:44:51] Sending request 'initialize - (0)'.
No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)
[Info  - 13:44:51] Connection to server got closed. Server will restart.
[Trace - 13:44:51] Sending request 'initialize - (0)'.
No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)
[Info  - 13:44:51] Connection to server got closed. Server will restart.
[Trace - 13:44:51] Sending request 'initialize - (0)'.
No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)
[Info  - 13:44:51] Connection to server got closed. Server will restart.
[Trace - 13:44:51] Sending request 'initialize - (0)'.
No 'hie.yaml' found. Try to discover the project type!
haskell-language-server-wrapper: /Users/felixlinker/git/tamarin-prover/lib/accountability/tamarin-prover-accountability.cabal: hGetContents: invalid argument (invalid byte sequence)
[Error - 13:44:51] Connection to server got closed. Server will not be restarted.

[fendor]

Hi, thank you for your bug report!

It seems like HLS is working correctly, and this looks like it is somehow a VSCode-haskell issue.

I can't seem to reproduce on arch linux.

[fendor]

I am not entirely sure whether this works on MacOs, but try running VSCode from the terminal with LANG=C.UTF-8 code . after cding to the root of the project.
I am curious whether this makes a difference.

[felixlinker]

Yes! This worked! I executed the above command and the extension was notably doing "different" things. First noticable difference was that I got flashed with the notification whether I want to install hls-2.5.0.0, and after doing that, the server worked fine.

Do you have any idea how I could persist this setting?

[felixlinker]

I get an error from my diff script, though, which reports:

felixlinker@MBP-von-Felix tamarin-prover % git diff
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LC_ALL = (unset),
        LC_CTYPE = "UTF-8",
        LANG = "C.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

I presume that this is not an issue with your extension, however, it seems like launching vscode this way may have unintended side-effects.

[fendor]

This hints to me that your locale settings may be messed up, persisting LANG=C.UTF-8 is the wrong idea :D

Rather, figure out why the locale is wrong. Unfortunately, I don't have a Mac available right now, look in the settings whether everything's correct. Also, what is the output of locale?

[felixlinker]

In my standard zhs shell, I get:

LANG=""
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

In the vscode shell (when launching vscode as suggested by you above), I get:

LANG="C.UTF-8"
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

In the vscode shell (when launching it normally), I get:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

Sidenotes: A colleague of mine runs into the same problems on their Mac in the tamarin-prover repo. Also, my Mac is rather new and I did very little configuration of it so far. Both these things hint towards this not being a "totally me" issue. (Of course, it could still be, just saying that it might not be the most likely thing...)

[felixlinker]

I worked out a slightly better fix. Setting the following setting "terminal.integrated.detectLocale": "off" did the trick for me.

@fendor Maybe this is worth documenting in the README?

[fendor]

I'd be happy to merge updates to the documentation :)

[felixlinker]

Done!