Running `arduino-cli lib install NTPClient_Generic` hangs forever on 100% CPU

[urish]

Describe the problem

Installing the NTPClient_Generic library through Arduino CLI hangs forever

To reproduce

Just run arduino-cli lib install NTPClient_Generic and watch it hangs.

After seeing it on my Ubuntu machine, I also verified this reproduces on a pristine ubuntu-20.04 docker container:

docker run -it --entrypoint "/bin/bash" ubuntu:20.04

then inside the container I ran:

apt update
apt install wget
wget https://downloads.arduino.cc/arduino-cli/nightly/arduino-cli_nightly-latest_Linux_64bit.tar.gz
tar zxf arduino-cli_nightly-latest_Linux_64bit.tar.gz
./arduino-cli -v lib install NTPClient_Generic

Expected behavior

Should install the library successfully, or at least quit with an error.

Arduino CLI version

arduino-cli Version: nightly-20230402 Commit: 7eae0bd Date: 2023-04-02T01:32:57Z

Operating system

Linux

Operating system version

Ubuntu 20.04

Additional context

Running the same command with verbose output, this is what I see:

$ ./arduino-cli -v lib install NTPClient_Generic
INFO[0000] Config file not found, using default values
INFO[0000] arduino-cli version nightly-20230402
INFO[0000] Updating libraries index
INFO[0000] Starting download                             url="https://downloads.arduino.cc/libraries/library_index.tar.bz2"
Downloading index: library_index.tar.bz2 downloaded
INFO[0002] Updating index                                url="https://downloads.arduino.cc/packages/package_index.tar.bz2"
INFO[0002] Starting download                             url="https://downloads.arduino.cc/packages/package_index.tar.bz2"
Downloading index: package_index.tar.bz2 downloaded
INFO[0002] Loading hardware from: /root/.arduino15/packages
INFO[0002] Checking signature                            index=/root/.arduino15/package_index.json signatureFile=/root/.arduino15/package_index.json.sig trusted=true
Downloading missing tool builtin:serial-discovery@1.4.0...
INFO[0002] Starting download                             url="https://downloads.arduino.cc/discovery/serial-discovery/serial-discovery_v1.4.0_Linux_64bit.tar.gz"
builtin:serial-discovery@1.4.0 downloaded
INFO[0003] Installing tool                               Tool="builtin:serial-discovery@1.4.0"
Installing builtin:serial-discovery@1.4.0...
INFO[0003] Skipping tool configuration.                  Tool="builtin:serial-discovery@1.4.0"
Skipping tool configuration....
INFO[0003] Tool installed                                Tool="builtin:serial-discovery@1.4.0"
builtin:serial-discovery@1.4.0 installed
Downloading missing tool builtin:mdns-discovery@1.0.9...
INFO[0003] Starting download                             url="https://downloads.arduino.cc/discovery/mdns-discovery/mdns-discovery_v1.0.9_Linux_64bit.tar.gz"
builtin:mdns-discovery@1.0.9 downloaded
INFO[0003] Installing tool                               Tool="builtin:mdns-discovery@1.0.9"
Installing builtin:mdns-discovery@1.0.9...
INFO[0003] Skipping tool configuration.                  Tool="builtin:mdns-discovery@1.0.9"
Skipping tool configuration....
INFO[0003] Tool installed                                Tool="builtin:mdns-discovery@1.0.9"
builtin:mdns-discovery@1.0.9 installed
Downloading missing tool builtin:serial-monitor@0.13.0...
INFO[0003] Starting download                             url="https://downloads.arduino.cc/monitor/serial-monitor/serial-monitor_v0.13.0_Linux_64bit.tar.gz"
builtin:serial-monitor@0.13.0 downloaded
INFO[0004] Installing tool                               Tool="builtin:serial-monitor@0.13.0"
Installing builtin:serial-monitor@0.13.0...
INFO[0004] Skipping tool configuration.                  Tool="builtin:serial-monitor@0.13.0"
Skipping tool configuration....
INFO[0004] Tool installed                                Tool="builtin:serial-monitor@0.13.0"
builtin:serial-monitor@0.13.0 installed
Downloading missing tool builtin:ctags@5.8-arduino11...
INFO[0004] Starting download                             url="https://downloads.arduino.cc/tools/ctags-5.8-arduino11-pm-x86_64-pc-linux-gnu.tar.bz2"
builtin:ctags@5.8-arduino11 downloaded
INFO[0005] Installing tool                               Tool="builtin:ctags@5.8-arduino11"
Installing builtin:ctags@5.8-arduino11...
INFO[0005] Skipping tool configuration.                  Tool="builtin:ctags@5.8-arduino11"
Skipping tool configuration....
INFO[0005] Tool installed                                Tool="builtin:ctags@5.8-arduino11"
builtin:ctags@5.8-arduino11 installed
INFO[0005] Loading tools from dir: /root/.arduino15/packages/builtin/tools
INFO[0005] Loaded tool                                   tool="builtin:ctags@5.8-arduino11"
INFO[0005] Loaded tool                                   tool="builtin:mdns-discovery@1.0.9"
INFO[0005] Loaded tool                                   tool="builtin:serial-discovery@1.4.0"
INFO[0005] Loaded tool                                   tool="builtin:serial-monitor@0.13.0"
INFO[0005] Loading libraries index file                  index=/root/.arduino15/library_index.json
INFO[0005] Adding libraries dir                          dir=/root/Arduino/libraries location=user
INFO[0005] Executing `arduino-cli lib install`

at which point it hangs forever

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details

[cmaglie]

Thanks for the report @urish.

The problem here is that the NTPClient_Generic depends on Timezone_Generic that depends on Wire, but Wire is not available in the Library Manager (because the Wire library is usually part of the libraries bundled within the platforms).

Normally, a missing dependency will result in a No valid dependencies solution found error, but in this particular case, since the Timezone_Generic has a lot of dependencies, the resolver is taking a lot of time checking all the possible combinations in the hope to find a suitable Wire library. This is not optimal since the algorithm complexity is exponential with the number of releases and the number of dependencies: it will eventually complete, but it will take a very long time.

https://github.com/khoih-prog/Timezone_Generic/blob/main/library.properties#L12

@per1234 do you think we could add a check in the library repository engine to reject libraries with missing dependencies?

In the meantime, I'll try to work out a fix to speed up the dependencies resolution in this particular case.

[cmaglie]

Fixed by #2140

[urish]

Thanks for the super quick turnaround!

[rgilton]

Still seeing this issue with arduino-cli version 0.32.2, which I believe includes the fix in #2140.

[cmaglie]

I confirm it, it's happening again, something changed in the index, I'll check this out.