Bug: OpenBLAS compile for Android doesn‘t work in Ubuntu 22.04

[FranzKafkaYu]

What happened?

I have followed the guide and install OpenBLAS first：

sudo apt install libopenblas-dev  

and I used the command to compile:

mkdir build-android-openblas && cd build-android-openblas && cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=latest -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS .. && make -j4  

and I see these warnings:

-- Could NOT find BLAS (missing: BLAS_LIBRARIES) 
CMake Warning at ggml/src/CMakeLists.txt:234 (message):
  BLAS not found, please refer to
  https://cmake.org/cmake/help/latest/module/FindBLAS.html#blas-lapack-vendors
  to set correct GGML_BLAS_VENDOR

I checked CMakeCache.txt and found that the OpenBLAS library didn't get the right path.

Name and Version

version：
commit 97bdd26 (HEAD, tag: b3400)

What operating system are you seeing the problem on?

No response

Relevant log output

No response

[jeroen-mostert]

Per #8006, these options have changed names. Use GGML_BLAS and GGML_BLAS_VENDOR instead. The build guide has been updated to document this, so if you came across old documentation, please point it out so it can be updated.

[FranzKafkaYu]

oppps,it's my fault,not a issue.

[FranzKafkaYu]

@jeroen-mostert I have updated this issue，can you help me? What is the right way to build OpenBLAS in Ubuntu 22?

[jeroen-mostert]

I don't have an Ubuntu install at the ready. Check where the package installs the headers and pass the -DBLAS_INCLUDE_DIRS option if necessary (for example, you might need /usr/include/openblas).

[smcnally]

On Ubuntu 22.04, I started having build issues this week with make. Running make LLAMA_CUDA=1 or make GGML_CUDA=1 failed with multiple Makefile errors. While reviewing the Makefile, I recloned the repo into a clean subdir, ran make GGML_CUDA=1 again and successfully built functioning binaries.

I'll build OpenBLAS on a clean 22.04 instance without CUDA installed if that helps you troubleshoot. First please let us know here if following Jeroen's comment and these updated build instructions works in your current clone or a new, clean one. e.g. use

-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS

instead of

-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS

and pass the -DBLAS_INCLUDE_DIRS options as appropriate for your environment.

[FranzKafkaYu]

On Ubuntu 22.04, I started having build issues this week with make. Running make LLAMA_CUDA=1 or make GGML_CUDA=1 failed with multiple Makefile errors. While reviewing the Makefile, I recloned the repo into a clean subdir, ran make GGML_CUDA=1 again and successfully built functioning binaries.

I'll build OpenBLAS on a clean 22.04 instance without CUDA installed if that helps you troubleshoot. First please let us know here if following Jeroen's comment and these updated build instructions works in your current clone or a new, clean one. e.g. use

-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS

instead of

-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS

and pass the -DBLAS_INCLUDE_DIRS options as appropriate for your environment.

yes,I am using -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS flags but the Problem is that the build system can't locate OpenBLAS related libraries and headers.

I am using Ubuntu 22,and I used apt to install OpenBLAS:

sudo apt install libopenblas-dev   
sudo apt install libopenblas-base  
sudo apt install libopenblas64-dev  

while when I use dpkg to check what have been installed：

$dpkg -L libopenblas-dev
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/libopenblas-dev
/usr/share/doc/libopenblas-dev/BACKERS.md
/usr/share/doc/libopenblas-dev/CONTRIBUTORS.md.gz
/usr/share/doc/libopenblas-dev/README.md.gz
/usr/share/doc/libopenblas-dev/USAGE.md.gz
/usr/share/doc/libopenblas-dev/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libopenblas-dev
/usr/share/doc/libopenblas-dev/changelog.Debian.gz

there are no headers nor libraries.
And I checked my system,found that there are some libraries related OpenBLAS:

/usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so  

these libraries are coming from libopenblas0-pthread

but I still can't find related headers:

dpkg -L  libopenblas0-pthread
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/openblas-pthread
/usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
/usr/share
/usr/share/doc
/usr/share/doc/libopenblas0-pthread
/usr/share/doc/libopenblas0-pthread/changelog.Debian.gz
/usr/share/doc/libopenblas0-pthread/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libopenblas0-pthread
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so.0  

Append:

Although I found cblas.h in my system:

libopenblas64-pthread-dev:amd64: /usr/include/x86_64-linux-gnu/openblas64-pthread/cblas.h  
libopenblas-pthread-dev:amd64: /usr/include/x86_64-linux-gnu/openblas-pthread/cblas.h  

and set env:

 export BLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu/openblas64-pthread  

it doesn's work,same error here.
so what is the right way to build with OpenBLAS?any detailed guidences ？

[jeroen-mostert]

I tested it. On a fresh install of Ubuntu 22.04, you need:

sudo apt install build-essential cmake pkg-config libopenblas-dev

After this:

rm -rf ~/llama.cpp.build ; cmake -B ~/llama.cpp.build -S ~/llama.cpp -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cd ~/llama.cpp.build
make -j

No further tweaks should be necessary.

Note that using a separate build dir and completely removing it before invoking cmake is highly recommended, because cmake will otherwise leave cruft that can cause subsequent runs to fail due to inappropriately cached results (and will also interfere with git). I suspect this might be what's tripping up your current build.

As an aside, the distro-supplied OpenBLAS will likely not be optimized for the architecture you're running on, so another option is to build OpenBLAS from source (this is easy since it has no dependencies other than what's already included in build-essential) and do a sudo make install PREFIX=/usr/local. cmake will then find and prefer this version. However, be aware that there is no make uninstall, so removing this build cleanly requires manual action.

[FranzKafkaYu]

Note that using a separate build dir and completely removing it before invoking cmake is highly recommended, because cmake will otherwise leave cruft that can cause subsequent runs to fail due to inappropriately cached results (and will also interfere with git). I suspect this might be what's tripping up your current build.

Thank u very much sir,I found that I can compile llama.cpp with OpenBLAS for Linux x86_64 in my Ubuntu,but When I execute cross-platform compile for Android arm64v8 it still report can't find OpenBLAS,details:

$ cmake -B ~/llama.cpp.build.android -S ~/llama.cpp -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=latest -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod  

-- Using latest available ANDROID_PLATFORM: 33.
-- The C compiler identification is Clang 14.0.7
-- The CXX compiler identification is Clang 14.0.7
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/franzkafka/Desktop/android/ndk/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/franzkafka/Desktop/android/ndk/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Found OpenMP_C: -fopenmp=libomp (found version "5.0") 
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0") 
-- Found OpenMP: TRUE (found version "5.0")  
-- OpenMP found
-- Could NOT find BLAS (missing: BLAS_LIBRARIES) 
CMake Warning at ggml/src/CMakeLists.txt:250 (message):
  BLAS not found, please refer to
  https://cmake.org/cmake/help/latest/module/FindBLAS.html#blas-lapack-vendors
  to set correct GGML_BLAS_VENDOR


-- Using llamafile
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- ARM detected
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /home/franzkafka/llama.cpp.build.android  

[jeroen-mostert]

Ah, I see. You kinda sorta mentioned this in your first post through the commands, but it's easy to miss. This issue is misnamed; it's not about building with OpenBLAS on Ubuntu 22.04, your issue is specifically about building for Android using OpenBLAS. This is a completely different scenario since obviously we can't use the system's OpenBLAS installation, you need to build OpenBLAS for Android separately and make it available to CMake. This is not specific to llama.cpp, nor even specific to Ubuntu.

I have zero experience building anything for Android and I won't be using this issue as an opportunity to learn. The linked instructions may be out of date and need some tweaking. Then, if you have a build, making things available to CMake is its own challenge. After that, of course, you also have to get OpenBLAS on the Android system, since it's not there by default. Static linking might help there, but last time I tried it (in a different context) CMake kind of insisted on a dynamic link to the BLAS libs even when building GGML statically, so that's yet another hurdle to clear.

Consider if you really need BLAS; if not just building with CPU backend support should be sufficient to get things working on Android. Note that our Android build instructions do not mention using any BLAS implementation; I don't know how much, if any, acceleration is to be expected from using OpenBLAS instead, it may well be a regression.