ArrowKDB

[nmcdonnell]

Arrowkdb should work with earlier libarrow versions, say 5.0 (or maybe even 3.0) since your are building from source.

What is your distro?

If you’re able to install libarrow.so.800 you should also be able to install libparquet.so.800.  It’ll be two separate yum commands similar to https://github.com/KxSystems/arrowkdb#linux (although that’s apt on ubuntu)

See https://arrow.apache.org/install/#c-and-glib-c-packages-for-debian-gnulinux-ubuntu-and-centos:

Something like this from centos 7:

sudo yum install -y epel-release || sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(cut -d: -f5 /etc/system-release-cpe | cut -d. -f1).noarch.rpm sudo yum install -y https://apache.jfrog.io/artifactory/arrow/centos/$(cut -d: -f5 /etc/system-release-cpe | cut -d. -f1)/apache-arrow-release-latest.rpm sudo yum install -y --enablerepo=epel arrow-devel # For C++ sudo yum install -y --enablerepo=epel parquet-devel # For Apache Parquet C++

 

[kxnoob]

> I had to change somethings in the config and some environment variables to make it works but it builds successfully

Yeah – I had to change the CMakeLists to point to that libarrow12 (ARROW_LIBRARY and PARQUET_LIBRARY). I tried pointing to a libarrow9 version in a different miniconda environment. I also had to force it to use C++ 17.

 

Unfortunately, I am not able to install libarrow 9 on my system outside of conda. The most my system is able to yum install is libarrow8. It installs  libarrow.so.800 to /usr/lib64 but not a libparquet.so.800.

Do you have any suggestions for any workarounds to this? I’m not an expert on CMake or building C++ packages and whatnot.

[nmcdonnell]

Hi,

I suspect you’re missing the libarrow dependencies.
Could run `ldd $QHOME/l64/arrowkdb.so`?
It should return something like:

ldd arrowkdb.so
linux-vdso.so.1 (0x00007ffed21f9000)
libarrow.so.900 => /lib/x86_64-linux-gnu/libarrow.so.900 (0x00007f92d5ded000)
libparquet.so.900 => /lib/x86_64-linux-gnu/libparquet.so.900 (0x00007f92d5ac2000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f92d58e0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f92d58c5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f92d56d3000)
libbrotlienc.so.1 => /lib/x86_64-linux-gnu/libbrotlienc.so.1 (0x00007f92d564a000)
libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f92d563a000)
libutf8proc.so.2 => /lib/x86_64-linux-gnu/libutf8proc.so.2 (0x00007f92d55ed000)
libre2.so.5 => /lib/x86_64-linux-gnu/libre2.so.5 (0x00007f92d557c000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f92d5576000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f92d529f000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f92d528c000)
liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f92d5269000)
libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f92d525e000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f92d5242000)
libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f92d5199000)
libcurl.so.4 => /lib/x86_64-linux-gnu/libcurl.so.4 (0x00007f92d5107000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f92d4fb8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f92d4f93000)
/lib64/ld-linux-x86-64.so.2 (0x00007f92d7c87000)
libthrift-0.13.0.so => /lib/x86_64-linux-gnu/libthrift-0.13.0.so (0x00007f92d4edc000)
libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f92d4eb9000)
libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f92d4e90000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f92d4e6f000)
librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f92d4e4d000)
libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007f92d4ddf000)
libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f92d4dcc000)
libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f92d4d39000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f92d4cec000)
libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f92d4c96000)
liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f92d4c83000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f92d4b01000)
libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f92d492b000)
libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007f92d48f4000)
libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007f92d48ba000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f92d4834000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f92d4757000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f92d4726000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f92d471f000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f92d4710000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f92d46f2000)
libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f92d46d5000)
libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f92d4690000)
libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f92d455a000)
libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f92d4544000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f92d453b000)
libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f92d452f000)
libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f92d449c000)
libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f92d43f5000)
libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f92d43bd000)
libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007f92d43a2000)
libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f92d4396000)
libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007f92d436c000)
libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f92d435a000)
libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f92d430c000)
libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f92d41e3000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f92d41a6000)

 

[kxnoob]

Hello, thank you for your response!

I got the following output.

 

$ ldd $QHOME/l64/arrowkdb.so linux-vdso.so.1 (0x00007ffd3e3fc000) libarrow.so.1200 => <>/miniconda3/lib/python3.9/site-packages/pyarrow/libarrow.so.1200 (0x00007fbaf4ee9000) libparquet.so.1200 => <>/miniconda3/lib/python3.9/site-packages/pyarrow/libparquet.so.1200 (0x00007fbaf4760000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fbaf43cb000) libm.so.6 => /lib64/libm.so.6 (0x00007fbaf4049000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fbaf3e31000) libc.so.6 => /lib64/libc.so.6 (0x00007fbaf3a6c000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fbaf3868000) librt.so.1 => /lib64/librt.so.1 (0x00007fbaf3660000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbaf3440000) /lib64/ld-linux-x86-64.so.2 (0x00007fbaf7c8a000)