KX Community

Find answers, ask questions, and connect with our KX Community around the world.

Home Forums kdb+ embedPy installation error

  • embedPy installation error

    Posted by mmv456 on March 26, 2024 at 3:06 pm

    Hi everyone, I’m starting out installing embedPy, and I’m doing so via downloading and installing the latest release (as of now it’s 1.5.0) to run on my local machine.

    I unzipped the 1.5.0 release and am currently trying to run the tests (q test.q), when I get the following error. I’m not sure what it means and I’m wondering if anyone can add some input?

    M:\M\embedPy_windows-1.5.0>q test.q

    KDB+ 4.0 2022.01.14 Copyright (C) 1993-2022 Kx Systems

    ++++++++++++++++++++++++++++++++++++++++++++++++++++

    Loading custom startup. Customize in C:/q/q.q

    Start timestamp (local): 2024.03.26D10:48:22.584873000

    Full display precisions: \P 0

    C:/q/help.q

    C:/q/c/odbc.k

    C:/q/ps.k

    'libpython

    [5] M:\M\embedPy_windows-1.5.0\p.q:14:

    .P.env:not H~P;

    .p:(:./p 2:(init;3))[L;H;B]]

    ^

    q.p))

    • This discussion was modified 2 months ago by  mmv456.
    • This discussion was modified 2 months ago by  mmv456.
    jlucid replied 1 day, 1 hour ago 4 Members · 10 Replies
  • 10 Replies
  • rocuinneagain

    Member
    March 27, 2024 at 1:57 pm

    Related thread: embedPy installation: os error with kdb and python – Stack Overflow

    I would try:

    python -m pip install libpython 

    If still failing see output from https://pypi.org/project/find-libpython/

    pip install find_libpython 

    >>> from find_libpython import find_libpython
    >>> find_libpython()

  • mmv456

    Member
    March 27, 2024 at 6:13 pm

    It looks like I didn’t install pip correctly, so after making sure I updated my environment variables and running

    C:\Users\a623>python -m ensurepip

    I was able to run the following:

    C:\Users\a623>python -m pip install libpython

    Collecting libpython

    Downloading libpython-0.2.tar.gz (15 kB)

    Collecting requests

    Downloading requests-2.31.0-py3-none-any.whl (62 kB)

    |████████████████████████████████| 62 kB 408 kB/s

    Collecting idna<4,>=2.5

    Downloading idna-3.6-py3-none-any.whl (61 kB)

    |████████████████████████████████| 61 kB 55 kB/s

    Collecting urllib3<3,>=1.21.1

    Downloading urllib3-2.2.1-py3-none-any.whl (121 kB)

    |████████████████████████████████| 121 kB 3.3 MB/s

    Collecting certifi>=2017.4.17

    Downloading certifi-2024.2.2-py3-none-any.whl (163 kB)

    |████████████████████████████████| 163 kB ...

    Collecting charset-normalizer<4,>=2

    Downloading charset_normalizer-3.3.2-cp39-cp39-win_amd64.whl (100 kB)

    |████████████████████████████████| 100 kB 5.5 MB/s

    Using legacy 'setup.py install' for libpython, since package 'wheel' is not installed.

    Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, libpython

    Running setup.py install for libpython ... done

    Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 idna-3.6 libpython-0.2 requests-2.31.0 urllib3-2.2.1

    WARNING: You are using pip version 21.1.3; however, version 24.0 is available.

    You should consider upgrading via the 'C:\Users\a623\AppData\Local\Programs\Python\Python39\python.exe -m pip install --upgrade pip' command.

    Then, I ran the test.q script, which weirdly gives me the same error as before:

    M:\M\embedPy_windows-1.5.0>q test.q

    KDB+ 4.0 2022.01.14 Copyright (C) 1993-2022 Kx Systems

    ++++++++++++++++++++++++++++++++++++++++++++++++++++

    Loading custom startup. Customize in C:/q/q.q

    Start timestamp (local): 2024.03.27D13:50:41.738296000

    Full display precisions: \P 0

    C:/q/help.q

    C:/q/c/odbc.k

    C:/q/ps.k

    'libpython

    [5] M:\M\embedPy_windows-1.5.0\p.q:14:

    .P.env:not H~P;

    .p:(:./p 2:(init;3))[L;H;B]]

    ^

    I’m not sure why I’m getting the same error, because it looks like I have pip set up correctly and installed libpython, as you can see above.

    When I try to reinstall libpython I get the error below:

    M:\M\embedPy_windows-1.5.0>python -m pip install libpython

    Requirement already satisfied: libpython in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (0.2)

    Requirement already satisfied: requests in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from libpython) (2.31.0)

    Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (3.3.2)

    Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (2.2.1)

    Requirement already satisfied: idna<4,>=2.5 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (3.6)

    Requirement already satisfied: certifi>=2017.4.17 in c:\users\a623\appdata\local\programs\python\python39\lib\site-packages (from requests->libpython) (2024.2.2)

    WARNING: You are using pip version 21.1.3; however, version 24.0 is available.

    You should consider upgrading via the 'C:\Users\a623\AppData\Local\Programs\Python\Python39\python.exe -m pip install --upgrade pip' command.

    Any idea what’s going on here?

    • This reply was modified 2 months ago by  mmv456. Reason: updated some code
    • rocuinneagain

      Member
      April 2, 2024 at 8:08 am

      Can you try https://pypi.org/project/find-libpython/

      pip install find_libpython 
      >>> from find_libpython import find_libpython
      >>> find_libpython()

      What is the output?

      • user931206

        Member
        May 13, 2024 at 10:21 am

        Hey
        I have a similar issue.
        Most likely caused by how our UNIX team force users specific environments when working with python.
        I have the same issue and the find_libpython() returns an empty string for me.
        Any idea how to fix this ?

        • mmv456

          Member
          May 14, 2024 at 1:24 pm

          Hello, I still haven’t been able to figure it out. Not sure if my team wants to continue using embedPy but if you figure it out let me know. I reached out to someone who works at kdb and they recommended taking a look at the FAQs link in the github: https://github.com/KxSystems/embedPy/blob/master/docs/faq.md. The first question/answer is related to the libpython error I’ve been getting and may help you. It looks like we need to enable python to allow shared libraries, which is not enabled if python is installed using the recommended method.

          • This reply was modified 2 weeks ago by  mmv456.
          • rocuinneagain

            Member
            May 14, 2024 at 1:57 pm

            Going forward PyKX under q is the recommended replacement for embedPy.

            *It is not operational yet on Windows

            • jlucid

              Member
              May 25, 2024 at 6:27 pm

              PyKX under q requires the pykx license feature flag right, so there is a license fee to use it. This is the part where you are using python within the q interpreter (loading pykx.q). You can use PyKx free when sending data from python to kdb, when you set export PYKX_UNLICENSED=”true”. Whereas embedPy is free to use (apache 2.0 license). This is my understanding, please correct me if I am wrong

            • rocuinneagain

              Member
              May 27, 2024 at 12:47 pm

              There is a docs page on PyKX modes of operation.

              The 3 modes are:

              1. Unlicensed mode – used for IPC query and publish. No license needed.
              2. Licensed modeimport pykx in Python session.
              3. PyKX under q\l pykx.q in a q session.

              Both 2 & 3 require a KX license which includes insights.lib.pykx and insights.lib.embedq licensing flags.

            • jlucid

              Member
              May 27, 2024 at 9:05 pm

              Thanks for providing clarification on the modes Rian. So the replacement for the embedPy part is a licensed model. Noting that Pykx has more advanced functionality than embedPy.

              Still, I hope they continue to maintain embedPy for users who only require what it offers, while keeping costs limited to the kdb license.

          • user931206

            Member
            May 20, 2024 at 4:01 pm

            Hey

            for me it was the UNIX team which messed up.
            They installed a new version of python and forgot to configure the shared library folder.

Log in to reply.