KX Community

Find answers, ask questions, and connect with our KX Community around the world.

Home Forums PyKX pykx incompatible with subprocess and multiprocess – core dumps

  • pykx incompatible with subprocess and multiprocess – core dumps

    Posted by hodgidav on June 16, 2023 at 12:00 am

    Hi,

     

    If I try to spawn a child python process after importing pykx the child will core dump.

     

    $ python –version

    Python 3.10.10

    $ pip list

    Package         Version

    ————— ——–

    ms.version      2.9.0

    numpy           1.24.3

    pandas          1.5.3

    pip             23.1.2

    pykx            1.5.4

    python-dateutil 2.8.2

    pytz            2022.7.1

    setuptools      65.5.0

    six             1.16.0

     

     

    $ cat parent.py

    print("parent start")

    import pykx

    import subprocess

    proc=subprocess.Popen([‘/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/bin/python’,’child.py’],stdout=subprocess.PIPE,stderr=subprocess.PIPE);proc.communicate();print(proc.wait())

    print("parent end")

    import sys

    sys.exit(69)

     

    $ cat child.py

    print(‘Child start’)

    import sys

    print(‘loading pykx’)

    #import pykx

    print(‘Child end’)

    sys.exit(42)

     

    $ python parent.py

    parent start

    42 #as expected

    parent end

     

    If I also import pykx in the child it fails:

    $ cat child.py

    print(‘Child start’)

    import sys

    print(‘loading pykx’)

    import pykx

    print(‘Child end’)

    sys.exit(42)

     

    $ python parent.py

    parent start

    -11

    parent end

    stracing the child reveals the coredump here:

    lstat("/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/lib/python3.10/site-packages/pykx/lib/l64/libq.so", {st_mode=S_IFREG|0755, st_size=813136, …}) = 0

    — SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} —

    +++ killed by SIGSEGV (core dumped) +++

     

    Stracing the parent shows what it is supposed to look like:

    lstat("/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/lib/python3.10/site-packages/pykx/lib/l64/libq.so", {st_mode=S_IFREG|0755, st_size=813136, …}) = 0

    pipe2([3, 4], O_CLOEXEC)                = 0

    fstat(3, {st_mode=S_IFIFO|0600, st_size=0, …}) = 0

    ioctl(3, TCGETS, 0x7ffe05620b40)        = -1 ENOTTY (Inappropriate ioctl for device)

    lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)

    ioctl(3, TCGETS, 0x7ffe05620bc0)        = -1 ENOTTY (Inappropriate ioctl for device)

    pipe2([5, 6], O_CLOEXEC)                = 0

    rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], = 0

    vfork()                                 = 349938

    rt_sigprocmask(SIG_SETMASK, [], NULL, = 0

    close(6)                                = 0

    close(4)                                = 0

    read(5, "", 50000)                      = 0

    close(5)                                = 0

    lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)

    fstat(3, {st_mode=S_IFIFO|0600, st_size=0, …}) = 0

    read(3, "", 8192)                       = 0

    close(3)                                = 0

    wait4(349938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 349938

    open("/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/lib/python3.10/site-packages/pykx/lib/l64/libq.so", O_RDONLY|O_CLOEXEC) = 3

    read(3, "177ELF211���������3�>�1���300206������"…, 832) = 832

    fstat(3, {st_mode=S_IFREG|0755, st_size=813136, …}) = 0

    mmap(NULL, 3684984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f8e4ccd9000

    Similarly if I switch from subprocess to multiprocess and set to spawn instead of fork it still fails.

     

    This time we dont even need to load pykx in the child:

    $ cat child.py

    print(‘Child start’)

    import sys

    print(‘loading pykx’)

    #import pykx

    print(‘Child end’)

    sys.exit(42)

     

    $ cat parentMP.py

    import os

    import sys

    import multiprocessing

    import pykx

    def task():

        os.system("/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/bin/python -u child.py>/tmp/hodgidav2/out 2>&1")

     

    if __name__ == ‘__main__’:

            multiprocessing.set_start_method(‘spawn’,force=True)

            print("parent start")

            # create the process

            process = multiprocessing.Process(target=task)

            # start the process

            process.start()

            # wait for the process to complete

            process.join()

            print("parent end")

            sys.exit(69)

     

    $ python parentMP.py

    parent start

    parent end

     

    $ cat /tmp/hodgidav2/out

    cat: /tmp/hodgidav2/out: No such file or directory

     

    If I remove pykx from parentMP.py then it works as expected:

    $ cat parentMP.py

    import os

    import sys

    import multiprocessing

    #import pykx

    def task():

        os.system("/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/bin/python -u child.py>/tmp/hodgidav2/out 2>&1")

     

    if __name__ == ‘__main__’:

            multiprocessing.set_start_method(‘spawn’,force=True)

            print("parent start")

            # create the process

            process = multiprocessing.Process(target=task)

            # start the process

            process.start()

            # wait for the process to complete

            process.join()

            print("parent end")

            sys.exit(69)

     

    $ python parentMP.py

    parent start

    parent end

     

    $ cat /tmp/hodgidav2/out

    Child start

    loading pykx

    Child end

     

    hodgidav replied 2 months, 3 weeks ago 3 Members · 3 Replies
  • 3 Replies
  • kshepherd

    Member
    June 19, 2023 at 12:00 am

    This has to do with the fact that PyKX uses a few environment variables to stop certain pieces from being loaded twice if it is reimported. The parent process loads PyKX and the env vars are set, when the child process is created these env vars are passed along to the child process, which causes PyKX in the child process to only partially load and segfault.

    This can be fixed by loading PyKX in the parent process after the child process is spawned.

    parent.py

     

    print("parent start")   
    import subprocess  
    proc=subprocess.Popen(     ['/a/stor164ncs2.new-york.ms.com/sc34770/s180992/hodgidav/myenv/bin/python','child.py'],     stdout=subprocess.PIPE,     stderr=subprocess.PIPE ) 
    import pykx # import PyKX after spawning the child process 
    proc.communicate() 
    print(proc.wait())  
    print("parent end")  
    import sys  
    sys.exit(69)

     

  • rocuinneagain

    Member
    July 21, 2023 at 12:00 am

    PyKX 1.6.1 has been released. Full details on the release can be found here.Highlights: Added sorted, grouped, parted, and unique. As methods off of Tables and Vectors. Added PyKXReimport class to allow subprocesses to reimport PyKX safely. Details here. Added environment variables to specify a path to libpython. Details here. Fixed memory leaks in QConnection and PyKX as a server. Fixed bug in Jupyter Notebook magic command. Full details including many other fixes included here. import pykx as kx with kx.PyKXReimport(): output = subprocess.run( (str(Path(sys.executable).as_posix()), ‘-c’, ‘import pykx; print(pykx.q(“til 10”))’), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, ).stdout.strip() assert output == “0 1 2 3 4 5 6 7 8 9”

  • rocuinneagain

    Member
    July 21, 2023 at 12:00 am
    PyKX 1.6.1 has been released. Full details on the release can be found here.
    Highlights:
    • Added sortedgroupedparted, and unique. As methods off of Tables and Vectors.
    • Added PyKXReimport class to allow subprocesses to reimport PyKX safely. Details here.
    • Added environment variables to specify a path to libpython. Details here.
    • Fixed memory leaks in QConnection and PyKX as a server.
    • Fixed bug in Jupyter Notebook magic command.
    Full details including many other fixes included here.

    import pykx as kx

    with kx.PyKXReimport():

    output = subprocess.run(

    (str(Path(sys.executable).as_posix()), ‘-c’, ‘import pykx; print(pykx.q(“til 10”))’),

    stdout=subprocess.PIPE,

    stderr=subprocess.STDOUT,

    text=True,

    ).stdout.strip()

    assert output == “0 1 2 3 4 5 6 7 8 9”

     

Log in to reply.