KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums KX Solutions questions about tuning a NUMA server

  • questions about tuning a NUMA server

    Posted by leguan on December 29, 2021 at 12:00 am

    Hello everyone,

    i have some questions about tuning q processes on NUMA server (linux kernel version is above 3.x).

    1. According to https://code.kx.com/q/kb/linux-production/  [1]  it is suggested to disable interleave and enable zone_reclaim, however https://code.kx.com/q/kb/cpu-affinity/#activating-numa [2]  suggests

    a) “numactl –interleave=all –physcpubind=0,1,2 q”

    Q1. Why allows the process to access all node’s memory instead of bind the process to local memory only?

    b) “echo 0 > /proc/sys/vm/zone_reclaim_mode”  which lets the process closes zone-reclaim and takes other numa nodes’s memory.

    Q2. Is zone_reclaim_mode=1  the correct answer according to [1] ?

    The attached shows one of our q process,  on a numa-enabled server, binds to mem 0-3,

    Q3. Is it identical to “interleave=all” ?

    Q4. If i want this process access local mem only,  shall i execute  “numactl –interleave=0” ?

    Q5. following up Q4, How to start a q process and put numactl command’s to work in kx-platform?  which Reversed Parameter shall be edited? or, should i “run”/start a process first, then  execute system “numactl –interleave=0 –physcpubind=0 ” inside process’s console ?

    thank you and the best,

    leguan,

     

     

    leguan replied 2 months, 3 weeks ago 2 Members · 4 Replies
  • 4 Replies
  • leades

    Member
    February 18, 2022 at 12:00 am

    Hello,

    Reverting back with feedback for some of the questions:

    Q1) Keeping the process within one node is the best option for performance. However, if the working set exceeds one node’s memory capacity, it then has to spill in to other nodes.

    Q2) No, this rarely helps with recent kernels. This discussion will be removed altogether from code.kx.com. The default for modern systems is 0, which should be fine. (Regardless, this setting has no effect if confining the process to a single node)

    Q3) No, the default NUMA policy attempts to allocate from the current node, and from others if no free memory is available. See https://man7.org/linux/man-pages/man2/set_mempolicy.2.html for more details.

    Q4) You would usually express that as “numactl -N 0 -m 0” (allocate from node 0, and also run on the cores that belong to node 0)

    I hope this information helps.

    Many Thanks

    Luke

  • leguan

    Member
    February 22, 2022 at 12:00 am

    Hello Luke ,

    Thank you very much for the above helpful answers.

    I would like to ask how to start a q instance/task in kx-platform with specified numactrl command?

    best,

     

    leguan

  • leades

    Member
    February 24, 2022 at 12:00 am

    Hello,

    You could try “numactl –cpubind=0 –membind=0,1 <command to start the q process>”. Let me know if this works.

    Many Thanks

    Luke

  • leguan

    Member
    February 25, 2022 at 12:00 am

    Hello Luck,

    Thank you for the answer.

    “numactl –cpubind=0 –membind=0,1 q blablabla…”

    This is a good practice to start a raw-kdb, to my understanding.

    Well, what i am trying to ask is :

    when we wanna start a task of an instance in kx-platform(w r using v4.5.0), we can  right-click on the task and select “RUN”.   Therefore which instance’s RESERVED PARAM(e.g host/port/taskSet/enableIPSharding/memLimit …) i can edit so that when we “RUN” the task, it works just as start a raw-kdb process in the way as you suggested.

    the best and wish you a great weekend!

    leguan

Log in to reply.