-
questions about tuning a NUMA server
Hello everyone,
i have some questions about tuning q processes on NUMA server (linux kernel version is above 3.x).
1. According to https://code.kx.com/q/kb/linux-production/ [1] it is suggested to disable interleave and enable zone_reclaim, however https://code.kx.com/q/kb/cpu-affinity/#activating-numa [2] suggests
a) “numactl –interleave=all –physcpubind=0,1,2 q”
Q1. Why allows the process to access all node’s memory instead of bind the process to local memory only?
b) “echo 0 > /proc/sys/vm/zone_reclaim_mode” which lets the process closes zone-reclaim and takes other numa nodes’s memory.
Q2. Is zone_reclaim_mode=1 the correct answer according to [1] ?
The attached shows one of our q process, on a numa-enabled server, binds to mem 0-3,
Q3. Is it identical to “interleave=all” ?
Q4. If i want this process access local mem only, shall i execute “numactl –interleave=0” ?
Q5. following up Q4, How to start a q process and put numactl command’s to work in kx-platform? which Reversed Parameter shall be edited? or, should i “run”/start a process first, then execute system “numactl –interleave=0 –physcpubind=0 ” inside process’s console ?
thank you and the best,
leguan,
-
4 Replies
-
Hello,
Reverting back with feedback for some of the questions:
Q1) Keeping the process within one node is the best option for performance. However, if the working set exceeds one node’s memory capacity, it then has to spill in to other nodes.
Q2) No, this rarely helps with recent kernels. This discussion will be removed altogether from code.kx.com. The default for modern systems is 0, which should be fine. (Regardless, this setting has no effect if confining the process to a single node)
Q3) No, the default NUMA policy attempts to allocate from the current node, and from others if no free memory is available. See https://man7.org/linux/man-pages/man2/set_mempolicy.2.html for more details.
Q4) You would usually express that as “numactl -N 0 -m 0” (allocate from node 0, and also run on the cores that belong to node 0)
I hope this information helps.
Many Thanks
Luke
-
Hello Luke ,
Thank you very much for the above helpful answers.
I would like to ask how to start a q instance/task in kx-platform with specified numactrl command?
best,
leguan
-
Hello,
You could try “numactl –cpubind=0 –membind=0,1 <command to start the q process>”. Let me know if this works.
Many Thanks
Luke
-
Hello Luck,
Thank you for the answer.
“numactl –cpubind=0 –membind=0,1 q blablabla…”
This is a good practice to start a raw-kdb, to my understanding.
Well, what i am trying to ask is :
when we wanna start a task of an instance in kx-platform(w r using v4.5.0), we can right-click on the task and select “RUN”. Therefore which instance’s RESERVED PARAM(e.g host/port/taskSet/enableIPSharding/memLimit …) i can edit so that when we “RUN” the task, it works just as start a raw-kdb process in the way as you suggested.
the best and wish you a great weekend!
leguan
Log in to reply.