-
C API, calling k within a thread
I am running a single threaded C++ program which publishes tables A and B.
I am using the k function below to publish the updates asynchronously.
k(-handle, (S)”.u.upd”, ks((S)table_name), data, (K)0)
I am not explicitly calling r0(data) on the data because calling the k function decrements the reference count.
I have confirmed this to be the case by running m4(0) before and after the call to k.
My question is, do I need to change anything in terms of memory management if I want to move the publishing of tables A and B into separate threads? Is it ok to call k from within a thread? Can both threads call k at the same time for instance? Just want to be aware of the dangers if any
Thanks
-
5 Replies
-
How is the thread started? It is only safe to call k(…) from a thread started by q itself. So it’s OK to call from C functions called by .z.pg regardless of multithreaded input mode, but it should not be called from a std::thread.
-
Ok thanks, so to clarify, we are not calling k from a q process, via a shared library. We are running a C++ process where different threads are responsible for publishing data to different q processes using the k function. The threads are created using std::thread. Before the threads are created, the handles to the different q processes are opened and then subsequently passed to the threads to use during the async publish.
This has been working well for us so far, in that data is being received by the q processes, and we can see that m4 (which returns the memory for the current thread) is showing no memory leaks. Are you saying there could be a danger with this setup?
We have been looking at the API reference doc, but it doesn’t mention this restriction in regards to k, see https://code.kx.com/q/interfaces/capiref/#k-evaluate. Please let us know if there is another reference we should be using.
-
I was just looking through old discussions, and can came across this one
where Charlie was recommending calling k() within a thread because the call blocks until the send buffer receives the message, if the buffer is full, the call will block until it is cleared. So this would imply its ok to call k within a thread, unless I am reading this wrong. What do you think? -
In the context of business information APIs, it’s crucial to ensure that each q process operates independently with its own memory space. This independence in memory management necessitates careful consideration of data serialization and deserialization when transmitting information between processes. The distinct memory spaces for each q process alleviate concerns related to thread safety within a single process.
To guarantee smooth operations and prevent conflicts, especially in scenarios where multiple processes attempt to access or modify shared resources, a robust system design is essential. Consider implementing mechanisms to manage errors effectively, isolating failures in one q process to prevent any adverse impact on others. Additionally, for enhanced reliability, it may be prudent to explore the incorporation of a supervision system. This system can actively monitor the health of q processes and, if necessary, initiate restarts to maintain uninterrupted functionality.
In the realm of business information API, these architectural considerations ensure the efficient and secure exchange of data between independent q processes, fostering a resilient and responsive system.
-
My question is about calling the k function, in the C API, from different threads within a non q process. A C or C++ process in this case. The q processes which are receiving the data are all independent and single threaded
Log in to reply.