-
How to notify RDB that there’s a new table in the tp that it can subscribe to?
One of our data plants it’s a self-updating data plant that handles column additions/deletions (type changes still manual)
The problem I am facing now is that .u.sub subscribes the first time the rdb connects to the tp and then it doesn’t check if new tables end up in the tp so if we are being sent a new table, the schema is set in the tp, but the rdb doesn’t know about it.
How can I notify the rdb that it needs to subscribe to this new table ?
Should I just send an async call with the name of the table and make the rdb subscribe to it?
Could that lead to loss of data?
-
2 Replies
-
Hi ,
Assuming vanilla kdb-tick, without any modifications to pub/sub logic (and this probably advised against), the TP would have to send some sort of signal to the RDB. The RDB would also have to be configured with some logic/callback to then subscribe to new table and instantiate it’s schema. As for data loss, the RDB may also need a customized function to read from the tick log. However, it doesn’t want to replay every entry in the tick log, jut the ones pertaining to the new table (this can be quite expensive depending on size of tick log and can potentially block RDB for significant period of time)There are possible structures that are more advantageous – like using a segmented tick plant where each table has it’s own tick log. In any event, while not supported by vanilla tick setup, it is possible with some custom configuration.
Hope this helps,
Megan
-
Based on your problem statement,
i would first ensure tp is sending the data as a table to subscriber’s and same is stored in tplog.
Next whenever TP has a new Table, tp should append the same in .u.w for each (or required) subscriber (rdb)
rdb on upd will simply insert. So a new table should appear without any data loss.
Thanks
Sujoy
Log in to reply.