-
Clearing symw
I have an intraday writer process which is writing data to an int partitioned database, the partitioning integer is constructed from the timestamp using `long$(.z.p) div 60*1e9, meaning I am partitioning data by one minute buckets. None of my tables have enumerated columns, all ints, shorts and chars, but still the internal count of distinct syms on my writer process is increasing, and with it the memory consumption. The reason is because my upsert to disk command needs to construct the directory path using the integer value, as shown below.
I understand that simply restarting the process will release the memory back to the OS, but is there a way for me to explicitly clear this memory build up within the process? Basically reset .Q.w[]`symw back to its value on startup. Also, I see this was asked back in 2015, https://community.kx.com/t5/kdb-and-q/symw-clean-up/m-p/7889, and the answer was no, so wondering if anything has changed, or if there is an alternative approach I can use for creating the path
` sv (`:/opt/kdb/database/;`$string[0];`table) `:/opt/kdb/database/0/table .Q.w[] used| 359968 heap| 67108864 peak| 67108864 wmax| 0 mmap| 0 mphy| 67193663488 syms| 670 symw| 28581 ` sv (`:/opt/kdb/database;`$string[1];`table) :/opt/kdb/database/1/table .Q.w[] used| 359968 heap| 67108864 peak| 67108864 wmax| 0 mmap| 0 mphy| 67193663488 syms| 671 symw| 28623 ` sv (`:/opt/kdb/database;`$string[2];`table) `:/opt/kdb/database/2/table .Q.w[] used| 359968 heap| 67108864 peak| 67108864 wmax| 0 mmap| 0 mphy| 67193663488 syms| 673 symw| 28691
-
2 Replies
-
No, you can’t reset symw, but you can mkdir and cd into those directories and just use `:table/ sym when saving/upserting.
-
Ok thanks for confirming the symw query and for the work around suggestion.
I will give the mkdir+cd combo a go
Log in to reply.