Re: How does nested columns/lists fragment memory?

[darrenwsun]

For 2, precisely it depends. If we tweak the example to let the first element to be an integer (or any other value of atomic type), we will see some effective garbage collection.

q)v:{(10;10000#"b")} each til 100000 
q).Q.w[] 
used| 1643008048 
heap| 1677721600 
peak| 1677721600 
wmax| 0 mmap| 0 
mphy| 7978725376 
syms| 665 
symw| 28405 

q).glob.t:([]a:`long$()) 
q)`.glob.t upsert flip enlist[`a]!enlist v[;0] 
`.glob.t 

q).Q.gc[] 
1543503872 

q).Q.w[] 
used| 1406896 
heap| 134217728 
peak| 1677721600 
wmax| 0 
mmap| 0 
mphy| 7978725376 
syms| 668 
symw| 28496

My explanation towards the different behaviors is that in the above example, since v[;0] is an int vector, its elements have to be in consecutive memory and thus it is a value copy from the original list. As such deleting v allows recycling the memory taken by v. While with the earlier example, v[;0] is a list of “references” to the elements in the original list v, so deleting v doesn’t remove all references to the memory blocks used by v (the references are now in .glob.t).

 

For 3, the shorter answer is that kdb uses copy-on-write.