• kdb+

  Possible memory leak with enumerated table dump?

  Posted by flying on September 23, 2022 at 12:00 am

  I have the following code fragment:

  E:test>q 
  KDB+ 3.6 2019.04.02 Copyright (C) 1993-2019 Kx Systems w32/ 8()core 4095MB xxxxxxxx xxxxxxxx xxx.xxx.xxx.xxx NONEXPIRE 
  Welcome to kdb+ 32bit edition For support please see http://groups.google.com/d/forum/personal-kdbplus Tutorials can be found at http://code.kx.com To exit, type \ To remove this startup msg, edit q.q 
  q)Syms:([A:`$/:"c"$(`int$"A")+til 26] NS:til 26) 
  q)T:{ ([] A:`Syms$upper x?`1; B:-1+x?2.; C:x?1000) }; 
  q)(persist:`:T.dat) set T 10000; 
  q).Q.w[] 
  used| 304208 
  heap| 67108864 
  peak| 67108864 
  wmax| 0 
  mmap| 0 
  mphy| 4294967295 
  syms| 668 
  symw| 24354 
  
  q)get persist 
  A B         C 
  ----------------- 
  L 0.1561184 314 .. 
  
  q).Q.w[] 
  used| 370736 
  heap| 67108864 
  peak| 67108864 
  wmax| 0 
  mmap| 0 
  mphy| 4294967295 
  syms| 668 
  symw| 24354 
  
  q)do[1000; get persist ]; 
  q).Q.w[] 
  used| 65906736 
  heap| 134217728 
  peak| 134217728 
  wmax| 0 
  mmap| 0 
  mphy| 4294967295 
  syms| 668 
  symw| 24354 
  
  q).Q.gc[] 
  0

  As seen from above, the “used” space as reported by `Q.w[] keeps increasing every single time `:T.dat file was read.

  Is this a sign of a memory leak somewhere caused by the enumerated column A in T.dat?

  flying replied 8 months, 1 week ago 2 Members · 2 Replies
• 2 Replies

• 

  rocuinneagain

  Member

  September 26, 2022 at 12:00 am

  I tested this in 3.6 2019.04.02 could reproduce it.

  The issue is not present in 3.6 2020.05.04.

  Release notes of fix:

  2019.05.24 FIX reading enums in log format could leak memory. e.g.

  q)h:hopen`:a set ();
  h enlist(`u;`sym?`a`b`c);
  hclose h;
  do[5;get`:a;0N!.Q.w[]`used]

   

• 

  flying

  Member

  September 27, 2022 at 12:00 am

  Thanks for pointing this out. 2020.05.04 seems good. But I don’t seem to be able to find a 32-bit q for this version. (My test environment, unfortunately, depends on a couple of 32-bit DLLs. So I had to find a 32-bit q to use.) Any luck that I can find a 32-bit build of 2020.05.04 (or newer) anywhere?

Log In to Reply