KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Clearing symw

  • Clearing symw

    Posted by jlucid on November 15, 2023 at 12:00 am

    I have an intraday writer process which is writing data to an int partitioned database, the partitioning integer is constructed from the timestamp using `long$(.z.p) div 60*1e9, meaning I am partitioning data by one minute buckets. None of my tables have enumerated columns, all ints, shorts and chars, but still the internal count of distinct syms on my writer process is increasing, and with it the memory consumption.  The reason is because my upsert to disk command needs to construct the directory path using the integer value, as shown below.

     

    I understand that simply restarting the process will release the memory back to the OS, but is there a way for me to explicitly clear this memory build up within the process? Basically reset .Q.w[]`symw back to its value on startup. Also, I see this was asked back in 2015, https://community.kx.com/t5/kdb-and-q/symw-clean-up/m-p/7889, and the answer was no, so wondering if anything has changed, or if there is an alternative approach I can use for creating the path

     

    ` sv (`:/opt/kdb/database/;`$string[0];`table)
    `:/opt/kdb/database/0/table
    .Q.w[]
    used| 359968
    heap| 67108864
    peak| 67108864
    wmax| 0
    mmap| 0
    mphy| 67193663488
    syms| 670
    symw| 28581
    
    ` sv (`:/opt/kdb/database;`$string[1];`table)
    
    :/opt/kdb/database/1/table
    .Q.w[]
    used| 359968
    heap| 67108864
    peak| 67108864
    wmax| 0
    mmap| 0
    mphy| 67193663488
    syms| 671
    symw| 28623
    
    ` sv (`:/opt/kdb/database;`$string[2];`table)
    `:/opt/kdb/database/2/table
    .Q.w[]
    used| 359968
    heap| 67108864
    peak| 67108864
    wmax| 0
    mmap| 0
    mphy| 67193663488
    syms| 673
    symw| 28691
    jlucid replied 7 months, 4 weeks ago 2 Members · 2 Replies
  • 2 Replies
  • adotsch

    Member
    November 16, 2023 at 12:00 am

    No, you can’t reset symw, but you can mkdir and cd into those directories and just use `:table/ sym when saving/upserting.

  • jlucid

    Member
    November 17, 2023 at 12:00 am

    Ok thanks for confirming the symw query and for the work around suggestion.

    I will give the mkdir+cd combo a go

Log in to reply.