KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Strange data type prevents table operation on what looks like a table

  • Strange data type prevents table operation on what looks like a table

    Posted by user931206 on February 15, 2024 at 12:00 am

    I have this table:

     

    tab:([]time:2?.z.P;col1:((`col11`col12!(1;3));(`col11`col12!(5;33)));col2:((`col11`col12!(1;3));(`col11`col12!(5;33)))) 
    time col1 col2 
    ----------------------------------------------------------------- 
    2015.10.24D22:29:46.626037056 `col11`col12!1 3 `col11`col12!1 3 2004.03.06D21:27:43.089887728 `col11`col12!5 33 `col11`col12!5 33

     

    I am doing the following operations on it :

    cols exec col1 from tab

    Now, I would expect that to work, but when I run a type on that exec operation i get a 0h even though it looks like an actual table (see below)

    col11 col12 
    ----------- 
    1 3 5 33

     

    It works if I do something like this:

    cols{x}each ?[tab;();();`col1]

    Does anyone know why that table looks like a table, but it’s not a table ?

    user931206 replied 3 months, 1 week ago 3 Members · 6 Replies
  • 6 Replies
  • roryk

    Member
    February 15, 2024 at 12:00 am
    Dropping into k can be useful to see the representation.
    Tab is
    
    +`time`col1`col2!(times;(`col11`col12!1 3;`col11`col12!5 33);(`col11`col12!1 3;`col11`col12!5 33))

    What’s interesting is that

    exec col1 from tab

    then produces

    (`col11`col12!1 3;`col11`col12!5 33)

     

    But pasting this into the repl gives back a table,

    +`col11`col12!(1 5;3 33)

    So it seems like there’s some automatic table conversion that’s not happening in exec, that does happen in other cases. Interesting! Not much of an answer, but might help to find one.

  • gyorokpeter-kx

    Member
    February 15, 2024 at 12:00 am

    q collapses dictionaries with matching keys into a table. However this doesn’t happen if the list in question is a table column.

    The pretty printing logic still does the conversion as part of printing the value, but doesn’t change the underlying type, which means functions that expect a “real” table will fail.

    You can force the collapsing by remaking the list using an each:

    q)type(::)each exec col1 from tab 98h q)cols(::)each exec col1 from tab `col11`col12

     

  • roryk

    Member
    February 15, 2024 at 12:00 am

    How come ‘exec col1 from tab’ doesn’t collapse, but tab `col1 does?

  • gyorokpeter-kx

    Member
    February 15, 2024 at 12:00 am

    It’s a property of some operations and not others. exec just picks the column out of the table without walking through it. The equivalent operation is tab[;`col1].  However tab[`col1] is an odd operation because normally the first index to a table is the row and not the column, which implies that q must do something extra under the hood to come up with the same result as tab[;`col1], which includes rebuilding the column.

    Similarly if you include a trivial condition in the exec such as exec col1 from tab where 1b then it acts the same as just exec col1 from tab . However if you add a nontrivial condition, even if it’s always true, such as exec col1 from tab where 0<=i then suddenly q has to walk through the column to build the answer which means it will be collapsed. Try different operations with t to see the difference in the time they take (I tried with 10000 rows and t:1000 to get a measurable result).

  • roryk

    Member
    February 15, 2024 at 12:00 am

    Interesting, thanks. So is tab `col1 semantically different to tab.col1? Unexpected (to me anyway)

  • gyorokpeter-kx

    Member
    February 15, 2024 at 12:00 am

    tab.col1 is another equivalent to tab[;`col1] (besides the limitations that it can’t be used everywhere). It’s tab`col1 that is the odd one out since it breaks the usual rules for indexing.

Log in to reply.