-
Strange data type prevents table operation on what looks like a table
I have this table:
tab:([]time:2?.z.P;col1:((`col11`col12!(1;3));(`col11`col12!(5;33)));col2:((`col11`col12!(1;3));(`col11`col12!(5;33)))) time col1 col2 ----------------------------------------------------------------- 2015.10.24D22:29:46.626037056 `col11`col12!1 3 `col11`col12!1 3 2004.03.06D21:27:43.089887728 `col11`col12!5 33 `col11`col12!5 33
I am doing the following operations on it :
cols exec col1 from tab
Now, I would expect that to work, but when I run a type on that exec operation i get a 0h even though it looks like an actual table (see below)
col11 col12 ----------- 1 3 5 33
It works if I do something like this:
cols{x}each ?[tab;();();`col1]Does anyone know why that table looks like a table, but it’s not a table ?
-
6 Replies
-
Dropping into k can be useful to see the representation. Tab is +`time`col1`col2!(times;(`col11`col12!1 3;`col11`col12!5 33);(`col11`col12!1 3;`col11`col12!5 33))
What’s interesting is that
exec col1 from tab
then produces
(`col11`col12!1 3;`col11`col12!5 33)
But pasting this into the repl gives back a table,
+`col11`col12!(1 5;3 33)
So it seems like there’s some automatic table conversion that’s not happening in exec, that does happen in other cases. Interesting! Not much of an answer, but might help to find one.
-
q collapses dictionaries with matching keys into a table. However this doesn’t happen if the list in question is a table column.
The pretty printing logic still does the conversion as part of printing the value, but doesn’t change the underlying type, which means functions that expect a “real” table will fail.
You can force the collapsing by remaking the list using an each:
q)type(::)each exec col1 from tab 98h q)cols(::)each exec col1 from tab `col11`col12
-
How come ‘exec col1 from tab’ doesn’t collapse, but tab `col1 does?
-
It’s a property of some operations and not others. exec just picks the column out of the table without walking through it. The equivalent operation is tab[;`col1]. However tab[`col1] is an odd operation because normally the first index to a table is the row and not the column, which implies that q must do something extra under the hood to come up with the same result as tab[;`col1], which includes rebuilding the column.
Similarly if you include a trivial condition in the exec such as exec col1 from tab where 1b then it acts the same as just exec col1 from tab . However if you add a nontrivial condition, even if it’s always true, such as exec col1 from tab where 0<=i then suddenly q has to walk through the column to build the answer which means it will be collapsed. Try different operations with t to see the difference in the time they take (I tried with 10000 rows and t:1000 to get a measurable result).
-
Interesting, thanks. So is tab `col1 semantically different to tab.col1? Unexpected (to me anyway)
-
tab.col1 is another equivalent to tab[;`col1] (besides the limitations that it can’t be used everywhere). It’s tab`col1 that is the odd one out since it breaks the usual rules for indexing.
Log in to reply.