Re: Lists, dictionaries, tables and lists of dictionaries

[rocuinneagain]

1. I might suggest to think about it the other way. Dictionaries are more like special paired lists.

q)dict:`a`b!1 2
q)lists:{(key x;value x)} dict
q)dict
a| 1
b| 2
q)lists
a b
1 2
q)dict `a
1
q)lists[0]?`a
0
q)lists[1] lists[0]?`a / The same as: dict `a
1

https://code.kx.com/q/ref/find/

 

2. Yes a list of conforming dictionaries is promoted a table

q)(`a`b!1 2;`a`b!1 2)
a b
---
1 2
1 2

Importantly in memory the way it is actually stored is ‘flipped’ so it is a dictionary of lists. (no longer a list of dictionaries)

q).Q.s1 (`a`b!1 2;`a`b!1 2)
"+`a`b!(1 1;2 2)"

 

This way the keys/column-names only need to be stored once for the whole table and not for each row.

The columns then are vectors which is more efficient and performant.

 

There are more details on indexing at depth here :

https://code.kx.com/q4m3/3_Lists/#38-iterated-indexing-and-indexing-at-depth

 

The “querying unstructured data” section of this blog may be of interest:
https://kx.com/blog/kdb-q-insights-parsing-json-files/

The code in it focuses on tables but can be adapted to lists/dictionaries as well:

q)asLists:sample cols sample
q)asLists[0;;`expiry]
17682D19:58:45.000000000
`
`long$()
,""
`long$()
0N
,""

q)@[`asLists;0;{(enlist[`]!enlist (::))(,)/:x}]
`asLists
q)asLists[0;;`expiry]
17682D19:58:45.000000000
::
::
::
::
::
::
q)fill:{n:count i:where (::)~/:y;@[y;i;:;n#x]}
q)fill[0Wn]asLists[0;;`expiry]
17682D19:58:45.000000000 0W 0W 0W 0W 0W 0W