Re: Performance using a table to combine where phrases

davidcrossey

Member

July 6, 2023 at 12:00 am

I would presume optimizations such as left to right sub-clause filtering as mentioned here https://code.kx.com/q4m3/9_Queries_q-sql/#9333-multiple-where-subphrases are lost when filtering using a table/dict lookup.

i.e. we need to check all of the columns specified in the lookup simlataneouly, instead of passing each consective filter to the next sub-phrase.

To visualize differently:

q)show tab:([]c1:`a`b`c;c2:1 2 3) c1 c2 ----- a 1 b 2 c 3 q)tab in 1#tab 100b q)select from tab where tab in 1#tab c1 c2 ----- a 1

Also worth having a look at the parse trees:

q)parse "select from PUBLICATIONS where date=2023.06.28,src=`market,PRICE_TYPE=`dev,INDEX1=`JPM,INDEX2=`,CONTRACT1=`2023.10.M,PROFILE=`,CONTRACT_TYPE=`BLA" ? `PUBLICATIONS 
,((=;`date;2023.06.28);(=;`src;,`market);(=;`PRICE_TYPE;,`dev);(=;`INDEX1;,`JPM);(=;`INDEX2;,`);(=;`CONTRACT1;,`2023.10.M);(=;`PROFILE;,`);(=;`CONTRACT_TYPE;,`BLA)) 0b () 
q)parse "select from PUBLICATIONS where ([]date;SRC;PRICE_TYPE;INDEX1;INDEX2;CONTRACT1;PROFILE;CONTRACT_TYPE) in 1#.eoh.f" ? `PUBLICATIONS 
,,(in;(+:;(!;,`date`SRC`PRICE_TYPE`INDEX1`INDEX2`CONTRACT1`PROFILE`CONTRACT_TYPE;(enlist;`date;`SRC;`PRICE_TYPE;`INDEX1;`INDEX2;`CONTRACT1;`PROFILE;`CONTRACT_TYPE)));(#;1;`.eoh.f)) 0b ()

KX Community

davidcrossey