[davidcrossey]

Hi Cristian,

A bunch of questions that might help us guide you;

• What version of JuypterQ do you have?
• What format is your licence? i.e. k[x,4,c].lic?
• What flags have you got in your Insights lic? (Can you share your banner information here?)
• What are your QLIC and QHOME env var pointing to?

q).z.l "" "2024.01.13" "2024.01.13" ,"1" "insights.lib.embedq insights.lib.pykx insights.lib.sql insights.lib.qlog ins.. ,"0" ,"0" "dcrossey..." ,"0" 
q)" " vs .z.l 4 "insights.lib.embedq" "insights.lib.pykx" "insights.lib.sql" "insights.lib.qlog" "insights.lib.kurl" "insights.lib.objstore" "insights.lib.bigquery" "insights.lib.restserver"

Have you installed your licence as per: Install kdb Insights Core – kdb products (kx.com) ?

[davidcrossey]

Yes it can be used on w64, however there are differences in terms of the libraries needed for linking: kdb/w64 at master KxSystems/kdb (github.com)

This repo: kxcontrib/capi (github.com) only supports Linux based environments. You could fork it and add Windows support yourself, or a more straight forward solution would be to setup WSL2 with a *nix distribution (such as Ubuntu), and use that environment for your local development – works with VS Code.

[davidcrossey]

Your usage of square brackets has created the projection on the right-hand side, which is expecting an x argument:

q).my.func:{[x;y;z] (x;y;z)} q)tab col1 col2 ——— a 1 b 2 c 3 / projection q){.my.func[x;y;`field]}'[flip value exec col1,col2 from tab] {.my.func[x;y;`field]}'[((`a;1);(`b;2);(`c;3))] q)type {.my.func[x;y;`field]}'[flip value exec col1,col2 from tab] 104h / right-hand side projection; note .’ as type (‘[x]) will return 106 q)type .'[flip value exec col1,col2 from tab] 104h / using apply-each without square brackets q)type {.my.func[x;y;`field]}.’flip value exec col1,col2 from tab 0h q){.my.func[x;y;`field]}.’flip value exec col1,col2 from tab `a 1 `field `b 2 `field `c 3 `field

Note the usage of apply-each .’

Here is a simplier example of your projection syntax:

/ missing x arg q)@'[1 2] @'[1 2] q)type @'[1 2] 104h / providing x arg q)@'[10+;1 2] 11 12 q)type @'[10+;1 2] 7h

Hope this helps

 

[davidcrossey]

Did you compile your code and link against the c.o object file too? Check out kdb/l64/c.o at master KxSystems/kdb GitHub If you need a different OS binary, check out the directories at the root of the repo.

More info here: C/C++ | Interfaces | kdb+ and q documentation – Kdb+ and q documentation (kx.com)

[davidcrossey]

I don’t believe sha256 is supported with -u/-U, however you could instead perhaps use .z.pw to carry out custom validation to the effect of:

1. Read your user:sha256 file in the callback when a connection attempt is made
2. Convert the plain text password from the user to sha256
3. Validate the user with 1b (success) or 0b (failure)

References:

• https://code.kx.com/q/releases/ChangesIn2.4/#zpw
• https://code.kx.com/q/ref/dotz/#zpw-validate-user

[davidcrossey]

I would presume optimizations such as left to right sub-clause filtering as mentioned here https://code.kx.com/q4m3/9_Queries_q-sql/#9333-multiple-where-subphrases are lost when filtering using a table/dict lookup.

i.e. we need to check all of the columns specified in the lookup simlataneouly, instead of passing each consective filter to the next sub-phrase.

To visualize differently:

q)show tab:([]c1:`a`b`c;c2:1 2 3) c1 c2 ----- a 1 b 2 c 3 q)tab in 1#tab 100b q)select from tab where tab in 1#tab c1 c2 ----- a 1

Also worth having a look at the parse trees:

q)parse "select from PUBLICATIONS where date=2023.06.28,src=`market,PRICE_TYPE=`dev,INDEX1=`JPM,INDEX2=`,CONTRACT1=`2023.10.M,PROFILE=`,CONTRACT_TYPE=`BLA" ? `PUBLICATIONS 
,((=;`date;2023.06.28);(=;`src;,`market);(=;`PRICE_TYPE;,`dev);(=;`INDEX1;,`JPM);(=;`INDEX2;,`);(=;`CONTRACT1;,`2023.10.M);(=;`PROFILE;,`);(=;`CONTRACT_TYPE;,`BLA)) 0b () 
q)parse "select from PUBLICATIONS where ([]date;SRC;PRICE_TYPE;INDEX1;INDEX2;CONTRACT1;PROFILE;CONTRACT_TYPE) in 1#.eoh.f" ? `PUBLICATIONS 
,,(in;(+:;(!;,`date`SRC`PRICE_TYPE`INDEX1`INDEX2`CONTRACT1`PROFILE`CONTRACT_TYPE;(enlist;`date;`SRC;`PRICE_TYPE;`INDEX1;`INDEX2;`CONTRACT1;`PROFILE;`CONTRACT_TYPE)));(#;1;`.eoh.f)) 0b ()

 

[davidcrossey]

You could use except each as follows:

q)d:(2023.03.12 2023.03.13)!(`data1.log`data2.log;“data3.log) q)d 2023.03.12| data1.log data2.log 2023.03.13| data3.log q)d except’ ` 2023.03.12| `data1.log`data2.log 2023.03.13| ,`data3.log

[davidcrossey]

If you mean remove the null key, you can drop the null using the following syntax:

q)show d:(4?(0Nd,3?.z.D))!1 2 3 4 2006.11.03| 1 2008.12.03| 2 | 3 | 4 q)enlist[0Nd] _ d 2006.11.03| 1 2008.12.03| 2

[davidcrossey]

Hi Jeremy,

IIRC, permissions are only available in the KX Platform Dashboards.

You could have a look through this wrapper module I worked on a while back to see if it suits your needs or can be extended to do so: https://github.com/davidcrossey/dash-direct-perms

Cheers,

David

[davidcrossey]

A non-keyed table is just a list of dictionaries, flipped:

 

q)tab:([]a:1 2 3;b:`a`b`c) 
tab 
a b 
--- 
1 a 
2 b
3 c 

q)`a`b!(1 2 3;`a`b`c) 
a| 1 2 3 
b| a b c 

q)flip `a`b!(1 2 3;`a`b`c) 
a b 
--- 
1 a 
2 b 
3 c 

q)tab~flip `a`b!(1 2 3;`a`b`c) 
1b

 

What you are doing here is creating a new table, with a column called tab. This new column has a record for each row of the original table as a dictionary

q)tab:([]a:1 2 3;b:`a`b`c) 
tab 
a b 
--- 
1 a 
2 b 
3 c 

q)([]tab) 
tab 
----------- 
`a`b!(1;`a) 
`a`b!(2;`b) 
`a`b!(3;`c)

You can also enlist the table to create a single record in the new table:

q)([] enlist tab) 
tab 
-------------------- 
+`a`b!(1 2 3;`a`b`c)

If you want to create a table from a table, typically you would utilise query filtering to select a subset of columns, or joins from another table. What is your use case here?

[davidcrossey]

You’ll get a type error without the each, because you need to create a scalar from each vector (i.e. each pair of symbols)

 

q)t:([]sym:`ORCL`APPL;src:`L`R) t 
sym src 
-------- 
ORCL L 
APPL R 

q) update symsrc:` sv (sym,'src) from t 
'type [0] 
update symsrc:` sv (sym,'src) from t ^ 

q) update symsrc:` sv'(sym,'src) from t 
sym src symsrc 
--------------- 
ORCL L ORCL.L 
APPL R APPL.R

 

Bonus tip – as shown above, you can avoid using a lambda by using parenthesis so the parser doesn’t interpret the comma as a new column statement.

[davidcrossey]

Check out https://code.kx.com/dashboards/datasources/#client-pivot

[davidcrossey]

might be worth checking if the objects are <64MB too

“During that return of memory, q checks if the capacity of the object is ?64MB. If it is and g is 1, the memory is returned immediately to the OS; otherwise, the memory is returned to the thread-local heap for reuse.

Executing .Q.gc[] additionally attempts to coalesce pieces of the heap into their original allocation units and returns any units ?64MB to the OS.” – System commands in q | Basics | kdb+ and q documentation – Kdb+ and q documentation (kx.com)

[davidcrossey]

Hi ,

Thanks for raising.

In the real world there should be quotes before trades. I assume given this is an exercise, the data has been randomly generated across a time period and hence you are seeing the above.

I wouldn’t be too concerned, as long as you can see how aj works with some other option_id that’s what matters.

[davidcrossey]

Have you included the k.h header file in your code?

There is a good whitepaper from  using the capi you can follow here