rocuinneagain
Forum Replies Created
-
The syntax you need is:
refServiceHandle(`getOptionRef;uniqueOpts)
This will pass the local variable down to the remote function as a parameter.
I’ve added a pull request to have this documented better:
https://github.com/KxSystems/docs/pull/85
Q for Mortals has some notes already:
https://code.kx.com/q4m3/1_Q_Shock_and_Awe/#119-interprocess-communication-101
-
metaon an empty table will never showC. This is because kdb+ does not have a dedicated datatype for lists of lists. The empty list will be()which is of type0h. On a populated table meta inspects the first item in a list of lists and uses it’s type to populate it’s result. It cannot do this on an empty table.q)tb:([] a:1 2;b:("wo";"rd")) q)tb a b ------ 1 "wo" 2 "rd" q)meta tb c| t f a -| ----- a| j b| C q)delete from `tb `tb q)tb a b --- q)meta tb c| t f a -| ----- a| j b| q).Q.s1 tb "+`a`b!(`long$();())" -
The sym file contains enumerated symbols for an symbol column in any table.
The reason you can select it is that if q fails to find a column of that name it will try to use a global variable , which sym will be.
q)`:tab/ set .Q.en[`:.] ([] a:1 2 3;b:`a`b`c) `:tab/ q)tab:get `:tab/ q)tab a b --- 1 a 2 b 3 c q)select sym from tab sym --- a b c q)sym //sym is in memory `a`b`c 2 q)nonsense:`one`two`three q)select nonsense from tab //nonsense is not a column but a global variable nonsense -------- one two three q)delete sym from `. /delete sym from memory `. q)tab / now any sym columns show indexes as they cannot resolve their text a b --- 1 0 2 1 3 2
Review some documentation on enumeration and on disk data:
- https://code.kx.com/q/basics/enumerations/
- https://code.kx.com/q4m3/14_Introduction_to_Kdb+/#1428-working-with-sym-files
-
kdb+ does not implement full regex.
https://code.kx.com/q/basics/regex/
It supports
?*[]^If you want full support you can use the C interface to bring in external libraries
- https://code.kx.com/q/basics/regex/#regex-libraries
- https://code.kx.com/developer/libraries/regular-expression-pcre2/
- https://github.com/KxSystems/ffi/blob/master/examples/pcre.q
-
Try this to confirm the file is valid:
//Bytes per record q)sum 3 3 2 2 4 14 //Bytes in file q)hcount `:file1.txt ?? //Confirm bytes in file has no remainder if divided by bytes in record q)0~hcount[`:file1.txt] mod sum 3 3 2 2 4 ??
If last line returns
0bthen your file is not a valid size for those fixed length messages.You can see quickly at the end of the file if the last record looks valid.
//Inspect the last records in the file to see visually if records look correct
q)-2#(sum 3 3 2 2 4) cut `char$read1
The issue could start anywhere in the file so you may need to do further investigation if it is still not obvious after checking the above.
-
One item to note is that .j.j will put processing load on to the kdb+ process.
JSON also only has a limited set of datatypes which means you will need to write more q code to perform parsing/casting
- https://www.w3schools.com/js/js_json_datatypes.asp
- https://kx.com/blog/kdb-q-insights-parsing-json-files/
Using the c.flip distributes the serialisation work across your java processes/threads and minimizes the work kdb+ will have to do to deserialise/parse/cast.
kdb+ does have a multithreaded input queue mode (but it does have restrictions)
-
I added extra columns and it did not slow down the operation for me.
Is your data in memory or on disk?data:([] dt:`s#.z.p+0D00:00:01*til 800000;mmm3:800000?1000.0) data:data,'flip (`$"col",/:string til 30)!30#enlist til 800000 f:`dt; w:(-1000000; 0)+:data[`dt]; result: wj[w;f;data;(data;(max;`mmm3);(min;`mmm3))]
-
On my laptop this took 765 milliseconds – what runtime are you seeing?
data:([] dt:`s#.z.p+0D00:00:01*til 800000;mmm3:800000?1000.0) f:`dt; w:(-1000000; 0)+:data[`dt]; \ts result: wj[w;f;data;(data;(max;`mmm3);(min;`mmm3))]
-
So you are only joining on time windows? no other column?
You could apply the sorted attribute.
https://code.kx.com/q/ref/set-attribute/#sorted
The ‘datetime’ datatype is deprecated. ‘timestamp’ should be used instead.
https://code.kx.com/q/basics/datatypes/#temporal
If you can share a minimal reproducible example in q that will help us answer your questions.
-
The documentation does have a note on attributes to apply:
t and q are simple tables to be joined (q should be sorted `sym`time with `p# on sym)
Have you applied this?
-
There are useful comparisons on this page: https://code.kx.com/pykx/1.3/comparisons.html
You can expose python function to q using embedpy
You can do it all in a q file or as I like to do split out to a .py and a .q
Such as: https://github.com/rianoc/qXML
kdb+ can also use websockets natively if needed
-
- Is QLIC set?
- Was this licence issued specifying it was for a windows machine?
-
What location is your licence file?
Have you set the QLIC environment variable?
-
Have you installed embedpy?
https://code.kx.com/q/ml/embedpy/#build-and-install
Can you load and test it?
\l p.q p)print(1+1)