rocuinneagain
Forum Replies Created
-
kdb+ does not implement full regex.
https://code.kx.com/q/basics/regex/
It supports
?*[]^If you want full support you can use the C interface to bring in external libraries
- https://code.kx.com/q/basics/regex/#regex-libraries
- https://code.kx.com/developer/libraries/regular-expression-pcre2/
- https://github.com/KxSystems/ffi/blob/master/examples/pcre.q
-
Try this to confirm the file is valid:
//Bytes per record q)sum 3 3 2 2 4 14 //Bytes in file q)hcount `:file1.txt ?? //Confirm bytes in file has no remainder if divided by bytes in record q)0~hcount[`:file1.txt] mod sum 3 3 2 2 4 ??
If last line returns
0bthen your file is not a valid size for those fixed length messages.You can see quickly at the end of the file if the last record looks valid.
//Inspect the last records in the file to see visually if records look correct
q)-2#(sum 3 3 2 2 4) cut `char$read1
The issue could start anywhere in the file so you may need to do further investigation if it is still not obvious after checking the above.
-
One item to note is that .j.j will put processing load on to the kdb+ process.
JSON also only has a limited set of datatypes which means you will need to write more q code to perform parsing/casting
- https://www.w3schools.com/js/js_json_datatypes.asp
- https://kx.com/blog/kdb-q-insights-parsing-json-files/
Using the c.flip distributes the serialisation work across your java processes/threads and minimizes the work kdb+ will have to do to deserialise/parse/cast.
kdb+ does have a multithreaded input queue mode (but it does have restrictions)
-
I added extra columns and it did not slow down the operation for me.
Is your data in memory or on disk?data:([] dt:`s#.z.p+0D00:00:01*til 800000;mmm3:800000?1000.0) data:data,'flip (`$"col",/:string til 30)!30#enlist til 800000 f:`dt; w:(-1000000; 0)+:data[`dt]; result: wj[w;f;data;(data;(max;`mmm3);(min;`mmm3))]
-
On my laptop this took 765 milliseconds – what runtime are you seeing?
data:([] dt:`s#.z.p+0D00:00:01*til 800000;mmm3:800000?1000.0) f:`dt; w:(-1000000; 0)+:data[`dt]; \ts result: wj[w;f;data;(data;(max;`mmm3);(min;`mmm3))]
-
So you are only joining on time windows? no other column?
You could apply the sorted attribute.
https://code.kx.com/q/ref/set-attribute/#sorted
The ‘datetime’ datatype is deprecated. ‘timestamp’ should be used instead.
https://code.kx.com/q/basics/datatypes/#temporal
If you can share a minimal reproducible example in q that will help us answer your questions.
-
The documentation does have a note on attributes to apply:
t and q are simple tables to be joined (q should be sorted `sym`time with `p# on sym)
Have you applied this?
-
There are useful comparisons on this page: https://code.kx.com/pykx/1.3/comparisons.html
You can expose python function to q using embedpy
You can do it all in a q file or as I like to do split out to a .py and a .q
Such as: https://github.com/rianoc/qXML
kdb+ can also use websockets natively if needed
-
- Is QLIC set?
- Was this licence issued specifying it was for a windows machine?
-
What location is your licence file?
Have you set the QLIC environment variable?
-
Have you installed embedpy?
https://code.kx.com/q/ml/embedpy/#build-and-install
Can you load and test it?
\l p.q p)print(1+1)
-
Everything looks correct in that screenshot. You just need to open your web browser and enter
http://localhost:10001in your address bar.For starting KX Developer I wrote a simple
developer.batfile which I can double click to start both the developer and have my web browser open to the correct URL. You could do similar for KX Dashboards.My
developer.batcontains:call C:developerconfigconfig.bat start "developer" "q" C:developerlauncher.q_ start "" http:localhost:%developer_PORT%/ax
-
You need to run both commands in the same ssh terminal. It does not work when you do it in 2 different ones like you screenshot.
-
To launch
pyqyou need to call thepyqexecutable instead of thepythonexecutable.https://code.kx.com/q/interfaces/pyq/#quick-start
This:
$ pyq
Not this:
$ python
Related questions raised on stackoverflow.com
- kdb – How do i start KX Developer – Stack Overflow
- kdb – How do I install PYQ in Conda or Pycharm? – Stack Overflow
- kdb – KX Developer launcher error how can i fix it? – Stack Overflow
For Developer
https://code.kx.com/developer/getting-started/#starting-developer
You need to start it with
source /path-to-install-dir/config/config.profile q /path-to-install-dir/launcher.q_
For your SSH error this may help: How to install libssh2-1 on Ubuntu 20.04 (Focal Fossa)? (devmanuals.net)