rocuinneagain
Forum Replies Created
-
This is a qSQL statement:
https://code.kx.com/q/ref/delete/
q)delete ps from x
qSQL has some differences to q.
“delete is a qSQL query template and varies from regular q syntax.”
In qSQL statements backticks are not needed for looking up columns in a table or keys in a dictionary.
These are normal q statements:
q)load `:flatT //Uses ` to declare a symbol, this case is known as a hsym, denoting a filepath as it starts with :
q)`flatT in key `. //` used to declare a symbol `flatT and also to refer to . the default namespacecode.kx.com
delete query | Reference | kdb+ and q documentation - kdb+ and q documentation
delete is a qSQL template that removes rows or columns from a table, entries from a dictionary, or objects from a namespace.
-
‘ form of each https://code.kx.com/q/ref/maps/#each
q){x[y]:99;x}'[(1 2 3;4 5 6;7 8 9);0 1 2]
99 2 3
4 99 6
7 8 99Amend at @ version https://code.kx.com/q/ref/apply/#amend-amend-at
q){@[x;y;:;99]}'[(1 2 3;4 5 6;7 8 9);0 1 2]
99 2 3
4 99 6
7 8 99
q)@[;;:;99]'[(1 2 3;4 5 6;7 8 9);0 1 2]
99 2 3
4 99 6
7 8 99 -
- except https://code.kx.com/q/ref/except/
- ‘ form of each https://code.kx.com/q/ref/maps/#each
q)tab:([] timestamp:1 2 3;timeWin:(1 2 3;1 2 4 5;3 1 1 3 5))
q)tab
timestamp timeWin
-------------------
1 1 2 3
2 1 2 4 5
3 3 1 1 3 5
q)update timeWin:except'[timeWin;timestamp] from tab
timestamp timeWin
-----------------
1 2 3
2 1 4 5
3 1 1 5 -
Yes with right to left evaluation it is being saved then sorted.
The reason this may have been done is that sorting in memory will temporarily use extra memory to perform the operation.
https://code.kx.com/q/ref/asc/#sorting-data-on-disk
Sorting on disk is very memory efficient as it does not bring the full table in to memory.
The trade off is it is slower as each column gets written twice.
q)tab:flip (`$/:.Q.an)!{1000000?10000.0}each .Q.an q)\ts `a xasc `:tab/ set tab q) 1824 25168800 q)\ts `:tab/ set `a xasc tab 1165 536874192 q)536874192%25168800 21.33094 //Sorting in memory uses 21x memory than sorting on disk -
1. qSQL has some differences to q.
https://code.kx.com/q/ref/select/
“select is a qSQL query template and varies from regular q syntax.”
https://code.kx.com/q/basics/qsql/#where-phrase
“Efficient Where phrases start with their most stringent tests.”
Each where clause is it’s own expression. You can see inside [] the expression are executed left to right but within each expression the code is executed right to left.
q)[(show 2;show 1);(show 4;show 3);(show 6;show 5)];
1
2
3
4
5
62.
https://code.kx.com/q/basics/funsql/#group-by-column
“b is a column name. The result is a dictionary.”
https://code.kx.com/q/basics/funsql/#group-by-columns
“b is a list of column names. … returns a dictionary”
https://code.kx.com/q/basics/funsql/#group-by-a-dictionary
“b is a dictionary. Result is a dictionary in which the key is a table with columns as specified by b …”
If you parse the queries you can see that sym:sym is group by dictionary
q)parse"exec first open, last close by sym from daily where sym like \"*L\""
?
`daily
,,(like;`sym;"*L")
,`sym
`open`close!((*:;`open);(last;`close))vs:
q)parse"exec first open, last close by sym:sym from daily where sym like \"*L\""
?
`daily
,,(like;`sym;"*L")
(,`sym)!,`sym
`open`close!((*:;`open);(last;`close)) -
PyKX converts a q CharVector to Python NumPy array of bytes by default.
https://code.kx.com/pykx/2.5/pykx-under-q/upgrade.html#functional-differences
q).pykx.eval["lambda x: print(type(x))"][test]; <class 'numpy.ndarray'> q).pykx.eval["lambda x: print(x)"][test]; [b'<' b't' b'a' b'b' b'l' b'e' b'>' b'<' b't' b'r' b'>' b'<' b't' b'h' ...You can create a helper to convert to strings as you require:
q)b2s:.pykx.eval["lambda x: x.tobytes().decode('UTF-8')"] q).pykx.print pd[`:read_html]b2s[test] [ First Name Last Name Age 0 John Doe 30 1 Jane Smith 25 2 Emily Jones 22]More info on default conversions:
https://code.kx.com/pykx/2.5/pykx-under-q/intro.html#function-argument-types
And:
https://code.kx.com/pykx/2.5/user-guide/fundamentals/text.html
-
@laura – This was a format testing reply – can be deleted. Thanks
-
<div>See these sections of docs on:</div>
Order of evaluation also matters: Left-of-Right Evaluation
abs(price)=({abs max x};price)Will run in order:
- ({abs max x};price)
- (price)=({abs max x};price)
- abs(price)=({abs max x};price)
whereas:
abs[price]=({abs max x};price)Will run in order:
- ({abs max x};price)
- abs[price]
- abs[price]=({abs max x};price)
This tries to pass 2 arguments to all which only takes one argument so you get a rank error
https://code.kx.com/q/ref/all-any/#all
all[col=a;col2=b]
-
You could make windows like so:
q)start:0D q)duration:1D q)length1:0D00:20 q)length2:0D00:10 q)windows:{flip (0;y-1)+\:y*til `long$x div y}'[duration;(length1;length2)] q)(first;last)@\:windows 0 0D00:00:00.000000000 0D00:19:59.999999999 0D23:40:00.000000000 0D23:59:59.999999999 q)(first;last)@\:windows 1 0D00:00:00.000000000 0D00:09:59.999999999 0D23:50:00.000000000 0D23:59:59.999999999 q)t:([] time:100?1D;sym:100?`A`B`C) time sym ------------------------ 0D11:37:38.441374450 A 0D16:23:13.911677598 A 0D03:40:38.809463381 C 0D12:50:31.973013281 B 0D10:58:33.142468035 C 0D00:11:36.601854264 BQuery for small tables for each sym for each windows:
syms:exec distinct sym from t q){[x;y;z] select from x where sym=y,time within z}[t](.)/:syms cross enlist each windows 0 +`time`sym!(`timespan$();`symbol$()) +`time`sym!(,0D03:19:44.361107647;,`A) +`time`sym!(0D03:20:29.103351384 0D03:28:31.211699992;`A`A) ... +`time`sym!(,0D20:47:50.484825968;,`B) +`time`sym!(,0D21:14:37.927203476;,`B) +`time`sym!(0D21:39:42.773356139 0D21:37:00.400066673;`B`B) +`time`sym!(`timespan$();`symbol$()) +`time`sym!(,0D22:09:45.092769563;,`B) -
The is a kdb+ Solace interface
- https://solace.com/kx/
- https://github.com/KxSystems/solace/tree/master
- https://code.kx.com/q/wp/solace/
One of the getPayloadAs* functions may be what you need
https://github.com/KxSystems/solace/blob/77dd10edefa014f39b6c5ed9a693bdcad7685509/q/solace.q#L30
q)msg "\PAR1\025\004\025\010...
q).solace.getPayloadAsBinary msg
-
, rather than ; separating the arguments of fbis. Otherwise runs fine:
q)fbis:{[c_;x0;p_]{[xn ;c;p] xn-((xn xexp p)-c)%(p*xn xexp p-1)}[;c_;p_]\[x0]}
q)fbis[2;1;3]
1
1.333333
1.263889
1.259933
1.259921
1.259921
-
https://code.kx.com/q/ref/asc/#sorting-data-on-disk
“xasc can sort data on disk directly, without loading the entire table into memory.”
If you interrupt an on disk xasc you could be in a state where some columns were sorted but not others.
-
There is a docs page on PyKX modes of operation.
The 3 modes are:
- Unlicensed mode – used for IPC query and publish. No license needed.
- Licensed mode – import pykx in Python session.
- PyKX under q – \l pykx.q in a q session.
Both 2 & 3 require a KX license which includes insights.lib.pykx and insights.lib.embedq licensing flags.
- The kdb Insights Personal Edition free trial license includes these flags.
- For commercial licences with PyKX enabled contact your KX sales representative or sales@kx.com. Alternately apply through https://kx.com/book-demo.
-
Node.js is required for the upload server.
If node is not installed, dashboards will work as normal without the upload component.
Documentation will be updated to detail this.
-
Going forward PyKX under q is the recommended replacement for embedPy.
*It is not operational yet on Windows