rocuinneagain
Forum Replies Created
-
To load the SQL interpreter you can run:
l s.k
Otherwise when you run your first ‘s)’ query this would be done automatically.
-
Hi Michael,
Hopefully this helps.
Sample Datasource:
([] description:enlist “ROMANI 4.875 01/24”;Volume:enlist 1.6385)
‘Template Text’ in ‘Text’ widget:
{{#each this}} {{description}} <br> Volume:{{toPrecision Volume 2}}m {{/each}}
-
This blog on JSON parsing may also have some useful tips for you as it covers some areas of nested data with irregular keys
https://kx.com/blog/kdb-q-insights-parsing-json-files/
An example for your ask is to use a prototype dictionary of default values
// Prototype of default values if lookup fails q)p:`a`b`c`d!("X";99;99;99) q)p a| "X" b| 99 c| 99 d| 99 // Actual dictionary q)d:`a`b`c!("";2;3) q)d a| "" b| 2 c| 3 // Failed lookup uses null of type of first key // a is first key, it is type char, null char is "" q)d`d "" // Prototype can be used by appending your dict to it q)(p,d)`d 99 q)(p,d)`b 2 // Handles vectors of keys nicely q)(p,d)`d`b`a 99 2 "" -
Another kdb implementation of the game: https://github.com/psaris/wordle
-
I didn’t fully read the question, got distracted by the idea of a q implementation of the game.
-
Is there a reason you cannot use -g 1 ?
Running .Q.gc[] so often is not the best approach but if you needed to a timer could check and run if after .z.pg for you:
runGC:0b .z.pg:{ bigresult: value x;runGC::1b; bigresult }; .z.ts:{if[runGC;.Q.gc[];runGC::0b]} t 1https://code.kx.com/q/ref/dotz/#zts-timer
To run it less often you could set a threshold and in the if statement check memory usage
-
You could use the python statsmodels.tsa.vector_ar.vecm.JohansenTestResult by importing it through EmbedPy and passing the data as a dataframe using mltab2df.
Below is an example based of a similar python version: http://web.pdx.edu/~crkl/ceR/Python/example14_3.py
$pip install statsmodels q)l p.q q)l ml/ml.q q).ml.loadfile`:init.q q)data:flip `YEAR`Y`C!"IFF"$flip 1_-12_{{x where not ""~/:x}" " vs x} each "rn" vs .Q.hg "http://web.pdx.edu/~crkl/ceR/data/usyc87.txt" q)coint_johansen:.p.import[`statsmodels.tsa.vector_ar.vecm;`:coint_johansen] q)pd:.ml.tab2df[data][`:set_index;"YEAR"] q)res:coint_johansen[pd;0;2] q)res[`:lr1]` 31.78169 12.17119 -1.566747e-012 q)res[`:lr2]` 19.6105 12.17119 -1.566747e-012 q)res[`:cvm]` 18.8928 21.1314 25.865 12.2971 14.2639 18.52 2.7055 3.8415 6.6349 q)res[`:cvt]` 27.0669 29.7961 35.4628 13.4294 15.4943 19.9349 2.7055 3.8415 6.6349 q){flip y!(x@/:hsym y)@:`}[res;`lr1`lr2`cvm`cvt] lr1 lr2 cvm cvt ---------------------------------------------------------------------------- 31.78169 19.6105 18.8928 21.1314 25.865 27.0669 29.7961 35.4628 12.17119 12.17119 12.2971 14.2639 18.52 13.4294 15.4943 19.9349 -1.566747e-012 -1.566747e-012 2.7055 3.8415 6.6349 2.7055 3.8415 6.6349 -
The day part is an integral portion of the datatype and will display by default.
In a UI/display time it can be dropped if required:
q)2_string .z.n /Atom "14:41:40.125906000" q)2_/:string 2#.z.n /List "14:42:00.701751000" "14:42:00.701751000" q)update 2_/:string time from ([] time:2#.z.n) /Column in table time -------------------- "14:42:38.625329000" "14:42:38.625329000" // All columns of timespan type in a table q)dropDays:{c:where -16h=type each first x;$[count c;![x;();0b;c!{((/:;_);2;($:;x))}each c];x]} q)dropDays ([] time:2#.z.n;b:1.1 1.2) time b ------------------------ "14:47:37.376270000" 1.1 "14:47:37.376270000" 1.2 -
As in the example it is a nested generic list the items need to be dealt with one at a time. As the list could have many different tables or even different datatypes within it.
q)cols each .[dsEg;(`html;`body)] a b q).[dsEg;(`html;`body);{cols each x}] doctype| ,"html" html | `text`body!(,"test";,`a`b)The use of :: may be useful to you if you have not been using it
https://code.kx.com/q/ref/apply/#nulls-in-i
It allows you to skips levels
q).[dsEg;(`html;`body;::;`a)] d f g //Better shown on an item with multiple entries in the list q)dsEg2:(`doctype`html)!(enlist "html";`text`body!(enlist"test";2#enlist ([]a: `d`f`g;b: 23 43 777))); q).[dsEg2;(`html;`body;::;`a)] d f g d f g
.Q.s1 may also be useful to you as it can help show the underlying structure of an item better than the console at times.
https://code.kx.com/q/ref/dotq/#qs1-string-representation
q).[dsEg;(`html;`body;::;`a)] d f g //Looks like a symbol list type 11h but is in fact a single item egeneric list type 0h q){-1 .Q.s1 x;} .[dsEg;(`html;`body;::;`a)] ,`d`f`g //.Q.s1 output can be ugly but always shows exact structure -
They can be on separate machines/environments.
You can see in the White Paper that they use Kdb+ running on an ec2 instance ‘ip-172-31-70-197.ec2.internal’ and it connects to Solace broker running on ‘mr2ko4me0p6h2f.messaging.solace.cloud’
As long as your network settings are setup so that the 2 hosts can communicate over the needed ports then you will be successful.
-
You can
1. Upload the .tgz to the server (SCP/FTP etc)
2. Extract it
3. Import the package through the UI
https://code.kx.com/platform/release_management/#import-from-a-different-location
-
The traditional tick.q is a single threaded process but there are many areas you can investigate to ensure performance is at it’s best:
- Use taskset to ensure other processes are not using the same core/recourses as the TP
- As the TP persists data to a log file ensuring that you have evaluated your hardware configuration is important to validate that your storage disk is not bottlenecking your system
- Chained tickerplants can be used to balance the flow of data in a system, particularly when there are many subscribers
- Async broadcast can be used to optimise publishing the same data to multiple subscribers
- Ensure the system is setup to follow best practices outlined in Linux production notes
- Use Unix domain sockets when opening connections on localhost to reduce CPU usage
- Unix domain sockets are also available from C feedhandlers
- Whitepaper on Kdb+ tick profiling
KX Platform:
- Messaging performance tuning
- Tickerplant template
- Process instances core affinity
-
Yes your understanding is correct for those 4 items.
These tutorial videos may be useful:
-
q)tab:([] a:`a`b`c;b:1 2 3) //Create a table q)tab 0 //Index in to the table to row 0. A dictionary is returned a| `a b| 1 q)til 3 0 1 2 q)count tab // tab contains 3 rows 3 q)til count tab // 'til' returns numbers up to entered value 0 1 2 q){show tab x}each til count tab //Using 'each' and 'show' in a lambda to display each row a| `a b| 1 a| `b b| 2 a| `c b| 3- “A simple table is a list of dictionaries” https://code.kx.com/q/learn/dicts-tables/
- https://code.kx.com/q/kb/faq/#row-indexing
- https://code.kx.com/q4m3/6_Functions/#618-anonymous-functions-and-lambda-expressions
-
Kdb+ does have inbuilt HTTP functions for GET (.Q.hg) and POST (.Q.hp)
These blog posts may be of interest:
Another option would be to use embedPy.
You can use it to expose Python functions to q
- Web Scraping – A Kdb+ Use case – KX – (Related code: abin-saju/qblog)
When data is returned as JSON you can use .j.k to deserialize