Forum Replies Created

Page 16 of 21
  • rocuinneagain

    Member
    September 9, 2022 at 12:00 am in reply to: xbar millisecond

    A timespan can be used directly

    q)select avg col by 0D00:00:00.005 xbar time from t 
    time                         | col 
    -----------------------------| ---- 
    2022.09.09D07:40:23.110000000| 3 
    2022.09.09D07:40:23.115000000| 11.5 
    2022.09.09D07:40:23.120000000| 18

     

  • rocuinneagain

    Member
    September 6, 2022 at 12:00 am in reply to: Accumulators – Access additional list / column

    Yes that’s much cleaner – as you only ever compute one value c2 and only look back 1 step. this will do exactly as you need.

  • rocuinneagain

    Member
    September 6, 2022 at 12:00 am in reply to: Accumulators – Access additional list / column

    Yes in that case an accumulator is needed. One method you could choose would be to pass a table through to accumulate while also allowing you to look back to previous rows:

    q)update c2:1_@[;`c2]{y[`c2]:enlist $[(y[`c1][0]>last x[`c2]) or ((last x[`c])<last x[`c2]);y[`c1][0];last x`c2];x,y}/[enlist each {(1#0#x),x}update c2:0 from `c`c1#t] from t 
    c c1 c2 
    ---------- 
    30 10 10 
    40 20 20 
    25 5  20 
    20 25 25 
    4  5  5 
    4  4  4 
    4.5 3 4 
    4.5 3.5 4
  • You can use a second q process help you to debug.

    Set a Listening-port p and have the HTTP Post handler (.z.pp) print incoming message contents

     

    p 5000 .z.pp:{show x;x}

     

     

    Trying curl from command line

     

    $ curl -H ‘Content-type: application/json’ -d ‘{“text”:”Hello World”}’ localhost:5000

     

     

    The server q process prints:

     

    ” {“text”:”Hello World”}” `Host`User-Agent`Accept`Content-type`Content-Length!(“localhost:5000″;”curl/7.58.0″;”*/*”;”application/json”;”22″)

     

     

    From a client q process using .Q.hp

     

    q).Q.hp[“http://localhost:5000″;.h.ty`json] .j.j enlist[`text]!enlist”Hello World”

     

     

    The server q process prints:

     

    ” {“text”:”Hello World”}” `Accept-Encoding`Connection`Host`Content-type`Content-length!(“gzip”;”close”;”localhost:5000″;”application/json”;”22″)

     

     

    There are only slight differences in the headers.

    You may need to consult documentation of the server you are connecting to to confirm if it has specific header requirements.

    (Clients using KX Insights have access to kurl  with more options than .Q.hp)

  • rocuinneagain

    Member
    September 5, 2022 at 12:00 am in reply to: Accumulators – Access additional list / column

    Looking at your expected c2 this logic may be what you are looking for:

     

    q)update c2:fills ?[(c1>prev c1) or c<prev c1;c1;0N] from t 
    c c1 c2 
    -------- 
    30 10 10 
    40 20 20 
    25 5 20 
    20 25 25 
    4  5  5 
    4  4  4

     

  • rocuinneagain

    Member
    September 5, 2022 at 12:00 am in reply to: Accumulators – Access additional list / column

    changing c to prev[c] looks to be what was missing

     

    q)update c2:fills ?[(c1>prev c1) or prev[c]<prev c1;c1;0N] from t 
    c c1 c2 
    -------- 
    30 10 10 
    40 20 20 
    25 5 20 
    20 25 25 
    4  5  5 
    4  4  4 
    1  3  4

     

  • rocuinneagain

    Member
    August 29, 2022 at 12:00 am in reply to: Each Parallel issue

    m'[x] is not parallel execution

     

    There can be cases where the overhead of using multiple threads can make execution slower than single threaded. (.i.e performing many small operations)

    .Q.fc is available to help in some of these cases.

    https://code.kx.com/q/ref/dotq/#qfc-parallel-on-cut

     

    If you measure memory usage with ts note that it only sees usage in main thread. It does not sum usage from all threads. So you cannot compare results directly.

     

    q)\ts {til 10000000;x}'[til 1000] // \ts shows real memory usage 
    5395 134250848 
    q)\ts {til 10000000;x}':[til 1000] // \ts only sees memory usage in main thread - not the sum of threads 
    5612 33408

     

    q)\ts {x}'[til 100000] // Single core 
    8 4746288 
    
    q)\ts {x}':[til 100000] // Slower due to parallel overhead 
    13 4194864 
    
    q)\ts .Q.fc[{x}][til 100000] //Faster way to use multiple cores 
    1 3146512

     

     

  • rocuinneagain

    Member
    August 19, 2022 at 12:00 am in reply to: What is the difference between wj and wj1?

    You are seeing the same results as your t2 has an entry for every second

    If we swap til 9 for 2*til 9 we can create gaps and then you can see the differences where wj will bring in prevailing values but wj1 will not.

     

    t1:([]sym:3#`ibm;time:07:01:01 07:01:03 07:01:05;price:100 101 105); 
    a:101 103 103 104 104 107 108 107 108; 
    b:98 99 102 103 103 104 106 106 107; 
    t2:([]sym:`ibm; time:07:00:58+2*til 9; ask:a; bid:b); 
    c:`sym`time; w:-2 1+:t1.time; 
    o1:wj[w;c;t1;(t2;(max;`ask);(min;`bid))]; 
    o2:wj1[w;c;t1;(t2;(max;`ask);(min;`bid))]; 
    o1~o2; 
    // 0b; 
    
    q)wj[w;c;t1;(t2;(::;`ask);(::;`bid))] 
    sym time price ask bid 
    ------------------------------------------ 
    ibm 07:01:01 100 101 103 103 98 99 102 
    ibm 07:01:03 101 103 103 104 99 102 103 
    ibm 07:01:05 105 103 104 104 102 103 103 
    
    q)wj1[w;c;t1;(t2;(::;`ask);(::;`bid))] 
    sym time price ask bid 
    ---------------------------------- 
    ibm 07:01:01 100 103 103 99 102 
    ibm 07:01:03 101 103 104 102 103 
    ibm 07:01:05 105 104 104 103 103

     

  • From: https://code.kx.com/q/kb/ssl/#tls-client-mode

     

    By default, kdb+ will try to verify the servers certificate against a trusted source, using the certificates from SSL_CA_CERT_FILE or SSL_CA_CERT_PATH to verify the servers certificate. If you don’t wish to verify a servers certificate, set

    $ export SSL_VERIFY_SERVER=NO
  • rocuinneagain

    Member
    August 3, 2022 at 12:00 am in reply to: Is KDB suitable for 24*7 system?

    In the basic kdb+ tick the RDB does do the write at EOD.

    No data is lost, it will buffer in the outbound queue from the TP process.

    As long as you have a safe memory overhead for this buffer then the system will continue to run.

    Queries will also buffer while the RDB is performing the write.

     

    More advanced architectures avoid the RDB being blocked, such as all the KX platforms. .e.g https://code.kx.com/insights/1.1/

  • You could pass it in to a projection

     

    q)f:{[xF] g:{[xG] xG + 2};
           h:{[g;xH] g[xH] + 3}[g];
           g[xF] + h[xF]} 
    q)f[40] 
    87

     

    Or composition in this basic case

     

    q)f:{[xF] g:{[xG] xG + 2};
            h:+[3] g @;
            g[xF] + h[xF]} 
    q)f[40] 
    87

     

     

  • rocuinneagain

    Member
    July 12, 2022 at 12:00 am in reply to: qpython to select columns from table

    Some ways:

    q("{((),x)#y}", numpy.string_("iq"), numpy.string_("t")) 
    q("{value"select ",string[x],"from ",string y}", "iq", "t") 
    q("select %s from %s" % ("iq","t")) 
    q("{?[y;();0b;{x!x}(),x]}", numpy.string_("iq"), numpy.string_("t"))

     

     

    1. Using # https://code.kx.com/q/ref/take/
    2. Using value https://code.kx.com/q/ref/value/
    3. Python string formatting
    4. Functional select https://code.kx.com/q/basics/funsql/

    Functional selects are the most powerful way https://code.kx.com/q/wp/parse-trees/

  • rocuinneagain

    Member
    July 12, 2022 at 12:00 am in reply to: qpython to select columns from table

    See docs for more on how the library handles difference between symbols and strings:

    https://qpython.readthedocs.io/en/latest/type-conversion.html#string-and-symbols

    (Answer above updated to use numpy.string_ to pass symbols as needed)

  • rocuinneagain

    Member
    July 11, 2022 at 12:00 am in reply to: How to extract remote data

    Do you have any details of your specific issue or error?

    This request is vague. https://stackoverflow.com/help/how-to-ask

  • rocuinneagain

    Member
    July 11, 2022 at 12:00 am in reply to: Assigning QDictionary in qpython to kdb

    You want to store it in a global variable in q?

    Then you can use a lambda and :: or set

     

    q('{`mydict set x}', qdict) 
    q('{mydict::x}', qdict)

     

     

Page 16 of 21