KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Large Scale WindowJoins Questions Re: Large Scale WindowJoins Questions

  • Laura

    Administrator
    November 30, 2022 at 12:00 am

    I have 800,000 rows of data in a table with the following meta:

    c              | t f a 
    ---------------| ----- 
    col1           | n     
    col2           | s     
    col3           | d     
    mmm3           | f     
    col4           | f     
    col5           | s     
    col6           | i     
    col7           | i     
    col8           | j     
    col9           | f     
    col10          | j     
    dt             | p   s 
    col11          | f     
    col12          | f     
    col13          | j     
    col14          | j     
    col15          | f     
    col16          | f     
    col17          | f     
    col18          | f     
    col19          | f     
    col20          | f     
    col21          | f     
    col22          | f     
    col23          | j     
    col24          | j

    Im attempting to get a rolling 5minute min/max using wj. (i recognize the window isn’t such for 5minutes, im just writing up an example)

    f:`dt; w:(-1000000; 0)+:data[`dt];
    data: wj[w;f;data;(data;(max;`mmm3);(min;`mmm3))];

    The wj works just fine, but takes insanely long to complete.