KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Timing/Memory surprise on 3 versions of <Conditional function applied to lists> Re: Timing/Memory surprise on 3 versions of <Conditional function applied to lists>

  • rocuinneagain

    Member
    October 12, 2021 at 12:00 am

    This is not really a suitable application of scan. Scan is an accumulator where it is useful when the calculation of a subsequent calculation depends of the result of the previous calculation. This has an overhead as the calculation is computed item by item and each result must be passed back in to the next calculation. As you never use the variable ‘x’ inside the scan it is an indication it is not the best use-case. This blog  has some visualisations which aim to show how scan functions internally.

     

    Your second version is fastest as it operates on ‘x`b’ as a vector rather than inside ‘each’.

    One other possible variation is shown below:

     

     

    q)\ts {?[x`b; last each x`a; first each x`a]}t 
    16 4746672 
    q)\ts {((first;last) x`b)@'x`a}t 
    7 4746640

     

    It’s goal is to avoid calculating ‘last each’ and ‘first each’ for every row.

    Instead it uses each both (‘) to apply first or last after it is known which function is needed.