KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Each Parallel issue

  • Each Parallel issue

    Posted by k-mambo on August 29, 2022 at 12:00 am

    Follows https://code.kx.com/q/basics/peach/,

    The result of m':[x] is exactly the same as m'[x]. If no secondary tasks are available, performance is the same as well.

    but m’:[x] more fast and less memory used. Is not m'[x] parallel execuation??…

    k-mambo replied 8 months, 1 week ago 2 Members · 1 Reply
  • 1 Reply
  • rocuinneagain

    Member
    August 29, 2022 at 12:00 am

    m'[x] is not parallel execution

     

    There can be cases where the overhead of using multiple threads can make execution slower than single threaded. (.i.e performing many small operations)

    .Q.fc is available to help in some of these cases.

    https://code.kx.com/q/ref/dotq/#qfc-parallel-on-cut

     

    If you measure memory usage with ts note that it only sees usage in main thread. It does not sum usage from all threads. So you cannot compare results directly.

     

    q)\ts {til 10000000;x}'[til 1000] // \ts shows real memory usage 
    5395 134250848 
    
    q)\ts {til 10000000;x}':[til 1000] // \ts only sees memory usage in main thread - not the sum of threads 5612 33408

     

    q)\ts {x}'[til 100000] // Single core 
    8 4746288 
    
    
    q)\ts {x}':[til 100000] // Slower due to parallel overhead 13 4194864
    q)\ts .Q.fc[{x}][til 100000] //Faster way to use multiple cores 1 3146512

     

     

Log in to reply.