KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ the advantage of random segmentation Re: the advantage of random segmentation

  • rocuinneagain

    Member
    January 2, 2023 at 12:00 am

    It depends on the data/access-patterns/hardware. May make sense in your use case but not in others.

    If you had a DB with a very large number of unique values in the `p# field then it can make sense to group them together to make the attribute more efficient and have the disk doing more sequential reads which are faster.

    There are some notes on estimating attribute overheads on:

    https://aquaq.co.uk/adventure-in-retrieving-memory-size-of-kdb-object/

     

    In some IoT schemas where there can be millions of unique identifiers even these approaches would not be enough and introducing a hash column/lookup can be needed to not have the DB be slowed by larger attributes and small random read reads.