KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ How can i understand the code below

  • How can i understand the code below

    Posted by chan_chenyanj on February 24, 2022 at 12:00 am
    .quantQ.IO.SAVEsPLAYtAB:{[tabPath;pvar;table]
    
      // tabPath -- path where to store the table
    
      // pvar -- variable to sort and index on for fast querying
    
    // table -- name of the table to save
    
    :@[;pvar; `p#] pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath]
    
        get table
    
    };

    how should i understand this function? In particular, what is :@ mean, what is `p#, why ` before sv? the question might be simple but new to q not sure i follow

     

    chan_chenyanj replied 9 months, 1 week ago 4 Members · 3 Replies
  • 3 Replies
  • awilson

    Member
    February 24, 2022 at 12:00 am

    Code reads right to left

    .Q.en[tabPath] get table
    // Fetch table (defined by sym/hsym 'table') and enumerate any symbol columns (to sym file located in directory tabPath)
    (` sv (tabPath; ` ; table; ` ) ) set ...
    // Save the table (splayed) to a directory defined by '/'-separated path (tabPath/table/)
    @[;pvar; `p#] pvar xasc ...
    // Sort by pvar and apply the parted attribute to this column in the table
    : ...
    // Return the result (the path to the saved table)
    ` sv (scalar from vector) joins elements of a filepath.
    e.g.
    q)` sv (`:path/to/dir; `filename)
    `:path/to/dir/filename
    q)` sv (`:path/to/dir; `filename; `)
    `:path/to/dir/filename/

    It is explained here https://code.kx.com/q/ref/sv/#filepath-components

    `p# asserts that the array/column is parted (i.e. common values are adjacent)

    It is explained here https://code.kx.com/q/ref/set-attribute/#grouped-and-parted

  • Laura

    Administrator
    February 24, 2022 at 12:00 am

    A lot to unpack here but lets break it down.

    .quantQ.IO.SAVEsPLAYtAB:{[tabPath;pvar;table]   // tabPath -- path where to store the table   // pvar -- variable to sort and index on for fast querying   // table -- name of the table to save   @[;pvar;`p#] pvar xasc (`sv (tabPath; ` ; table; `)) set .Q.en[tabPath] get table}

    The three arguments are all symbols: respectively a file path, a table column name, and a table name. (It were best to state that in the comments.) Ill come to how we see that.

    Lets set up a small table so we can watch what happens.

    q)atable:([]ab:til 3;cd:`x`y`z;de:string`tom`**bleep**`harry) q)get `atable ab cd de ------------- 0 x "tom" 1 y "**bleep**" 2 z "harry" q)tabPath:`:path/to/table

    The argument to get is the name of the table; the result is the table, which becomes the argument to .Q.en[tabPath].

    That is, binary .Q.en is projected onto tabPath to make a unary. The effect is the same as if we had assigned the result of get table to t and then called .Q.en[tabPath;t]. What .Q.en does is too extensive to discuss here but you can read up on it. The work it does is side effects; its result is the table.

    Which then gets passed as the right argument to set, which is quite a workhorse. Its syntax is overloaded, so we examine its left argument (` sv (tabPath; ` ; table; ` ) ) to see how set is used here.

    The sv keyword is being called in this form, which is how we know that tabPath and table must both be symbols. It returns a filehandle symbol as the left argument of set.

    From this we see that set is being called with syntax til set y // serialize y to fil. It returns its left argument: the symbol filepath to which it has written the table; this becomes the right argument to xasc, which will sort the table on disc by the column named by pvar.

    The result is again a reference to the table, which becomes the argument to @[;pvar;`p#], a projection of Apply At onto second and third arguments pvar and `p#. The effect is to set the partitioned attribute on the column named by pvar. The result of Apply At is the table reference. Your code uses Explicit return : to make that the result of .quantQ.IO.SAVEsPLAYtAB. That is unnecessary: a functions result is the result of evaluating its last (here the single) expression.

  • eohara

    Member
    February 24, 2022 at 12:00 am

    Hi,

     

    The : in :@ is an explicit return (see Function notation Basics kdb+ and q documentation – Kdb+ and q documentation (kx.com)).

    The @ is an amend operator (see Amend, Amend At Reference kdb+ and q documentation – Kdb+ and q documentation (kx.com)).

    `p# is the parted attribute (see Tables | Q For Mortals | A textbook forkdb+ and the q programming language – Q for Mortals (kx.com)).

    ` is the first argument of sv here as we wish to build a filepath to a splayed table (see sv scalar from vector | Reference | kdb+ and q documentation – Kdb+ and q documentation (kx.com)).

    To bring that altogether, what is happening in your function is:

    1. get table
      1. This returns the table data for the given table name
    2. .Q.en[tabPath] get table
      1. This enumerates the output of 1) against a sym file in “tabPath”, and returns the enumerated table
    3. (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table
      1. Writes the table from 2) to the filepath, and returns the filepath if successful e.g. if tabPath is `:/home/kx/db and table is `trade, then (`  sv (tabPath; ` ; table; ` ) ) will return `:/home/kx/db/trade/
        1. The trailing slash after trade means the table will be saved splayed
    4. pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table
      1. Sorts the written table in 3) on the column in the pvar variable
    5. :@[;pvar; `p#] pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table
      1. Amends the sorted table on disk, adding a parted attribute to the column in the pvar variable

     

Log in to reply.