KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Confused by serialization result of char list. (Aka “string”)

  • Confused by serialization result of char list. (Aka “string”)

    Posted by MilanGill on February 26, 2025 at 4:35 pm

    I’m investigating the serialization format for various KDB datatypes as part of some work to maintain an existing library for interfacing with a KDB system.

    This library implements a KDB compatible serialization, such that data can be read from and written to disk by both KDB and the language runtime which uses this library.

    I have written the string (list of char) “ab” to a file.

    I did this using the Q commands:

    my_data: "ab"
    `:my_data set my_data


    • This discussion was modified 4 days, 16 hours ago by  MilanGill.
    • This discussion was modified 4 days, 16 hours ago by  MilanGill.
    • This discussion was modified 4 days, 16 hours ago by  MilanGill.
    • This discussion was modified 4 days, 16 hours ago by  MilanGill.
    MilanGill replied 2 days, 15 hours ago 2 Members · 5 Replies
  • 5 Replies
  • MilanGill

    Member
    February 26, 2025 at 4:38 pm

    … this forum is so buggy …

    • MilanGill

      Member
      February 26, 2025 at 4:40 pm

      When I read the data back using a hex editor, I find a very unusual serialization format.

      The first two bytes of data are different to what I would typically expect. `FE, 20` instead of `FF 01`. Most of the other types of data which I have serialized begin with a header containing `FF 01`.

      Following this there is a byte `0A` which I believe corresponds to “list of char”.

      Then there are a sequence of 13 zeros. This doesn’t make much sense to me, since I would expect to see a length. (With value 2.)

      Finally, I see characters “a” and “b”. (Which is the original data.)

  • ss1

    Member
    February 26, 2025 at 6:13 pm

    kdb+ on-disk format is not documented and subject to change. kdb+ should be used to read/write its data.

    A shared object for kdb+ can be written, if a 3rd party needs access to data (e.g. q script reads data from disk & calls c function with data, or vice-versa) , https://code.kx.com/q/interfaces/using-c-functions/ , or kdb+ can use IPC to send the data to/from a 3rd party.

    • MilanGill

      Member
      February 28, 2025 at 5:15 pm

      The reason we are trying to do this is we have other processes written in other languages which we want to be able to read data written to disk by Q/KDB.

      These are obscure languages, and there are no clients directly supported/provided by KDB (the organization).

Log in to reply.