KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ Process requests before continuing with script

  • Process requests before continuing with script

    Posted by victor_wong on August 4, 2024 at 7:22 am

    I would like to start two data feeds in a script and wait for them to connect before continuing, like so:

    h:(); .z.po:{h,:x};
    system each {"q ",x," -p 0W &"} each ("feed1.q";"feed2.q");
    / in feed{1,2}.q: hopen `::<port>
    while[2>count h;system"sleep 1"];
    / continue script...

    However, the connections aren’t accepted while the script is running, so the while loop doesn’t terminate. Is there a way to process the connection requests before continuing with the script?

    And is it possible to start non-interactive q processes (like above) that don’t terminate without opening a port?

    pgyorok replied 1 month, 3 weeks ago 3 Members · 4 Replies
  • 4 Replies
  • pgyorok

    Member
    August 5, 2024 at 9:11 am

    In order to process incoming connections, the q interpreter must be in its main loop, which will not happen if it is executing code (e.g. the while loop). So if you have code that must be run after such interaction, you will have to come up with a dependency system to execute the code after certain conditions are fulfilled. One example of this is inithook (kdb/q/inithook at main · finos/kdb · GitHub).

    The q process will stay alive as long as it has at least one input handle open – that could be the IPC port but also stdin, so if you make sure that your process has a stdin (e.g. by running it in docker with the -it options) it will stay alive. However there should not be a problem with opening a port just to keep it alive. Use \p 0W to open a random port without the risk of hitting a “port already in use” error. If the concern is random processes connecting, you could use firewall rules to block incoming connections or set the .z.ac/.z.pw handlers to always return false so all connections are rejected.

  • rocuinneagain

    Member
    August 6, 2024 at 10:46 am

    You can use a timer and wrap the rest of the code in a function.

    h:(); .z.po:{h,:x};
    {system "q ",x," -p 0W &"} each ("feed1.q";"feed2.q");
    .z.ts:{if[2=count h;system"t 0";main[]]}
    \t 1000
    main:{[] show "Rest of code" }

    Or move the code out to a different file and only load that once the handles are open:

    h:(); .z.po:{h,:x};
    {system "q ",x," -p 0W &"} each ("feed1.q";"feed2.q");
    .z.ts:{if[2=count h;system"t 0";system"l main.q"]}
    \t 1000
  • victor_wong

    Member
    August 10, 2024 at 5:59 am

    Thank you both for your detailed explanations and helpful advice. I also tried to see if I could use the deferred response feature (-30!) to suspend execution and return to the main loop, but I couldn’t find a way to establish a connection to the process’s own port; sending a message to the 0 handle doesn’t trigger .z.pg.

    • pgyorok

      Member
      August 12, 2024 at 9:17 am

      Handle 0 triggers .z.ps, not .z.pg.

Log in to reply.