KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ embedPy/ BeautifulSoup Re: embedPy/ BeautifulSoup

  • simon_watson_sj

    Member
    July 16, 2021 at 12:00 am

    Hi All,

    I got a response from Conor at KX about this. It’s due to the python library having output in a nonstandard python data type.

    To get around this, you convert the output to standard Python data types when in python prior to bringing it over to Q. The key to his solution was the python function he defines to do this where you can see he changes the output to a string python type:

    q)p)def func(x):return str(x)

    Also note that the [<] symbol in  )qfunc[<] below is just a better way of telling embedPy to interpret the object in the memory space as a Q object rather than a Python one (which would look like ‘foreign’ when represented by Q if you recall from the docs).

    His full solution is as follows (based on the example_html doc I defined in my initial post):

    q)bs4[`:BeautifulSoup][example_html;”html.parser”]

    {[f;x]embedPy[f;x]}[foreign]enlist

    q)bs4[`:BeautifulSoup][example_html;”html.parser”]`

    foreign

    q)bs4[`:BeautifulSoup][example_html;”html.parser”]

    {[f;x]embedPy[f;x]}[foreign]enlist

    q)bs:bs4[`:BeautifulSoup][example_html;”html.parser”]

    q)result:bs[`:find_all][“a”;`href pykw 1b]

    q)p)def func(x):return str(x)

    q)qfunc:.p.get`func

    q)qfunc[<]each result`

    “<a href=”http://somegreatsite.com“>Link Name</a>”

    “<a href=”mailto:support@yourcompany.com“>support@yourcompany.com</a>”