[rocuinneagain]

Some inbuilt functions do make assumptions around how data is stored for segmented databases.

The functions assume each date is stored in the segment entry matching modulus of the date by the number of par.txt entries .i.e round robin.

See related thread:

Solved: .Q.par Doesn’t Provide the Correct Result in the S… – KX Community – 14220

And warning: https://code.kx.com/q/database/segment/#considerations

“Partition data correctly: data for a particular date must reside in the partition for that date.”

 

However for querying and normal operations where these functions are not called there is no such requirement.

 

Symlinking is used often in kdb+ systems for flexibility around storage layouts.

[rocuinneagain]

As this ran without error you should now be able to use your query: >>> pykx.q(‘l s.k_’) pykx.Identity(pykx.q(‘::’)) Should now run: kx.q.sql(‘select * from trades’)

[rocuinneagain]

Can you run:

 

>>> pykx.q('l s.k_')

 

If that still fails check

 

>>> import os 
>>> os.environ["QHOME"] 
'/home/user/folder/venv/lib/python3.10/site-packages/pykx/lib' 
>>> os.listdir(os.environ["QHOME"]) 
['write.q', 'm64arm', 'p.q', 'l64arm', 'l64', 'kxic.k', '_update_marker', 'bq.q_', 'csvutil.q', 'kurl.q_', 'q.k', 's.k_', 'w64', 'qlog.q_', 'm64', 'objstor.q_', 'read.q', 'rest.q_', 'kurl.sidecar.q_']

 

PyKX 1.6.0 includes a utility for inspecting these common areas pykx.util.debug_environment(detailed=True)

[rocuinneagain]

I have tested and 1.6.0 will allow the import to continue after this error.

 

This runs for me:

from prefect import flow
from prefect.deployments import Deployment
import os
os.environ["QARGS"] = "--unlicensed"

import pykx as kx

def setup():
    print("testing pykx")
    masterConn = kx.SyncQConnection( host='localhost', port=5000, timeout=3.0 )
    user = masterConn('.z.u')
    print('User = ' + user.py())
    port = masterConn('5000')
    print('Port = ' + str(port.py()))
    masterConn.close()
    global conn
    conn = kx.SyncQConnection( host='localhost', port=port.py())
    user = conn('.z.u')
    print('User = ' + user.py())

@flow()
def test_pykx():
    setup()
    logger = get_run_logger()
    logger.info("Initializing initial arguments")
    dailyDates = conn('dailyDates:(.z.d-7)+til 7')
    logFile = conn('inLogFile:`')


def deploy():
    deployment = Deployment.build_from_flow(
        flow=test_pykx,
        name="prefect-example-deployment"
    )
    deployment.apply()

if __name__ == "__main__":
    deploy()

 

If you want to run in licensed mode you need to move the import of pykx to avoid a nosocket error

from prefect import flow
from prefect.deployments import Deployment

def setup():
    import pykx as kx
    print("testing pykx")
    masterConn = kx.SyncQConnection( host='localhost', port=5000, timeout=3.0 )
    user = masterConn('.z.u')
    print('User = ' + user.py())
    port = masterConn('5000')
    print('Port = ' + str(port.py()))
    masterConn.close()
    global conn
    conn = kx.SyncQConnection( host='localhost', port=port.py())
    user = conn('.z.u')
    print('User = ' + user.py())

@flow()
def test_pykx():
    setup()
    logger = get_run_logger()
    logger.info("Initializing initial arguments")
    dailyDates = conn('dailyDates:(.z.d-7)+til 7')
    logFile = conn('inLogFile:`')


def deploy():
    deployment = Deployment.build_from_flow(
        flow=test_pykx,
        name="prefect-example-deployment"
    )
    deployment.apply()

if __name__ == "__main__":
    deploy()

 

 

[rocuinneagain]

Can you run: >>> pykx.q(‘l s.k_’) If that still fails check >>> import os >>> os.environ[“QHOME”] ‘/home/user/folder/venv/lib/python3.10/site-packages/pykx/lib’ >>> os.listdir(os.environ[“QHOME”]) [‘write.q’, ‘m64arm’, ‘p.q’, ‘l64arm’, ‘l64’, ‘kxic.k’, ‘_update_marker’, ‘bq.q_’, ‘csvutil.q’, ‘kurl.q_’, ‘q.k’, ‘s.k_’, ‘w64’, ‘qlog.q_’, ‘m64’, ‘objstor.q_’, ‘read.q’, ‘rest.q_’, ‘kurl.sidecar.q_’] PyKX 1.6.0 includes a utility for inspecting these common areas pykx.util.debug_environment(detailed=True)

[rocuinneagain]

https://code.kx.com/q/ref/maps/#each

' operates the same as each in these cases

 

 

(first'') dict 
a| 1 `a 
b| 2 `b 
c| 3 `c 

{first each x} each dict 
a| 1 `a 
b| 2 `b 
c| 3 `c 

(first')dict 
a| 1 
b| 2 
c| 3 

first each dict 
a| 1 
b| 2 
c| 3

 

[rocuinneagain]

To use sql library you will need the correct flag in your license:

>>> print('n'.join(pykx.q('.z.l 4').py().decode().split())) 
insights.lib.embedq 
insights.lib.pykx 
insights.lib.sql ##<<------ Do you see this? 
insights.lib.qlog 
insights.lib.kurl 
insights.lib.objstore 
insights.lib.bigquery 
insights.lib.restserver

 

[rocuinneagain]

To use sql library you will need the correct flag in your license: >>> print(‘n’.join(pykx.q(‘.z.l 4’).py().decode().split())) insights.lib.embedq insights.lib.pykx insights.lib.sql ##<<—— Do you see this? insights.lib.qlog insights.lib.kurl insights.lib.objstore insights.lib.bigquery insights.lib.restserver

[rocuinneagain]

Yes there is a warning on: https://code.kx.com/q/database/segment/#considerations

“Partition data correctly: data for a particular date must reside in the partition for that date.“

[rocuinneagain]

3.11 support has just been added – we’ll get those docs updated. Thanks!

 

https://code.kx.com/pykx/1.6/changelog.html#pykx-160

• Added support for Python 3.11.
  • Support for pyarrow in this python version is currently in Beta.

[rocuinneagain]

PyKX is now available on PyPI

You can pip install pykx without any username/password.

Full details: Installation – PyKX

[rocuinneagain]

1. What version of PyKX as you using?
2. Can you run this example?

>>> import pykx as kx 
>>> kx.__version__ '1.6.0' 
>>> kx.q('trades: ([]sym:`ibm`msft`apple`samsung;mcap:2000 4000 9000 6000;ex:`nasdaq`nasdaq`DAX`Dow)') 
pykx.Identity(pykx.q('::')) 
>>> kx.q.sql('select * from trades') 
pykx.Table(pykx.q(' 
sym mcap ex 
------------------- 
ibm 2000 nasdaq 
msft 4000 nasdaq 
apple 9000 DAX 
samsung 6000 Dow 
'))

 

• Creating q code block with syntax highlighting in a message 
• Adding inline code to your questions & messages

 

[rocuinneagain]

PyKX is now available on PyPI

You can pip install pykx without any username/password.

Full details: Installation – PyKX

[rocuinneagain]

https://code.kx.com/insights/1.5/enterprise/walkthrough/query.html#scratchpad

[rocuinneagain]

These previous discussions have good information:

• https://community.kx.com/t5/kdb-and-q/C-convert-large-decimal-to-KDB/m-p/11049
• https://community.kx.com/t5/kdb-and-q/Big-Integers-larger-than-long/m-p/228
• https://github.com/manifoldfinance/qBigInt
• https://github.com/adotsch/c-extensions-to-q