Re: Need help to read hdf. file written in python to kdb+?

rocuinneagain

Member

June 2, 2022 at 12:00 am

If you run through the example in q then inspect the created file in python you will see what the interface expects

https://code.kx.com/q/interfaces/hdf5/examples/#create-a-dataset

The table columns are stored individually inside groups

>>>data = h5.File('experiments.h5', 'r') 
>>> data['experiment2_tables'] <HDF5 group "/experiment2_tables" (1 members)> 
>>> data['experiment2_tables/tab_dset'] <HDF5 group "/experiment2_tables/tab_dset" (5 members)> 
>>> data['experiment2_tables/tab_dset/class'] <HDF5 dataset "class": shape (10000,), type "<i2">

(The filename must end in ‘.h5’)

For you to store data from Python you should match this style using groups for columns.

import h5py as h5 
import pandas as pd 
df = pd.DataFrame({"AA":[1, 2], "BB":[3, 4], "CC":[5, 6]}) 
f = h5.File('forKX.h5','w') 
project = f.create_group("project") 
table = project.create_group("table") for col in df.columns: table[col] = df[col].to_numpy() 
f.close()

kdb+ still does not know you intend this data to be a table.

As outline in the docs https://code.kx.com/q/interfaces/hdf5/hdf5-types/#tables-and-dictionaries

Attributes would be needed.

Without the attributes you can reshape in to a table like so:

q){flip x!{.hdf5.readData["forKX.h5";"project/table/",string x]} each x}`AA`BB`CC 
AA BB CC 
-------- 
1 3 5 
2 4 6

KX Community

rocuinneagain