[hari__]

the dimension size for the above one is 768

[hari__]

Schema config: “

'columns': [{
'name': 'stock_name',
'pytype': 'str'
},
{
'name': 'Date',
'pytype': 'str'
},
{'name': 'embeddings',
                       'vectorIndex': {'dims': len(tables[table_name][stock_names[0]]['embeddings'].iloc[0]), 'metric': 'CS', 'type': 'flat'} 
                    }

I’m using kdb ai cloud version, and the outliers are negative values below -30e-3. No major outliers as this is an output of distilbert.  The distances look off, manually calculating it, it seems like its not scaled (dot product and cosine similarity give me similar distances). The distance is as high as 60