KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums KDB.AI KDB.AI Cosine similarity Re: KDB.AI Cosine similarity

  • hari__

    Member
    January 19, 2024 at 12:00 am

    Schema config: “

    'columns': [{
    'name': 'stock_name',
    'pytype': 'str'
    },
    {
    'name': 'Date',
    'pytype': 'str'
    },
    {'name': 'embeddings',
                           'vectorIndex': {'dims': len(tables[table_name][stock_names[0]]['embeddings'].iloc[0]), 'metric': 'CS', 'type': 'flat'} 
                        }
    I’m using kdb ai cloud version, and the outliers are negative values below -30e-3. No major outliers as this is an output of distilbert.  The distances look off, manually calculating it, it seems like its not scaled (dot product and cosine similarity give me similar distances). The distance is as high as 60