KX Community

Find answers, ask questions, and connect with our KX Community around the world.
KX Community Guidelines

Home Forums kdb+ md5 – getting the original string back. Re: md5 – getting the original string back.

  • Laura

    Administrator
    June 16, 2023 at 12:00 am

    The other comments address well that md5 is a hashing function which is implicitly non-reversible. It’s also not a very secure hashing algorithm as it’s vulnerable to both collision attacks and length extension attacks (more reading here for interest).

    Very much dependent on your use case, if there’s a known fixed list of strings that the original messages can be you can “decode” the data. Take the scenario where there are 2 users in a message chat, Alice and Bob, and we have a table that has those users md5 hashed to hide their identity, but we know prior that the users are Alice and Bob. You can see who sent what like this:

    t:([]time:10#.z.p;users:10?`Alice`Bob;message:10?10); 
    t:update users:{md5 x} each string users from t; 
    lookup:(md5 "Alice";md5 "Bob")!`Alice`Bob; 
    t:update users:lookup[users] from t;

    This can be extended to any number of users (and other use cases) but has the prerequisite of there being a known fixed list of users.

    If however, your use case is that you are storing large amounts of data and don’t want a lookup like this, I’d suggest not using MD5 for your encryption at rest but encrypting using public/private keys so you are able to decrypt your data while still having security at rest. You could use say OpenSSL and integrate with KDB/Q (example of encrypting/decrypting data here).

    Unfortunately (or rather fortunately) if you don’t know what the values are in your table and are trying to retrieve them knowing they’ve been md5 hashed this isn’t possible outside of brute-force attacks, the likes of which underpin password cracking.

    I hope one of these responses addresses your use case!