jattwick
Forum Replies Created
-
Thank you! Another related question – after doing the initial query on the partition, when I do
100#rawData
That still worked (and was fast, and did not increase the memory usage). If you’re saying we are not actually loading the partition into memory, shouldn’t this not work? Is there a way to partially load the columns in the partition?
-
Thank you ! the lsq won’t fit our use case since we want the coefficients to be strictly greater than zero.
We’ll have a look at embedPy – was just hoping somebody had possibly written a native kdb implementation already.
-
The weird thing for me is we have a process that writes in this fashion, running several times a day. In some cases, it fails (which makes sense given the explanation you have given), but it does work in some cases as well. might lean towards just re-architecting the datatypes for this table.
-
What precisely does at a process level mean? Does that mean only if a single port is multithreading (i.e. peaching and writing to the sym file the lock will be used)? Or I have two distinct ports both writing to the sym file, this built in locking won’t work?
My hypothesis is in the latter (multiple ports writing to the sym file at once), there is no locking of the sym file and we are seeing some concurrent writes. Solution for us to write our own lock that can handle these cases.