-
Im working in memory.
I load the file in its 800,000 x 7. Then run a bunch of updates to make the remaining 23 columns (30 or so total).
And then run the wj. And pass back the resulting table.
Im trying to simulate what it would be doing in a tp. Like as more data comes in the wj is going to continuously run slower until it hits max file for the day, 800k rows. So I figured loading the whole file in and doing it all at once would be a decent enough way to test what it’d be doing.