How I rewrote my spreadsheet from ReactTable to Vanilla JS
In this video I’m showing ReactTable implementation of my spreadsheet for , what problems I did have with it and how I solved it by rewriting the table in Vanilla JS.
In AMS Pilot I have a spreadsheet, very similar to Google Spreadsheet, but customized to the target audience needs.
I wrote the original version with ReactTable(Tanstack Table), it seems a very good library for that use case.
Problem 1. Correctness
Filtering functionality didn’t work as expected. Because filtering works ‘before’ the rows grouping happens, it didn’t do what the users expected. The users expect to be able to filter what they see, and they see the aggregated data, so that’s what they expect to be able to filter.
It was possible to fix of course. But it was so hard, that it would require a lot of work on my side to get done correctly. And especially make it the way so it doesn’t hurt the performance.
Problem 2. Performance
Performance was not great. The spreadsheet sometimes has around 60k rows, and since React is immutable, if I want to change a single row, I have to change the entire array and it will cause all sorts of recalculations.
And of course you can say that I could introduce pagination of some sort. But the spreadsheet has to recalculate the aggregated data every time something changes. So it has to know about all the data, even if it doesn’t display it(and in fact, it doesn’t display all the rows at once, it uses Row Virtualisation technique.)
Solution. Vanilla JS
So I sat down and rewrote the entire thing with Vanilla JS. You can see the end result on the video.
Correctness was solved, because I wrote it the way I wanted. The filters are working the way the customers expect them to work.
Performance was solved as well. You can see on the video how much faster it is compared to React version.
SOA(Structure of Arrays). Because I didn’t use table library anymore, I could afford to structure the data in a way which is more compact in memory, and more importantly more efficient for CPU cache. If you want more details on this, I recommend Andrew Kelley’s great talk on Data Oriented Design.
Reaggregate use case
In the video I show the actual code, specifically for a use case of coloring a row.
Here I want to touch on some other use cases and how they are implemented in Vanilla JS, because they have some interesting implementation details.
On the top page I have some controls that impact everything else.
Whenever a user changes date range or top filter, then the whole data set needs to be recalculated, because the metrics will be different now.
Here is how the code for date picker change looks like:
The interesting thing is, steps 1, 2, 3 happen only in memory. And the actual live DOM touching happens only in step 4.
Let’s go through each of the steps in more details.
Recreate elements
I have this variable called elements. It contains all the DOM top-level nodes, that I need access to.
By having this variable, I don’t need to ever query the DOM. I can just refer to memory reference and that’s it.
So recreating new elements is very simple now:
I just take the old elements, abort all the listeners on it with AbortController*, and create new elements in memory.
*- I use Abort Signals when adding listeners to the DOM like this:
*- I’m still not sure if I need to do it, since the Internet says that modern browsers remove listeners automagically whenever a DOM element is removed. From my profiling I see that the browser is not as efficient at this as I am when I do it explicitly.
Refiltering data by date
This is self explanatory, I filter the data by date range. I use library Arquero to do that in an efficient manner.
Reaggregate and render in memory
I fold the code to avoid too many details and to be able to show it on the screen, because the entire function is too long.
The idea here is just aggregate the data and render it in memory.
It’s very straightforward. The code is stupidly simple top-to-bottom procedure.
I don’t use any OOP or layers of unnecessary abstractions. But that’s a too long topic to get into here.
Replace elements in DOM
Replacing in-memory elements into the DOM is one line. I specifically use browser’s native replaceChildren(), because it’s super fast.
As I understand it, it does roughly the same stuff what React is doing by diffing the DOM and apply the difference. But the native implementation actually replaces the entire DOM, and is implemented in C++ instead of JS.
My understand may be incorrect, but so far it stands strong, because in practice I see it’s running very fast, faster than React version.
By having the DOM touching being isolated to a single line, browser layout and repaint is isolated in a single place as well, and it’s fast:
Next use case: Row sorting
This is another use case that I think the browser’s API is very good at. Row sorting, especially if we consider that the rows are aggregated and can be expanded/collapsed with children inside, seems to be a pretty expensive operation.
But I made it work fairly fast and here is how. I used appendChild() native method. I used a special case of this function:
Let’s start with the high-level implementation of sorting:
Now let’s go through each step.
Apply changes to state
I just calculate what will be the next column and sort direction. Pretty self-explanatory.
Save on the server
Saving the state is very simple as well. Just using the Fetch API. The utility function fetchExpectOk() does what you expect it to.
Render thead cells
Here I just re-render thead cells for columns that are affected. So that the little ‘arrow’ icon is displayed:
Render tbody rows
I sort the rows and put them into <tbody> in a correct order. And the appendChild() function is doing its magic here by actually moving <tr> elements, and not ever removing them from the DOM.
The end result is working very fast.