Replies: 3 comments 1 reply
-
I would add that when I try to replicate this example here:
I get this error: |
Beta Was this translation helpful? Give feedback.
-
Not without recompiling Perspective from source. We perform this transformation for Perspective's "Export to HTML" feature, as browsers restrict the use of the Web Worker API from
If your dataset is so large that simply passing it in this manner is visibly impacting rendering, I do not think inlining the engine will help. Loading large JSON into Perspective's engine and back may or may not be fast, but it is many times slower than what it was designed for, e.g. loading data via Arrow format and only deserializing from the query what is visible on the screen as the user interacts via virtualization, and surely slower than the cost of copying the value between workers. Data encoded to |
Beta Was this translation helpful? Give feedback.
-
Thanks for your help and support. Even when passing the data through the main thread with json objects, perspective table is still faster than duckdb (which was also leaking memory for me) and all other solutions for me. It's just that as you can see I have tried many strategies, and using the perspective API directly seemed like the simplest and best solution I could think of since I already have a worker performing analog actions. I am building a react app that handles high frequency updates for 80 000 rows * 20 columns. It is quite fast already, but indeed the functionnalities are complex and I am building the app with the worst conditions possible (all the data at the same time in multiple unique components) to make the app as fast as possible. I am indeed using virtualization. I will go through the examples again and thank you, indeed I started using ArrayBuffer yesterday and it helps to gain speed (thanks for the demo). Thanks again and have a great day! |
Beta Was this translation helpful? Give feedback.
-
Hi,
Thanks for the wonderful library. I have a complex react setup where I am trying to use perspective tables in my own worker to perform complex queries with big data.
My problem is that I cannot use the perspective.worker() in a background thread (i.e. my own worker) so the data needs to pass through the main thread and it is blocking my app rendering.
Is there any way to simply use the perspective table and view, in javascript, without spawning the worker? I thought using perspective.table() would do it but no.
I have been trying to solve this for more than a week, so if you know how to resolve my problem, a line of code or two showing the solution would be really appreciated.
These are the things I have tried so far and why it didn't work:
comlink:
forces to pass through the main thread
MessageChannel:
the perspective worker is a custom worker and cannot be assigned new channels or commands
use perspective.table() without the worker:
that didnt work because table() is not a function of the perspective module, it seems to only be accessible by the worker.
use the prespective api (table_api.js):
I thought that was a very good idea, but it seems even the perspective api table() command is waiting for a worker as an argument! see here:
try to create the perspective worker in the main thread and pass it through useContext:
I want to use it in a non-component file and it was giving me an error, this is the least desirable way to make it work, it will be slow and pass through the main thread.
The ideal scenario (I think) would be to be able to use directly the table and view commands in a api command fashion, but I think these commands are expecting a worker as an argument (by looking in the files "table_api.js" and "view_api.js").
Thank you!!
Beta Was this translation helpful? Give feedback.
All reactions