Pregraphics formatters: IncrementalFormatter
#447
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
Ensure `.boundingRect()` calcs and `.draw_last_datum()` do geo-sizing based on source data instead of presuming some `1.0` unit steps in some spots; we need this to support an epoch index as is needed for overlays. Further, clean out a bunch of old bounding rect calc code and add some commented code for trying out `QRectF.united()` on the path + last datum curve segment. Turns out that approach is slower as per eyeballing the added profiler points.
Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
goodboy
added
viz
(sub-)systems
goodboy
commented
Jan 30, 2023
| # print(f'{array_key} append len: {append_length}') | ||
| new_x = x_out[-append_length - 2:] # slice_to_head] | ||
| new_y = y_out[-append_length - 2:] # slice_to_head] | ||
| print(f'{array_key} append len: {append_length}') |
There was a problem hiding this comment.
FYI this was a failed attempt at trying to get QPainterPath "appends" working by using graphics obj apis do do the appends at the path level instead of via a full redraw).
I'm leaving the commented code in for the future whenever we decide to take another try at it.
guilledk
approved these changes
Jan 30, 2023
77 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In support of #420 (and the general effort towards having sanely sized PRs for others to review 😂,) this introduces a "middleware" layer for our curve rendering graphics engine.
The main idea here is to formalize machinery for conducting post
numpy.array(normally read fromShmArraybuffers), pre graphics-QPainterPathformatting and incremental update, the main purpose of which is to speed up curve rendering cycles on real-time updated data flows.The main abstractions to review include:
piker.ui._pathops.IncrementalFormattertype hierarchy from ab1f155 which introduces a the incremental update subsystem as mentioned above and which is primarily used from ourui._flow.Rendererlayer: the subsys which does the work of transforming shm buffer data to the finalQPainterPathgraphics objects expected byQt.BarItemsandCurve) to remove a bunch of old cruft-code and re-implement.boundingRect()methods to be sample step agnostic by relying on geometry calcs alone.Further details about the (rather large) set of changes can be seen in commit msgs.