You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a interface for DmlHandler and implement a new InMemoryDmlHandler which can mock the functionality of DmlHandler using in memory data, for the purposes of local indexer testing. This is necessary because unit testing an indexer is expected to be quick, with high iteration speed. This is not possible when running against an actual Postgres instance in a docker as tear down takes time, set up takes time, and solutions like In Memory PG have various attributes we do not want, such as custom parsers. The first implementation of this can be rough, to cover main features. Later, it can be extended or scrapped based on our needs.
Each DmlHandler function should work, return data, and error out where possible, emulating the behavior of using an actual Postgres instance.
The text was updated successfully, but these errors were encountered:
Leaving context on #809 before I go on a 2 week leave.
The code is in a pretty messy in-between state right now. I've done a couple iterations on it to try and find the right abstraction. As a result, most of the code is loose with typing and names. So, I'll first cover things at a high level and then mention the specific decisions made so far in the code.
The data that the InMemoryDmlHandler needs to be stored in a form such that we can:
Enrich input data for fields that need it (Such as serial data, or columns with default values)
Know what columns are primary keys for uniqueness constraints
Search throw rows and column values for select or upsert matching
Return modified data
Be reasonably extendable for any potential features to implement, like type assertions.
So, I broke the problem down into three different problems:
Collecting specification for tables
Enriching data
Performing actions on data
The 1st should be a responsibility of some higher level class which performs this on construciton, with the schema AST as input. The AST contains all information we need, and we just need to continue improving the storage of the information in the AST for our needs. So far, I went with a simple interface, but perhaps we can represent the data in a more extensible form, like configurable classes representing columns.
The 2nd needs too be done at a class level, as things like counters for serial data, need to be available to any row or class when enriching. Maybe this should be its own class since the scope of work here will grow in the future. It could even own validation of input data.
The 3rd I think should be properties of the row, in conjunction with private functions of the data class. The row should have helpful functions like primaryKey() which can be used to compare against other rows. And a private function in the data class which then uses them for things like find.
That is my current state of thought regarding the InMemoryDmlHandler. The code is roughly along those lines, but with somewhat messy abstractions and names that need renaming. But, the insert and select functions to actually work, and I have a unit test for them.
For next steps, I would say this work can just sit back until I come back. But, most of the underlying feature work is there, so implementing the remaining functions shouldn't be too hard. So, if an urgency is there, someone can pick this up and get it to work with a bit of trouble, and start using it for getting a local indexer environment set up.
Create a interface for DmlHandler and implement a new InMemoryDmlHandler which can mock the functionality of DmlHandler using in memory data, for the purposes of local indexer testing. This is necessary because unit testing an indexer is expected to be quick, with high iteration speed. This is not possible when running against an actual Postgres instance in a docker as tear down takes time, set up takes time, and solutions like In Memory PG have various attributes we do not want, such as custom parsers. The first implementation of this can be rough, to cover main features. Later, it can be extended or scrapped based on our needs.
Each DmlHandler function should work, return data, and error out where possible, emulating the behavior of using an actual Postgres instance.
The text was updated successfully, but these errors were encountered: