Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add any existing ext::* scalar to a type's __casttype__ #686

Merged
merged 6 commits into from
Jul 11, 2023

Conversation

scotttrinh
Copy link
Collaborator

Closes #685

The type system resolves ancestor types to scalars calling them "material scalars" and we were not getting any ext::* scalars in our subquery that resolves these.

Some additional updates to make error messages a bit more useful since they currently print [object Object] instead of anything useful 😬

@scotttrinh scotttrinh requested a review from jaclarke July 10, 2023 18:05
@@ -127,9 +127,15 @@ export function $resolveOverload(
throw new Error(
`No function overload found for ${
funcName.includes("::")
? `'e.${funcName.split("::")[1]}()'`
? `'e.${funcName.split("::").join(".")}()'`
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supports submodules including ext::pgvector in this case. It was printing e.pgvector() instead of e.ext.pgvector.cosine_distance().

Note: we're not dropping std and math here any more 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I guess the previous logic was to strip out the std or default as they get aliased into the top level and we didn't have nested modules, but even that's broken because of the cal and math modules (and any non default user modules) 🤦.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I couldn't quickly think of a maintainable way to update this to still remove std and default other than hardcoding a filter in here, but maybe that's fine?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it's fine to always show the full module + function name in error messages. You can probably also shadow the top level aliases of the stdlib functions anyway, and I think it's not worth trying to handle all that correctly just for an error message.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I wonder if it's even worth trying to typescript-ify the function names at all 🤔 I think it would be maybe more clear if the error message was pointing to the actual EdgeQL function that was called rather than trying to construct what it probably looks like at the call site? In other words, just use funcName here. Thoughts?

@scotttrinh scotttrinh changed the title Add exemplary test case Add any existing ext::* scalar to a type's __casttype__ Jul 10, 2023
@@ -15,7 +15,7 @@ export async function setupTests() {
}

async function cleanupData(client: Client) {
await client.execute(`reset schema to initial`);
await client.execute(`delete PgVectorTest`);
Copy link
Collaborator Author

@scotttrinh scotttrinh Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops! I erroneously thought reset schema to initial just deleted all the user defined objects, but it actually... resets the schema to the initial state before any migrations were run. 🤦

Still need to figure out a query that will delete all objects of user-defined types!

@@ -127,7 +127,7 @@ export async function getTypes(
material_scalars := (
SELECT ScalarType
FILTER
(.name LIKE 'std::%' OR .name LIKE 'cal::%')
(.name LIKE 'std::%' OR .name LIKE 'cal::%' OR .name LIKE 'ext::%')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for now, but potentially a future extension may add non base-type scalars. So maybe the filter should check that the type has no non-abstract ancestors?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So maybe the filter should check that the type has no non-abstract ancestors?

Isn't that what AND NOT .is_abstract is doing on line 131? Or do you think there is some way to narrow the search space in ext:: specifically?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's checking that the type itself isn't abstract. I meant checking the ancestors are all abstract as well like this:

select schema::ScalarType {
  name,
} filter not .abstract
  and not exists .enum_values
  and not exists (select .ancestors filter not .abstract)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, maybe I don't understand after all: where would this query go?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of the current material_scalars query.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so don't even filter on the name at all? We don't need the shape here, right? At the very least we also want at least the id since that's what we actually consume. I'll make the change and maybe add a test for std, cal, ext.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaclarke992f2f0 (#686)

Does this seem right? I left the name filtering in, but do you think that's unnecessary?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a little poking around in my local database and yeah, it seems fine to remove the name filtering here. The only thing it was filtering out was cfg::Memory and that seems fine to include in these "materials".

Base automatically changed from test-multiple-versions to master July 11, 2023 17:34
@scotttrinh scotttrinh force-pushed the 685-resolve-ext-as-materials branch from 66b316d to dba99dd Compare July 11, 2023 17:35
Checked the results locally, and they seem sensible. Only additional
scalar is the `cfg::Memory` scalar, which seems fine to include here
@scotttrinh scotttrinh merged commit 1f4fdcf into master Jul 11, 2023
@scotttrinh scotttrinh deleted the 685-resolve-ext-as-materials branch July 11, 2023 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ext::pgvector functions are not able to use scalars extend from ext::pgvector::vector
2 participants