Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIMS-1531: Property Quicksearch Backend #2306

Merged
merged 4 commits into from
Apr 8, 2024

Conversation

GrahamS-Quartech
Copy link
Contributor

🎯 Summary

PIMS-1531

This adds an endpoint that you can use to search across multiple fields of parcels and buildings and get results that loosely match the provided keyword. You can optionally supply a take param to limit the results (it applies to each repo so the final count is effectively doubled).

I tried to make this work without the queryBuilder, but I couldn't seem to get it working with the integer columns without casting them using TypeORM's Raw( ), and the behavior of that function seems to be pretty bugged.

In the future, maybe we could also optionally limit/expand which fields are searched across but I think this is fine for our current needs.

🔰 Checklist

  • I have read and agree with the following checklist and am following the guidelines in our Code of Conduct document.
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation where required.
  • I have tested my changes to the best of my ability.
  • My changes generate no new warnings.

Copy link

github-actions bot commented Apr 6, 2024

🚀 Deployment Information

The Express API Image has been built with the tag: 2306. Please make sure to utilize this specific tag when promoting these changes to the TEST and PROD environments during the API deployment. For more updates please monitor Image Tags Page on Wiki.

Copy link

codeclimate bot commented Apr 6, 2024

Code Climate has analyzed commit 4287803 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 94.7%.

View more on Code Climate.

Copy link
Collaborator

@dbarkowsky dbarkowsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be easier to do if we made a view? I'm thinking we probably need to change the other materialized view to just a plain view anyway. It doesn't seem to be auto-updating.
A view that just has the fields we need for this fuzzy search could be useful

import { Building } from '@/typeorm/Entities/Building';
import { Parcel } from '@/typeorm/Entities/Parcel';

const propertiesFuzzySearch = async (keyword: string, limit?: number) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should give this a default of undefined. In testing, it required me to fill the second argument with undefined manually.

Comment on lines +6 to +18
const parcels = await AppDataSource.getRepository(Parcel)
.createQueryBuilder('parcel')
.leftJoinAndSelect('parcel.Agency', 'agency')
.leftJoinAndSelect('parcel.AdministrativeArea', 'adminArea')
.leftJoinAndSelect('parcel.Evaluations', 'evaluations')
.leftJoinAndSelect('parcel.Fiscals', 'fiscals')
.where(`parcel.pid::text like '%${keyword}%'`)
.orWhere(`parcel.pin::text like '%${keyword}%'`)
.orWhere(`agency.name like '%${keyword}%'`)
.orWhere(`adminArea.name like '%${keyword}%'`)
.orWhere(`parcel.address1 like '%${keyword}%'`)
.take(limit)
.getMany();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you wanted to do this without the query builder, this seems to be working for me:

 const parcels = await AppDataSource.getRepository(Parcel).find({
    relations: {
      Agency: true,
      AdministrativeArea: true,
      Evaluations: true,
      Fiscals: true,
    },
    where: [
      {
        Agency: {
          Name: Like(`%${keyword}%`),
        },
      },
      {
        AdministrativeArea: {
          Name: Like(`%${keyword}%`),
        },
      },
      {
        Address1: Like(`%${keyword}%`),
      },
      !isNaN(parseInt(keyword)) ? { PID: parseInt(keyword) } : undefined,
      !isNaN(parseInt(keyword)) ? { PIN: parseInt(keyword) } : undefined,
    ],
  });

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should result in PID and PIN only working for exact rather than partial matches though right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, seems to be. Should PIDs and PINs partially match?
Also, I think this is case-specific. There's ILike provided by TypeORM as well, which is case-insensitive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dunno, I assumed they should but I guess you could argue they should be exact. Maybe a Lindsay question.

@GrahamS-Quartech
Copy link
Contributor Author

Would this be easier to do if we made a view? I'm thinking we probably need to change the other materialized view to just a plain view anyway. It doesn't seem to be auto-updating. A view that just has the fields we need for this fuzzy search could be useful

Materialized views do not update until instructed to, so we would need a cronjob or some other mechanism to instruct it to. A regular view could be nice for convenience, though it probably wouldn't have any particular performance benefit.

@dbarkowsky dbarkowsky self-requested a review April 8, 2024 20:35
Copy link
Collaborator

@dbarkowsky dbarkowsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some more experimentation, I think what you had is the only way if we need partial matches on PID/PIN. Limitation of TypeORM.

@GrahamS-Quartech GrahamS-Quartech merged commit 66051c6 into main Apr 8, 2024
5 checks passed
@GrahamS-Quartech GrahamS-Quartech deleted the PIMS-1531-PropertyQuicksearchBackend branch April 8, 2024 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants