Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Locate Improvement ability to give hints where to look for pattern #18

Open
IlfirinPL opened this issue May 23, 2016 · 3 comments
Open

Comments

@IlfirinPL
Copy link

I think in some big resolution to improve performance it would be nice to give option to give hints where to start looking for pattern, when hints failed resume using standard approach.
I see two solutions
first give rectangle where to start to search (10,10| 200|200)
or general information about sector like TOP, CENTER,BOTTOM, LEFT, RIGHT ex. BOTTOM -RIGHT or TOP-CENTER.
And Information about which display.

@Tattoo
Copy link
Contributor

Tattoo commented May 24, 2016

A nice idea, although there are many implementation concerns:

  • Can pyautogui even target other screens than the main screen monitor?
  • How to deal with the fact that pyautogui handles coordinates differently between OSes?
  • What should happen if user gives incorrect coordinates/sector? Should the library then just start searching whole screen area? What is the worst case performance hit in this case?
  • What is the performance gain in best case?
  • How to enhance debugging that it's clear where the reference image was not found (in the hinted coordinates/sector or the whole screen)?
  • Should this be a separate keyword? At least it is very backwards incompatible to the current implementation. You could do this by having different "modes" to the locating, but that pattern is usually not a good idea because it sacrifices the usability of the library.

@IlfirinPL
Copy link
Author

I think it would be best option to add kwargs args to locate.
I see the support for multiple screen its no roadmap so maybe it will have to wait (https://github.com/asweigart/pyautogui/blob/master/docs/roadmap.rst)

About when selector failed using hint i think i should do fallback to whole search sinice its hint only.
Performance gain i think since area to expect to find is 1/9 (selecting region by example top-right) of typical search, decrease will be great event worst case scenario its not that bad 10/9 ( 111% typical use). So i think its much to gain.

There are also two other options that I notice when I start looking for some answers.
http://pyautogui.readthedocs.io/en/latest/screenshot.html

There are to options that could be useful also
Grayscale Matching (useGray=False)- with this enable its decrease search by around 30%
Pixel Matching( tolerance ) (tolerance=0)- it could help with some cases when colors change a little, much slower

@Tattoo
Copy link
Contributor

Tattoo commented May 24, 2016

Thanks for your input but we will measure the performance hits or gains empirically while implementing, as well as answer to the rest of the questions.

Grayscale matching in pyautogui is next to useless. In my experience, it just leads to not being able to find anything. Pixel matching is completely different thing and the way I understand it, not applicable to reference image recognition in any way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants