Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sniper v2.1 #5522

Merged
merged 9 commits into from
Sep 19, 2016
Merged

Sniper v2.1 #5522

merged 9 commits into from
Sep 19, 2016

Conversation

YvesHenri
Copy link

@YvesHenri YvesHenri commented Sep 18, 2016

This is an upgrade to the PR 5443 (#5443).

  • Social feed sharing is now enabled by default, since it helps the community and has no drawbacks. Whether to use its data or not has to be set in the new Sniper task config (mode = social).
  • You can now specify multiple sources (URLs) to gather pokemon data from, including the old PokemonGo-Map. Note that they have to reply JSON objects.
  • MoveToMapPokemon was kept because some users still use the "walk-to-snipe" feature. Deleting it in the near future should have no impact on this new Sniper.

Related issues: #3672 #3656 #3666 #3355 #3278 #3226 #5523

…Ls to gather pokemon information.

Social sharing is enabled by default. Whether to use it or not, you'll need to specify in the Sniper task config.
@solderzzc
Copy link
Contributor

solderzzc commented Sep 18, 2016

👍

Approved with PullApprove

@@ -32,6 +32,10 @@
- [Description](#description)
- [Options](#options)
- [Example](#example)
- [Sniping _(Sniper)_](#sniping-_-sniper-_)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why, but navigating from the links does not jump to the new sniper section, and the sublinks go to the previous one (for example, new sniper description goes to old sniper description).

- `missing` - Order by the target's pokedex missing status.
- `threshold` - Order by the threshold you have specified in the `catch` list.
- `expiration_timestamp_ms` - Order by the expiration time.
* `sources` - This should map a JSON param values from a given url. For example: different urls will provide different JSON response formats. If a param does not exist, you DO NOT have to specify it! Map bellow their corresponding values:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: bellow -> below

* `mode` - The mode on which the sniper will fetch the informations. (default: social)
- `social` - Information will come from the social network.
- `url` - Information will come from one or multiple urls.
* `bullets` - Each bullet corresponds to an attempt of catching a pokemon. (default: 1)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add an example here. I still don't understand what that means.
i understand that I will try to catch it only once (unlikely that this is its meaning :) ), and next option ensures that I won't miss.

"key": "results",
"mappings": {
"iv": { "param": "iv" },
"name": { "param": "namez" },
Copy link

@lucasbento lucasbento Sep 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo here, must be { "param": "name" }.

Also needs to add id param, e.g.: "id": { "param": "id" }

Copy link
Author

@YvesHenri YvesHenri Sep 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely. Accidently pasted the config I was using to test whether it would recognize wrong mappings or not (not a type tho). 😄

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A mapping should only be added if it really exists in the response AND the value is correct. A special case is the pokesnipers; it has an ID but the value is somewhat a custom ID, not the pokedex id, so just dont put it.

@lucasbento
Copy link

This is not working with https://pokesnipers.com/api/v1/pokemon.json, it throws this error after successfully fetching the source:

Traceback (most recent call last):
  File "pokecli.py", line 843, in <module>
    main()
  File "pokecli.py", line 202, in main
    bot.tick()
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/__init__.py", line 718, in tick
    if worker.work() == WorkerResult.RUNNING:
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 341, in work
    targets = self._get_pokemons_from_url()
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 428, in _get_pokemons_from_url
    return self._parse_pokemons(results_hash_map.values())
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 373, in _parse_pokemons
    pokemon_name = Pokemons.name_for(pokemon.get('pokemon_id'))
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/inventory.py", line 456, in name_for
    return cls.data_for(pokemon_id).name
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/inventory.py", line 452, in data_for
    return cls.STATIC_DATA[pokemon_id - 1]
IndexError: list index out of range
[2016-09-18 10:13:15] [MainThread] [sentry.errors] [ERROR] Sentry responded with an error: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128) (url: https://app.getsentry.com/api/90254/store/)
Traceback (most recent call last):
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync
    super(ThreadedHTTPTransport, self).send(data, headers)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send
    ca_certs=self.ca_certs,
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open
    response = self._open(req, data)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open
    '_open', req)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open
    return self.do_open(ValidHTTPSConnection, req)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1057, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1097, in _send_request
    self.endheaders(body)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 895, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
[2016-09-18 10:13:15] [MainThread] [sentry.errors.uncaught] [ERROR] [u'IndexError: list index out of range', u'  File "pokecli.py", line 843, in <module>', u'  File "pokecli.py", line 202, in main', u'  File "pokemongo_bot/__init__.py", line 718, in tick', u'  File "pokemongo_bot/cell_workers/sniper.py", line 341, in work', u'  File "pokemongo_bot/cell_workers/sniper.py", line 428, in _get_pokemons_from_url', u'  File "pokemongo_bot/cell_workers/sniper.py", line 373, in _parse_pokemons', u'  File "pokemongo_bot/inventory.py", line 456, in name_for', u'  File "pokemongo_bot/inventory.py", line 452, in data_for']
Sun Sep 18 10:13:15 BRT 2016 Pokebot  Stopped.
Press any button or wait 20 seconds to continue.

@YvesHenri
Copy link
Author

@lucasbento Please show me more logs. Did it successfuly validate this source?

@lucasbento
Copy link

@YvesHenri: yes, it did.

@lucasbento
Copy link

@YvesHenri here is all the important part of the log:

[2016-09-18 10:38:53] [MainThread] [PokemonGoBot] [INFO] Starting bot...
[2016-09-18 10:38:54] [MainThread] [CollectLevelUpReward] [INFO] Received level up reward:
[2016-09-18 10:38:54] [MainThread] [IncubateEggs] [INFO] Eggs incubating: [3.95/5.0 km] (Eggs left: 8, Incubating: 1)
[2016-09-18 10:38:54] [MainThread] [ShowBestPokemon] [INFO] *Best Pokemons* (Dragonite, CP 2332, IVCP 0.94, DPS 15.04, HP 126/126) | (Snorlax, CP 1862, IVCP 0.64, DPS 15.76, HP 200/200) | (Vaporeon, CP 1749, IVCP 0.91, DPS 11.1, HP 171/171) | (Vaporeon, CP 1677, IVCP 0.95, DPS 16.06, HP 165/165) | (Lapras, CP 1600, IVCP 0.92, DPS 12.66, HP 158/158)
[2016-09-18 10:38:54] [MainThread] [PokemonCatchWorker] [INFO] Skip ignored Slowpoke! (CP 451) (Potential 0.6) (A/D/S 11/10/6)
[2016-09-18 10:38:58] [MainThread] [    Sniper] [INFO] Fetching pokemons from the sources...
[2016-09-18 10:38:59] [MainThread] [    Sniper] [INFO] After merging, we've got 24 results
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO]
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Ran for 0:00:14
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Total XP Earned: 0  Average: 0.00/h
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Travelled 0.00km
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Visited 0 stops
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Encountered 0 pokemon, 0 caught, 0 released, 0 evolved, 0 never seen before ()
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Threw 0 pokeballs
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Earned 0 Stardust
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Hatched eggs 0
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Next egg hatches in 1.05 km
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO]
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Highest CP Pokemon:
[2016-09-18 10:39:01] [MainThread] [       cli] [INFO] Most Perfect Pokemon:
Traceback (most recent call last):
  File "pokecli.py", line 843, in <module>
    main()
  File "pokecli.py", line 202, in main
    bot.tick()
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/__init__.py", line 718, in tick
    if worker.work() == WorkerResult.RUNNING:
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 341, in work
    targets = self._get_pokemons_from_url()
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 428, in _get_pokemons_from_url
    return self._parse_pokemons(results_hash_map.values())
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/cell_workers/sniper.py", line 373, in _parse_pokemons
    pokemon_name = Pokemons.name_for(pokemon.get('pokemon_id'))
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/inventory.py", line 456, in name_for
    return cls.data_for(pokemon_id).name
  File "/Users/Lucas/Documents/Projects/Pokelast/pokemongo_bot/inventory.py", line 452, in data_for
    return cls.STATIC_DATA[pokemon_id - 1]
IndexError: list index out of range
[2016-09-18 10:39:01] [MainThread] [sentry.errors] [ERROR] Sentry responded with an error: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128) (url: https://app.getsentry.com/api/90254/store/)
Traceback (most recent call last):
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync
    super(ThreadedHTTPTransport, self).send(data, headers)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send
    ca_certs=self.ca_certs,
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open
    response = self._open(req, data)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open
    '_open', req)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open
    return self.do_open(ValidHTTPSConnection, req)
  File "/Users/Lucas/Documents/Projects/Pokelast/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1057, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1097, in _send_request
    self.endheaders(body)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 895, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
[2016-09-18 10:39:01] [MainThread] [sentry.errors.uncaught] [ERROR] [u'IndexError: list index out of range', u'  File "pokecli.py", line 843, in <module>', u'  File "pokecli.py", line 202, in main', u'  File "pokemongo_bot/__init__.py", line 718, in tick', u'  File "pokemongo_bot/cell_workers/sniper.py", line 341, in work', u'  File "pokemongo_bot/cell_workers/sniper.py", line 428, in _get_pokemons_from_url', u'  File "pokemongo_bot/cell_workers/sniper.py", line 373, in _parse_pokemons', u'  File "pokemongo_bot/inventory.py", line 456, in name_for', u'  File "pokemongo_bot/inventory.py", line 452, in data_for']
Sun Sep 18 10:39:01 BRT 2016 Pokebot  Stopped.
Press any button or wait 20 seconds to continue.

…lling a function that did not exist until then.

Fixed the param name in the new sniper doccs.
Fixed issue X for the old sniper that wouldnt catch after teleports.
Fixed issue where it wouldnt get the pokemon name from its id.
@YvesHenri
Copy link
Author

Fixed issue #5523 (old sniper)

…as been captured or not. It now defaults to false.
@lucasbento
Copy link

It is working pretty well now:

[2016-09-18 12:31:16] [MainThread] [    Sniper] [INFO] Validating sources: https://pokesnipers.com/api/v1/pokemon.json, https://pokewatchers.com/grab/...
[2016-09-18 12:31:17] [MainThread] [    Sniper] [INFO] Source 'https://pokesnipers.com/api/v1/pokemon.json' is good!
[2016-09-18 12:31:25] [MainThread] [    Sniper] [INFO] Source 'https://pokewatchers.com/grab/' contains errors. Details: Fetching has timed out!
[2016-09-18 12:31:25] [MainThread] [pokemongo_bot.health_record.bot_event] [INFO] Health check is enabled. For more information:
[2016-09-18 12:31:25] [MainThread] [pokemongo_bot.health_record.bot_event] [INFO] https://github.com/PokemonGoF/PokemonGo-Bot/tree/dev#analytics
[2016-09-18 12:31:25] [MainThread] [PokemonGoBot] [INFO] Starting bot...
[2016-09-18 12:31:26] [MainThread] [CollectLevelUpReward] [INFO] Received level up reward:
[2016-09-18 12:31:26] [MainThread] [IncubateEggs] [INFO] Eggs incubating: [0.77/5.0 km] (Eggs left: 8, Incubating: 1)
[2016-09-18 12:31:26] [MainThread] [ShowBestPokemon] [INFO] *Best Pokemons* (Dragonite, CP 2332, IVCP 0.94, DPS 15.04, HP 126/126) | (Snorlax, CP 1862, IVCP 0.64, DPS 15.76, HP 200/200) | (Vaporeon, CP 1749, IVCP 0.91, DPS 11.1, HP 171/171) | (Vaporeon, CP 1677, IVCP 0.95, DPS 16.06, HP 165/165) | (Lapras, CP 1600, IVCP 0.92, DPS 12.66, HP 158/158)
[2016-09-18 12:31:26] [MainThread] [    Sniper] [INFO] Fetching pokemons from the sources...
[2016-09-18 12:31:27] [MainThread] [    Sniper] [INFO] Source 'https://pokesnipers.com/api/v1/pokemon.json' returned 25 results
[2016-09-18 12:31:27] [MainThread] [    Sniper] [INFO] After merging, we've got 25 results
[2016-09-18 12:31:27] [MainThread] [    Sniper] [INFO] Poliwhirl is not listed to catch, nor a VIP and its IV sucks. Skipping...

@YvesHenri
Copy link
Author

Also fixes #5128.

@@ -245,7 +245,7 @@ def snipe(self, pokemon):
exists = True

# Also, if the IDs arent valid, update them!
if not pokemon['encounter_id'] or not pokemon['spawnpoint_id']:
if not pokemon['encounter_id'] or not pokemon['spawn_point_id']:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, is it really that simple? I spent hours looking at this yesterday and missed this completely.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, not that simple. Fixes the "does not exist" error, but still fails to actually catch anything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some over-zealous try/except behavior in pokemon_catch_worker seems to be the culprit. See PR #5542

Copy link
Author

@YvesHenri YvesHenri Sep 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Social info does not have the spawnpoint ID, only the encounter, and the correct key for social spawnpoint value is 'spawn_point_id', not 'spawnpoint_id' like it was. So under social mode, it would never hit that if, meaning that it would never fix the spawnpoint and catch the target. Pokemons without a spawnpoint are lured pokemons (but social pokemons arent really lured) and it would crash (error is suppressed) and skip. The "does not exist" is a failed attempt to find it nearby after teleporting to it.

@Gobberwart
Copy link
Contributor

Does not fix #5523... at least not all of it. See #5542

@cowst
Copy link

cowst commented Sep 19, 2016

Feedback:

  • social didn't work (or solve issue Mqtt connection refused - not authorised #5518 ) because I already burnt myself on mqtt with normal dev branch beforehand :p
  • multiple urls in principle seemed to work, but this times out (on browser it works, but responds with a little delay). Is there any other source I can try?

…pPokemon task). NOTE: Please note that there are many reasons a pokemon will "not exist" and they are: it could be already caught by you, expired, too far from the given coords and/or wrong coords/infos at all. Also, when using social and MoveToMapPokemon, the reason it skips the catch after it teleports back is due to the "spawnpoint_id" key, which doesnt exist in the social JSON data and the correct key value is "spawn_point_id" (its been fixed).

Minor text refactors on the new Sniper task.
@YvesHenri
Copy link
Author

@cowst As expected. This PR is not really a fix for your issue, I just thought that you mightve been wrong about the "empty list", so if you did, this new sniper would be catching it all, so its really a mqtt issue.
Also, pokewatchers is annoying because it has the "sucuri firewall", which is a bitch. Although, its not really an issue. The only problem is that the new sniper has to validate it on startup, that means it HAS to reply something by the time its trying to validate it, otherwise it will be removed from the source list and you it will no longer use it, unless you restart the bot untill it validates it.
I do plan to add empty replies/timed out sorces into a "pending sources" list and they'll be validated again after the startup validation.
I do not know any other source but this is why this update was made for. Everyone should find, setup and use their URLs in this new task. Find yours. :)

@cowst
Copy link

cowst commented Sep 19, 2016

Alright :)

About the pending sources, is it really going to help if this service is always so slow to respond?
I would rather increase the timeout if I know for sure it is just slow (but worth the wait ;) ).

@YvesHenri
Copy link
Author

@cowst I dont want users to COMPLETELY customize EVERYTHING, since it might lead to many kind of bugs due to wrong/bad customizings, which means lots of issues. It's a nightmare to track them all. Anyways, it has a decent 7 seconds of timeout when validating and 3 seconds when fetching.
Surely having a list of sources that are pending validations will help A LOT, though I'm not doing this untill this PR gets merged (dont ask me why it hasnt yet).

@solderzzc
Copy link
Contributor

solderzzc commented Sep 19, 2016

👍

Approved with PullApprove

@YvesHenri YvesHenri merged commit 969e5d1 into PokemonGoF:dev Sep 19, 2016
@cowst
Copy link

cowst commented Sep 19, 2016

@YvesHenri I didn't look at the code, but why aren't we skipping pokemons before teleporting?
Isn't it a waste?

Example:
[2016-09-19 21:14:59] [MainThread] [ Sniper] [INFO] Teleporting to meet Onix (39.163984; -119.765993)...
[2016-09-19 21:15:02] [MainThread] [ Sniper] [INFO] Yay! There really is a wild Onix nearby!
[2016-09-19 21:15:03] [MainThread] [ Sniper] [INFO] Teleporting back to the old position (49.8730329398; 8.61117880831)...
[2016-09-19 21:15:06] [MainThread] [PokemonCatchWorker] [INFO] Skip ignored Onix! (CP 412) (Potential 0.51) (A/D/S 2/11/10)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants