Mimic the request workflow of official application

[s0crat]

Short Description

Bot request workflow is not implemented in the same way as the official application do. It's a way for Niantic to detect bots. Below is the request workflow sent by my Android official application, latest version:

1st step:

[<-] Request for GetPlayer  {}
[->] Response for GetPlayer {
  // some stuff }

2d step:

[<-] Request for DownloadRemoteConfigVersion  { platform: 'ANDROID', app_version: 3300, '$unknownFields': [] }
[<-] Request for GetHatchedEggs  {}
[<-] Request for GetInventory  {}
[<-] Request for CheckAwardedBadges  {}
[<-] Request for DownloadSettings  {}
[->] Response for DownloadRemoteConfigVersion  {
  // some stuff }
[->] Response for GetHatchedEggs  {
  // some stuff }
[->] Response for GetInventory  {
  // some stuff }
[->] Response for CheckAwardedBadges  {
  // some stuff }
[->] Response for DownloadSettings  {
    "hash": "1234567890123456789012345678901234567890", // 40bytes hash
    // other settings sent by the server including "google_maps_api_key" }

3rd Step

[<-] Request for GetAssetDigest  { platform: 'ANDROID', app_version: 3300, '$unknownFields': [] }
[<-] Request for GetHatchedEggs  {}
[<-] Request for GetInventory  { last_timestamp_ms: '1234567890123', '$unknownFields': [] }
[<-] Request for CheckAwardedBadges  {}
[<-] Request for DownloadSettings  { hash: '1234567890123456789012345678901234567890', // 40bytes hash received previously
  '$unknownFields': [] }
[->] Response for GetAssetDigest  {
    "digest": [
        {
            "asset_id": "12345678-90ab-cdef-1234-567890abcdef/1234567890123456",
            "bundle_name": "pm1234",
            "version": "1234567890123456",
            "checksum": 1234567890,
            "size": 123456,
            "key": {
                "type": "Buffer",
                "data": [
                    12, 34, 567, 89, 012, 345, 678, 901, 234, 56, 78, 901, 23, 45, 678, 901
                ]
            }
        },
        {
            "asset_id": ...
            // repetead asset_id stuff
    ]
}
[->] Response for GetHatchedEggs  {
  // some stuff }
[->] Response for GetInventory  {
  // some stuff }
[->] Response for CheckAwardedBadges  {
  // some stuff }
[->] Response for DownloadSettings  {
  // only hash & timestamp, server doesn't resend all data each time }

Next steps:

[<-] Request for action
[<-] Request for GetHatchedEggs  {}
[<-] Request for GetInventory  { last_timestamp_ms: '1234567890123', '$unknownFields': [] }
[<-] Request for CheckAwardedBadges  {}
[<-] Request for DownloadSettings  { hash: '1234567890123456789012345678901234567890', '$unknownFields': [] }
[->] Response for LevelUpRewards  {

[->] Response for some action {
  // some stuff }
[->] Response for GetInventory  {
  // some stuff }
[->] Response for CheckAwardedBadges  {
  // some stuff }
[->] Response for DownloadSettings  {
  // some stuff }

Possible solution

Start with GetPlayer request only. For each other actions, implement thefollowing workflow:

[<-] Request some action (LevelUpRewards, Encounter, CatchPokemon, GetMapObjects, GetAssetDigest, MarkTutorialComplete, GetPlayer, etc. )
[<-] Request for GetHatchedEggs
[<-] Request for GetInventory
[<-] Request for CheckAwardedBadges
[<-] Request for DownloadSettings

Note that GetAssetDigest and GetPlayer are sent periodically (1 time each 5 min ?).

How it would help others

Implementing the right request workflow can help to mimic the official application behavior and make more difficult the detection of bots.

[s0crat]

Addtionnal information regarding GetAssetDigest can be found on reddit. GetAssetDigest and GetDownloadUrls must be implemented to be close of official app behavior.

[OursDesCavernes]

This seems more efficient that #4114
This way, we should be able to make a bot that does not get banned when walking arround your place ...

[takenek]

Awsome work @s0crat i hope this is a problem :-)

[sohje]

Aren't we already mimic some parts of this sequence?

request.get_player()
request.get_hatched_eggs()
request.get_inventory()
request.check_awarded_badges()
request.download_settings(hash="54b359c97e46900f87211ef6e6dd0b7f2a3ea1f5")

[s0crat]

The sequence you fingerpoint is located in app_simulation_login() function of src/pgoapi/pgoapi/pgoapi.py . Few comments:

1. In the current implementation, this sequence is only send one time, during the login sequence. It's not the right behavior.
2. It's not the right sequence of official app ver 0.33 (to be analysed for the new version 0.35) --> DownloadRemoteConfigVersion is missing
3. The call of DownloadSettings must be send first whitout parameters, the response include the right hash value to be used futher to recall the function (currently, in the pgoapi.py file the hash value is hardcoded)

[sohje]

Made some tests with pgoapi - calling DownloadSettings w/o params always returns same hash value, changing platfom to ADNROID, iOS, etc dont make any sense.

[s0crat]

Have you tried with different accounts ? I think this is linked to a specific account. Hash value retrieved with the account used for tests isn't the same as the value hardcoded. Moreover, DownloadSettings return lot of other stuffs related to a specific account (like the google_maps_api_key , etc.).

I guess the hash value is per account ...

[sohje]

Yeap, tried 3 accounts. Same hash value for all of them:

{'hash': u'2788184af4004004d6ab0740f7632983332106f6'...}

Values isn't the same as hardcoded hash value - 54b359c97e46900f87211ef6e6dd0b7f2a3ea1f5
Also, same google_maps_api key on every request.

[s0crat]

Same hash value for me. Maybe linked with the version of the app ?

[sohje]

Tried 0.33.0 and 0.35.0 - nothing changed.