Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'executable_path' field to the 'api.create_sessions()' method #1136

Merged
merged 1 commit into from
Apr 12, 2024

Conversation

Kkordik
Copy link
Contributor

@Kkordik Kkordik commented Apr 2, 2024

Adding executable_path can solve the issue:
EmptyResponseException: None -> TikTok returned an empty response
#1090

The example on how it works is in google colab:
https://colab.research.google.com/drive/14FV3Ja3rmrubQ1FVLNhByTz1n_MPPlKA#scrollTo=h9tG2EQM_Ekd

Edited: Forgot to mention that to actually solve the issue you should install an older version of Google Chrome and specify its executable path when calling the create_sessions method.

Version Result
115.0.5790.170 OK
121.0.6167.57 NG

(this solution and the table were provided by the author of the #1090 (comment))

@ekorian
Copy link
Contributor

ekorian commented Apr 2, 2024

Are you sure you that this doesn't only changes your browser version and device profile used for rate limiting ? Don't you get the Empty Response error back at one point ?

@Kkordik
Copy link
Contributor Author

Kkordik commented Apr 2, 2024

Are you sure you that this doesn't only changes your browser version and device profile used for rate limiting ? Don't you get the Empty Response error back at one point ?

@ekorian , You are right, with these changes you can choose a browser (chromium).
Now I don't get the Empty Response. You can try it in the colab notebook, that I have mentioned earlier.

This solution is based on the comments:
comment 1
comment 2 where the author explained:

Upon further investigation of this issue, it was determined that the above code (using executable_path) does not work on some versions of Google Chrome.
Specifically, we found the following behavior.

Version | Result

-- | --
115.0.5790.170 | OK
121.0.6167.57 | NG

@ekorian
Copy link
Contributor

ekorian commented Apr 2, 2024

I suspect that you get the Empty Response because of rate limiting policies at TikTok.
I get the same result if I use a different browser version, by using a different playwright version, and with no executable path: first it works, then Empty Response comes back again.

@Kkordik
Copy link
Contributor Author

Kkordik commented Apr 2, 2024

I suspect that you get the Empty Response because of rate limiting policies at TikTok. I get the same result if I use a different browser version, by using a different playwright version, and with no executable path: first it works, then Empty Response comes back again.

I don't think that the problem is in rate limiting policies. Because running the same example code, that retrieves 30 trending videos, all the time I was getting the Empty Response. And running the modified example code (colab link), that also retrieves 30 videos, I don't get the error.

If it was the rate limit (that is strange for 30 videos), it would not work in any case, would it?

Also changing the browser for FireFox as it is done in this pull request - #1129 workes for the author.

Copy link

sonarqubecloud bot commented Apr 6, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@lhphat02
Copy link
Contributor

lhphat02 commented Apr 9, 2024

merge pls! I really need it 🥺

@davidteather davidteather changed the base branch from main to 6.3.0 April 12, 2024 17:08
@davidteather
Copy link
Owner

LGTM :)

@davidteather davidteather merged commit d2c2635 into davidteather:6.3.0 Apr 12, 2024
1 check passed
davidteather added a commit that referenced this pull request Apr 12, 2024
* Add 'executable_path' field to the 'api.create_sessions()' method (#1136)
* Fixed retries loop (#1133)
* Add option to select which browser playwright uses (#1129)
* add browser selection option to create_sessions method
* fix spelling typo

---------

Co-authored-by: David Teather <34144122+davidteather@users.noreply.github.com>

* Update hashtag.py (#1126)

TikTok appears to have limited the number of posts that can be returned for a single hashtag search to 35 (e.g. bellingcat/tiktok-hashtag-analysis#28).

Similar to how the `User.liked` method works, I set the batch size to 35 videos per request.

* add executable path to str

* bump ver

---------

Co-authored-by: kkordik <99617240+Kkordik@users.noreply.github.com>
Co-authored-by: ekorian <korian.edeline@gmail.com>
Co-authored-by: Will Howes <39531537+willmhowes@users.noreply.github.com>
Co-authored-by: Tristan Lee <tristan@bellingcat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants