CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) #915

linglin506 · 2020-09-18T02:36:34Z

This error will be reported from 0:00 on September 18, 2020. It is found through debug that the https://twitter.com/i/search/timeline page does not exist. Can you fix it?By changing the IP, it can be determined that Twitter is not banning the IP

CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs

sukioral · 2020-09-18T03:21:16Z

I experienced the same problem today as well. Yesterday was still fine.

zhixideyu · 2020-09-18T03:40:08Z

今天用就有事

hcanalesmx · 2020-09-18T04:29:00Z

I am having the same issue

bigslick87 · 2020-09-18T05:06:41Z

Glad to see I am not the only one. I just discovered this today and was excited to use it. I hope they can fix it ASAP

hj5992 · 2020-09-18T06:58:37Z

getting the same error!

gab-santi · 2020-09-18T07:08:51Z

Getting the same error as well. Hope it gets fixed ASAP

tmantynen · 2020-09-18T07:27:40Z

I guess Twitter has deleted https://twitter.com/i/search/timeline

Twint is not the only Twitter scraper having the same problem. See: jonbakerfish/TweetScraper#92

Buaasinong · 2020-09-18T07:50:04Z

bad news

zhixideyu · 2020-09-18T07:54:03Z

我想Twitter已经删除了https://twitter.com/i/search/timeline

Twint并不是唯一遇到相同问题的Twitter抓取工具。请参阅：jonbakerfish / TweetScraper＃92

Does that mean the project is going to fail

yemen2016 · 2020-09-18T08:04:25Z

I am also having the same issue. The system was working fine yesterday!
I hope the problem will be solved soon.

Sebastokratos42 · 2020-09-18T08:13:04Z

I guess Twitter has deleted https://twitter.com/i/search/timeline

Twint is not the only Twitter scraper having the same problem. See: jonbakerfish/TweetScraper#92

Problem also appears using GetOldTweets3.

shelu16 · 2020-09-18T09:30:10Z

https://twitter.com/search?f=live&q=bug%20min_retweets%3A100%20until%3A2020-09-18%20since%3A2020-09-11&src=typed_query but it can work can we do this search here

vishal7892 · 2020-09-18T09:59:33Z

Even I am facing the same issue. Any luck?

TryAndErr0r · 2020-09-18T10:53:20Z

We need to find a new endpoint. I loaded the site and saw this link
New Api link?.
It is important to note that if you copy the request and past it to notepad you can see that the request is provided with headers and you can see that it provides a x-csrf-token and x-guest-token which are required. You can find these tokens in the cookie fields "gt" and "ct0".

This is somewhat the official twitter api and you need a bearer token to access this. The bearertoken is provided in the https://abs.twimg.com/responsive-web/client-web/main.6b1bdbc5.js file. Just search for "AAAAAA" and you can find it.

Now you can craft a GET request in postman with the x-csrf-token, the x-guest-token and the bearer token for authentification.

yunusemrecatalcam · 2020-09-18T13:29:41Z

Booom! I found a new endpoint, implemented the new parser for it. Check this out! #917
@pielco11

estebanpdl · 2020-09-18T17:47:01Z

Having the same issue as well. Just tried yesterday (2020-09-17) and everything worked fine. I tried Twint today (2020-09-18) and I got the same loop.

CRITICAL:root:twint.get:User:
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 27.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 64.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 125.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 216.0 secs

boshez · 2020-09-18T18:48:16Z

I thought I was the only person who was facing this. Hope they fix this.

hasobi · 2020-09-18T19:44:42Z

i'm having the same problem here.

guilherme-sal · 2020-09-18T20:10:56Z

Same problem here. Yesterday was working fine...

jeremy1392 · 2020-09-18T20:46:13Z

Everything is working except to fetch tweets. Yesterday was working fine

ivanlewin · 2020-09-18T20:58:29Z

Hi I just want to add that I also had no problems yesterday but today I can't get any data.

Also, I was trying other tool to see if it worked and they seem to be having similar issues

aradiayu · 2020-09-19T07:59:32Z

I think I'm the only one facing this 'twint' problem. I hope it is fixed soon

Dbroug01 · 2020-09-19T08:48:31Z

I am also facing this issue

tmantynen · 2020-09-19T09:49:09Z

Booom! I found a new endpoint, implemented the new parser for it. Check this out! #917
@pielco11

Big thanks to @yunusemrecatalcam

I'm not using Twint but instead my own similar Twitter scraper program writen in PHP. So I can't comment directly on @yunusemrecatalcam 's changes to Twint.

Anyway. Couple of days ago my own scraper stopped working just like all Twitter scrapers using i/search/timeline endpoint did. Following @yunusemrecatalcam 's example, I implemented similar changes to my program that @yunusemrecatalcam did to Twint in PR #917.

I haven't had the time to finished all changes, but preliminary impressions are mixed. The new mobile endpoint appears to function similarly to the old one, but Twitter appears to return a lot fewer tweets as a result to queries than previously. Queries that resulted hundreds of tweets earlier now result just tens of tweets. I can't yet comment on speed or rate limits etc. And again, these are just preliminary impressions.

(Edit: I incorrectly tagged @pielco11 when I should have tagged @yunusemrecatalcam. My apologies.)

adrival13 · 2020-09-19T20:19:21Z

Same problem
CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 27.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 64.0 secs

rayhannr · 2020-09-20T02:44:19Z

got the same problem though fetching followers and followings are just fine. anyway, is there any good alternative to twint?

shizia · 2020-09-20T06:30:32Z

今天用就有事

以前是都能用吗？才刚发现这个就发现好像出问题了

lukaspistelak · 2020-09-20T09:33:52Z

they changed end point url to 👍

https://twitter.com/explore/

debjyoti003 · 2021-05-12T08:26:22Z

after installing the new one, I am getting

~/.local/lib/python3.7/site-packages/twint/format.py in Tweet(config, t)
21 output = output.replace("{hashtags}", ",".join(t.hashtags))
22 output = output.replace("{cashtags}", ",".join(t.cashtags))
---> 23 output = output.replace("{replies}", t.replies_count)
24 output = output.replace("{retweets}", t.retweets_count)
25 output = output.replace("{likes}", t.likes_count)

TypeError: replace() argument 2 must be str, not int

debjyoti003 · 2021-05-13T13:38:26Z

Is there any specification for installing it on ubuntu?

wildangunawan · 2021-05-14T04:07:57Z

Is there any specification for installing it on ubuntu?

I have no problem installing it on Ubuntu 16.04 under Conda environtment. I used this command:

pip install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

vesuvius13 · 2021-05-15T17:32:24Z

I got the same issue and the previous solution doesn't work now

create a new virtual environment in anaconda and install python 3.6 in there.
conda create -n yourenvname python=3.6 anaconda
Then, follow these steps:-

git clone https://github.com/twintproject/twint.git
cd twint
pip3 install . -r requirements.txt
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Then try to run the script:-

!pip install nest_asyncio
import nest_asyncio
nest_asyncio.apply()

import twint
config = twint.Config()
config.Search = "Dogecoin"
config.Limit = 10
config.Store_csv = True
config.Output = "doge.csv"
twint.run.Search(config)

romellfudi · 2021-05-25T12:51:09Z

Hi everyone, when I run the following command:

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Does it work on MAC OS?
I have been trying to use it since 2019

lemiemie · 2021-05-26T19:43:19Z

Hi everyone, when I run the following command:
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Does it work on MAC OS?
I have been trying to use it since 2019

@romellfudi Yes it works on my macOS. It gave me error message at the beginning but after I deleted the "--user" part it works

zachlagden · 2021-08-11T08:20:23Z

@eabanoz @jiwoochoi-kor @hnodaro
Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
thank you so much!!! it worked

worked for me

hglite69 · 2021-08-15T06:22:21Z

Me Too ^-^

jpblopez · 2021-09-18T00:17:29Z

@eabanoz @jiwoochoi-kor @hnodaro
Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
thank you so much!!! it worked

Also worked for me, as of September 2021

asdrubalivan · 2021-10-27T02:40:30Z

^ This worked for me too!

tsaaii · 2021-11-24T23:57:08Z

upgrading twint works.

yehfanju · 2021-11-25T05:57:16Z

pip3 uninstall twint

@tsaaii can you normally scrape all the tweets without encountering limit/date constraints? Thanks!

tsaaii · 2021-11-25T14:03:53Z

it limits tweets on time line as mentioned in documentation. I haven't seen any date constraints so far.

yehfanju · 2021-11-25T16:43:34Z

@tsaaii Thanks for your reply! Did you clone the git repo as well or just pips upgrade? I can only get a few tweets after uninstall and reinstall.

tsaaii · 2021-11-25T17:35:12Z

clone the repo first, upgrade it next. It works for me. Make sure python version is 3.6.

yehfanju · 2021-11-25T20:18:16Z

@tsaaii Thanks! the 3.6 version and upgrade works for me but only for the latest tweets. Twint does not return anything when specifying since and until. Just want to know if you also cannot retrieve historical tweets or it's just me!

olivermj3 · 2021-12-30T00:21:46Z

This issue fixed after running this command:
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

However, following this upgrade, it resumed collect briefly (16 tweets recovered then the following print out before the error:

[!] No more data! Scraping will stop now.
found 0 deleted tweets in this search.

RefreshTokenException Traceback (most recent call last)
/var/folders/7z/866jzymn52zcbnymflj7xhbw0000gn/T/ipykernel_6392/906502173.py in
7 filename = str(str(target)+".csv")
8 c.Output = filename
----> 9 twint.run.Search(c)

~/.local/lib/python3.8/site-packages/twint/run.py in Search(config, callback)
408 config.Followers = False
409 config.Profile = False
--> 410 run(config, callback)
411 if config.Pandas_au:
412 storage.panda._autoget("tweet")

~/.local/lib/python3.8/site-packages/twint/run.py in run(config, callback)
327 raise
328
--> 329 get_event_loop().run_until_complete(Twint(config).main(callback))
330
331

~/.local/lib/python3.8/site-packages/twint/run.py in init(self, config)
34 # USAGE : to get a new guest token simply do self.token.refresh()
35 self.token = token.Token(config)
---> 36 self.token.refresh()
37 self.conn = db.Conn(config.Database)
38 self.d = datelock.Set(self.config.Until, self.config.Since)

~/.local/lib/python3.8/site-packages/twint/token.py in refresh(self)
67 else:
68 self.config.Guest_token = None
---> 69 raise RefreshTokenException('Could not find the Guest token in HTML')

RefreshTokenException: Could not find the Guest token in HTML

fwgood · 2021-12-30T00:22:06Z

已收到您的邮件，会尽快回复

rangersmyth74 · 2021-12-30T06:51:41Z

!pip install twint
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
import twint
import nest_asyncio
c = twint.Config()
c.Username = "github"
c.Search = "code"
nest_asyncio.apply()
twint.run.Search(c)

Hey all... I am having this problem now in Dec 2021. I have tried all the fixes, but none worked for me. I used Google Cloud Shell and I was able to get twint to work, but then I was getting the same errors.

I was wondering how to use the above date? Do I run it on the command line? Just wondering.

Thanks.

machlovi · 2022-02-01T19:24:52Z

I am facing the same issue ..I have tried all the solutions. Can anyone help me

fwgood · 2022-02-01T19:25:13Z

已收到您的邮件，会尽快回复

ArshdeepSandhu · 2022-02-22T11:33:37Z

@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.

Thank you very much Himanshu! Was really struggling with this.

ArshdeepSandhu · 2022-02-22T11:36:49Z

I am facing the same issue ..I have tried all the solutions. Can anyone help me

Have you tried what Himanshu suggested?

skswldndi · 2022-03-14T07:54:27Z

@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command
pip3 uninstall twint
And then reinstall twint using the following command
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
This should work.
Thank you very much Himanshu! Was really struggling with this.

This worked for me! Thanks.

fwgood · 2022-03-14T07:54:49Z

已收到您的邮件，会尽快回复

palavaryunusemre · 2022-04-04T14:45:49Z

@eabanoz @jiwoochoi-kor @hnodaro Aşağıdaki komutu kullanarak önceki twint kurulumunu kaldırın
pip3 uninstall twint
Ardından aşağıdaki komutu kullanarak twint'i yeniden yükleyin
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Bu çalışmalı.
Himanshu'ya çok teşekkür ederim! Bu konuda gerçekten zorlanıyordu.

this didn't work for me

fwgood · 2022-04-04T14:46:13Z

已收到您的邮件，会尽快回复

palavaryunusemre · 2022-04-04T14:48:48Z

已收到您的邮件，会尽快回复

thanks

Sebastokratos42 mentioned this issue Sep 18, 2020

HTTP Error, Gives 404 but the URL is working Mottl/GetOldTweets3#98

Open

robert-moore mentioned this issue Sep 19, 2020

Twint stopped working out of the blue #918

Open

weirdyang mentioned this issue Jan 1, 2022

CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) #1312

Open

CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) #915

CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) #915

Comments

linglin506 commented Sep 18, 2020

sukioral commented Sep 18, 2020

zhixideyu commented Sep 18, 2020

hcanalesmx commented Sep 18, 2020

bigslick87 commented Sep 18, 2020

hj5992 commented Sep 18, 2020

gab-santi commented Sep 18, 2020

tmantynen commented Sep 18, 2020

Buaasinong commented Sep 18, 2020

zhixideyu commented Sep 18, 2020

yemen2016 commented Sep 18, 2020

Sebastokratos42 commented Sep 18, 2020

shelu16 commented Sep 18, 2020

vishal7892 commented Sep 18, 2020

TryAndErr0r commented Sep 18, 2020 • edited Loading

yunusemrecatalcam commented Sep 18, 2020

estebanpdl commented Sep 18, 2020 • edited Loading

boshez commented Sep 18, 2020

hasobi commented Sep 18, 2020 • edited Loading

guilherme-sal commented Sep 18, 2020

jeremy1392 commented Sep 18, 2020

ivanlewin commented Sep 18, 2020 • edited Loading

aradiayu commented Sep 19, 2020

Dbroug01 commented Sep 19, 2020

tmantynen commented Sep 19, 2020 • edited Loading

adrival13 commented Sep 19, 2020

rayhannr commented Sep 20, 2020

shizia commented Sep 20, 2020

lukaspistelak commented Sep 20, 2020

debjyoti003 commented May 12, 2021

debjyoti003 commented May 13, 2021 • edited Loading

wildangunawan commented May 14, 2021

vesuvius13 commented May 15, 2021 • edited Loading

romellfudi commented May 25, 2021 • edited Loading

lemiemie commented May 26, 2021

zachlagden commented Aug 11, 2021

hglite69 commented Aug 15, 2021

jpblopez commented Sep 18, 2021 • edited Loading

asdrubalivan commented Oct 27, 2021

tsaaii commented Nov 24, 2021

yehfanju commented Nov 25, 2021

tsaaii commented Nov 25, 2021

yehfanju commented Nov 25, 2021

tsaaii commented Nov 25, 2021

yehfanju commented Nov 25, 2021

olivermj3 commented Dec 30, 2021

[!] No more data! Scraping will stop now. found 0 deleted tweets in this search.

fwgood commented Dec 30, 2021 via email

rangersmyth74 commented Dec 30, 2021

machlovi commented Feb 1, 2022

fwgood commented Feb 1, 2022 via email

ArshdeepSandhu commented Feb 22, 2022

ArshdeepSandhu commented Feb 22, 2022

skswldndi commented Mar 14, 2022

fwgood commented Mar 14, 2022 via email

palavaryunusemre commented Apr 4, 2022

fwgood commented Apr 4, 2022 via email

palavaryunusemre commented Apr 4, 2022

TryAndErr0r commented Sep 18, 2020 •

edited

Loading

estebanpdl commented Sep 18, 2020 •

edited

Loading

hasobi commented Sep 18, 2020 •

edited

Loading

ivanlewin commented Sep 18, 2020 •

edited

Loading

tmantynen commented Sep 19, 2020 •

edited

Loading

debjyoti003 commented May 13, 2021 •

edited

Loading

vesuvius13 commented May 15, 2021 •

edited

Loading

romellfudi commented May 25, 2021 •

edited

Loading

jpblopez commented Sep 18, 2021 •

edited

Loading

[!] No more data! Scraping will stop now.
found 0 deleted tweets in this search.