"login fail" error message

[squirrelB]

I was able to seamlessly get data from Fitbit using fitbitScraper on 5th Sept. But the same codes now resulting in a "login fail" message. Not sure what changed. Everything the same, worked one day but not anymore...

[dryoutao]

You are not alone squirrelB - would anyone please answer?

[squirrelB]

I'm trying to figure if there's something up with the "login" code. Tried changing rememberme to TRUE but didn't help. Will play around with the POST() code. I've used R mainly for statistical applications and httr is really new to me... so if anyone can figure it out, it'll be much appreciated!

[ghost]

It looks like upon fitbit's latest release they have changed some of their API structure (bit of a beginner here as well), and the login function no longer finds the relevant cookie (the one in the function that has the header "u" which no longer exists, at least from what I can see - I had 3 cookies with names "__cfduid", "JSESSION" and "fct").

I have tried the 3 cookies it returns in the existing fitbitscraper functions but they all fail.

[squirrelB]

Yeah. Same here. Got 3 cookies; tried them individually (assigned them to "cookie <- "yadidiaaycookiehere")... seemed like I could login. But the the subsequent Fitscraper function (like "get_daily_data" for instance) resulted in an error:
Error: lexical error: invalid char in json text.
<!--[if lt IE 7
(right here) ------^

Will try sign up to dev.fitbit.com and try httr; but heard that was complicated setting up the token and stuff.

[dryoutao]

Hi Both,

When I ran the following

httr::POST(url,
headers=headers,
body=body)

I got

Response [https://www.fitbit.com/login]
Date: 2017-09-11 05:49
Status: 403
Content-Type: text/html;charset=UTF-8
Size: 770 B
...

It's a 403 error message indicating login failure. Did you get the same message?

[ghost]

Yea I get the same response, status 403, anyone had any luck?

[squirrelB]

Yeah, session 403. From what I've read "403 Forbidden status in response to a request from a client for a web page or it may indicate that the server can be reached and process the request but refuses to take any further action. " (wiki)
I ran this after setting the header and body as it:

a <- httr::POST(url, headers=headers, body=body)
str(a)
List of 10
$ url : chr "https://www.fitbit.com/login"
$ status_code: int 403
etc, etc
3 cookies, but no "u" cookie so that cookie <- a$cookies$u ends up with cookie=null
I feel Fitbit has locked down our data and quite annoyed by it.

[PhillipKWood]

Alas. Do you think it would do any good to post this dismay in their community group discussion sections?

[jibanes]

Same exact problem here.

[squirrelB]

I gave up on the R packages for Fitbit and ended up (1) using the fitbit API website to get an access token, (2) running the curl code from the terminal to get data, and then (3) R.
curl -i -H "Authorization: yadiyadiyagobbledegook" https://api.fitbit.com/1/user/-/activities/heart/date/yyyy-mm-dd/1d/1min/time/hh:mm/hh:mm.json > mydata.json

Then in R:
d <- file("mydata.json", "r")
d2 <- readLines(d)
d3 <- jsonlite::fromJSON(d2[16]) ###line 1 to 15 were headers##

[neuropil]

Did that work?

[jibanes]

I think patching fitbitscraper might be better if you have a workaround for auth...

[vaschoepfer]

Any updates on this?

Or a more detailed solution with the api?

[squirrelB]

Hi. I got frustrated with Fitbit for not making it easier to get our data. And since this Fitbitscraper issue is not resolved as far as I know, I've made a tutorial on accessing data directly from the Fitbit API and also R codes for plotting the heart rate time course. In case it's useful to anyone, here it is
https://annofoneblog.wordpress.com/2017/10/19/your-heart-your-calories-your-sleep-your-data-how-to-extract-your-fitbit-data-and-make-graphs-using-r/

[vaschoepfer]

It is very useful! I did receive an error though and I can't work around it. On the R side, rather than the API side. It could very well be me. Everything looks right in the text file, but then again I'm not quite sure what I'm looking for. I've downloaded the .json a few times to try it, but get this error each time.

I'll keep working, but thank you!!

d <- file("mydata.json", "r") ###connecting to file; "r" mode is open to read in text mode##
d2 <- readLines(d)
Warning message:
In readLines(d) : incomplete final line found on 'mydata.json'
d3 <- fromJSON(d2[16]) ####relevant line is #16, read that in ##
Error: parse error: trailing garbage
:85.5,"weightUnit":"METRIC"}}HTTP/1.1 200 OK
(right here) ------^

Super quick edit- I had to change it to line 30, rather than line 16. But it works! You rock!

[squirrelB]

Ah, good to know. I guess the json file has variable number of headers. Mine has 14 lines of headers then a blank line before data starts. Thanks for the info.

[PhillipKWood]

That's very helpful. When I get to the part, though, where I'm asked to copy the parse response field, I dutifully copied: #scope=activity%20heartrate%20location%20nutrition%20profile%20settings%20sleep%20social%20weight&expires_in=604800 but received the note: Token response does not match the expected format; please check that you're using the correct OAuth 2.0 flow. I'm using Google Chrome to fill out the form- could that be the problem? thanks for any insights! Phil

…

On Thu, Oct 19, 2017 at 12:50 AM, squirrelB ***@***.***> wrote: Hi. I got frustrated with Fitbit for not making it easier to get our data. And since this Fitbitscraper issue is not resolved as far as I know, I've made a tutorial on accessing data directly from the Fitbit API and also R codes for plotting the heart rate time course. In case it's useful to anyone, here it is https://annofoneblog.wordpress.com/2017/10/19/your- heart-your-calories-your-sleep-your-data-how-to- extract-your-fitbit-data-and-make-graphs-using-r/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYCgXP7dC8gWJD2ssY90NxoOdECuwZ-Fks5stuMYgaJpZM4PRk_8> .

[squirrelB]

Hi, I suggest that you copy and paste the full web address into the "parse response window" and it'll parse it out for you. I'd probably ignore the specific instruction in the API website about the #scope. [... just to be super clear with the instruction, when you click the link in the 1:Authorize step, it opens a new browser window where you select the data you want and click "allow". That takes you to a google page with a long web address... copy that entire address and paste it to the 2:Parse window. I hope that made sense. I know it can be confusing] On Thu, Oct 19, 2017 at 11:00 AM, PhillipKWood <notifications@github.com> wrote:

…

That's very helpful. When I get to the part, though, where I'm asked to copy the parse response field, I dutifully copied: #scope=activity%20heartrate%20location%20nutrition% 20profile%20settings%20sleep%20social%20weight&expires_in=604800 but received the note: Token response does not match the expected format; please check that you're using the correct OAuth 2.0 flow. I'm using Google Chrome to fill out the form- could that be the problem? thanks for any insights! Phil On Thu, Oct 19, 2017 at 12:50 AM, squirrelB ***@***.***> wrote: > Hi. I got frustrated with Fitbit for not making it easier to get our data. > And since this Fitbitscraper issue is not resolved as far as I know, I've > made a tutorial on accessing data directly from the Fitbit API and also R > codes for plotting the heart rate time course. In case it's useful to > anyone, here it is > https://annofoneblog.wordpress.com/2017/10/19/your- > heart-your-calories-your-sleep-your-data-how-to- > extract-your-fitbit-data-and-make-graphs-using-r/ > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#20 (comment)- 337806388>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ AYCgXP7dC8gWJD2ssY90NxoOdECuwZ-Fks5stuMYgaJpZM4PRk_8> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AeTmUUyVd_LgfFwbUzGXhGdbNwf2vJ01ks5st3IUgaJpZM4PRk_8> .

-- K Mozhui

[ghost]

Really useful, squirrelB. Thanks a lot