Including custom URLs when reporting events with GA.

[cagryInside]

We start reporting SHA-1 hashed custom URLs as a secondary dimension as discussed #105 (comment). However, when the application first time installed and run in devices, it also reports "CUSTOM URL" event, even though there is no custom URL set.

[cagryInside]

With the current fix, there is one problem may occur if there is no region and no custom url defined at the same time.

 public static void reportEventWithCategory(String category, String action, String label) {
if (isAnalyticsActive()) {
Tracker tracker = Application.get().getTracker(Application.TrackerName.APP_TRACKER);
Tracker tracker2 = Application.get().getTracker(Application.TrackerName.GLOBAL_TRACKER);

String obaRegionName = getObaRegionName();

tracker.send(new HitBuilders.EventBuilder()
.setCategory(category)
.setAction(action)
.setLabel(label)

.setCustomDimension(1, obaRegionName)

.build());

When there is no region and no custom url defined, getObaRegionName() going to return null instead of "custom URL". And, we will push null as a secondary dimension. Since, this happens only one time (until the app finds the closest region when users install the app first time), we can disable logging, if getObaRegionName() returns null.
Another approach could be, instead of using null, we can push some specific string (e.g.: "NO Region").

@barbeau any comments are welcomed.

[barbeau]

Thanks @cagryInside! A few comments in line on commit.

[bbodenmiller]

Worth noting OBA iPhone currently reports null region in a few places. I think it'd be best to not discard logging events just because a region name can't be found.

[barbeau]

@bbodenmiller We're actually changing where the region name is logged, so that's no longer an issue. In OBA Android the region name is only null for a few seconds on first startup after install, and after that either the region or the custom API URL should always have a value.

[cagryInside]

Also, IOS sends two more events when the app starts and home button presses:
-> Show Experimental Regions : YES or NO
-> Set Region Automatically: YES or NO

So, in my understanding, we also shouldn't trigger these events when the app starts. These events should be triggered when the event actually happens. (e.g., : when the user clicks Show Experimental Regions button in settings).

@barbeau could you please confirm.

[barbeau]

@cagryInside Right, IMO:

• Set Region Manually
• Show Experimental Regions : YES or NO
• Set Region Automatically

...should NOT be triggered when the app starts (or, for iOS, home button presses). Instead, these should be triggered when those values change. So, if a user starts up OBA for first time and Tampa is automatically selected as a region, "Set Region Automatically" should be triggered. If their region never changes, either manually or automatically, after that initial startup, then no more analytics events of those types would be logged. Same with "Show Experimental Regions" - this is only logged when the user changes this setting.

[barbeau]

The main motivation behind capturing these items is to determine how often users actually change these settings, and how often regions are automatically switched.

[cagryInside]

@barbeau agreed. So, in OBA Android the following events will be much less than IOS.

[barbeau]

So, in OBA Android the following events will be much less than IOS.

Correct.

And, since those event labels have been heavily polluted on iOS by traffic that doesn't really capture what we originally wanted (IMO), we should probably consider changing them.

@bbodenmiller @aaronbrethorst what are your thoughts on this?

I believe that:

• Set Region Manually
• Show Experimental Regions : YES or NO
• Set Region Automatically

... analytics should NOT be triggered when the app starts (and, for iOS, home button presses), and should only be triggered when these values change. If we agree on this, we have a lot of bad analytics data for these categories, and we should discuss changing the labels so we can rely on these metrics.

[bbodenmiller]

When I setup the iOS analytics I believe I was thinking a bit differently and I'll have to go confirm exactly how it works (or doesn't work) - it's been awhile. Essentially, however, my thinking was that we should track when a setting change occurs but also how many sessions are running with specific settings set. For example it's relivent to know if users with experimental regions enabled are running less sessions, etc. I believe I had a way to differentiate between the set event and just recording the setting each session but I'll have to look in to it.

Ideally I think it'd be best to have the iOS & Android events with the same names operate the same between platforms so we can do straight comparisons & analysis on the roll up data. I'll try to set aside some time this weekend to look at both iOS & Android analytics to see where they differ and where I think things could be improved.

[bbodenmiller]

@barbeau if you guys can wait a bit, I think it'd be easiest to wait until I have a chance to review the iOS analytics a bit more.

[bbodenmiller]

I believe the setting of the settings occurs under the ui_action category.

[cagryInside]

In the current IOS configuration, events for region reporting set as shown in the table below. Also, current Android GA reporting was developed according to this.

Event Category	Event Action	Event Label	When?
ui_action	set_region	Set Region: + regionName	The user select a region
ui_action	button_press	Set region automatically	The user select preference
ui_action	button_press	Set region manually	The user select preference
app_settings	configured_region	API Region	App starts
app_settings	general	Set Region Automatically	App starts
app_settings	general	Show Experimental Regions	App starts
app_settings	configured_region_auto	Set Region Automatically: %@; Old Region: %@	Auto region config

So, according to discussion above, we might not need to report events when app starts.
The proposal:

Event Category	Event Action	Event Label	When?
ui_action	set_region	Set Region: + regionName	The user select a region
ui_action	button_press	Set region automatically	The user select preference
ui_action	button_press	Set region manually	The user select preference
app_settings	configured_region_auto	Set Region Automatically: %@; Old Region: %@	Auto region config

[bbodenmiller]

What do we gain vs. loss with not reporting the set settings at app startup? My thinking is that we should track when a setting change occurs but also how many sessions are running with specific settings set.

[barbeau]

Thanks @cagryInside for the table showing what is triggered when - I didn't realize we were logging these both under ui_actions and app_settings. We should really have a table like this on the OBA Analytics page, since right now the whole concept of logging "sessions" is missing from that documentation on iOS. I'd like to have the documentation sufficiently detailed so that someone could create GA reports without having to dig through the code.

@bbodenmiller It looks like your PR OneBusAway/onebusaway-iphone#310 is going to fix the current issue we noticed on iOS (OneBusAway/onebusaway-iphone#376) where the region events are being logged at least twice for reach "session" - once when the user opens the app, and again when the user gets a call or hits the home button (i.e., in applicationWillResignActive). On Android we currently have it set up to log only when the app starts up, and it looks like your PR fix would bring the two implementations in line with one another.

@cagryInside and I just discussed this further, and on Android we're going to match the current iOS events (including "session" events that are logged at startup) to provide consistency for now, so we can get another build into beta and resume testing before pushing to production (we need to get "before" data on OBA Android for a study).

However, @cagryInside will post a proposal for changing both Android and iOS analytics to simply things, which would also include changing the exact action for how these "session" events are logged so we can discard the old bad iOS data and start fresh. Proposal will be posted here:
OneBusAway/onebusaway#5

...since it impacts both Android and iOS projects.

[bbodenmiller]

@barbeau Thanks for the details. I agree we should improve the documentation to include all the analytics details. The current page is certainly a good start.

I look forward to seeing the proposal and certainly agree simplifications would be great. I have to admit iOS analytics were pretty much thrown together just so we had something.