Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telemetry #66

Merged
merged 47 commits into from
Aug 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
a20a460
Fully generic TableSet leveraging sqlite3 over Persistable instances!
matux Jul 23, 2022
e5da9ad
Generic SQL instruction set.
matux Jul 24, 2022
43e7fa6
Added TableSet tests
matux Jul 25, 2022
5c50dcf
Removed old database.
matux Jul 25, 2022
d0a7771
Telemetry data structures, removed Result, some documentation.
matux Jul 26, 2022
18706fb
Fully composable processes
matux Jul 28, 2022
2e0269a
Moved Config back to /src
matux Jul 28, 2022
18e9b4a
Refactored Body to carry Telemetry.
matux Jul 31, 2022
338b1cb
Tests now use AsyncNotifier instead of IsolatedNotifier for testability
matux Jul 31, 2022
27c0cd5
Added Record for persistent telemetry readings.
matux Jul 31, 2022
eddf495
New language extensions.
matux Jul 31, 2022
b024ad8
Added sorting to TableSet, telemetry is always sorted by timestamp.
matux Jul 31, 2022
7586d57
Added Telemetry class that manages telemetry readings.
matux Jul 31, 2022
f68c643
Renamed ext dir in Rollbar Dart to extension.
matux Jul 31, 2022
bb105fa
Report is now a mixin.
matux Jul 31, 2022
e618eae
Updated minimum Dart/Flutter support to 2.17/3.0
matux Jul 31, 2022
61b3141
Updates to language extensions
matux Jul 31, 2022
c9dd5a2
Removed logging file.
matux Jul 31, 2022
b25e12b
Removed generated Telemetry Mocks and wrote them manually.
matux Jul 31, 2022
08b1793
Renamed PersistentSender to PersistentHttpSender.
matux Jul 31, 2022
efea96a
Send telemetry on every occurrence.
matux Jul 31, 2022
0a247e9
Created a ZippedIterable in Rollbar Common.
matux Aug 1, 2022
19575e4
Removed config from PayloadRecord
matux Aug 1, 2022
e0f8a55
Mixin for deep equality of nested data structures
matux Aug 1, 2022
984fdb0
Send number of processors to Rollbar.
matux Aug 1, 2022
94d706e
Equatable/Serializable test suite for all our data structures.
matux Aug 1, 2022
6a4968b
Retrieve a sorted by timestamp snapshot of all telemetry events
matux Aug 1, 2022
a50174c
Fixed dynamic casting issue in Json in PlatformTransformer.
matux Aug 2, 2022
9f3fd12
Added manual deep test of a payload serialization roundtrip.
matux Aug 2, 2022
e93b996
Fixed some tests.
matux Aug 2, 2022
7de9aaa
Enabled persisting payloads by default.
matux Aug 2, 2022
030011f
Don't keep a Config on Telemetry.
matux Aug 2, 2022
01cd7e6
Forcing sqlite3_flutter_libs to 3.5.8 due to #65
matux Aug 2, 2022
383d53d
Use IsolatedNotifier only with Flutter.
matux Aug 2, 2022
5912afd
Use latest pana version again.
matux Aug 2, 2022
508462e
Gitignore local db files.
matux Aug 2, 2022
635f3eb
Share unique database between types with Persistence.
matux Aug 2, 2022
6ba0ea5
Telemetry must be in Notifier since it must share the Isolate memory.
matux Aug 2, 2022
e694959
Use proper platform-specific directory to store database file.
matux Aug 3, 2022
ca1f612
Added telemetry to example, logging to http sender, fixed timestamp
matux Aug 3, 2022
a7cb8e6
Added configurable lifetime threshold for all persistent records.
matux Aug 3, 2022
df6801a
All tests passing.
matux Aug 3, 2022
2ef97be
Renamed Event to Occurrence, added @internal attributes to internals.
matux Aug 3, 2022
e373514
Cleaning up exports and public api.
matux Aug 3, 2022
a858b8d
Cleaned up some tests
matux Aug 3, 2022
c2e27e5
Misc cleanup
matux Aug 3, 2022
5f29254
Adjustments to the telemetry API.
matux Aug 4, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
restore-keys: |
${{ runner.os }}-deps-${{ matrix.flutter_version }}-
- name: Enable pana
run: flutter pub global activate pana 0.21.12
run: flutter pub global activate pana
- name: Show Flutter environment
run: |
which flutter
Expand Down
4 changes: 1 addition & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,4 @@ local.properties

**/.DS_Store

rollbar_payloads.db
# rollbar_payloads.db-journal
# rollbar_payloads copy- backup.db
**/*.db
2 changes: 1 addition & 1 deletion build_tools/run-pana
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ function run-pana-checks {
PUB_CACHE=$HOME/.pub-cache
fi

flutter pub global activate pana 0.21.12
flutter pub global activate pana

mkdir -p pana-report

Expand Down
15 changes: 14 additions & 1 deletion rollbar_common/lib/rollbar_common.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,23 @@ library rollbar_common;
export 'src/extension/function.dart';
export 'src/extension/object.dart';
export 'src/extension/collection.dart';
export 'src/extension/string.dart';
export 'src/extension/mirror.dart';
export 'src/extension/math.dart';
export 'src/extension/date_time.dart';
export 'src/extension/environment.dart';

export 'src/extension/tuple.dart';
export 'src/identifiable.dart';
export 'src/serializable.dart';
export 'src/persistable.dart';

export 'src/zipped.dart';
export 'src/table_set.dart';
export 'src/data/payload_record.dart';
export 'src/data/breadcrumb_record.dart';

export 'src/http.dart';
export 'src/level.dart';
export 'src/tuple.dart';

export 'src/connectivity_monitor.dart';
121 changes: 121 additions & 0 deletions rollbar_common/lib/src/data/breadcrumb_record.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
import 'package:meta/meta.dart';
import '../../rollbar_common.dart'
show
Persistable,
PersistableFor,
SerializableFor,
UUID,
uuidGen,
Datatype,
JsonMap;

@sealed
@immutable
class BreadcrumbRecord implements Persistable<UUID> {
matux marked this conversation as resolved.
Show resolved Hide resolved
@override
final UUID id;
final String breadcrumb;
final DateTime timestamp;

static Map<String, Datatype> get persistingKeyTypes => {
'id': Datatype.uuid,
'breadcrumb': Datatype.text,
'timestamp': Datatype.integer,
};

BreadcrumbRecord({
UUID? id,
required this.breadcrumb,
DateTime? timestamp,
}) : id = id ?? uuidGen.v4obj(),
timestamp = timestamp ?? DateTime.now().toUtc();

BreadcrumbRecord copyWith({
UUID? id,
String? breadcrumb,
DateTime? timestamp,
}) =>
BreadcrumbRecord(
id: id ?? this.id,
breadcrumb: breadcrumb ?? this.breadcrumb,
timestamp: timestamp ?? this.timestamp);

@override
factory BreadcrumbRecord.fromMap(JsonMap map) => BreadcrumbRecord(
id: map.id,
breadcrumb: map.breadcrumb,
timestamp: map.timestamp,
);

@override
JsonMap toMap() => {
'id': id.toBytes(),
'breadcrumb': breadcrumb,
'timestamp': timestamp.microsecondsSinceEpoch,
};

/// Compares this [BreadcrumbRecord] to another [BreadcrumbRecord].
///
/// Comparison is timestamp-based.
///
/// If [other] is not a [BreadcrumbRecord] instance, an [ArgumentError] is
/// thrown.
///
/// Returns a value like a [Comparator] when comparing this to [other]. That
/// is, it returns a negative integer if this is ordered before [other], a
/// positive integer if this is ordered after [other], and zero if this and
/// [other] are ordered together.
///
/// The [other] argument must be a value that is comparable to this object.
@override
int compareTo(other) {
if (other is! BreadcrumbRecord) {
throw ArgumentError('Cannot compare between different types.', other);
}

return timestamp.compareTo(other.timestamp);
}

@override
String toString() => 'Record('
'id: ${id.uuid}, '
'breadcrumb: $breadcrumb, '
'timestamp: $timestamp)';

@override
bool operator ==(Object other) =>
identical(this, other) ||
(other is BreadcrumbRecord &&
other.id == id &&
other.breadcrumb == breadcrumb &&
other.timestamp == timestamp);

@override
int get hashCode => Object.hash(id, breadcrumb, timestamp);
}

@sealed
class SerializableBreadcrumbRecord implements SerializableFor {
const SerializableBreadcrumbRecord();

@override
BreadcrumbRecord fromMap(JsonMap map) => BreadcrumbRecord.fromMap(map);
}

@sealed
class PersistableBreadcrumbRecord implements PersistableFor {
const PersistableBreadcrumbRecord();

@override
Map<String, Datatype> get persistingKeyTypes =>
BreadcrumbRecord.persistingKeyTypes;
}

extension BreadcrumbRecordAttributes on JsonMap {
UUID get id => UUID.fromList(this['id'].whereType<int>().toList());
String get breadcrumb => this['breadcrumb'];
DateTime get timestamp => DateTime.fromMicrosecondsSinceEpoch(
this['timestamp'],
isUtc: true,
);
}
141 changes: 141 additions & 0 deletions rollbar_common/lib/src/data/payload_record.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import 'package:meta/meta.dart';
import '../../rollbar_common.dart'
show
Persistable,
PersistableFor,
SerializableFor,
UUID,
uuidGen,
Datatype,
JsonMap;

@sealed
@immutable
class PayloadRecord implements Persistable<UUID> {
matux marked this conversation as resolved.
Show resolved Hide resolved
@override
final UUID id;
final String accessToken;
final String endpoint;
final String payload;
final DateTime timestamp;

static Map<String, Datatype> get persistingKeyTypes => {
'id': Datatype.uuid,
'accessToken': Datatype.text,
'endpoint': Datatype.text,
'payload': Datatype.text,
'timestamp': Datatype.integer,
};

PayloadRecord({
UUID? id,
required this.accessToken,
required this.endpoint,
required this.payload,
DateTime? timestamp,
}) : id = id ?? uuidGen.v4obj(),
timestamp = timestamp ?? DateTime.now().toUtc();

PayloadRecord copyWith({
UUID? id,
String? accessToken,
String? endpoint,
String? payload,
DateTime? timestamp,
}) =>
PayloadRecord(
id: id ?? this.id,
accessToken: accessToken ?? this.accessToken,
endpoint: endpoint ?? this.endpoint,
payload: payload ?? this.payload,
timestamp: timestamp ?? this.timestamp);

@override
factory PayloadRecord.fromMap(JsonMap map) => PayloadRecord(
id: map.id,
accessToken: map.accessToken,
endpoint: map.endpoint,
payload: map.payload,
timestamp: map.timestamp);

@override
JsonMap toMap() => {
'id': id.toBytes(),
'accessToken': accessToken,
'endpoint': endpoint,
'payload': payload,
'timestamp': timestamp.microsecondsSinceEpoch,
};

/// Compares this [PayloadRecord] to another [PayloadRecord].
///
/// Comparison is timestamp-based.
///
/// If [other] is not a [PayloadRecord] instance, an [ArgumentError] is
/// thrown.
///
/// Returns a value like a [Comparator] when comparing this to [other]. That
/// is, it returns a negative integer if this is ordered before [other], a
/// positive integer if this is ordered after [other], and zero if this and
/// [other] are ordered together.
///
/// The [other] argument must be a value that is comparable to this object.
@override
int compareTo(other) {
if (other is! PayloadRecord) {
throw ArgumentError('Cannot compare between different types.', other);
}

return timestamp.compareTo(other.timestamp);
}

@override
String toString() => 'Record('
'id: ${id.uuid}, '
'accessToken: $accessToken, '
'endpoint: $endpoint, '
'payload: $payload, '
'timestamp: $timestamp)';

@override
bool operator ==(Object other) =>
identical(this, other) ||
(other is PayloadRecord &&
other.id == id &&
other.accessToken == accessToken &&
other.endpoint == endpoint &&
other.payload == payload &&
other.timestamp == timestamp);

@override
int get hashCode =>
Object.hash(id, accessToken, endpoint, payload, timestamp);
}

@sealed
class SerializablePayloadRecord implements SerializableFor {
const SerializablePayloadRecord();

@override
PayloadRecord fromMap(JsonMap map) => PayloadRecord.fromMap(map);
}

@sealed
class PersistablePayloadRecord implements PersistableFor {
const PersistablePayloadRecord();

@override
Map<String, Datatype> get persistingKeyTypes =>
PayloadRecord.persistingKeyTypes;
}

extension PayloadRecordAttributes on JsonMap {
UUID get id => UUID.fromList(this['id'].whereType<int>().toList());
String get accessToken => this['accessToken'];
String get endpoint => this['endpoint'];
String get payload => this['payload'];
DateTime get timestamp => DateTime.fromMicrosecondsSinceEpoch(
this['timestamp'],
isUtc: true,
);
}
Loading