System.InvalidCastException at LiteDB.PageService.GetPage[T]

[Amazia]

Hi,
I use snowplow-dotnet-tracker inside a Xamarin iOS application.
I encounter this exception that crash my application:

Application Specific Information:
*** Terminating app due to uncaught exception 'System.AggregateException', reason: 'System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. ---> System.InvalidCastException: Specified cast is not valid.
at (wrapper castclass) System.Object.__castclass_with_cache(object,intptr,intptr)
at LiteDB.PageService.GetPage[T] (System.UInt32 pageID) <0x101946b20 + 0x00107> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.PageService.GetFreePage[T] (System.UInt32 startPageID, System.Int32 size) <0x1019470a0 + 0x00043> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.DataService.Insert (LiteDB.CollectionPage col, System.Byte[] data) <0x101942100 + 0x0007f> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.InsertDocument (LiteDB.CollectionPage col, LiteDB.BsonDocument doc) <0x101932f40 + 0x001c7> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine+<>c__DisplayClass18_0.b__0 (LiteDB.CollectionPage col) <0x101938830 + 0x0008b> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func2[T,TResult] action) <0x10197b200 + 0x00100> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0 at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable1[T] docs) <0x101932d50 + 0x0016f> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.Insert (System.String collection, LiteDB.BsonDocument doc) <0x101932c80 + 0x00073> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteCollection1[T].Insert (T document) <0x10191e330 + 0x00103> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0 at Snowplow.Tracker.Storage.LiteDBStorage.Put (System.String item) <0x1018fee20 + 0x000d7> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0 at Snowplow.Tracker.Queues.PersistentBlockingQueue.Enqueue (System.Collections.Generic.List1[T] items) <0x1018ff660 + 0x0011b> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at Snowplow.Tracker.Emitters.AsyncEmitter.Input (Snowplow.Tracker.Models.Payload payload) <0x101908980 + 0x000b7> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at Snowplow.Tracker.Tracker+<>c__DisplayClass46_0.b__0 () <0x1018fdda0 + 0x0003f> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at System.Threading.Tasks.Task.InnerInvoke () <0x10047d850 + 0x0005f> in <b238a3153e534349ad10ed0787f2157a#7af7393f81f215f195fc3bd1f6181fba>:0
at System.Threading.Tasks.Task.Execute () <0x10047cf50 + 0x00043> in <b238a3153e534349ad10ed0787f2157a#7af7393f81f215f195fc3bd1f6181fba>:0
--- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidCastException: Specified cast is not valid.
at (wrapper castclass) System.Object.__castclass_with_cache(object,intptr,intptr)
at LiteDB.PageService.GetPage[T] (System.UInt32 pageID) <0x101946b20 + 0x00107> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.PageService.GetFreePage[T] (System.UInt32 startPageID, System.Int32 size) <0x1019470a0 + 0x00043> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.DataService.Insert (LiteDB.CollectionPage col, System.Byte[] data) <0x101942100 + 0x0007f> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.InsertDocument (LiteDB.CollectionPage col, LiteDB.BsonDocument doc) <0x101932f40 + 0x001c7> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine+<>c__DisplayClass18_0.b__0 (LiteDB.CollectionPage col) <0x101938830 + 0x0008b> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func2[T,TResult] action) <0x10197b200 + 0x00100> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0 at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable1[T] docs) <0x101932d50 + 0x0016f> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteEngine.Insert (System.String collection, LiteDB.BsonDocument doc) <0x101932c80 + 0x00073> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0
at LiteDB.LiteCollection1[T].Insert (T document) <0x10191e330 + 0x00103> in <b99fd1821def4ff9a5794b6636ed75ad#7af7393f81f215f195fc3bd1f6181fba>:0 at Snowplow.Tracker.Storage.LiteDBStorage.Put (System.String item) <0x1018fee20 + 0x000d7> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0 at Snowplow.Tracker.Queues.PersistentBlockingQueue.Enqueue (System.Collections.Generic.List1[T] items) <0x1018ff660 + 0x0011b> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at Snowplow.Tracker.Emitters.AsyncEmitter.Input (Snowplow.Tracker.Models.Payload payload) <0x101908980 + 0x000b7> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at Snowplow.Tracker.Tracker+<>c__DisplayClass46_0.b__0 () <0x1018fdda0 + 0x0003f> in <b17d47f385b747f9a219933145b544a2#7af7393f81f215f195fc3bd1f6181fba>:0
at System.Threading.Tasks.Task.InnerInvoke () <0x10047d850 + 0x0005f> in <b238a3153e534349ad10ed0787f2157a#7af7393f81f215f195fc3bd1f6181fba>:0
at System.Threading.Tasks.Task.Execute () <0x10047cf50 + 0x00043> in <b238a3153e534349ad10ed0787f2157a#7af7393f81f215f195fc3bd1f6181fba>:0 <---

And this one:
at (wrapper castclass) System.Object.__castclass_with_cache(object,intptr,intptr)
LiteDB.PageService.GetPage(uint pageID)
LiteDB.DataService.GetBlock(PageAddress blockAddress)
LiteDB.DataService.Read(PageAddress blockAddress)
LiteDB.LiteEngine.d__9.MoveNext()
LiteDB.LiteCollection.d__17.MoveNext()
System.Collections.Generic.EnumerableHelpers.ToArray(IEnumerable source, ref int length)
System.Linq.Buffer.Buffer(IEnumerable source)
System.Linq.OrderedEnumerable.ToList(int minIdx, int maxIdx)
System.Linq.OrderedPartition.ToList()
System.Linq.Enumerable.ToList(IEnumerable source)
Snowplow.Tracker.Storage.LiteDBStorage.TakeLast(int n)
Snowplow.Tracker.Queues.PersistentBlockingQueue.Peek(int count, int maxWait)
Snowplow.Tracker.Emitters.AsyncEmitter.loop()
System.Threading.ThreadHelper.ThreadStart_Context(object state)
System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, object state, bool preserveSyncCtx)
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, object state, bool preserveSyncCtx)
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, object state)
System.Threading.ThreadHelper.ThreadStart()

Thanks for helping

[BenFradet]

@ninjabear @jbeemster any one of you able to help here?

[ninjabear]

I think it's got to be one of these problems in LiteDB @BenFradet which is used to cache events before they are sent to collectors.

I think long term we should replace LiteDB, 1.0.1 and 1.0.2 were already triggered by LiteDB bugs. The problem is finding a .netstandard compatible db. It just needs to implement this interface.

The storage interface can also be implemented in client-side code to provide your own storage mechanism which may be the solution here. There's some code somewhere for an in memory implementation of the storage interface which may be of help.

[Amazia]

Thanks for your answer. I will look for implementing my own storage.
Have you considered at least to upgrade the version of LiteDB ?
When I was looking for the error I found this issue
So maybe upgrading the version of LiteDB could improve.
But when I tried to upgrade the liteDB version in my solution I got this issue:
Error MT2101: Can't resolve the reference 'System.Void LiteDB.BsonMapper::RegisterAutoId(System.Func2<!!0,System.Boolean>,System.Func3<LiteDB.LiteEngine,System.String,!!0>)', referenced from the method 'System.Void Snowplow.Tracker.Storage.LiteDBStorage::.ctor(System.String)' in 'LiteDB, Version=3.1.0.0, Culture=neutral, PublicKeyToken=4ee40123013c9f27'. (MT2101)

[ninjabear]

Hey @Amazia I dug up an example (I would look to implement your own!) of custom storage:

Implementing IStorage

using Snowplow.Tracker.Storage;
using System;
using System.Collections.Generic;
using System.Linq;

...

public class MemoryStorage : IStorage
    {
        private List<string> _items = new List<string>(1000);

        public int TotalItems => _items.Count;

        public bool Delete(List<string> idList)
        {
            var ids = (from i in idList select int.Parse(i)).OrderByDescending(x=>x).ToList();
            ids.ForEach(id => _items.RemoveAt(id));
            return true;
        }

        public void Put(string item)
        {
            _items.Insert(0, item);
        }

        public List<StorageRecord> TakeLast(int n)
        {
            int totalBack = Math.Min(_items.Count(), n);
            int startIndex = _items.Count() - totalBack;

            var last = new List<StorageRecord>(totalBack);

            for (int idx=startIndex; idx<_items.Count(); idx++)
            {
                var item = new StorageRecord { Id = idx.ToString(), Item = _items[idx] };
                last.Add(item);
            }

            return last;
        }
    }

Building the tracker using custom storage

var collectorHostname = "<collector>";
var port = 80;
var logger = new ConsoleLogger();
var dest = new SnowplowHttpCollectorEndpoint(collectorHostname, method: Snowplow.Tracker.Endpoints.HttpMethod.POST, l: logger, port: port);
var storage = new MemoryStorage();
var queue = new PersistentBlockingQueue(storage, new PayloadToJsonString());
var emitter = new AsyncEmitter(dest, queue, l: logger);
Tracker.Instance.Start(emitter, synchronous: false);

If you've already got a nice data store you can wire it up similarly to this. The team is tied up with other projects at the moment but we're always appreciative of PRs if you can find the time or it would make your life easier (thinking about #82 too).

[Amazia]

Thanks for your comment :)
I have implemented my own storage using SQLite. I will create a PR as soon as it will be validated in production and if it interests you.