Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.HeaderPage'

[ArefGihuni]

Closed the app and now it's happening.

   at LiteDB.PageService.GetPage[T](UInt32 pageID)
   at LiteDB.PageService.NewPage[T](BasePage prevPage)
   at LiteDB.IndexService.AddNode(CollectionIndex index, BsonValue key, Byte level, IndexNode last)
   at LiteDB.LiteEngine.UpdateDocument(CollectionPage col, BsonDocument doc)
   at LiteDB.LiteEngine.<>c__DisplayClass29_0.<Update>b__0(CollectionPage col)
   at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func`2 action)
   at LiteDB.LiteCollection`1.Update(T document)
   at MyAppAsync()
   at System.Threading.Tasks.Task.Execute()

[mbdavid]

Are you using master branch version or 3.1.1 from Nuget?

[ArefGihuni]

Yes, version 3.1.0 from Nuget .. 3.1.1 was too slow so I downgraded ..

it's a UWP windows 10 app

[ArefGihuni]

I started getting InvalidCastException
Is there any way to fix it?

LiteDB.PageService.GetPage<System.__Canon>(Void* InstParam, UInt32 pageID) in at 0:0 
LiteDB.TransactionService.Commit() in at 0:0 
LiteDB.LiteEngine.Commit() in at 0:0 
LiteDB.LiteEngine.Transaction<System.Int32>$catch$0() in at 0:0 
LiteDB.LiteEngine.Insert(String collection, IEnumerable$1<$BsonDocument> docs) in at 0:0 
LiteDB.LiteEngine.Insert(String collection, $BsonDocument doc) in at 0:0 
LiteDB.LiteCollection$1<System.__Canon>.Insert(__Canon document) in at 0:0 

[mbdavid]

this error occurs in some concurrency scenarios. I'm working on this to fix

[lidanger]

I have added the option FileOptions.WriteThrough when opening the data file at 2017.6.13.
The program runs very well utill the same problem appeared 3 days ago again.It's strange, isn't?

The version of LiteDB is 3.1.1.0 with a modify of adding the option FileOptions.WriteThrough.
The threading model is single instance.

The first exception is

System.Threading.SynchronizationLockException: The write lock is being released without being held.
at System.Threading.ReaderWriterLockSlim.ExitWriteLock()
at LiteDB.LockService.b__17_2()
at LiteDB.LockService.<>c__DisplayClass11_0.b__0()
at LiteDB.LockControl.Dispose()
at LiteDB.LiteCollection`1.Insert(T document)

Then, It came after the program being restarted

System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.HeaderPage'.
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.PageService.NewPage[T](BasePage prevPage)
at LiteDB.PageService.GetFreePage[T](UInt32 startPageID, Int32 size)
at LiteDB.IndexService.AddNode(CollectionIndex index, BsonValue key, Byte level, IndexNode last)
at LiteDB.LiteEngine.InsertDocument(CollectionPage col, BsonDocument doc)
at LiteDB.LiteEngine.<>c__DisplayClass61_0.b__0(CollectionPage col)
at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func2 action) at LiteDB.LiteEngine.Insert(String collection, IEnumerable1 docs)
at LiteDB.LiteCollection1.Insert(IEnumerable1 docs)

@mbdavid

[mbdavid]

Hi @lidanger, I'm working on it in "nojournal" branch. It's not finish, but you can check.

[lidanger]

@mbdavid, that's good.I'll try.

[trichling]

Hi,

I have a similar issue, here is my stack trace:

System.InvalidCastException: 'Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.IndexPage'.'

at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.PageService.GetFreePage[T](UInt32 startPageID, Int32 size)
at LiteDB.IndexService.AddNode(CollectionIndex index, BsonValue key, Byte level, IndexNode last)
at LiteDB.LiteEngine.InsertDocument(CollectionPage col, BsonDocument doc)
at LiteDB.LiteEngine.<>c__DisplayClass33_0.b__0(CollectionPage col)
at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func2 action) at LiteDB.LiteEngine.Upsert(String collection, IEnumerable1 docs)
at LiteDB.LiteEngine.Upsert(String collection, BsonDocument doc)
at LiteDB.LiteCollection`1.Upsert(T document)

Thanks for investigating

[JasonGrass]

Hi, I have a similar issue too , here is my stack trace:
version is 4.1.1

System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'.
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data, BsonReader bsonReader)
at LiteDB.LiteEngine.d__9.MoveNext()
at LiteDB.LiteEngine.d__13.MoveNext()
at LiteDB.LiteCollection1.<Find>d__17.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source)
at xxxCache.<>c__DisplayClass29_0.b__0(LiteDatabase db)
at xxxCache.Execute[TReturn](Func`2 execute)

[mbdavid]

Hi @JasonGrass, I never got this error anymore (on v4.x). Can you use LiteDB.Demo project (inside LiteDB source) to make a replication of this error?

[JasonGrass]

Here is the broken db file ，
But it can be read by using shell command.
I do not know how it been broken.

CoursewareCache.zip

[os-alan]

Same thing. Got this exception recently. Version is 4.1.1 (from Nuget)

System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.IndexPage'.
   at LiteDB.PageService.GetPage[T](UInt32 pageID)
   at LiteDB.IndexService.<FindAll>d__13.MoveNext()
   at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data, BsonReader bsonReader)
   at LiteDB.LiteEngine.<Find>d__9.MoveNext()
   at LiteDB.LiteEngine.<Find>d__13.MoveNext()
   at LiteDB.LiteCollection`1.<Find>d__17.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)

And here's the code producing it:
var chunk = liteCollection.Find(x => x.DocumentId == documentId, skip: allItems.Count, limit: chunkSize);

[mbdavid]

Hi @guys, this code/database already has broken and only with this I can replicate this broken problem. I never got this problem in v4.x. How are using LiteDB over concurrency? Are web/mobile or desktop app? How write operations are made? If possible, create a sample example using LiteDB.Demo project. I made this project just to catch this kind concurrency of problem.

[os-alan]

Hi @mbdavid!
Frankly speaking I'm not getting that issue since that moment and cannot reproduce it, though I tried. It is a desktop app (WPF) and I use write/update/delete records from different threads. Will keep you posted, if I face it again.

[mbdavid]

Hi @os-alan, if you could reproduce this error in v4 please send to me. After corrupt database I can test anymore.

[MarjieAVolk]

Hey there - My team's product has been using LiteDB for a little while without seeing this problem.
But just recently we updated from v3 something to v4.1.1.0, and also we switched over to using LiteDB's concurrency features (keeping a single instance of the database open and accessing it on multiple threads). Just after doing this, we started getting error reports from our users, with both this error message (StrangeLoopGames/EcoIssues#5548) and a variety of similar messages (StrangeLoopGames/EcoIssues#5610, StrangeLoopGames/EcoIssues#5707).

It's possible the people who are seeing this had an existing .db/.journal file and it was upgraded to v4. I haven't been sure what to ask them. Would it be helpful for me to get their .db files and share them here?

[mbdavid]

Hi @MarjieAVolk, will be very helpfull if you can send me a copy of this corruped database to check. Did you know if this datafile was updated from v3? In v3 you don't have this problem?

[MarjieAVolk]

Yes, in v3 I did not have the problem. But I also was not using concurrency in the same way. I will ask these people for their db files. Thanks.

[MarjieAVolk]

Okay, I got one of the .db files that is seeing the error:
Game.zip

This is the exact stack trace they reported:

Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.HeaderPage'.

Stack:
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.TransactionService.PersistDirtyPages()
at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func2 action) 
at LiteDB.LiteEngine.Insert(String collection, BsonDocument doc, BsonType autoId) 
at LiteDB.LiteCollection1.Insert(T document)
at Eco.Core.Utils.AtomicAction.SimpleAtomicAction.TryApply()
at Eco.Core.Utils.AtomicAction.MultiAtomicAction.TryApply()
at Eco.Core.Utils.AtomicAction.MultiAtomicAction.TryApply()
at Eco.Gameplay.Components.CraftingComponent.CreateWorkOrder(Player player, Recipe recipe, Int32 quantity)

[MarjieAVolk]

Any updates on this? Did that file have the same issue?

[mbdavid]

Hi @MarjieAVolk, not yet, sorry. I will try check this soon.

[MarjieAVolk]

Here's another one:

Game.zip

Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'.

Stack:
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.PageService.GetFreePage[T](UInt32 startPageID, Int32 size)
at LiteDB.DataService.Insert(CollectionPage col, Byte[] data)
at LiteDB.LiteEngine.InsertDocument(CollectionPage col, BsonDocument doc, BsonType autoId)
at LiteDB.LiteEngine.<>c__DisplayClass22_0.b__0(CollectionPage col)
at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func2 action) at LiteDB.LiteEngine.Insert(String collection, BsonDocument doc, BsonType autoId) at LiteDB.LiteCollection1.Insert(T document)
at Eco.Core.Utils.AtomicAction.SimpleAtomicAction.TryApply()
at Eco.Core.Utils.AtomicAction.MultiAtomicAction.TryApply()
at Eco.Core.Utils.AtomicAction.MultiAtomicAction.TryApply()
at Eco.Gameplay.Items.UsableItemUtils.PlayerDeleteBlock(Vector3i blockPosition, Player player, Boolean addToInventory, BlockItem fallbackGiveItem, IAtomicAction[] additionalActions)
at Eco.Gameplay.Items.ToolItem.PlayerDeleteBlock(Vector3i blockPosition, Player player, Boolean addToInventory, Single calorieMultiplier, BlockItem fallbackGiveItem, IAtomicAction[] additionalActions)
at Eco.Mods.TechTree.ShovelItem.OnActLeft(InteractionContext context)
at Eco.Gameplay.Interactions.InteractionExtensions.ExecuteUntilSuccess(IEnumerable`1 results)
at Eco.Gameplay.Players.Player.PlayerInteract(InteractionInfo info)

[MarjieAVolk]

I see these are different stack traces, so maybe I should file separate bugs for each one?

[mbdavid]

Both error are caused with same problem: looking for wrong page. Thats happend when, for some reason, page are corrupted and one page are linking to a wrong page (with different type). What collection name are you inserting/finding?

[MarjieAVolk]

I knew I should have copied the full stack traces, haha.

Okay, for the first one I sent, it will be a CraftAction. For the second one, it is a PickUpAction.

[mbdavid]

Hi @MarjieAVolk, is this second database corrupted by an bug that was "added" in commit on 11/dec here:
https://github.com/mbdavid/LiteDB/commits/master/LiteDB/Utils/Extensions/DictionaryExtensions.cs
This bugs occurs only sometimes depending on concurrency... it's was reverted (fixed) and are present in v4.1.2... can you test on this version to try this error again?

[MarjieAVolk]

You are saying the error is fixed in 4.1.2? I haven't really been able to reproduce these errors on my own machine, I only get reports of them from our players. Do you want me to check whether the second database works when run on 4.1.2?

[mbdavid]

If you can not reproduce the error, you can only update your source code to use 4.1.2 to check if you will get more users error report (with datafiles created in v4.1.2). In datafiles with this error in old version is not possible do anything because datafile are corrupted. If this problem occurs again (i hope no :) you can open another issue about this because this error is not same as in this issue (same error message but different reasons)

[MarjieAVolk]

Okay, we haven't had complaints of this for a long time, but we just got another one. We are still on 4.1.2, so it might not be relevant to the current release, but here it is just in case:

2018-07-20 22:54:05.7874|ERROR:Unhandled exception: System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'. 
at LiteDB.PageService.GetPage[T](UInt32 pageID) at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data) at LiteDB.LiteEngine.<Find>d__9.MoveNext() 
at LiteDB.LinqExtensions.<Batch>d__01.MoveNext()
at LiteDB.LiteEngine.InsertBulk(String collection, IEnumerable1 docs, Int32 batchSize, BsonType autoId) 
at LiteDB.LiteEngine.Shrink(String password, IDiskService tempDisk) 
at LiteDB.LiteDatabase.Shrink(String password) 
at LiteDB.LiteDatabase.Shrink() 
at LiteDbExplorer.Windows.DatabasePropertiesWindow.ButtonShrink_Click(Object sender, RoutedEventArgs e)

The bad .db file:
GameDB.zip

[FlorianWeikert]

Hi, we're using v4.1.4 and since yesterday get the same error on different tables.
Attached the broken file:
Broken.zip

[js-seth-h]

Same Issue. LiteDB Ver 4.1.4 from nuget.
And furthermore, in my case, database broken was occur when system power off accidentally.
My software logs and Window Event log tell me that not-logged system turn down happen in gap of tasks (less than 5 minutes, and reason of turn off is unknown, yet).
After that, DB broken. I get same error "exception: System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'."
And my Software still open database at the time of window turn off.
I guess that writing operation not finished because window problem.

[djarvis]

Hi.

I am seeing this same exception under heavy-ish load with multiple threads. Using v4.1.4 in a .NET Core web api project running in a Linux docker container hosted in Azure.

Does anyone know of any safe threshold for concurrent tasks where I may not run into this issue?

[matkuki]

Hi @mbdavid ,
I get the same error Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage' when trying to read an entire collection from a database file that was corrupted when I copied the file without closing my application first.

My question is: Is there a way to skip the corrupted items?
Example that throws an error:

var col = db.GetCollection<MyDbItem>(TABELA_STATES);
var all = col.FindAll();
foreach (var i in all) //THIS LINE THROWS THE ERROR WHEN IT REACHES THE CORRUPT ITEM!
{
    // Some code
}

Thanks

[FlorianWeikert]

Hi @mbdavid,
is there any update on this issue? Or some kind of usefull workaround?

Thanks

[mbdavid]

Hi @scr4py4rd, sorry about this problem. I try reproduce this error for a long time and get no way to reproduce... so was not possible to fix yet.

I about away from LiteDB project for some time because I need focus in my full work job. I believe that I will back next month. I writing v5 and my plans is fix all this in this new version because v5 works quite different in a more robust memory/manage way...

[djarvis]

Having had time pass allowed for some more observation. Though still not very clear, the issue tends to surface when some other process or thread does a file-read on the database file, like perhaps a file-copy, during which time the LiteDB library is active with it.

[FlorianWeikert]

Hi @mbdavid, thanks for your reply.

Do you have a plan, when v5 will be ready?

[junbujianwpl]

This is our exception stack. We use the newest version from Nuget. It seems like a tough bug surviving several versions.

Unhandled Exception System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'. at LiteDB.PageService.GetPage[T](UInt32 pageID) at LiteDB.PageService.GetFreePage[T](UInt32 startPageID, Int32 size) at LiteDB.DataService.Insert(CollectionPage col, Byte[] data) at LiteDB.LiteEngine.InsertDocument(CollectionPage col, BsonDocument doc, BsonType autoId) at LiteDB.LiteEngine.<>c__DisplayClass33_0.<Upsert>b__0(CollectionPage col) at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func '2 action) at LiteDB.LiteEngine.Upsert(String collection, IEnumerable'1 docs, BsonType autoId) at LiteDB.LiteCollection'1.Upsert(IEnumerable '1 documents) at LiteDB.LiteCollection'1.Upsert(T document)

[FlorianWeikert]

Tried your current v5 branch and get a similiar exception.

[jmahlitz]

I can safely reproduce this error on my HDD.
For that i use a 50 mb csv file and upload it concurrently with 20x Task.Run(() => method);

In method i do:

1. I get the connection via new LiteDatabase(path);
2. I upload files with Upload(Guid.New(), "file500" stream);

After starting the application and seeing file size of the db growing (in explorer) i close the console process.
After restarting the application and trying to read from the db, i then get the error when doing a FindAll().

[jmahlitz]

@scr4py4rd : If i use only one connection without disposing it, i am not able to produce the error. Maybe you can try using it as singleton.

[FlorianWeikert]

@jmahlitz : I use it in an asp core application, define it as singleton in the core dependency injection IServiceCollection and use it in mode exclusive.

Attached my testcode to crash the file:
DatabaseJobTests.zip

I create a new database without any special parameter, create 3 collections with 5000 entries and start from multiple other threads read&write access.

With version 5, I get with attached code an error in 2 of 3 runs:
at LiteDB.Engine.HeaderPage.ReadContent(BinaryReader reader, Boolean utcDate) at LiteDB.Engine.BasePage.ReadPage(BinaryReader reader, Boolean readContent, Boolean utcDate) at LiteDB.Engine.LogFileService.ReadPages(Boolean readContent)+MoveNext() at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext()
at LiteDB.LinqExtensions.ForEach[T](IEnumerable1 source, Action2 action)+MoveNext()
at LiteDB.Engine.DataFileService.WritePages(IEnumerable1 pages) at LiteDB.Engine.WalService.Checkpoint(HeaderPage header, Boolean lockReserved) at LiteDB.Engine.LiteEngine.Shutdown() at LiteDB.Engine.LiteEngine.Dispose() at KMA.TraXability.Tests.SystemMaintenanceTests.DatabaseBackupJobTests.DbCloneWhileWriting() in C:\src\KMA.TraXability\Test\KMA.TraXability.Tests\SystemMaintenanceTests\DatabaseBackupJobTests.cs:line 153

With version 4 I can't reproduce the crash, it appears only in production system every ~4-6 days (6000 write / 12000 read operations per day)

[shorstok]

@mbdavid Any workarounds / suggestions for now? I'm having this issue too, after database was corrupted by sudden power-out

[hazems]

+1 I have this issue too, thanks

[maynardflies]

I get this problem as well, For me it happens when I am trying to drop the collection and the error is slightly different, "Unable to cast object of type 'LiteDB.IndexPage' to type 'LiteDB.DataPage'.". The database shouldn't be in use by anything else

[jjxtra]

Seeing this in 4.1.4...

[oddgames-david]

This was using 4.1.4, in Unity on an Android device.

data.zip

[otavioreis]

I had the same problem yesterday. "Não é possível converter um objeto do tipo 'LiteDB.EmptyPage' no tipo 'LiteDB.DataPage'.
I think the data got corrupted after a power interruption that turned off the computer where the file was being saved. My software has a worker that access the DB file regularly.
Do you have any way to identify the corrupted data page to avoid it? Or something that could help me to identify that the file is corrupted because I am planning to mirror all the data on physical files, and use them to restore the database if it gets corrupted.

data.zip

[jjxtra]

Saw this error enough that I quickly went back to sqlite, no errors since the migration back

[NidhiBisht]

Getting this issue

[hesamfaraji]

i have the same issue in version 4.1.4.0
stack trace:

[InvalidCastException: Unable to cast object of type 'LiteDB.HeaderPage' to type 'LiteDB.CollectionPage'.]
LiteDB.PageService.GetPage(UInt32 pageID) +226
LiteDB.CollectionService.Get(String name) +96
LiteDB.LiteEngine.GetCollectionPage(String name, Boolean addIfNotExits) +37
LiteDB.d__9.MoveNext() +261
LiteDB.d__13.MoveNext() +140
LiteDB.d__17.MoveNext() +153
System.Linq.Buffer1..ctor(IEnumerable1 source) +120
System.Linq.d__1.MoveNext() +175
System.Linq.WhereSelectEnumerableIterator2.MoveNext() +87 System.Linq.<DistinctIterator>d__631.MoveNext() +220
System.Web.UI.WebControls.ListControl.PerformDataBinding(IEnumerable dataSource) +276
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +290
System.Web.UI.WebControls.ListControl.PerformSelect() +37
System.Web.UI.WebControls.BaseDataBoundControl.DataBind() +74
Pulse.panel.bindJobGroups() in C:\Users\faraji-h\Documents\projects4\b\CEF2\Pulse\panel.aspx.cs:55
Pulse.d__2.MoveNext() in C:\Users\faraji-h\Documents\projects4\b\CEF2\Pulse\panel.aspx.cs:46
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58
Pulse.<Page_Load>d__0.MoveNext() in C:\Users\faraji-h\Documents\projects4\b\CEF2\Pulse\panel.aspx.cs:31
System.Runtime.CompilerServices.<>c.b__6_0(Object state) +52
System.Web.<>c__DisplayClass22_0.b__0() +20
System.Web.Util.SynchronizationHelper.SafeWrapCallback(Action action) +90
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58
System.Web.Util.WithinCancellableCallbackTaskAwaiter.GetResult() +31
System.Web.UI.d__246.MoveNext() +397
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58
System.Web.Util.WithinCancellableCallbackTaskAwaiter.GetResult() +31
System.Web.UI.d__523.MoveNext() +4866

[Waxavi]

Hello, any update on this issue, is there any workaround?

[JKamsker]

This issue should have been fixed a year ago, i will close this issue for now. Please let me know if the problem persists anyway.