[Bug]: Failed to weave assembly error when opening a unity project.

[tomkrikorian]

What happened?

When opening our unity project, we get the Failed to weave assembly error all the time on one of our internal unity package called Opuscope.Core.Runtime. This package does not not have any dependence on realm btw.

Repro steps

Tried to reproduce on a new empty project but cannot reproduce the issue so far. Will update if can repro.

Version

10.6.0

What SDK flavour are you using?

MongoDB Realm (i.e. Sync, auth, functions)

What type of application is this?

Unity

Client OS and version

MacOS & Unity Editor 2021.1.27f1

Code snippets

No response

Stacktrace of the exception/crash you're getting

First stacktrace when opening the project : 

[Opuscope.Core.Runtime] Failed to weave assembly. If the error persists, please report it to https://github.com/realm/realm-dotnet/issues: System.IO.IOException: Sharing violation on path /Users/tomkrikorian/rainbow/src/Client.Core/UnityProject/Library/ScriptAssemblies/Opuscope.Core.Runtime.dll
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in <695d1cc93cca45069c528c15c9fdd749>:0 
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at Mono.Cecil.ModuleDefinition.GetFileStream (System.String fileName, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00006] in <800f825115904118ad11eac7876ca36c>:0 
  at Mono.Cecil.ModuleDefinition.Write (System.String fileName, Mono.Cecil.WriterParameters parameters) [0x00006] in <800f825115904118ad11eac7876ca36c>:0 
  at RealmWeaver.ResolutionResult.SaveModuleUpdates () [0x00006] in <800f825115904118ad11eac7876ca36c>:0 
  at RealmWeaver.UnityWeaver.WeaveAssemblyCore (System.String assemblyPath, System.Collections.Generic.IEnumerable`1[T] references, System.String framework, System.String targetOSName) [0x0009f] in <800f825115904118ad11eac7876ca36c>:0 
UnityEngine.Debug:LogError (object)
RealmWeaver.UnityWeaver/UnityLogger:Error (string,Mono.Cecil.Cil.SequencePoint)
RealmWeaver.UnityWeaver:WeaveAssemblyCore (string,System.Collections.Generic.IEnumerable`1<string>,string,string)
RealmWeaver.UnityWeaver/<>c__DisplayClass19_0:<WeaveAllAssemblies>b__1 ()
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Second stacktrace when using the weave assembly button in the realm unity window : 

[Opuscope.Core.Runtime] Failed to weave assembly. If the error persists, please report it to https://github.com/realm/realm-dotnet/issues: System.BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid.
  at Mono.Cecil.PE.ImageReader.ReadImage () [0x00013] in <800f825115904118ad11eac7876ca36c>:0 
  at Mono.Cecil.PE.ImageReader.ReadImage (Mono.Disposable`1[T] stream, System.String file_name) [0x00007] in <800f825115904118ad11eac7876ca36c>:0 
  at Mono.Cecil.ModuleDefinition.ReadModule (Mono.Disposable`1[T] stream, System.String fileName, Mono.Cecil.ReaderParameters parameters) [0x00006] in <800f825115904118ad11eac7876ca36c>:0 
  at Mono.Cecil.ModuleDefinition.ReadModule (System.String fileName, Mono.Cecil.ReaderParameters parameters) [0x0006c] in <800f825115904118ad11eac7876ca36c>:0 
  at RealmWeaver.WeaverAssemblyResolver.Resolve (System.String assemblyPath, System.Collections.Generic.IEnumerable`1[T] references) [0x00033] in <800f825115904118ad11eac7876ca36c>:0 
  at RealmWeaver.UnityWeaver.WeaveAssemblyCore (System.String assemblyPath, System.Collections.Generic.IEnumerable`1[T] references, System.String framework, System.String targetOSName) [0x00013] in <800f825115904118ad11eac7876ca36c>:0 
UnityEngine.Debug:LogError (object)
RealmWeaver.UnityWeaver/UnityLogger:Error (string,Mono.Cecil.Cil.SequencePoint)
RealmWeaver.UnityWeaver:WeaveAssemblyCore (string,System.Collections.Generic.IEnumerable`1<string>,string,string)
RealmWeaver.UnityWeaver/<>c__DisplayClass19_0:<WeaveAllAssemblies>b__1 ()
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Relevant log output

No response

[nirinchev]

Hey, thanks for the report! The first error looks like there are multiple processes trying to write to the same file. I am not sure what may be causing it as we've never seen this issue, but would be very interested in a repro. One thing that is interesting is - is this happening only when the editor is launched? I.e. if you go ahead and change some code, thus forcing a recompilation, do you still get an error from the Realm weaver or does it go away? If it's happening only at launch, it may indicate that we're racing with the compiler which is trying to recompile your code while we're trying to save the woven assembly.

[tomkrikorian]

Hi,
It's happening when we launch the editor yes. When changing some code, it works again and the error goes away.
Still trying to repro on my side on a new project but getting nowhere sadly.

[tomkrikorian]

Looking at the weaving code, maybe a fix would be to wait for the editor to finish compiling or updating the AssetDatabase before weaving ?
Like this (very dirty maybe though) :

 private static async Task<int> WeaveAllAssemblies()
        {
            var assembliesWoven = 0;
            try
            {
                while(EditorApplication.isCompiling || EditorApplication.isUpdating)
                {
                    await Task.Delay(300);
                }
                EditorApplication.LockReloadAssemblies();
                var weavingTasks = GetAssemblies()
                    .Select(assembly => Task.Run(() =>
                    {
                        if (!WeaveAssemblyCore(assembly.outputPath, assembly.allReferences, "Unity Editor", GetTargetOSName(Application.platform)))
                        {
                            return null;
                        }

                        string sourceFilePath = assembly.sourceFiles.FirstOrDefault();
                        if (sourceFilePath == null)
                        {
                            return null;
                        }

                        return sourceFilePath;
                    }));

                var weaveResults = await Task.WhenAll(weavingTasks);

                foreach (var result in weaveResults.Where(r => r != null))
                {
                    AssetDatabase.ImportAsset(result, ImportAssetOptions.ForceUpdate);
                    assembliesWoven++;
                }
            }
            catch (Exception ex)
            {
                UnityLogger.Instance.Error($"[Realm] Failed to weave assemblies. If the error persists, please report it to https://github.com/realm/realm-dotnet/issues: {ex}");
            }
            finally
            {
                EditorApplication.UnlockReloadAssemblies();
                if (assembliesWoven > 0)
                {
                    AssetDatabase.Refresh();
                }
            }
            return assembliesWoven;
        }

[nirinchev]

Hey Tom, yeah, that is pretty much in line with what I was thinking. It's going to be tough to verify the fix without a repro case, but if you're up for it, I can send you a .tgz package you can test with your main project.

[nirinchev]

Here's a link to the package from the PR with your proposed changes. If you have the time, please give it a try and let me know if it fixes the issue for you.

[tomkrikorian]

Thanks, I'll try this package then ! Will let you know asap if it fixes the issue for us :)

[tomkrikorian]

Issue is still here sadly even with this PR. I'll try to find some time to work on my repro project but looks like it's a hard one to fix.

[nirinchev]

If you're able to share your project privately, I can provide you with a secure upload link. Happy to also sign an NDA if you're concerned about it. I imagine it should be something silly/easy to fix, but without a repro case, it's hard to iterate.

[tomkrikorian]

Checked with the company and we're ok to send you the project privately. You can reach me at tom@minsar.com for the upload link :)

[tomkrikorian]

Updating the ticket with new informations, this only happens on MacOS (M1 & Intel). Never on Windows.

[nirinchev]

Hey Tom, thanks for the update and sorry for the long wait for a place to upload your project - there's some delay with setting up an account for me and all the folks who can help are off for Thanksgiving 😕 I'll send you an email as soon as everything is setup.

[tomkrikorian]

I managed to find a fix in the end. Looks like it was the array of Task.Run that was causing the issue for some reason. Don't know why it only happens on MacOS 🤷 See #2737