Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finish up the implementation of Binary Archives #207

Merged
merged 3 commits into from
Apr 7, 2021

Conversation

expenses
Copy link
Contributor

@expenses expenses commented Apr 6, 2021

As mentioned in gfx-rs/gfx#3716 (comment), we can use MTLBinaryArchives to cache pipelines to disk. This feature was really not clear to me, and seems to be best documented in the WWDC 2020 video Build GPU Binaries with Metal.

Binary Archives were partially implemented, but could not be used so this PR adds that functionality. I modified the circle example to use one but it's possible that making a new example would make more sense. I've tested that the binary archive is indeed written to with the write format, but I've had some trouble creating a pipeline with MTLPipelineOption::FailOnBinaryArchiveMiss to check that they're loaded okay.

@expenses
Copy link
Contributor Author

expenses commented Apr 6, 2021

I've had some trouble creating a pipeline with MTLPipelineOption::FailOnBinaryArchiveMiss to check that they're loaded okay.

Specfically, if I create a new pipeline function like:

pub fn new_render_pipeline_state_with_fail_on_binary_archive_miss(
        &self,
        descriptor: &RenderPipelineDescriptorRef,
    ) -> Result<RenderPipelineState, String> {
        unsafe {
            let pipeline_options =
            MTLPipelineOption::FailOnBinaryArchiveMiss;

            let pipeline_state: *mut MTLRenderPipelineState = try_objc! { err =>
                msg_send![self, newRenderPipelineStateWithDescriptor:descriptor
                                                                options:pipeline_options
                                                               error:&mut err]
            };

            Ok(RenderPipelineState::from_ptr(pipeline_state))
        }
    }

and use that, the example crashes with:

2021-04-06 09:16:20.147 circle[5835:37733] -[MTLIGAccelDevice newRenderPipelineStateWithDescriptor:options:error:]: unrecognized selector sent to instance 0x7f9101025000 fatal runtime error: Rust cannot catch foreign exceptions

I'm installing XCode now to see if that'll give me any more hints.

@expenses
Copy link
Contributor Author

expenses commented Apr 6, 2021

Hmm, while running it with XCode I get a different error when pipeline_state_descriptor.set_binary_archives(&[binary_archive]); is uncommented:

2021-04-06 17:00:09.345083+0200 circle[22665:266363] -[MTLDebugBinaryArchive findProgramObjectForFunction:]: unrecognized selector sent to instance 0x7faffb57d230
2021-04-06 17:00:09.346245+0200 circle[22665:266363] [General] An uncaught exception was raised
2021-04-06 17:00:09.346360+0200 circle[22665:266363] [General] -[MTLDebugBinaryArchive findProgramObjectForFunction:]: unrecognized selector sent to instance 0x7faffb57d230
2021-04-06 17:00:09.346492+0200 circle[22665:266363] [General] (
	0   CoreFoundation                      0x00007fff2079a6af __exceptionPreprocess + 242
	1   libobjc.A.dylib                     0x00007fff204d23c9 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff2081cc85 -[NSObject(NSObject) __retain_OA] + 0
	3   CoreFoundation                      0x00007fff2070207d ___forwarding___ + 1467
	4   CoreFoundation                      0x00007fff20701a38 _CF_forwarding_prep_0 + 120
	5   Metal                               0x00007fff28680850 -[MTLCompiler getProgramObjectForFunction:variantKey:requiredBinaryKeys:sourceBinaryArchives:] + 275
	6   Metal                               0x00007fff28680a4e -[MTLCompiler getProgramObject:destinationArchive:sourceBinaryArchives:variantKey:requiredBinaryKeys:failOnMiss:] + 106
	7   Metal                               0x00007fff28683066 __128-[MTLCompiler newRenderPipelineStateWithDescriptorInternal:options:reflection:destinationBinaryArchive:error:completionHandler:]_block_invoke + 201
	8   libdispatch.dylib                   0x000000010f8180b0 _dispatch_client_callout + 8
	9   libdispatch.dylib                   0x000000010f8293b2 _dispatch_lane_barrier_sync_invoke_and_complete + 135
	10  Metal                               0x00007fff286827a1 -[MTLCompiler newRenderPipelineStateWithDescriptorInternal:options:reflection:destinationBinaryArchive:error:completionHandler:] + 1134
	11  Metal                               0x00007fff286067e3 -[MTLCompiler newRenderPipelineStateWithDescriptor:options:reflection:error:completionHandler:] + 27
	12  Metal                               0x00007fff28651876 -[_MTLDevice newRenderPipelineStateWithDescriptor:options:reflection:error:] + 76
	13  MetalTools                          0x00007fff20dd7e6c -[MTLDebugDevice _newRenderPipelineStateWithDescriptor:options:reflection:error:] + 203
	14  libMTLCapture.dylib                 0x000000010f6f5d52 GTMTLGuestAppClient_initGlobalTransportWithHostURL + 244033
	15  circle                              0x000000010f31514d _ZN65_$LT$$LP$A$C$B$RP$$u20$as$u20$objc..message..MessageArguments$GT$6invoke17hf844f02f9a0b7cfeE + 141
	16  circle                              0x000000010f31cf16 _ZN4objc7message8platform15send_unverified17h22c33f2417098a93E + 134
	17  circle                              0x000000010f316653 _ZN5metal6device9DeviceRef25new_render_pipeline_state17h07a88b66f05fd32aE + 515
	18  circle                              0x000000010f28de09 _ZN6circle22prepare_pipeline_state17hf56cd230a3b82b84E + 1225
	19  circle                              0x000000010f28ce57 _ZN6circle4main17h58b9bd1379235028E + 1751
	20  circle                              0x000000010f28eeee _ZN4core3ops8function6FnOnce9call_once17h5179fd1d748d4770E + 14
	21  circle                              0x000000010f28fba1 _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h3f472f45e72faa27E + 17
	22  circle                              0x000000010f28fc34 _ZN3std2rt10lang_start28_$u7b$$u7b$closure$u7d$$u7d$17h0e1d107c7afde1d0E + 20
	23  circle                              0x000000010f34d3e4 _ZN3std2rt19lang_start_internal17h86f505dc7de50d93E + 644
	24  circle                              0x000000010f28fc11 _ZN3std2rt10lang_start17h66ec750870131ffdE + 65
	25  circle                              0x000000010f28def2 main + 34
	26  libdyld.dylib                       0x00007fff20643621 start + 1
	27  ???                                 0x0000000000000001 0x0 + 1

This doesn't happen when running normally with METAL_DEVICE_WRAPPER_TYPE=1 (the Metal validation layer thingy).

Seems to be the same error as https://developer.apple.com/forums/thread/672049.

@expenses
Copy link
Contributor Author

expenses commented Apr 7, 2021

I've had some trouble creating a pipeline with MTLPipelineOption::FailOnBinaryArchiveMiss to check that they're loaded okay.

This is working now, so we can probably go ahead and merge this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants