Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault #488

Closed
Finn-Sueberkrueb opened this issue Oct 5, 2022 · 6 comments · Fixed by #502
Closed

Segmentation fault #488

Finn-Sueberkrueb opened this issue Oct 5, 2022 · 6 comments · Fixed by #502
Assignees
Labels
gc garbage collection

Comments

@Finn-Sueberkrueb
Copy link

Thank you very much for this wonderful package!

The following minimal example.

using Enzyme;

function objective!(x, loss, R)
    for i in 1:1000
        #y = R * x
        y = zeros(3)
        y[1] = R[1,1] * x[1] + R[1,2] * x[2] + R[1,3] * x[3]

        loss[] = y[1]
    end
    return nothing
end;

x = zeros(3)
R = [1.0 0.0 0.0
    0.0 1.0 0.0
    0.0 0.0 1.0]
loss = Ref(0.0)
dloss = Ref(1.0)

for i in 1:1000
    loss[] = 0.0
    dloss[] = 1.0
    autodiff(Reverse, objective!, Duplicated(x, zero(x)), Duplicated(loss, dloss), R)
end

Raises the following segmentation fault. The error occurs after varying number of AutoDiff calls.

signal (11): Segmentation fault: 11
in expression starting at /Users/src/minimal_example.jl:26
gc_mark_loop at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
_jl_gc_collect at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_gc_collect at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_gc_pool_alloc_noinline at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_gc_alloc at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
_new_array_ at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
_new_array at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_alloc_array_1d at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
Allocations: 56871453 (Pool: 56818646; Big: 52807); GC: 61

Version Info

Julia Version 1.8.0
Commit 5544a0fab76 (2022-08-17 13:38 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.3.0)
  CPU: 10 × Apple M1 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
  Threads: 1 on 8 virtual cores

[7da242da] Enzyme v0.10.6 `https://github.com/EnzymeAD/Enzyme.jl.git#main`
@Finn-Sueberkrueb
Copy link
Author

Error on a x86 machine

signal (11): Segmentation fault
in expression starting at /home/user/minimal_example.jl:24
page_metadata at /buildworker/worker/package_linux64/build/src/gc.h:457 [inlined]
gc_setmark_pool at /buildworker/worker/package_linux64/build/src/gc.c:806 [inlined]
gc_setmark at /buildworker/worker/package_linux64/build/src/gc.c:813 [inlined]
gc_mark_loop at /buildworker/worker/package_linux64/build/src/gc.c:2714
_jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:3039
jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:3248
maybe_collect at /buildworker/worker/package_linux64/build/src/gc.c:882 [inlined]
jl_gc_pool_alloc at /buildworker/worker/package_linux64/build/src/gc.c:1209
jl_gc_alloc_ at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_gc_alloc at /buildworker/worker/package_linux64/build/src/gc.c:3290
_new_array_ at /buildworker/worker/package_linux64/build/src/array.c:127 [inlined]
_new_array at /buildworker/worker/package_linux64/build/src/array.c:193 [inlined]
jl_alloc_array_1d at /buildworker/worker/package_linux64/build/src/array.c:441
unknown function (ip: 0x7f0fe80984cb)
macro expansion at /home/user/.julia/packages/Enzyme/2K2vv/src/compiler.jl:5699 [inlined]
enzyme_call at /home/user/.julia/packages/Enzyme/2K2vv/src/compiler.jl:5461
unknown function (ip: 0x7f0fe80b4b1a)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:713
CombinedAdjointThunk at /home/user/.julia/packages/Enzyme/2K2vv/src/compiler.jl:5438
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:713
autodiff at /home/user/.julia/packages/Enzyme/2K2vv/src/Enzyme.jl:302
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:713
autodiff at /home/user/.julia/packages/Enzyme/2K2vv/src/Enzyme.jl:330
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
top-level scope at /home/user/minimal_example.jl:27
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:876
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:830
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:944
eval at ./boot.jl:373 [inlined]
include_string at ./loading.jl:1196
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
_include at ./loading.jl:1253
include at ./Base.jl:418
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
exec_options at ./client.jl:292
_start at ./client.jl:495
jfptr__start_38732.clone_1 at /opt/julia-1.7.2/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1788 [inlined]
true_main at /buildworker/worker/package_linux64/build/src/jlapi.c:559
jl_repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:701
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x400808)
Allocations: 28616321 (Pool: 28603117; Big: 13204); GC: 33

Version Info

Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

@wsmoses wsmoses added the gc garbage collection label Oct 5, 2022
@wsmoses
Copy link
Member

wsmoses commented Oct 5, 2022

@vchuravy this seems to occur, though nondeterminstically, and I'd guess GC related since we do cache the (shadow) julia object in the loop

@wsmoses
Copy link
Member

wsmoses commented Oct 6, 2022

mod = ; ModuleID = 'text'
source_filename = "text"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-linux-gnu"

; Function Attrs: nofree nosync nounwind readnone speculatable willreturn
declare void @llvm.dbg.declare(metadata, metadata, metadata) #0

; Function Attrs: noreturn
declare void @ijl_bounds_error_ints({} addrspace(12)*, i64*, i64) local_unnamed_addr #1

define private void @julia_objective__1863_inner.1({} addrspace(10)* nocapture nonnull readonly align 16 dereferenceable(40) %0, {} addrspace(10)* nocapture nofree nonnull writeonly align 8 dereferenceable(8) %1, {} addrspace(10)* nocapture nonnull readonly align 16 dereferenceable(40) %2) local_unnamed_addr #2 !dbg !5 {
entry:
  call void @llvm.dbg.value(metadata {} addrspace(10)* null, metadata !22, metadata !DIExpression(DW_OP_deref)), !dbg !23
  call void @llvm.dbg.value(metadata {} addrspace(10)* null, metadata !21, metadata !DIExpression(DW_OP_deref)), !dbg !23
  call void @llvm.dbg.value(metadata {} addrspace(10)* null, metadata !20, metadata !DIExpression(DW_OP_deref)), !dbg !23
  call void @llvm.dbg.value(metadata {} addrspace(10)* %0, metadata !20, metadata !DIExpression(DW_OP_deref)), !dbg !23
  call void @llvm.dbg.value(metadata {} addrspace(10)* %1, metadata !21, metadata !DIExpression(DW_OP_deref)), !dbg !23
  call void @llvm.dbg.value(metadata {} addrspace(10)* %2, metadata !22, metadata !DIExpression(DW_OP_deref)), !dbg !23
  %3 = bitcast {} addrspace(10)* %2 to double addrspace(13)* addrspace(10)*
  %4 = addrspacecast double addrspace(13)* addrspace(10)* %3 to double addrspace(13)* addrspace(11)*
  %5 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %4, align 16
  %6 = bitcast {} addrspace(10)* %0 to double addrspace(13)* addrspace(10)*
  %7 = addrspacecast double addrspace(13)* addrspace(10)* %6 to double addrspace(13)* addrspace(11)*
  %8 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %7, align 16
  %9 = bitcast {} addrspace(10)* %1 to double addrspace(10)*
  br label %L2.i, !dbg !25

L2.i:                                             ; preds = %idxend.i, %entry
  %value_phi.i = phi i64 [ 1, %entry ], [ %28, %idxend.i ]
  %10 = call noalias nonnull {} addrspace(10)* inttoptr (i64 140080130477952 to {} addrspace(10)* ({} addrspace(10)*, i64)*)({} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 140079781262464 to {}*) to {} addrspace(10)*), i64 noundef 3), !dbg !26
  %11 = bitcast {} addrspace(10)* %10 to { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(10)*, !dbg !37
  %12 = addrspacecast { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(10)* %11 to { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(11)*, !dbg !37
  %13 = getelementptr inbounds { i8 addrspace(13)*, i64, i16, i16, i32 }, { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(11)* %12, i64 0, i32 1, !dbg !37
  %14 = load i64, i64 addrspace(11)* %13, align 8, !dbg !37, !tbaa !49, !range !54
  %.not.not = icmp eq i64 %14, 0, !dbg !55
  br i1 %.not.not, label %oob.i, label %idxend.i, !dbg !66

oob.i:                                            ; preds = %L2.i
  %.phi.trans.insert = bitcast {} addrspace(10)* %10 to double addrspace(13)* addrspace(10)*
  %.phi.trans.insert8 = addrspacecast double addrspace(13)* addrspace(10)* %.phi.trans.insert to double addrspace(13)* addrspace(11)*
  %.pre = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %.phi.trans.insert8, align 8, !dbg !67, !tbaa !70
  %15 = load double, double addrspace(13)* %5, align 8, !dbg !72, !tbaa !74
  %16 = load double, double addrspace(13)* %8, align 8, !dbg !77, !tbaa !74
  %17 = fmul double %15, %16, !dbg !78
  store double %17, double addrspace(13)* %.pre, align 8, !dbg !67, !tbaa !74
  %18 = alloca i64, align 8, !dbg !81
  store i64 1, i64* %18, align 8, !dbg !81
  %19 = addrspacecast {} addrspace(10)* %10 to {} addrspace(12)*, !dbg !81
  call void @ijl_bounds_error_ints({} addrspace(12)* %19, i64* noundef nonnull align 8 %18, i64 noundef 1) #1, !dbg !81
  unreachable, !dbg !81

idxend.i:                                         ; preds = %L2.i
  %20 = bitcast {} addrspace(10)* %10 to double addrspace(13)* addrspace(10)*
  %21 = addrspacecast double addrspace(13)* addrspace(10)* %20 to double addrspace(13)* addrspace(11)*
  %22 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %21, align 8, !tbaa !70, !nonnull !4
  %23 = bitcast double addrspace(13)* %22 to i8 addrspace(13)*, !dbg !83
  %24 = shl nuw i64 %14, 3, !dbg !83
  call void @llvm.memset.p13i8.i64(i8 addrspace(13)* noundef nonnull align 8 %23, i8 noundef 0, i64 %24, i1 noundef false), !dbg !84
  %25 = load double, double addrspace(13)* %5, align 8, !dbg !72, !tbaa !74
  %26 = load double, double addrspace(13)* %8, align 8, !dbg !77, !tbaa !74
  %27 = fmul double %25, %26, !dbg !78
  store double %27, double addrspace(13)* %22, align 8, !dbg !67, !tbaa !74
  store double %27, double addrspace(10)* %9, align 8, !dbg !86, !tbaa !92
  %.not5 = icmp eq i64 %value_phi.i, 1000, !dbg !95
  %28 = add nuw nsw i64 %value_phi.i, 1, !dbg !100
  br i1 %.not5, label %julia_objective__1863_inner.exit, label %L2.i, !dbg !101

julia_objective__1863_inner.exit:                 ; preds = %idxend.i
  ret void, !dbg !102
}

; Function Attrs: argmemonly nofree nosync nounwind willreturn
declare void @llvm.lifetime.start.p0i8(i64 immarg, i8* nocapture) #3

; Function Attrs: argmemonly nofree nosync nounwind willreturn
declare void @llvm.lifetime.end.p0i8(i64 immarg, i8* nocapture) #3

; Function Attrs: nofree nosync nounwind willreturn
declare i8* @llvm.stacksave() #4

; Function Attrs: nofree nosync nounwind willreturn
declare void @llvm.stackrestore(i8*) #4

; Function Attrs: nofree nosync nounwind readnone speculatable willreturn
declare void @llvm.dbg.value(metadata, metadata, metadata) #0

; Function Attrs: argmemonly nofree nounwind willreturn writeonly
declare void @llvm.memset.p13i8.i64(i8 addrspace(13)* nocapture writeonly, i8, i64, i1 immarg) #5

declare {} addrspace(10)* @ijl_box_int64(i64)

declare {} addrspace(10)* @jl_f_apply_type({} addrspace(10)*, {} addrspace(10)**, i32)

declare {} addrspace(10)* @jl_gc_alloc_typed(i8*, i64, {} addrspace(10)*)

declare {} addrspace(10)* @julia.gc_alloc_obj({}**, i64, {} addrspace(10)*)

; Function Attrs: argmemonly nofree nounwind willreturn writeonly
declare void @llvm.memset.p10i8.i64(i8 addrspace(10)* nocapture writeonly, i8, i64, i1 immarg) #5

declare void @julia.write_barrier({} addrspace(10)*, ...)

; Function Attrs: argmemonly nofree nounwind willreturn writeonly
declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i1 immarg) #5

declare void @free(i8*)

; Function Attrs: alwaysinline
define void @diffejulia_objective__1863_inner_1wrap({} addrspace(10)* %0, {} addrspace(10)* %1, {} addrspace(10)* %2, {} addrspace(10)* %3, {} addrspace(10)* %4) #6 !dbg !103 {
entry:
  %gcframe22 = alloca [4 x {} addrspace(10)*], align 16
  %gcframe22.sub = getelementptr inbounds [4 x {} addrspace(10)*], [4 x {} addrspace(10)*]* %gcframe22, i64 0, i64 0
  %5 = bitcast [4 x {} addrspace(10)*]* %gcframe22 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(32) %5, i8 0, i32 32, i1 false), !tbaa !109
  %6 = getelementptr inbounds [4 x {} addrspace(10)*], [4 x {} addrspace(10)*]* %gcframe22, i64 0, i64 2
  %7 = alloca [8000 x i8], align 16
  %.sub = getelementptr inbounds [8000 x i8], [8000 x i8]* %7, i64 0, i64 0
  %thread_ptr = call i8* asm "movq %fs:0, $0", "=r"() #9
  %ppgcstack_i8 = getelementptr i8, i8* %thread_ptr, i64 -8
  %ppgcstack = bitcast i8* %ppgcstack_i8 to {}****
  %pgcstack = load {}***, {}**** %ppgcstack, align 8
  %8 = bitcast [4 x {} addrspace(10)*]* %gcframe22 to i64*
  store i64 8, i64* %8, align 16, !tbaa !109
  %9 = getelementptr inbounds [4 x {} addrspace(10)*], [4 x {} addrspace(10)*]* %gcframe22, i64 0, i64 1
  %10 = bitcast {} addrspace(10)** %9 to {}***
  %11 = load {}**, {}*** %pgcstack, align 8
  store {}** %11, {}*** %10, align 8, !tbaa !109
  %12 = bitcast {}*** %pgcstack to {} addrspace(10)***
  store {} addrspace(10)** %gcframe22.sub, {} addrspace(10)*** %12, align 8
  %13 = bitcast {} addrspace(10)* %4 to double addrspace(13)* addrspace(10)*
  %14 = addrspacecast double addrspace(13)* addrspace(10)* %13 to double addrspace(13)* addrspace(11)*
  %15 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %14, align 16
  %"'ipc.i" = bitcast {} addrspace(10)* %1 to double addrspace(13)* addrspace(10)*
  %16 = bitcast {} addrspace(10)* %0 to double addrspace(13)* addrspace(10)*
  %"'ipc4.i" = addrspacecast double addrspace(13)* addrspace(10)* %"'ipc.i" to double addrspace(13)* addrspace(11)*
  %17 = addrspacecast double addrspace(13)* addrspace(10)* %16 to double addrspace(13)* addrspace(11)*
  %"'ipl.i" = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %"'ipc4.i", align 16
  %18 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %17, align 16
  %"'ipc7.i" = bitcast {} addrspace(10)* %3 to double addrspace(10)*
  %19 = bitcast {} addrspace(10)* %2 to double addrspace(10)*
  call void @llvm.lifetime.start.p0i8(i64 8000, i8* nonnull %.sub)
  call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 16 dereferenceable(8000) %.sub, i8 0, i64 8000, i1 false), !dbg !111
  %_malloccache.i = bitcast [8000 x i8]* %7 to {} addrspace(10)**, !dbg !111
  %20 = call noalias nonnull dereferenceable(8000) dereferenceable_or_null(8000) i8* inttoptr (i64 140080140341536 to i8* (i64)*)(i64 8000), !dbg !111
  call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(8000) %20, i8 0, i64 8000, i1 false), !dbg !111
  %_malloccache12.i = bitcast i8* %20 to double*, !dbg !111
  %21 = call noalias nonnull dereferenceable(8000) dereferenceable_or_null(8000) i8* inttoptr (i64 140080140341536 to i8* (i64)*)(i64 8000), !dbg !111
  call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(8000) %21, i8 0, i64 8000, i1 false), !dbg !111
  %_malloccache18.i = bitcast i8* %21 to i64*, !dbg !111
  br label %L2.i.i, !dbg !111

L2.i.i:                                           ; preds = %idxend.i.i, %entry
  %iv.i = phi i64 [ %iv.next.i, %idxend.i.i ], [ 0, %entry ]
  %iv.next.i = add nuw nsw i64 %iv.i, 1, !dbg !113
  %22 = call noalias nonnull {} addrspace(10)* inttoptr (i64 140080130477952 to {} addrspace(10)* ({} addrspace(10)*, i64)*)({} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 140079781262464 to {}*) to {} addrspace(10)*), i64 noundef 3) #10, !dbg !113
  %23 = getelementptr inbounds [4 x {} addrspace(10)*], [4 x {} addrspace(10)*]* %gcframe22, i64 0, i64 3
  store {} addrspace(10)* %22, {} addrspace(10)** %23, align 8
  %24 = call {} addrspace(10)* inttoptr (i64 140080130477952 to {} addrspace(10)* ({} addrspace(10)*, i64)*)({} addrspace(10)* addrspacecast ({}* inttoptr (i64 140079781262464 to {}*) to {} addrspace(10)*), i64 3), !dbg !113
  %25 = getelementptr inbounds {} addrspace(10)*, {} addrspace(10)** %_malloccache.i, i64 %iv.i, !dbg !113
  store {} addrspace(10)* %24, {} addrspace(10)** %6, align 16
  store {} addrspace(10)* %24, {} addrspace(10)** %25, align 8, !dbg !113, !invariant.group !119
  %26 = bitcast {} addrspace(10)* %24 to i8 addrspace(13)* addrspace(10)*, !dbg !113
  %27 = load i8 addrspace(13)*, i8 addrspace(13)* addrspace(10)* %26, align 8, !dbg !113
  call void @llvm.memset.p13i8.i64(i8 addrspace(13)* noundef align 1 dereferenceable(24) %27, i8 0, i64 24, i1 false), !dbg !113
  %28 = bitcast {} addrspace(10)* %22 to { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(10)*, !dbg !120
  %29 = addrspacecast { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(10)* %28 to { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(11)*, !dbg !120
  %30 = getelementptr inbounds { i8 addrspace(13)*, i64, i16, i16, i32 }, { i8 addrspace(13)*, i64, i16, i16, i32 } addrspace(11)* %29, i64 0, i32 1, !dbg !120
  %31 = load i64, i64 addrspace(11)* %30, align 8, !dbg !120, !tbaa !49, !range !54
  %.not.not.i = icmp eq i64 %31, 0, !dbg !126
  br i1 %.not.not.i, label %oob.i.i, label %idxend.i.i, !dbg !130

oob.i.i:                                          ; preds = %L2.i.i
  %.phi.trans.insert.i = bitcast {} addrspace(10)* %22 to double addrspace(13)* addrspace(10)*
  %.phi.trans.insert8.i = addrspacecast double addrspace(13)* addrspace(10)* %.phi.trans.insert.i to double addrspace(13)* addrspace(11)*
  %.pre.i = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %.phi.trans.insert8.i, align 8, !dbg !131, !tbaa !70
  %32 = load double, double addrspace(13)* %15, align 8, !dbg !133, !tbaa !74
  %33 = load double, double addrspace(13)* %18, align 8, !dbg !134, !tbaa !74
  %34 = fmul double %32, %33, !dbg !135
  store double %34, double addrspace(13)* %.pre.i, align 8, !dbg !131, !tbaa !74
  %35 = alloca i64, align 8, !dbg !136
  store i64 1, i64* %35, align 8, !dbg !136
  %36 = addrspacecast {} addrspace(10)* %22 to {} addrspace(12)*, !dbg !136
  call void @llvm.lifetime.end.p0i8(i64 8000, i8* nonnull %.sub)
  call void @ijl_bounds_error_ints({} addrspace(12)* %36, i64* noundef nonnull align 8 %35, i64 noundef 1) #11, !dbg !136
  unreachable

idxend.i.i:                                       ; preds = %L2.i.i
  %37 = bitcast {} addrspace(10)* %22 to double addrspace(13)* addrspace(10)*
  %38 = addrspacecast double addrspace(13)* addrspace(10)* %37 to double addrspace(13)* addrspace(11)*
  %39 = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %38, align 8, !tbaa !70, !nonnull !4
  %40 = bitcast double addrspace(13)* %39 to i8 addrspace(13)*, !dbg !138
  %41 = shl nuw i64 %31, 3, !dbg !138
  call void @llvm.memset.p13i8.i64(i8 addrspace(13)* noundef nonnull align 8 %40, i8 noundef 0, i64 %41, i1 noundef false) #10, !dbg !139
  %42 = load double, double addrspace(13)* %15, align 8, !dbg !133, !tbaa !74
  %43 = load double, double addrspace(13)* %18, align 8, !dbg !134, !tbaa !74
  %44 = fmul double %42, %43, !dbg !135
  store double %44, double addrspace(13)* %39, align 8, !dbg !131, !tbaa !74, !alias.scope !141, !noalias !144
  %45 = getelementptr inbounds i64, i64* %_malloccache18.i, i64 %iv.i, !dbg !146
  store i64 %41, i64* %45, align 8, !dbg !146, !invariant.group !148
  %46 = getelementptr inbounds double, double* %_malloccache12.i, i64 %iv.i, !dbg !146
  store double %42, double* %46, align 8, !dbg !146, !tbaa !74, !invariant.group !149
  store double %44, double addrspace(10)* %19, align 8, !dbg !146, !tbaa !92, !alias.scope !141, !noalias !144
  %.not5.i = icmp eq i64 %iv.next.i, 1000, !dbg !150
  br i1 %.not5.i, label %invertidxend.i.i, label %L2.i.i, !dbg !153

invertidxend.i.i:                                 ; preds = %idxend.i.i, %invertidxend.i.i
  %"iv'ac.i.0" = phi i64 [ %60, %invertidxend.i.i ], [ 999, %idxend.i.i ]
  %47 = load double, double addrspace(10)* %"'ipc7.i", align 8
  store double 0.000000e+00, double addrspace(10)* %"'ipc7.i", align 8, !dbg !146, !tbaa !92, !alias.scope !144, !noalias !141
  %48 = getelementptr inbounds {} addrspace(10)*, {} addrspace(10)** %_malloccache.i, i64 %"iv'ac.i.0"
  %49 = bitcast {} addrspace(10)** %48 to double addrspace(13)* addrspace(10)**
  %50 = load double addrspace(13)* addrspace(10)*, double addrspace(13)* addrspace(10)** %49, align 8
  %"'ipc9_unwrap.i" = addrspacecast double addrspace(13)* addrspace(10)* %50 to double addrspace(13)* addrspace(11)*
  %"'il_phi3_unwrap.i" = load double addrspace(13)*, double addrspace(13)* addrspace(11)* %"'ipc9_unwrap.i", align 8, !tbaa !70
  %51 = load double, double addrspace(13)* %"'il_phi3_unwrap.i", align 8
  store double 0.000000e+00, double addrspace(13)* %"'il_phi3_unwrap.i", align 8, !dbg !131, !tbaa !74, !alias.scope !144, !noalias !141
  %52 = fadd fast double %51, %47
  %53 = getelementptr inbounds double, double* %_malloccache12.i, i64 %"iv'ac.i.0"
  %54 = load double, double* %53, align 8, !dbg !133, !tbaa !74, !invariant.group !149
  %m1diffe.i = fmul fast double %54, %52
  %55 = load double, double addrspace(13)* %"'ipl.i", align 8, !dbg !134, !tbaa !74, !alias.scope !144, !noalias !141
  %56 = fadd fast double %55, %m1diffe.i
  store double %56, double addrspace(13)* %"'ipl.i", align 8, !dbg !134, !tbaa !74, !alias.scope !144, !noalias !141
  %57 = getelementptr inbounds i64, i64* %_malloccache18.i, i64 %"iv'ac.i.0", !dbg !139
  %58 = load i64, i64* %57, align 8, !dbg !139, !invariant.group !148
  %"'ipc15_unwrap.i" = bitcast double addrspace(13)* %"'il_phi3_unwrap.i" to i8 addrspace(13)*, !dbg !139
  call void @llvm.memset.p13i8.i64(i8 addrspace(13)* noundef nonnull align 8 %"'ipc15_unwrap.i", i8 noundef 0, i64 %58, i1 noundef false) #10, !dbg !139
  %59 = icmp eq i64 %"iv'ac.i.0", 0
  %60 = add nsw i64 %"iv'ac.i.0", -1
  br i1 %59, label %diffejulia_objective__1863_inner.1.exit, label %invertidxend.i.i

diffejulia_objective__1863_inner.1.exit:          ; preds = %invertidxend.i.i
  call void @llvm.lifetime.end.p0i8(i64 8000, i8* nonnull %.sub)
  call void @free(i8* nonnull %20), !dbg !154
  call void @free(i8* nonnull %21), !dbg !154
  %61 = load {} addrspace(10)*, {} addrspace(10)** %9, align 8, !tbaa !109
  %62 = bitcast {}*** %pgcstack to {} addrspace(10)**
  store {} addrspace(10)* %61, {} addrspace(10)** %62, align 8, !tbaa !109
  ret void
}

; Function Attrs: inaccessiblemem_or_argmemonly
declare void @ijl_gc_queue_root({} addrspace(10)*) #7

; Function Attrs: allocsize(1)
declare noalias nonnull {} addrspace(10)* @ijl_gc_pool_alloc(i8*, i32, i32) #8

; Function Attrs: allocsize(1)
declare noalias nonnull {} addrspace(10)* @ijl_gc_big_alloc(i8*, i64) #8

declare noalias nonnull {} addrspace(10)** @julia.new_gc_frame(i32)

declare void @julia.push_gc_frame({} addrspace(10)**, i32)

declare {} addrspace(10)** @julia.get_gc_frame_slot({} addrspace(10)**, i32)

declare void @julia.pop_gc_frame({} addrspace(10)**)

; Function Attrs: argmemonly nofree nounwind willreturn writeonly
declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i1 immarg) #5

attributes #0 = { nofree nosync nounwind readnone speculatable willreturn }
attributes #1 = { noreturn }
attributes #2 = { "probe-stack"="inline-asm" }
attributes #3 = { argmemonly nofree nosync nounwind willreturn }
attributes #4 = { nofree nosync nounwind willreturn }
attributes #5 = { argmemonly nofree nounwind willreturn writeonly }
attributes #6 = { alwaysinline "probe-stack"="inline-asm" }
attributes #7 = { inaccessiblemem_or_argmemonly }
attributes #8 = { allocsize(1) }
attributes #9 = { nounwind }
attributes #10 = { mustprogress willreturn }
attributes #11 = { mustprogress noreturn willreturn }

!llvm.module.flags = !{!0, !1}
!llvm.dbg.cu = !{!2}

!0 = !{i32 2, !"Dwarf Version", i32 4}
!1 = !{i32 2, !"Debug Info Version", i32 3}
!2 = distinct !DICompileUnit(language: DW_LANG_Julia, file: !3, producer: "julia", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !4, nameTableKind: None)
!3 = !DIFile(filename: "/home/wmoses/git/Enzyme.jl/s.jl", directory: ".")
!4 = !{}
!5 = distinct !DISubprogram(name: "objective!", linkageName: "julia_objective!_1863", scope: null, file: !3, line: 5, type: !6, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !18)
!6 = !DISubroutineType(types: !7)
!7 = !{!8, !9, !10, !15, !10}
!8 = !DICompositeType(tag: DW_TAG_structure_type, name: "Nothing", align: 8, elements: !4, runtimeLang: DW_LANG_Julia, identifier: "140079777217184")
!9 = !DICompositeType(tag: DW_TAG_structure_type, name: "#objective!", align: 8, elements: !4, runtimeLang: DW_LANG_Julia, identifier: "140079998295888")
!10 = !DIDerivedType(tag: DW_TAG_typedef, name: "Array", baseType: !11)
!11 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !12, size: 64, align: 64)
!12 = !DICompositeType(tag: DW_TAG_structure_type, name: "jl_value_t", file: !13, line: 71, align: 64, elements: !14)
!13 = !DIFile(filename: "julia.h", directory: "")
!14 = !{!11}
!15 = !DICompositeType(tag: DW_TAG_structure_type, name: "RefValue", size: 64, align: 64, elements: !16, runtimeLang: DW_LANG_Julia, identifier: "140079783828064")
!16 = !{!17}
!17 = !DIBasicType(name: "Float64", size: 64, encoding: DW_ATE_unsigned)
!18 = !{!19, !20, !21, !22}
!19 = !DILocalVariable(name: "#self#", arg: 1, scope: !5, file: !3, line: 5, type: !9)
!20 = !DILocalVariable(name: "x", arg: 2, scope: !5, file: !3, line: 5, type: !10)
!21 = !DILocalVariable(name: "loss", arg: 3, scope: !5, file: !3, line: 5, type: !15)
!22 = !DILocalVariable(name: "R", arg: 4, scope: !5, file: !3, line: 5, type: !10)
!23 = !DILocation(line: 0, scope: !5, inlinedAt: !24)
!24 = distinct !DILocation(line: 0, scope: !5)
!25 = !DILocation(line: 6, scope: !5, inlinedAt: !24)
!26 = !DILocation(line: 459, scope: !27, inlinedAt: !30)
!27 = distinct !DISubprogram(name: "Array;", linkageName: "Array", scope: !28, file: !28, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!28 = !DIFile(filename: "boot.jl", directory: ".")
!29 = !DISubroutineType(types: !4)
!30 = distinct !DILocation(line: 468, scope: !27, inlinedAt: !31)
!31 = distinct !DILocation(line: 588, scope: !32, inlinedAt: !34)
!32 = distinct !DISubprogram(name: "zeros;", linkageName: "zeros", scope: !33, file: !33, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!33 = !DIFile(filename: "array.jl", directory: ".")
!34 = distinct !DILocation(line: 585, scope: !32, inlinedAt: !35)
!35 = distinct !DILocation(line: 583, scope: !32, inlinedAt: !36)
!36 = distinct !DILocation(line: 8, scope: !5, inlinedAt: !24)
!37 = !DILocation(line: 151, scope: !38, inlinedAt: !39)
!38 = distinct !DISubprogram(name: "size;", linkageName: "size", scope: !33, file: !33, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!39 = distinct !DILocation(line: 95, scope: !40, inlinedAt: !42)
!40 = distinct !DISubprogram(name: "axes;", linkageName: "axes", scope: !41, file: !41, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!41 = !DIFile(filename: "abstractarray.jl", directory: ".")
!42 = distinct !DILocation(line: 116, scope: !43, inlinedAt: !44)
!43 = distinct !DISubprogram(name: "axes1;", linkageName: "axes1", scope: !41, file: !41, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!44 = distinct !DILocation(line: 285, scope: !45, inlinedAt: !46)
!45 = distinct !DISubprogram(name: "eachindex;", linkageName: "eachindex", scope: !41, file: !41, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!46 = distinct !DILocation(line: 352, scope: !47, inlinedAt: !48)
!47 = distinct !DISubprogram(name: "fill!;", linkageName: "fill!", scope: !33, file: !33, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!48 = distinct !DILocation(line: 589, scope: !32, inlinedAt: !34)
!49 = !{!50, !50, i64 0}
!50 = !{!"jtbaa_arraylen", !51, i64 0}
!51 = !{!"jtbaa_array", !52, i64 0}
!52 = !{!"jtbaa", !53, i64 0}
!53 = !{!"jtbaa"}
!54 = !{i64 0, i64 9223372036854775807}
!55 = !DILocation(line: 83, scope: !56, inlinedAt: !58)
!56 = distinct !DISubprogram(name: "<;", linkageName: "<", scope: !57, file: !57, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!57 = !DIFile(filename: "int.jl", directory: ".")
!58 = distinct !DILocation(line: 382, scope: !59, inlinedAt: !61)
!59 = distinct !DISubprogram(name: ">;", linkageName: ">", scope: !60, file: !60, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!60 = !DIFile(filename: "operators.jl", directory: ".")
!61 = distinct !DILocation(line: 654, scope: !62, inlinedAt: !64)
!62 = distinct !DISubprogram(name: "isempty;", linkageName: "isempty", scope: !63, file: !63, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!63 = !DIFile(filename: "range.jl", directory: ".")
!64 = distinct !DILocation(line: 879, scope: !65, inlinedAt: !46)
!65 = distinct !DISubprogram(name: "iterate;", linkageName: "iterate", scope: !63, file: !63, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!66 = !DILocation(line: 352, scope: !47, inlinedAt: !48)
!67 = !DILocation(line: 966, scope: !68, inlinedAt: !69)
!68 = distinct !DISubprogram(name: "setindex!;", linkageName: "setindex!", scope: !33, file: !33, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!69 = distinct !DILocation(line: 9, scope: !5, inlinedAt: !24)
!70 = !{!71, !71, i64 0}
!71 = !{!"jtbaa_arrayptr", !51, i64 0}
!72 = !DILocation(line: 925, scope: !73, inlinedAt: !69)
!73 = distinct !DISubprogram(name: "getindex;", linkageName: "getindex", scope: !33, file: !33, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!74 = !{!75, !75, i64 0}
!75 = !{!"jtbaa_arraybuf", !76, i64 0}
!76 = !{!"jtbaa_data", !52, i64 0}
!77 = !DILocation(line: 924, scope: !73, inlinedAt: !69)
!78 = !DILocation(line: 385, scope: !79, inlinedAt: !69)
!79 = distinct !DISubprogram(name: "*;", linkageName: "*", scope: !80, file: !80, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!80 = !DIFile(filename: "float.jl", directory: ".")
!81 = !DILocation(line: 924, scope: !73, inlinedAt: !82)
!82 = distinct !DILocation(line: 10, scope: !5, inlinedAt: !24)
!83 = !DILocation(line: 354, scope: !47, inlinedAt: !48)
!84 = !DILocation(line: 966, scope: !68, inlinedAt: !85)
!85 = distinct !DILocation(line: 353, scope: !47, inlinedAt: !48)
!86 = !DILocation(line: 39, scope: !87, inlinedAt: !89)
!87 = distinct !DISubprogram(name: "setproperty!;", linkageName: "setproperty!", scope: !88, file: !88, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!88 = !DIFile(filename: "Base.jl", directory: ".")
!89 = distinct !DILocation(line: 57, scope: !90, inlinedAt: !82)
!90 = distinct !DISubprogram(name: "setindex!;", linkageName: "setindex!", scope: !91, file: !91, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!91 = !DIFile(filename: "refvalue.jl", directory: ".")
!92 = !{!93, !93, i64 0}
!93 = !{!"jtbaa_mutab", !94, i64 0}
!94 = !{!"jtbaa_value", !76, i64 0}
!95 = !DILocation(line: 477, scope: !96, inlinedAt: !98)
!96 = distinct !DISubprogram(name: "==;", linkageName: "==", scope: !97, file: !97, type: !29, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !4)
!97 = !DIFile(filename: "promotion.jl", directory: ".")
!98 = distinct !DILocation(line: 883, scope: !65, inlinedAt: !99)
!99 = distinct !DILocation(line: 11, scope: !5, inlinedAt: !24)
!100 = !DILocation(line: 883, scope: !65, inlinedAt: !99)
!101 = !DILocation(line: 11, scope: !5, inlinedAt: !24)
!102 = !DILocation(line: 0, scope: !5)
!103 = distinct !DISubprogram(name: "objective!", linkageName: "julia_objective!_1863", scope: null, file: !3, line: 5, type: !6, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !2, retainedNodes: !104)
!104 = !{!105, !106, !107, !108}
!105 = !DILocalVariable(name: "#self#", arg: 1, scope: !103, file: !3, line: 5, type: !9)
!106 = !DILocalVariable(name: "x", arg: 2, scope: !103, file: !3, line: 5, type: !10)
!107 = !DILocalVariable(name: "loss", arg: 3, scope: !103, file: !3, line: 5, type: !15)
!108 = !DILocalVariable(name: "R", arg: 4, scope: !103, file: !3, line: 5, type: !10)
!109 = !{!110, !110, i64 0}
!110 = !{!"jtbaa_gcframe", !52, i64 0}
!111 = !DILocation(line: 6, scope: !103, inlinedAt: !112)
!112 = distinct !DILocation(line: 0, scope: !103)
!113 = !DILocation(line: 459, scope: !27, inlinedAt: !114)
!114 = distinct !DILocation(line: 468, scope: !27, inlinedAt: !115)
!115 = distinct !DILocation(line: 588, scope: !32, inlinedAt: !116)
!116 = distinct !DILocation(line: 585, scope: !32, inlinedAt: !117)
!117 = distinct !DILocation(line: 583, scope: !32, inlinedAt: !118)
!118 = distinct !DILocation(line: 8, scope: !103, inlinedAt: !112)
!119 = distinct !{}
!120 = !DILocation(line: 151, scope: !38, inlinedAt: !121)
!121 = distinct !DILocation(line: 95, scope: !40, inlinedAt: !122)
!122 = distinct !DILocation(line: 116, scope: !43, inlinedAt: !123)
!123 = distinct !DILocation(line: 285, scope: !45, inlinedAt: !124)
!124 = distinct !DILocation(line: 352, scope: !47, inlinedAt: !125)
!125 = distinct !DILocation(line: 589, scope: !32, inlinedAt: !116)
!126 = !DILocation(line: 83, scope: !56, inlinedAt: !127)
!127 = distinct !DILocation(line: 382, scope: !59, inlinedAt: !128)
!128 = distinct !DILocation(line: 654, scope: !62, inlinedAt: !129)
!129 = distinct !DILocation(line: 879, scope: !65, inlinedAt: !124)
!130 = !DILocation(line: 352, scope: !47, inlinedAt: !125)
!131 = !DILocation(line: 966, scope: !68, inlinedAt: !132)
!132 = distinct !DILocation(line: 9, scope: !103, inlinedAt: !112)
!133 = !DILocation(line: 925, scope: !73, inlinedAt: !132)
!134 = !DILocation(line: 924, scope: !73, inlinedAt: !132)
!135 = !DILocation(line: 385, scope: !79, inlinedAt: !132)
!136 = !DILocation(line: 924, scope: !73, inlinedAt: !137)
!137 = distinct !DILocation(line: 10, scope: !103, inlinedAt: !112)
!138 = !DILocation(line: 354, scope: !47, inlinedAt: !125)
!139 = !DILocation(line: 966, scope: !68, inlinedAt: !140)
!140 = distinct !DILocation(line: 353, scope: !47, inlinedAt: !125)
!141 = !{!142}
!142 = distinct !{!142, !143, !"primal"}
!143 = distinct !{!143, !" diff: %"}
!144 = !{!145}
!145 = distinct !{!145, !143, !"shadow_0"}
!146 = !DILocation(line: 39, scope: !87, inlinedAt: !147)
!147 = distinct !DILocation(line: 57, scope: !90, inlinedAt: !137)
!148 = distinct !{}
!149 = distinct !{}
!150 = !DILocation(line: 477, scope: !96, inlinedAt: !151)
!151 = distinct !DILocation(line: 883, scope: !65, inlinedAt: !152)
!152 = distinct !DILocation(line: 11, scope: !103, inlinedAt: !112)
!153 = !DILocation(line: 11, scope: !103, inlinedAt: !112)
!154 = !DILocation(line: 0, scope: !103)

@wsmoses
Copy link
Member

wsmoses commented Oct 6, 2022

Looks like we found a bug in Julia itself! JuliaLang/julia#47075 (comment) Working on a fix now. Will also make a workaround here.

@wsmoses
Copy link
Member

wsmoses commented Oct 6, 2022

Check if this works around it for you? #494

@Finn-Sueberkrueb
Copy link
Author

The error message has changed a bit now

signal (11): Segmentation fault: 11
in expression starting at REPL[7]:1
maybe_collect at /Users/julia/src/gc.c:918 [inlined]
jl_gc_big_alloc_inner at /Users/julia/src/gc.c:984 [inlined]
jl_gc_big_alloc_noinline at /Users/julia/src/gc.c:1023
jl_gc_alloc_ at /Users/julia/src/./julia_internal.h:438 [inlined]
jl_gc_alloc at /Users/julia/src/gc.c:3475 [inlined]
ijl_gc_alloc_typed at /Users/julia/src/gc.c:4078
Allocations: 21057107 (Pool: 21030147; Big: 26960); GC: 28
Julia Version 1.9.0-DEV.1527
Commit c1c2a69b82 (2022-10-06 14:23 UTC)
Platform Info: macOS (arm64-apple-darwin21.6.0)

Enzyme v0.10.7 `https://github.com/EnzymeAD/Enzyme.jl.git#fixww`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gc garbage collection
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants