Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with BatchNorm/InstanceNorm after Conv1D on GPU #1280

Closed
DeanLym opened this issue Jul 12, 2020 · 0 comments · Fixed by #1397
Closed

Error with BatchNorm/InstanceNorm after Conv1D on GPU #1280

DeanLym opened this issue Jul 12, 2020 · 0 comments · Fixed by #1397

Comments

@DeanLym
Copy link

DeanLym commented Jul 12, 2020

I tried to use BatchNorm and InstanceNorm after Conv1D. It works on cpu but fails on gpu.

(BatchNorm works fine on gpu after Conv2D)

Here is a minimal example reproducing the bug.

using Flux

device = gpu
xin = rand(Float32, 10, 2, 4) |> device

model = Chain(
    Conv((3,), 2=>2, pad=1),
    BatchNorm(2),
) |> device

loss(x) = Flux.mse(model(x), x)

ps = params(model)

loss, back = Flux.pullback(ps) do
    loss(xin)
end

Here is the error message

MethodError: no method matching Float32(::ForwardDiff.Dual{Nothing,Float32,1})
Closest candidates are:
  Float32(::Real, !Matched::RoundingMode) where T<:AbstractFloat at rounding.jl:200
  Float32(::T) where T<:Number at boot.jl:715
  Float32(!Matched::Int8) at float.jl:60
  ...
in top-level scope at Conv1DNorm.jl:19
in pullback at Zygote\iFibI\src\compiler\interface.jl:172
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in #9 at Conv1DNorm.jl:20 
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in loss at Conv1DNorm.jl:13 
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in Chain at Flux\IjMZL\src\layers\basic.jl:38 
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in applychain at Flux\IjMZL\src\layers\basic.jl:36 
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in applychain at Flux\IjMZL\src\layers\basic.jl:36 
in _pullback at Zygote\iFibI\src\compiler\interface2.jl
in BatchNorm at Flux\IjMZL\src\layers\normalise.jl:214 
in broadcasted at base\broadcast.jl:1232 
in _pullback at ZygoteRules\6nssF\src\adjoint.jl:47 
in adjoint at Zygote\iFibI\src\lib\lib.jl:175 
in _pullback at ZygoteRules\6nssF\src\adjoint.jl:47 
in adjoint at Zygote\iFibI\src\lib\broadcast.jl:193 
in broadcast_forward at Zygote\iFibI\src\lib\broadcast.jl:181
in materialize at base\broadcast.jl:820
in copy at base\broadcast.jl:854 
in getindex at base\broadcast.jl:564 
in _broadcast_getindex at base\broadcast.jl:604 
in _broadcast_getindex_evalf at base\broadcast.jl:631 
in  at Zygote\iFibI\src\lib\broadcast.jl:175
in  at CUDA\sjcZt\src\broadcast.jl:21
@bors bors bot closed this as completed in 7e9a180 Feb 4, 2021
@bors bors bot closed this as completed in #1397 Feb 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant