add smoke test for ffjord

[prbzrg]

I just added some smoke tests for ffjord to reveal errors related to some adtypes
It reveals errors that reported in #588, #610, #615, #624
This PR can be merged when these errors fixed

[ChrisRackauckas]

Instead, just set @test_broken for now.

[prbzrg]

Should I expand the tests and unroll the loop? (to use both @test and @test_broken)

[ChrisRackauckas]

Are all of the tests broken?

[prbzrg]

Regardless of adtype, using

regularize=true & monte_carlo=false

make errors and using

GalacticOptim.AutoForwardDiff()
GalacticOptim.AutoReverseDiff()
GalacticOptim.AutoTracker()

as adtype is also broken.
Overall, 14 of 20 tests are broken.

[ChrisRackauckas]

as adtype is also broken.

What do you mean?

[prbzrg]

In each #610, #615, #624, I tested the same code with different adtype and reported errors.
AutoZygote and AutoFiniteDiff works fine, but other adtypes make different errors.

[ChrisRackauckas]

oh with FFJORD, yes.

[ChrisRackauckas]

It looks like this still isn't catching some of the errors?

[prbzrg]

If we only use @test, it could reproduce all four issues, but also make testing fail, you can see the errors in CI of first commit:
https://github.com/SciML/DiffEqFlux.jl/runs/3628543892?check_suite_focus=true#step:6:312
https://github.com/SciML/DiffEqFlux.jl/runs/3628543892?check_suite_focus=true#step:6:636
https://github.com/SciML/DiffEqFlux.jl/runs/3628543892?check_suite_focus=true#step:6:969
https://github.com/SciML/DiffEqFlux.jl/runs/3628543892?check_suite_focus=true#step:6:1350

I think we have two choices:

1. Only use @test and wait until all four issues fixed to merge this.
2. Use both @test and @test_broken by unrolling the loop. We can merge this sooner, but it doesn't log errors to CI, and it increases line of codes (more code to read and review)

I don't know whether there is any other choices. By second one, if someone wants to fix the errors, just changes it to @test and investigate the problems, but I prefer shorter codes.
Which way should we go?

[ChrisRackauckas]

(2) is a better way to go, because it's always better to fix and change from broken than have things documented in unmerged PRs.

[prbzrg]

OK, I made the loop unrolled and also converted section comments to testsets.
Furthermore, I notice something; there is a test for FFJORD with regularizers

DiffEqFlux.jl/test/cnf_test.jl

Lines 103 to 127 in 34c022f

	###
	# test for default multivariate distribution and FFJORD with regularizers
	###
	nn = Chain(Dense(1, 1, tanh))
	data_dist = Beta(7, 7)
	data_train = Float32.(rand(data_dist, 1, 100))
	tspan = (0.0f0, 1.0f0)
	ffjord_test = FFJORD(nn, tspan, Tsit5())
	function loss_adjoint(θ)
	logpx, λ₁, λ₂ = ffjord_test(data_train, θ, true)
	return mean(@. -logpx + 0.1 * λ₁ + λ₂)
	end
	optfunc = GalacticOptim.OptimizationFunction((x, p) -> loss_adjoint(x), GalacticOptim.AutoZygote())
	optprob = GalacticOptim.OptimizationProblem(optfunc, 0.01f0 .* ffjord_test.p)
	res = GalacticOptim.solve(optprob, ADAM(0.1), cb=cb, maxiters=300)
	θopt = res.minimizer
	data_validate = Float32.(rand(data_dist, 1, 100))
	actual_pdf = pdf.(data_dist, data_validate)
	learned_pdf = exp.(ffjord_test(data_validate, θopt; monte_carlo=false)[1])
	@test totalvariation(learned_pdf, actual_pdf) / size(data_validate, 2) < 0.40

but it doesn't use regularize=true!

[prbzrg]

I want to update other tests to more look like tests of this PR:

1. Use Float32 distribution instead of converting samples
2. Use DiffEqFlux.sciml_train instead of OptimizationFunction -> OptimizationProblem -> solve

Can I update other tests in this PR?

[prbzrg]

I made the updates. If you disagree, I will move them to new PR.

[prbzrg]

Tests are failing. I just undo two last commits and will change other tests in a new PR.

[ChrisRackauckas]

Thanks!