Don't try and convert to FloatX except if Integer or AbstractFloat (option 1)

[oxinabox]

One option to fix the issues with tying to use ForwardDiff on a rrule pullback that makes use of ProjectTo
See discussion on JuliaDiff/ForwardDiff.jl#538.

Which fixes the following test that fails in FluxML/Zygote.jl#1035

  using Zygote
  dx, dy = diaghessian(f34, xs, y)
  @test size(dx) == size(xs)
  @test vec(dx) ≈ diag(hessian(x -> f34(x,y), xs))
  @test dy ≈ hessian(y -> f34(xs,y), y)

This option basically says:
All subtypes of Real represent the same subspace: all of Real.
So if you give me one I am not familar with (e.g. Dual) I am just going to leave it alone.
But if you give a AbstractFloat subtype, I will fix the precision to match what i saw in the primal pass.
And if you give me an Integer I will make it a Float.

So for this particular case if you give a ProjectTo{Float64}(::Dual{Int}) it will return a Dual{Int} because it has no idea what a Dual is or if it is Integery or Floaty, but it trust that it is a good Real.
This works fine.

It is a bit more code than I expected as need to make sure complex works.

[codecov-commenter]

Codecov Report

Merging #417 (127bae5) into master (424a0b7) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #417      +/-   ##
==========================================
+ Coverage   92.25%   92.32%   +0.06%     
==========================================
  Files          14       14              
  Lines         775      782       +7     
==========================================
+ Hits          715      722       +7     
  Misses         60       60              

Impacted Files	Coverage Δ
src/projection.jl	95.71% <100.00%> (+0.14%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 424a0b7...127bae5. Read the comment docs.

[mcabbott]

What's this aimed at? The real types we preserve, we also already preserve their complex versions.

Maybe recursing into Complex can wait until someone has a stranger real type which needs it. Then we will also know whether ProjectTo(zero(T)) is sufficient for this strange type.

[oxinabox]

I thought we needed it for ProjectTo(1.0 + 1im)(Complex(Dual(1.0, 2.0), Dual(1.0, 2.0)))
But that gets caught by (::ProjectTo{<:Number})(dx::Number) = dx

We would need it if we wanted ProjectTo(1.0 + 1im)(Complex(Dual(1, 2), Dual(1, 2))) to return a Complex{Dual{Float64}}
rather than a Complex{Dual{Int}} which it currently does.
but that is not needed right now, so I guess we can add it back later

[mcabbott]

Is this true? project_type should be Real, Complex, or Number. I get super-confused by dispatch with type parameters on LHS like {T}(.... But not important really.

[oxinabox]

for e.g. Float32 or ComplexF64 etc it is not the case that it is Real, Complex, or Number.
I could add those in as alternative special cases instead of this.

[oxinabox]

I would not be shocked if this come backs to bite us, but we can remove it if and when that happens

[mcabbott]

Oh right, I wasn't thinking clearly, that's exactly the use case.

[oxinabox]

@mcabbott do you approve this PR?
By ColPrac i need someone to approve so i can merge it, so I can finish the Zygote PR

[mcabbott]

This seems fine for now, to make Zygote work. We knew these details may need tweaking.

They may still need more. I'm not so sure whether or not we should do JuliaDiff/ForwardDiff.jl#538 (and possibly things like it elsewhere); if we do then we may also want to make this more strict again, so that x::Float32 forces the numbers inside the Dual to be Float32. Or maybe not.