function^n for iterated functions

[stevengj]

As suggested in #34251 (comment) by @StefanKarpinski and motivated more recently by #39031 for dirname, this PR defines function^n as an iterated function.

For example, (dirname^2)("/foo/bar/baz") == dirname(dirname("/foo/bar/baz")) == "/foo", and ((x -> (x + 2/x)/2)^10)(1.234) performs 10 iterations of Newton's method for sqrt(2), giving 1.414213562373095 ≈ sqrt(2).

(Note that you must call f^2 as (f^2)(x), not f^2(x), because the latter is parsed as f^(2(x)).)

Using literal_pow, I made f^0 == identity, f^1 == f, and f^2 == f ∘ f, up to f^4, for improved type inferability and inlining. (Note that f^-1 already calls inv(f), generally giving a MethodError, from literal_pow, and the interpretation as an iterated function is consistent with f^-1 denoting an inverse function.)

Keyword arguments of f^n are passed through to f. I also changed the identity function to ignore keyword arguments, and changed f ∘ g to pass any keywords through to both functions — previously, keywords were not allowed in either case, while the documentation for f ∘ g incorrectly stated that keywords were passed to g, so the new behavior is backwards compatible and more useful.

[oscardssmith]

The traditional issue with this has been that sin^2(x) is common notation for sin(x)^2

[stevengj]

Yes, but the iterated-function notation is far more common and consistent — there are only a handful of functions like sin that commonly use the squaring notation, and even they are inconsistent: sin⁻¹(x) is standard notation for asin(x), not 1/sin(x). Moreover, we already have sin(x)^2, so there would not be much point in defining sin^2 with that meaning.

In consequence, settling on the iterated-function meaning doesn't bother me here; it is more general, more consistent, and far more useful (since it is vastly more concise and clear than any existing alternative) than mere exponentiation of the result.

[simeonschaub]

I am not a big fan of using ^ for this, since right now, * and ∘ are separate operators and there are packages like ApproxFun, where for Fun objects both of these actually have a different meaning that should not be confused. This also limits this syntax for iterated application to callable objects that are <: Function, which ∘ is not limited to. What do you think about using ^ + unicode suffix for this instead? ∘ is already non-ASCII, so I don't think it would be unreasonable to use Unicode for this.

[simeonschaub]

Maybe this should be a type parameter instead, so application can be type stable?

[stevengj]

I thought about that — it depends on how this is used. If n is almost always a hard-coded literal value, then a type parameter is definitely the way to go. On the other hand, if people use an n determined dynamically a lot, it would create a lot of dynamic dispatch.

[stevengj]

Maybe one could get the best of both words? Use n::TN where TN is Val{N} if a literal argument was used, and where TN is Int otherwise?

[simeonschaub]

Is it really that common to construct a lot of these with dynamic n? The uses I had in mind was mostly with higher order functions like map, where the cost of calling the functor is much more important than that of constructing and dynamically passing it. Making this dependent on literal_pow sounds interesting, but I can still imagine use cases, where n is not a literal, but will still pretty much always constant propagate. I am also not 100% sure whether we want to make literal_pow more special, since it's not an uncommon source of confusion already.

[stevengj]

What do you think about using ^ + unicode suffix for this instead?

I can't think of anything offhand that wouldn't be a bit ugly and confusing…

ApproxFun already overloads ^, so they would probably want to keep doing their own behavior. You can always call IteratedFunction(f, n) explicitly if you want to force this behavior or have a non-Function object.

[simeonschaub]

I think superscript o would make sense for this: f ^ᵒ n.

ApproxFun already overloads ^, so they would probably want to keep doing their own behavior.

I am mainly concerned that x ^ ::Integer now doesn't always mean iterated multiplication anymore. I would be fine with f ^ n if we also defined f * g for function composition, but that's probably too late now.

[JeffBezanson]

I don't think this is the right definition. Arguments to the composed function should only be passed to inner. If it were possible for inner to return something that caused keyword arguments to be passed to outer, then that would be the right thing to do. Since that isn't possible, the outer function should never be passed keyword arguments.

The situation is similar to that with varargs: the inner function can take any number of arguments, but the outer function can only accept the inner function's returned value.

[StefanKarpinski]

Possible Unicode symbol option is ∘̂.

[uniment]

concerned that x ^ ::Integer now doesn't always mean iterated multiplication anymore. I would be fine with f ^ n if we also defined f * g for function composition

Isn't composition enough of a multiplication-like operator? After all, it has the same precedence:

julia> let prec=Base.operator_precedence; prec(:⋅) ≡ prec(:×) ≡ prec(:∘) ≡ prec(:*) ≡ prec(:⋆) end
true

The natural question should be: which type of "multiplication" should ^ distribute? Dot product $(\cdot)$? Cross product $(\times)$? Composition $(\circ)$? Convolution $(\ast)$? Cross-correlation $(\star)$? Note that convolution powers seem to be written $f^{*n}$.

I've seen iterated functions written $f^n$ plenty, and many more times I've seen an inverse $f^{-1}$ consistent with this, and constructing such an object is useful. Insisting that ^ distribute specifically the * character seems a rather artificial constraint which mathematical convention doesn't abide by; the world of basic math seems mostly agreed that raising a function to a power should refer to repeated composition $(\circ)$, even though raising a number to a power is repeated multiplication $(\cdot)$.

I'll add that $x\mapsto\cos(x)^2$ can anyway be easily composed as $\left(x\mapsto x^2\right)\circ\cos$, so for the trig folks to steal $\cos^2$ for this is rather rude (and a convention they themselves don't even respect when they write $\cos^{-1}=\arccos$).

FWIW, here's what ChatGPT thinks:

Where f is a function and x is a variable, what is (f^3)(x)? Does this operation have a name?

The notation (f^3)(x) means "the function f applied to x, then applied again to the result, and then applied a third time to the result." In other words, it is the composition of f with itself three times, which can also be written as f(f(f(x))). This operation is commonly referred to as "function composition" or "iterated function application."

Note that the notation (f^3)(x) should not be confused with the notation f(x)^3, which means "the result of applying the function f to x, and then raising the result to the third power."

[jariji]

Insisting that ^ distribute specifically the * character seems a rather artificial constraint which mathematical convention doesn't abide by

I think in programming we need to be a bit more selective about our notation than in math, because while mathematicians have the benefit of context, culture, and critical thinking, computers take the notation literally.

For programming, I prefer to specify relationships between symbols, such as x ^ i means "* iterated". That way we can specify that interface with computer tooling so it's easy to understand and validate that it holds using PropCheck.jl or InterfaceSpecs.jl.

[uniment]

I prefer to specify relationships between symbols, such as x ^ i means "* iterated".

While I empathize with the sentiment, in the case of * it seems that we already don't follow the policy you propose: matrix-matrix multiplication isn't merely + iteration, and for strings and regular expressions we have no + operator to iterate over. So imposing that ^ should only and always distribute * would be a policy specific to ^.

I think of the ^ character to mean primarily not * iteration but instead superscript so I don't see a problem, but maybe $\LaTeX$ gave me brain damage 😆

[jariji]

Btw, array languages have function iteration syntax, but they also allow n to be negative, which uses the inverse of the function.

Here I'm using https://github.com/JuliaMath/InverseFunctions.jl.

using InverseFunctions, Test
struct IteratedFunction{F}
    f::F
    n::Int
end
((;f,n)::IteratedFunction)(x) =
    if n == 0
        x
    elseif n > 0
        foldl((y,f)->f(y), [f for _ in 1:n]; init=x);
    else # if n < 0
        (inverse(f) ∘̂ (-n))(x)
    end

f ∘̂ n = IteratedFunction(f,n)

@test (exp∘̂1)(1) ≈ 2.718281828459045
@test (exp∘̂2)(1) ≈ 15.154262241479262
@test (exp∘̂3)(1) ≈ 3.814279104760214e6
@test (exp∘̂(-1))(ℯ) ≈ 1.0
@test (exp∘̂(-2))(ℯ) ≈ 0

[uniment]

Yes, I'm enthusiastic about the possibility of f^-1 referring to inverse(f). 🤩

In #24990 I show an example where I write inverse(exp ∘ _^2); with that and this it could be written (exp ∘ _^2)^-1. (contingent on lowering to typed partially applied functions; currently it must be written inverse(exp ∘ Fix2(^, 2)))

I forgot to mention before, but here's another reason that iterated function composition is a coherent use of ^:

Where a linear transformation f(x::Vector)::Vector can be represented by a matrix 𝐀 such that f(x) == 𝐀*x, then repeating the transformation 𝐀^n * x (for example, if 𝐀 is a rotation matrix) can be written (f^n)(x). Indeed in math notation we write $\mathbf A^n\vec x = (f^n)(\vec x)$, since iterated multiplication of a matrix is directly analogous to iterated composition of a function.

[oschulz]

Regarding f^-1 and inverse (I maintain InverseFunctions, together with @devmotion ):

A while ago I created a package FunctionChains that might be very suitable to host something like this (if it doesn't go into the language itself). FunctionChains supports InverseFunctions.inverse out of the box, so we can already do:

julia> using FunctionChains, InverseFunctions, FillArrays

julia> f = setinverse(x -> x * 2, x -> x / 2)

julia> g = fchain(Fill(f, 3))

julia> g(2)
16

julia> inverse(g)(16)
2.0

So in principle, this

∘̂(f, n::Integer) = n >= 0 ? fchain(Fill(f, n)) : fchain(Fill(inverse(f), -n))

would already do the trick:

julia> (f∘̂ 3)(2)
16

julia> (f∘̂ -3)(16)
2.0

julia> inverse(f∘̂ 3)(16)
2.0

FunctionChains has a few extra tricks, so this works too then

julia> with_intermediate_results(f∘̂ 3, 2)
3-element Vector{Int64}:
  4
  8
 16

and also

julia> using ChangesOfVariables

julia> f = setladj(x -> x * 2, _ -> log(2))

julia> with_logabsdet_jacobian(f∘̂3, 2)
(16, 2.0794415416798357)

It would certainly be better to use a dedicated IteratedFunction (as suggested by @jariji ) instead of fchain(Fill(...)) though - FillArrays would be kind of a heavy dependency. I'd be happy to add something like IteratedFunction to FunctionChains if people are interested - we could also move FunctionChains into JuliaMath or so, to make it more of a community package (I hadn't widely advertised the package so far).

[uniment]

Point of note, ∘̂ currently has an undesirable operator precedence for this purpose.

julia> let prec=Base.operator_precedence;  prec(:∘̂), prec(:^)  end
(12, 15)

[oschulz]

Point of note, ∘̂ currently has an undesirable operator precedence for this purpose.

Ah, yes - also, it's hard to type.

Since ^ is problematich (see discussion above) ... hm, what else might be a good fit?

[jariji]

I like the ∘̂ character for this. I wonder if we could add a new precedence class that requires parentheses to disambiguate. At least for a release or two. That's what Guy Steele advocated in his Juliacon talk: require parentheses unless the operator is learned by first-year university.

[uniment]

I don't believe ^ is problematic (see discussion above).

[oschulz]

I don't believe ^ is problematic (see discussion above).

Well, I'll be happy with whatever the language experts on this thread would prefer to go with.

If we do it in a package though, anything like ^(::Function,::Integer) or even ^(::Function,::Integer) would certainly be type-piracy though. So that would have to be done in the language (i.e. this PR).

Question, if we do this in the language, though, not in a package, how do get f^-n to call on InverseFunctions.inverse, though?

[uniment]

I can imagine three possible paths at the moment:

1. Include InverseFunctions in the language and have Base.:^(::Function, n::Integer) call InverseFunctions.inverse for n<0.

2. Don't include InverseFunctions in the language but still have ^ call InverseFunctions.inverse, and if the package isn't loaded throw an error. (Admittedly this creates an interesting codependency between the language and a package, but maybe we can carve out an exemption for this case?)

3. Declare in the language the function Base.inverse with no methods (leave it up to the user to define methods), and have ^ call inverse. Then, when a user loads InverseFunctions, it could run a loop to check if, for example, hasmethod(Base.inverse, Tuple{typeof(exp)}): if not, then define it as Base.inverse(::typeof(exp)) = InverseFunctions.inverse(exp). Does it count as type piracy if you define a method for a Base function on Base types, but you only define it if it isn't already defined?

[mbauman]

Using ^ is problematic for exactly the same reason that Base.:^(::Function, ::Int) is pirate-y and non-ideal — dispatching on ::Function is a code-smell as you'd ideally want to also support iterated constructors and callables. But you can't define it in general because * is not ∘.

Using an alternative like ^ᵒ or ∘̂ is possible right now in any package. Or a package could define its own independent ^ function that people could choose to use (shadowing the base definition) with using InverseFunctions: ^.

[uniment]

Base.:^(::Function, ::Int) is pirate-y

Not if it's implemented in Base.

support iterated constructors and callables

1. Is this an actual use case?
2. Is it not satisfied by calling the IteratedFunction constructor implemented in this PR, as mentioned above by @stevengj?
3. I understand that duck typing is preferred over dispatching on ::Function for good reasons (such as map), but is this case not an exception to that heuristic?

My suspicion is that a cost/benefit analysis will find the inconveniency of always needing to type ^ᵒ or ∘̂ outweighs that of occasionally needing to call the IteratedFunction constructor (heck, I'd rather type IteratedFunction than ^\^o<tab> or \circ<tab>\hat<tab> 😅). Not to mention, the mental load for f^n is lower because everybody already knows what it means from linear algebra.

Ok I'll step back now.

[oschulz]

I'll add this to FunctionChains (oschulz/FunctionChains.jl#13) as f ∘̂ n so people can play with it. Question is, must negative n be supported? It's pretty much impossible to do in a type-stable fashion.