feat(scripts): polynomial and rational approximations

[p0mvn]

Progress towards: #3013

Math Operations Approximations

Context

This is a set of scripts to approximate a mathematical operation using polynomial
and rational approximations.

The main script is in its respective function of main.py. It does the following:

1. Computes polynomial and rational approximations of a given function (e^x by default),
   returning the coefficients.

2. Computes (x,y) coordinates for every kind of approximation given the same x coordinates.
   Plots the results for rough comparison.

3. Plots the results for rough comparison.

4. Computes the max error for every approximation given the same x coordinates.

5. Computes and plots max errors for every approximation with a varying number of parameters.

In other words, this script runs various approximation methods, plots their results and deltas
from actual function values. It can be configured to print the maximum error.
The exact behavior is controlled by the global variables at the top of main.py.

The following are the resources used to create this:

• https://xn--2-umb.com/22/approximation
• https://sites.tufts.edu/atasissa/files/2019/09/remez.pdf

In main.py, there is also an exponent_approximation_choice script.

This is a shorter and simpler version of main that isolates the 13-parameter
Chebyshev Rational approximation of e^x. We are planning to use it in production.
Therefore, we need to perform coefficient truncations to 36 decimal points
(the max osmomath supported precision). This truncation is applied
to exponent_approximation_choice but not main.

Configuration

Several parameters can be changed on the needs basis at the
top of main.py.

Some of the parameters include the function we are approximating, the [x_start, x_end] range of
the approximation, and the number of terms to be used. For the full parameter list, see main.py.

Usage

Assuming that you are in the root of the repository and have Sympy installed:

# Create a virtual environment.
python3 -m venv ~/approx-venv

# Start the environment
source ~/approx-venv/bin/activate

# Install dependencies in the virtual environment.
pip install -r scripts/approximations/requirements.txt

# Run the script.
python3 scripts/approximations/main.py

# Run tests
python3 scripts/approximations/rational_test.py

Example Output

Plotting Functions

This script plots the results to sanity check that they look right.

On the screenshot below, "50000 terms Equispaced poly" aims to be the most accurate representation of the function $$e^x$$

Visually, the results are as expected where "Chebyshev Rational" is plotted as the most accurate, followed by "Chebyshev Poly" and, lastly, "Equispaced Poly".

Plotting Approximation Errors For Each Approximation

Plotting Chosen Approximation Errors

13 parameter Chebyshev Rational errors with coefficient truncations done to match osmomath.BigDec precision of 36:

[ValarDragon]

If this is too slow, lmk, we can easily speed this up

[p0mvn]

I think it's fine and not noticeable. Out of curiosity - what would be your suggestion to speed this up? Horner's method or some kind of factorization trick?

[p0mvn]

The graph for errors is still TBD

[p0mvn]

The graph for errors is still TBD

In the latest commit 3cf279e, I added the ability to print the max error for each approximation. The result looks something like this:

Max Error on [0, 1]
10 coordinates equally spaced on the X axis
3 polynomial terms and (3, 3) rational terms used for approximations.


Equispaced Poly: 0.014336181311823015
Chebyshev Poly: 0.009865548570433536
Chebyshev Rational: 7.428807818232741e-06

Please let me know if this is sufficient with regards to benchmarking errors

[p0mvn]

Made the suggested change to perform apples to apples comparison by having:

polynomial_num_terms = numerator_terms + denominator_terms

Now, working on better error plots across various combinations of terms to determine the optimal one

[ValarDragon]

Nice!

[ValarDragon]

LGTM, just need an issue to measure truncating the solved coefficients before plotting/computing error. Happy to merge now and track that in an issue

[p0mvn]

LGTM, just need an issue to measure truncating the solved coefficients before plotting/computing error. Happy to merge now and track that in an issue

This should already be achieved here:

osmosis/scripts/approximations/main.py

Lines 179 to 181 in e6784a3

	# Truncate the coefficients to osmomath precision.
	coef_numerator = [sp.Float(sp.N(coef, osmomath_precision + 1), osmomath_precision + 1) for coef in coef_numerator]
	coef_denominator = [sp.Float(sp.N(coef, osmomath_precision + 1), osmomath_precision + 1) for coef in coef_denominator]

The 3rd plot in the PR description is made from that.

I'm in the process of writing Go code right now. In that PR, I will also hard code the coefficients I'm using in this script and replot the error graph