Skip to content

Commit

Permalink
Merge pull request #442 from gmlueck/gmlueck/fast-uses-half-precision
Browse files Browse the repository at this point in the history
Fix defs of "fast_length" and "fast_normalize"
  • Loading branch information
gmlueck authored Sep 14, 2023
2 parents a9ef77c + 6457d05 commit e63b8b7
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions adoc/chapters/programming_interface.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -25656,7 +25656,7 @@ _Constraints:_ Available only if [code]#GeoFloat# is a _float geometric type_
as defined above.

_Returns:_ The length of vector [code]#p# computed as:
[code]#+sqrt((half)(pow(p[0],2) + pow(p[1],2) + ...))+#.
[code]#+half_precision::sqrt(pow(p[0],2) + pow(p[1],2) + ...)+#.

The return type is [code]#GeoFloat# if the input type is scalar. Otherwise,
the return type is [code]#GeoFloat::value_type#.
Expand All @@ -25676,7 +25676,8 @@ _Constraints:_ Available only if [code]#GeoFloat# is a _float geometric type_
as defined above.

_Returns:_ A vector in the same direction as [code]#p# but with a length of 1
computed as [code]#+p*rsqrt((half)(pow(p[0],2) + pow(p[1],2) + ... ))+#.
computed as
[code]#+p * half_precision::rsqrt(pow(p[0],2) + pow(p[1],2) + ...)+#.

The result shall be within 8192 ulps error from the infinitely precise result
of
Expand Down

0 comments on commit e63b8b7

Please sign in to comment.