Add simple methods to convert to and from bytes for `ZZ` and finite fields #37343

GiacomoPope · 2024-02-14T17:09:00Z

I often have to work with the conversion of bytes / integers and currently do this by working with int types and then casting things to ZZ or elements of finite fields.

This PR is a small addition to Integer and FiniteField types which allow this with simply functions which internally convert to / from int for the user so that things can be done naively from SageMath types.

There's a TODO in the code which acknowledges that a faster method for to_bytes might be to work straight from the gmp object in cython for the integers, but it wasn't obvious to me how to do this.

Examples

sage: ZZ.from_bytes(b'\x00\x10', byteorder='big')
16
sage: ZZ.from_bytes(b'\x00\x10', byteorder='little')
4096
sage: ZZ.from_bytes(b'\xfc\x00', byteorder='big', is_signed=True)
-1024
sage: ZZ.from_bytes(b'\xfc\x00', byteorder='big', is_signed=False)
64512
sage: ZZ.from_bytes([255, 0, 0], byteorder='big')
16711680
sage: type(_)
<class 'sage.rings.integer.Integer'>

sage: (1024).to_bytes(2, byteorder='big')
b'\x04\x00'
sage: (1024).to_bytes(10, byteorder='big')
b'\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00'
sage: (-1024).to_bytes(10, byteorder='big', is_signed=True)
b'\xff\xff\xff\xff\xff\xff\xff\xff\xfc\x00'
sage: x = 1000
sage: x.to_bytes((x.bit_length() + 7) // 8, byteorder='little')
b'\xe8\x03'

sage: F = GF(65537)
sage: a = F.random_element()
sage: a.to_bytes()
b'\x00\n\x86'
sage: F.from_bytes(_)
2694
sage: a
2694

sage: F = GF(3^10)
sage: a = F.random_element(); a
z10^8 + z10^7 + 2*z10^5 + 2*z10^4 + 2*z10^3 + z10^2 + z10 + 1
sage: a.to_bytes()
b'$\xf7'
sage: F.from_bytes(_)
z10^8 + z10^7 + 2*z10^5 + 2*z10^4 + 2*z10^3 + z10^2 + z10 + 1

📝 Checklist

The title is concise, informative, and self-explanatory.
The description explains in detail what this PR is about.
I have created tests covering the changes.
I have updated the documentation accordingly.

GiacomoPope · 2024-02-14T17:49:28Z

I decided to implement the below, so this comment is redundant

An extension of this PR would be to do the same for finite fields, where the length of the bytes is set by the characteristic automatically... Not sure this needs to be done in the same PR though as i imagine people may have opinions about this

grhkm21 · 2024-02-15T09:34:37Z

Linting fails

grhkm21 · 2024-02-15T10:46:19Z

Looks good to me

GiacomoPope · 2024-02-15T13:13:09Z

Added the review label back in because I added more features to the PR

GiacomoPope · 2024-02-15T17:09:07Z

@grhkm21 @yyyyx4 if either of you have time to look at the extended PR to include FF as well as ZZ

grhkm21

Small changes.

src/sage/rings/finite_rings/element_base.pyx

src/sage/rings/finite_rings/finite_field_base.pyx

src/sage/rings/integer.pyx

grhkm21 · 2024-02-15T17:24:57Z

src/sage/rings/integer.pyx

+        TODO: should we convert straight from the gmp type in cython? This is definitely
+        possible but I'm not sure the cleanest way to do this


I can have a look at that.

Cool, wanna look before merging the PR?

GiacomoPope · 2024-02-15T17:29:24Z

Committed changes. Thanks for your review.

grhkm21 · 2024-02-15T17:30:51Z

Looks good to me, but changed to s: needs work because I will work on it.

If I don't finish in a day or two then better merge it first.

tscrim · 2024-02-20T08:43:54Z

Note that one-line descriptions should be one line (=sentence). So the part about the internal bit should go on its own paragraph.

Also please use .. TODO::.

GiacomoPope · 2024-02-20T09:47:51Z

Thanks Travis. I made changes from your feedback. Since it was five days since Gareth suggested working on the optimisation for conversion to GMP I may remove the needs work if that's OK? I dont think it's an important optimisation for the method.

tscrim

Thanks. Yes, I think you can remove the needs work. (Either way, that isn't anything that requires this to be held up as it can always be done on a subsequent PR.)

I think the OUTPUT: blocks are gratuitous, but if you want to keep them, either no capital letter for the bullet point or remove the bullet and make it a full sentence with a period/full-stop.

Just as a comment: I also dislike the blanklines between input blocks as I find it more complicated to parse quickly. However, this is up to individual authors, but I feel compelled to mention it from time to time.

src/sage/rings/integer.pyx

src/sage/rings/finite_rings/element_base.pyx

src/sage/rings/integer.pyx

grhkm21 · 2024-02-20T12:34:59Z

Hi sorry, I’m three days behind on all GitHub discussions 🥲 very sorry for the lack of update. I’ll do it on a separate PR

GiacomoPope · 2024-02-20T12:41:04Z

I removed the needs work tab and commited the suggested changes. I added the "needs review" back in simply as I didnt want to label my own PR as "positive review". Will let the CI run and I suppose someone else can add the final label when they have time to double check the latest commit.

grhkm21 · 2024-02-20T22:46:42Z

src/sage/rings/finite_rings/element_base.pyx

+        """
+        Return an array of bytes representing an integer.
+
+        Internally relies on the python ``int.to_bytes()`` method.


Is there a syntax to link Python methods, similar to :meth:`asdf`? Maybe more advanced devs like @tscrim will know

I am not sure. It might work as other :meth:`class_name.method_name` things, but that is more of a sphinx thing than anything sage-specific...

grhkm21

Otherwise it looks good to me, provided CI pass (missing the three ubuntu Conda ones)

… `ZZ` and finite fields I often have to work with the conversion of bytes / integers and currently do this by working with `int` types and then casting things to `ZZ` or elements of finite fields. This PR is a small addition to `Integer` and `FiniteField` types which allow this with simply functions which internally convert to / from `int` for the user so that things can be done naively from SageMath types. There's a TODO in the code which acknowledges that a faster method for `to_bytes` might be to work straight from the `gmp` object in cython for the integers, but it wasn't obvious to me how to do this. ### Examples ```py sage: ZZ.from_bytes(b'\x00\x10', byteorder='big') 16 sage: ZZ.from_bytes(b'\x00\x10', byteorder='little') 4096 sage: ZZ.from_bytes(b'\xfc\x00', byteorder='big', is_signed=True) -1024 sage: ZZ.from_bytes(b'\xfc\x00', byteorder='big', is_signed=False) 64512 sage: ZZ.from_bytes([255, 0, 0], byteorder='big') 16711680 sage: type(_) <class 'sage.rings.integer.Integer'> ``` ```py sage: (1024).to_bytes(2, byteorder='big') b'\x04\x00' sage: (1024).to_bytes(10, byteorder='big') b'\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00' sage: (-1024).to_bytes(10, byteorder='big', is_signed=True) b'\xff\xff\xff\xff\xff\xff\xff\xff\xfc\x00' sage: x = 1000 sage: x.to_bytes((x.bit_length() + 7) // 8, byteorder='little') b'\xe8\x03' ``` ```py sage: F = GF(65537) sage: a = F.random_element() sage: a.to_bytes() b'\x00\n\x86' sage: F.from_bytes(_) 2694 sage: a 2694 ``` ```py sage: F = GF(3^10) sage: a = F.random_element(); a z10^8 + z10^7 + 2*z10^5 + 2*z10^4 + 2*z10^3 + z10^2 + z10 + 1 sage: a.to_bytes() b'$\xf7' sage: F.from_bytes(_) z10^8 + z10^7 + 2*z10^5 + 2*z10^4 + 2*z10^3 + z10^2 + z10 + 1 ``` ### 📝 Checklist - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have created tests covering the changes. - [x] I have updated the documentation accordingly. URL: sagemath#37343 Reported by: Giacomo Pope Reviewer(s): Giacomo Pope, grhkm21, Travis Scrimshaw

src/sage/rings/integer_ring.pyx

Co-authored-by: Travis Scrimshaw <clfrngrown@aol.com>

github-actions · 2024-03-06T17:41:51Z

Documentation preview for this PR (built with commit e51474d; changes) is ready! 🎉

GiacomoPope added t: enhancement p: minor / 4 s: needs review labels Feb 14, 2024

grhkm21 added s: positive review and removed s: needs review labels Feb 15, 2024

GiacomoPope changed the title ~~Add simple methods to convert Integers to and from bytes~~ Add simple methods to convert to and from bytes for ZZ and finite fields Feb 15, 2024

GiacomoPope added s: needs review and removed s: positive review labels Feb 15, 2024

grhkm21 reviewed Feb 15, 2024

View reviewed changes

grhkm21 added s: needs work and removed s: needs review labels Feb 15, 2024

tscrim requested changes Feb 20, 2024

View reviewed changes

src/sage/rings/integer.pyx Outdated Show resolved Hide resolved

src/sage/rings/integer.pyx Outdated Show resolved Hide resolved

src/sage/rings/finite_rings/element_base.pyx Outdated Show resolved Hide resolved

src/sage/rings/integer.pyx Outdated Show resolved Hide resolved

GiacomoPope removed the s: needs work label Feb 20, 2024

GiacomoPope added the s: needs review label Feb 20, 2024

grhkm21 reviewed Feb 20, 2024

View reviewed changes

grhkm21 approved these changes Feb 20, 2024

View reviewed changes

grhkm21 added s: positive review and removed s: needs review labels Feb 20, 2024

tscrim reviewed Feb 22, 2024

View reviewed changes

src/sage/rings/integer_ring.pyx Outdated Show resolved Hide resolved

GiacomoPope and others added 3 commits February 22, 2024 10:06

Update element_base.pyx

daf93c0

Update finite_field_base.pyx

b5af144

Update src/sage/rings/integer_ring.pyx

f6bb7c9

Co-authored-by: Travis Scrimshaw <clfrngrown@aol.com>

tscrim approved these changes Feb 23, 2024

View reviewed changes

GiacomoPope added 2 commits February 25, 2024 22:39

Merge branch 'develop' into encoding_to_bytes

f3fbff7

Merge branch 'develop' into encoding_to_bytes

e51474d

vbraun merged commit 9991759 into sagemath:develop Mar 25, 2024
13 checks passed

mkoeppe added this to the sage-10.4 milestone Mar 25, 2024

github-actions bot removed the s: positive review label Mar 26, 2024

GiacomoPope deleted the encoding_to_bytes branch April 1, 2024 01:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add simple methods to convert to and from bytes for `ZZ` and finite fields #37343

Add simple methods to convert to and from bytes for `ZZ` and finite fields #37343

GiacomoPope commented Feb 14, 2024 •

edited

Loading

GiacomoPope commented Feb 14, 2024 •

edited

Loading

grhkm21 commented Feb 15, 2024

grhkm21 commented Feb 15, 2024

GiacomoPope commented Feb 15, 2024

GiacomoPope commented Feb 15, 2024

grhkm21 left a comment

grhkm21 Feb 15, 2024

GiacomoPope Feb 15, 2024

grhkm21 Feb 15, 2024

GiacomoPope commented Feb 15, 2024

grhkm21 commented Feb 15, 2024

tscrim commented Feb 20, 2024

GiacomoPope commented Feb 20, 2024

tscrim left a comment

grhkm21 commented Feb 20, 2024

GiacomoPope commented Feb 20, 2024

grhkm21 Feb 20, 2024

tscrim Feb 21, 2024 •

edited

Loading

grhkm21 left a comment •

edited

Loading

github-actions bot commented Mar 6, 2024

		TODO: should we convert straight from the gmp type in cython? This is definitely
		possible but I'm not sure the cleanest way to do this

Add simple methods to convert to and from bytes for ZZ and finite fields #37343

Add simple methods to convert to and from bytes for ZZ and finite fields #37343

Conversation

GiacomoPope commented Feb 14, 2024 • edited Loading

Examples

📝 Checklist

GiacomoPope commented Feb 14, 2024 • edited Loading

grhkm21 commented Feb 15, 2024

grhkm21 commented Feb 15, 2024

GiacomoPope commented Feb 15, 2024

GiacomoPope commented Feb 15, 2024

grhkm21 left a comment

Choose a reason for hiding this comment

grhkm21 Feb 15, 2024

Choose a reason for hiding this comment

GiacomoPope Feb 15, 2024

Choose a reason for hiding this comment

grhkm21 Feb 15, 2024

Choose a reason for hiding this comment

GiacomoPope commented Feb 15, 2024

grhkm21 commented Feb 15, 2024

tscrim commented Feb 20, 2024

GiacomoPope commented Feb 20, 2024

tscrim left a comment

Choose a reason for hiding this comment

grhkm21 commented Feb 20, 2024

GiacomoPope commented Feb 20, 2024

grhkm21 Feb 20, 2024

Choose a reason for hiding this comment

tscrim Feb 21, 2024 • edited Loading

Choose a reason for hiding this comment

grhkm21 left a comment • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Mar 6, 2024

Add simple methods to convert to and from bytes for `ZZ` and finite fields #37343

Add simple methods to convert to and from bytes for `ZZ` and finite fields #37343

GiacomoPope commented Feb 14, 2024 •

edited

Loading

GiacomoPope commented Feb 14, 2024 •

edited

Loading

tscrim Feb 21, 2024 •

edited

Loading

grhkm21 left a comment •

edited

Loading