Implemented a function for smarter debug formatting. #606

andrei-papou · 2019-03-24T19:01:18Z

PR for #398

How it works:

let arr = Array3::from_elem((15, 2, 15), 1.0);
println!("{:?}", arr);

[[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 ...,
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]]] shape=[15, 2, 15], strides=[30, 15, 1], layout=C (0x1), const ndim=3

let arr = Array3::from_elem((3, 3, 3), 1.0);
println!("{:?}", arr);

[[[1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0]],
 [[1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0],
  [1.0, 1.0, 1.0]]] shape=[3, 3, 3], strides=[9, 3, 1], layout=C (0x1), const ndim=3

The number of elements before ellipses is managed by the PRINT_ELEMENTS_LIMIT constant.

LukeMathWalker · 2019-03-25T21:49:47Z

src/arrayformat.rs

 };
 use crate::dimension::IntoDimension;

+#[derive(Debug)]
+enum ArrayDisplayMode {
+    // Array is small enough to me printed without omitting any values.


Typo: to me printed

src/arrayformat.rs

LukeMathWalker · 2019-03-25T22:03:09Z

The output looks quite nice!
I have left some comments pointing at a bunch of functions in std that could help in visually denoising the new function.
It would also be great to have some tests for these new routines, both for the selection of printing mode and the formatting itself.

- fixed typo in ArrayDisplayMode comment - updated ArrayDisplayMode constructor to use shape instead of array - fixed output for complex dimensions

SuperFluffy · 2019-03-27T10:50:46Z

src/lib.rs

@@ -149,6 +149,8 @@ mod array_serde;
 mod arrayformat;
 mod data_traits;

+pub use arrayformat::PRINT_ELEMENTS_LIMIT;


I'd refer to not reexport this public constant at the top level of the crate, but rather reexport it from some config or integration_test module.

As far as I can tell the motivation for making it pub is entirely for making it usable within the integration tests at tests/format.rs?

Awesome. I think that's better. I understand your motivation to have a way of configuring formatting similar to numpy's set_printoptions, but we wouldn't be able to actually accomplish that through const PRINT_ELEMENT_LIMIT. I am actually not sure right now if that was possible at all for a user of the library to accomplish at compile time.

SuperFluffy · 2019-03-27T10:54:05Z

I personally don't like that the implementation detail PRINTS_ELEMENTS_LIMIT is made public. Can the integration tests at tests/format.rs be made into unit tests under arrayformat.rs? Or is there a reason to have them as integration tests?

andrei-papou · 2019-03-27T11:18:41Z

@SuperFluffy I can move them to arrayformat.rs. But as far as I understand to convert those tests to real unit tests I should test array_format_v2 directly, not Display, Debug, etc. implementations. And I'm not really sure how to mock f: &mut fmt::Formatter and mut format: F arguments for the array_format_v2 function. Any help here is appreciated.

Although I agree that those tests should be unit tests, it would be good to give a user some control over PRINT_ELEMENTS_LIMIT. For example, numpy allows configuring this number using set_printoptions (see edgeitems argument).

andrei-papou · 2019-04-08T14:34:25Z

@LukeMathWalker @SuperFluffy any more comments on the changes? I think all the previous issues are fixed

LukeMathWalker · 2019-04-12T07:35:51Z

Sorry for the late reply @andrei-papou, it has been a crazy week.
I will try to set aside some time this weekend to go over the PR one last time.

LukeMathWalker · 2019-04-16T21:11:36Z

I have opened a PR with some observations @andrei-papou.

Smart debug formatting review

andrei-papou · 2019-04-17T11:17:41Z

@LukeMathWalker I tried to make the function more modular by extracting parts of the functionality to other functions and by reducing the number of boolean flags. I also got rid of the explicit second for loop. I hope the code is a bit more readable now. Please take a look

…sional

LukeMathWalker · 2019-04-23T09:11:00Z

I have opened a PR on your branch, trying out a different approach to solve this issue @andrei-papou - let me know what you think of it.

A different approach

LukeMathWalker · 2019-04-25T07:02:48Z

I am happy to merge this into master. Do you want to have a look @jturner314?

jturner314 · 2019-04-25T16:17:13Z

@LukeMathWalker Sure, I'm happy to review it, but feel free to go ahead and merge it if it looks good to you. I won't be able to review it until late next week or so. (It's the end of the semester right now, so I'm busy with exams.) Thanks for working with @andrei-papou on this PR.

LukeMathWalker · 2019-04-26T08:27:17Z

I am happy to call the shots and go ahead with this one - I'd like to avoid consuming your time on PRs I am confident reviewing (we need it for all the others were I have little context 👀).
Good luck with your exams @jturner314!

jturner314 · 2019-04-26T23:55:48Z

Thank you. :)

andrei.papou added 2 commits March 24, 2019 21:53

Implemented a function for smarter debug formatting.

fe4365c

Fixed existing tests

2450f50

andrei-papou mentioned this pull request Mar 24, 2019

Smarter Debug formatting for arrays #398

Closed

LukeMathWalker reviewed Mar 25, 2019

View reviewed changes

src/arrayformat.rs Outdated Show resolved Hide resolved

LukeMathWalker reviewed Mar 25, 2019

View reviewed changes

src/arrayformat.rs Outdated Show resolved Hide resolved

LukeMathWalker reviewed Mar 25, 2019

View reviewed changes

src/arrayformat.rs Outdated Show resolved Hide resolved

LukeMathWalker mentioned this pull request Mar 26, 2019

Are we ndarray yet? #597

Open

31 tasks

andrei-papou added 6 commits March 26, 2019 14:34

Some fixes:

3474d3f

- fixed typo in ArrayDisplayMode comment - updated ArrayDisplayMode constructor to use shape instead of array - fixed output for complex dimensions

Implemented axis shrinking for n-dim arrays

2a8a7c7

All the PR issues are fixed. Tests are needed.

1b5da67

Fixed the tests

fbf8cac

Added tests for 1- and 2-dimensional array with overflow

51c4e11

Try to fix 1.31 build failing

0d11708

SuperFluffy reviewed Mar 27, 2019

View reviewed changes

PRINT_ELEMENTS_LIMIT is now private

18c868a

LukeMathWalker added 8 commits April 16, 2019 20:39

Remove dead code

3aa8abe

Rename format_array_v2 to format_array

17f86bb

Simplify code using ndim

d48c5a9

Access last element directly

6c636dc

Add a test to specify behaviour for zero-dimensional arrays

88d53e3

Add a test for empty arrays

b962a7b

Test zero-length axes

29a8f70

We already know that ndim > 0

27c4a92

Andrew and others added 2 commits April 17, 2019 10:43

Merge pull request #2 from LukeMathWalker/smart-debug-formatting-review

49b4338

Smart debug formatting review

Made the formatting logic more modular

b535f28

Simplify element access

2a8a3b0

LukeMathWalker mentioned this pull request Apr 20, 2019

Is there a way to use index_axis on a reference to ArrayBase? #620

Closed

LukeMathWalker added 8 commits April 20, 2019 17:03

Almost there

5fd9b2f

Clone for the win

f60185c

Cast to 1d array

198d689

First and last elements have to be special cased. Done for multidimen…

c1036fb

…sional

Extract in a separate function

5336b10

Tests are passing

4fa8a62

Minor improvements

e659390

Formatting

0abff2a

LukeMathWalker and others added 3 commits April 23, 2019 20:16

Remove closure

e049abd

Use custom enum

83c9f08

Merge pull request #3 from LukeMathWalker/refactoring-smarter-debugging

72e05d7

A different approach

LukeMathWalker merged commit 61b4835 into rust-ndarray:master Apr 26, 2019

bluss mentioned this pull request Sep 10, 2019

Provide a way to print the whole array #705

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented a function for smarter debug formatting. #606

Implemented a function for smarter debug formatting. #606

andrei-papou commented Mar 24, 2019

LukeMathWalker Mar 25, 2019

LukeMathWalker commented Mar 25, 2019

SuperFluffy Mar 27, 2019

andrei-papou Mar 28, 2019

SuperFluffy Mar 28, 2019

SuperFluffy commented Mar 27, 2019

andrei-papou commented Mar 27, 2019

andrei-papou commented Apr 8, 2019

LukeMathWalker commented Apr 12, 2019

LukeMathWalker commented Apr 16, 2019

andrei-papou commented Apr 17, 2019

LukeMathWalker commented Apr 23, 2019

LukeMathWalker commented Apr 25, 2019

jturner314 commented Apr 25, 2019

LukeMathWalker commented Apr 26, 2019

jturner314 commented Apr 26, 2019

Implemented a function for smarter debug formatting. #606

Implemented a function for smarter debug formatting. #606

Conversation

andrei-papou commented Mar 24, 2019

LukeMathWalker Mar 25, 2019

Choose a reason for hiding this comment

LukeMathWalker commented Mar 25, 2019

SuperFluffy Mar 27, 2019

Choose a reason for hiding this comment

andrei-papou Mar 28, 2019

Choose a reason for hiding this comment

SuperFluffy Mar 28, 2019

Choose a reason for hiding this comment

SuperFluffy commented Mar 27, 2019

andrei-papou commented Mar 27, 2019

andrei-papou commented Apr 8, 2019

LukeMathWalker commented Apr 12, 2019

LukeMathWalker commented Apr 16, 2019

andrei-papou commented Apr 17, 2019

LukeMathWalker commented Apr 23, 2019

LukeMathWalker commented Apr 25, 2019

jturner314 commented Apr 25, 2019

LukeMathWalker commented Apr 26, 2019

jturner314 commented Apr 26, 2019