NPT-14408: Unable to see data when importing tensorboard logs.

[odinsbane]

I am not able to view metrics from tensorboard generated logs.

1. To reproduce

Train a tensorflow model with keras using the tesnorboad callback.

import keras
import numpy

inp = keras.layers.Input( (16, ) )
op = keras.layers.Dense(4)(inp)

mdl = keras.models.Model( inputs=[inp], outputs = [op] )

x = numpy.random.random((128, 16))
y = (numpy.random.random((128, 4))>0.75) * 1.0

mdl.compile()

metrics = [ keras.metrics.BinaryAccuracy(name = "ba")]
mdl.compile( optimizer = keras.optimizers.Adam(0.00001), loss="mean_squared_error", metrics = metrics)

callbacks = [ keras.callbacks.TensorBoard(log_dir='./junk-logs'), keras.callbacks.EarlyStopping(patience=5, monitor="loss", mode='min')]

mdl.fit( x, y, callbacks=callbacks, epochs=100)

Then I try to synch the logs with the neptune.

export NEPTUNE_PROJECT="project/name"
export NEPTUNE_API_TOKEN="xxx"
./neptune-env/bin/neptune tensorboard junk-logs

The result is a new entry in my project, but I cannot find any of the values that were tracked.

Expected behavior

I can see graphs for the metrics in tensorboard, but I cannot see the graphs in neptune

3. Environment

Default environment that was installed when I used ./neptune-env/bin/pip install neptune-tensorboard

./neptune-env/bin/python --version

> Python 3.10.13

./neptune-env/bin/pip list

Package Version

---

absl-py 2.0.0
arrow 1.3.0
astunparse 1.6.3
attrs 23.2.0
boto3 1.34.14
botocore 1.34.14
bravado 11.0.3
bravado-core 6.1.1
cachetools 5.3.2
certifi 2023.11.17
charset-normalizer 3.3.2
click 8.1.7
contourpy 1.2.0
cycler 0.12.1
exceptiongroup 1.2.0
execnet 2.0.2
filelock 3.13.1
flatbuffers 23.5.26
fonttools 4.47.0
fqdn 1.5.1
fsspec 2023.12.2
future 0.18.3
gast 0.5.4
gitdb 4.0.11
GitPython 3.1.40
google-auth 2.26.1
google-auth-oauthlib 1.2.0
google-pasta 0.2.0
grpcio 1.60.0
h5py 3.10.0
idna 3.6
iniconfig 2.0.0
isoduration 20.11.0
Jinja2 3.1.2
jmespath 1.0.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.20.0
jsonschema-specifications 2023.12.1
keras 2.15.0
kiwisolver 1.4.5
libclang 16.0.6
Markdown 3.5.1
MarkupSafe 2.1.3
matplotlib 3.8.2
ml-dtypes 0.2.0
monotonic 1.6
mpmath 1.3.0
msgpack 1.0.7
neptune 1.8.6
neptune-tensorboard 1.0.2
networkx 3.2.1
numpy 1.26.3
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
oauthlib 3.2.2
opt-einsum 3.3.0
packaging 23.2
pandas 2.1.4
pillow 10.2.0
pip 23.0.1
pluggy 1.3.0
protobuf 4.23.4
psutil 5.9.7
pyasn1 0.5.1
pyasn1-modules 0.3.0
PyJWT 2.8.0
pyparsing 3.1.1
pytest 7.4.4
pytest-xdist 3.5.0
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 6.0.1
referencing 0.32.1
requests 2.31.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.16.2
rsa 4.9
s3transfer 0.10.0
setuptools 65.5.0
simplejson 3.19.2
six 1.16.0
smmap 5.0.1
swagger-spec-validator 3.0.3
sympy 1.12
tbparse 0.0.8
tensorboard 2.15.1
tensorboard-data-server 0.7.2
tensorboardX 2.6.2.2
tensorflow 2.15.0.post1
tensorflow-estimator 2.15.0
tensorflow-io-gcs-filesystem 0.35.0
termcolor 2.4.0
tomli 2.0.1
torch 2.1.2
triton 2.1.0
types-python-dateutil 2.8.19.14
typing_extensions 4.9.0
tzdata 2023.4
uri-template 1.3.0
urllib3 2.0.7
webcolors 1.13
websocket-client 1.7.0
Werkzeug 3.0.1
wheel 0.41.2
wrapt 1.14.1

This is a 64bit linux, I think centos with 4.18 kernel.

[SiddhantSadangi]

Thanks for reporting this @odinsbane 🙇

I was able to reproduce this and have created a ticket for the engineering team.
I'll keep you updated on the progress and reach out if we need any further information.

[AleksanderWWW]

@SiddhantSadangi @odinsbane
I've identified the issue to be a gap in what we log in the integration automatically (the field tensors was missing and actually it was the only field to which any data was logged in this example). I believe that #71 will sort things out