Skip to content

Commit

Permalink
fix throttling
Browse files Browse the repository at this point in the history
  • Loading branch information
jynx10 committed Aug 10, 2021
1 parent e4ad51f commit 80b7503
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions catalyst/loggers/comet.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,13 @@ def log_metrics(
prefix_parameters = [stage_key, loader_key, scope]
prefix = self._format_prefix(prefix_parameters)

self.experiment.log_metrics(
metrics,
step=global_batch_step,
epoch=global_batch_step,
prefix=prefix,
)
if global_batch_step % 10 == 0:
self.experiment.log_metrics(
metrics,
step=global_batch_step,
epoch=global_batch_step,

This comment has been minimized.

Copy link
@DN6

DN6 Aug 10, 2021

Collaborator

I would change this to epoch=global_epoch_step

This comment has been minimized.

Copy link
@DN6

DN6 Aug 10, 2021

Collaborator

Also, global_batch_step doesn't necessarily have to be in steps of 10? e.g. global_batch_step = [11, 21, 31]

In that case, nothing would get logged. I think we can handle the throttling logic outside of the integration. It is something that the Catalyst user should have control over.

This comment has been minimized.

Copy link
@DN6

DN6 Aug 10, 2021

Collaborator

Or have it be configurable through a parameter

if global_batch_step % self.logging_frequency == 0

prefix=prefix,
)

def log_image(
self,
Expand Down

0 comments on commit 80b7503

Please sign in to comment.