You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm using the progressbar to show the progress of my AI model training. Therefore I have three bars, "epochs", "training", "validation". First the "training" bar is filling up (like a few thousand steps). Then "training" stops and "validation" is filling up. After that is finished, I restart both and advance the "epoch" bar once and want to restart the "training" bar. But thats when the error occurs: The running clock of the previously finished and resetted "training" bar does not start again. The advancement etc works, but the clock timer keeps showing 0:00:00.
This is how it looks in the terminal. You can see, the yellow clock of "training" remains, while the loop clearly started and was also advanced many times. Same happens in the "validation" task.
Here is my code snippet:
fromrich.progressimportBarColumn, Progress, TextColumn, TimeElapsedColumn, TimeRemainingColumndefinit_progressbar() ->Progress:
returnProgress(
TextColumn("[progress.description]{task.description}"),
TextColumn(" "),
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
TextColumn(" "),
BarColumn(),
TextColumn(" "),
TextColumn("[progress.download]{task.completed:>5.0f}/{task.total:>5.0f}"),
TextColumn(" "),
TimeElapsedColumn(),
TextColumn(" "),
TimeRemainingColumn(),
)
progress=init_progressbar()
progress.start()
epoch_task=progress.add_task("[red]Epoch", total=num_epochs)
train_task=progress.add_task("[blue]Training", total=num_train_data, start=False)
valid_task=progress.add_task("[green]Validation", total=num_valid_data, start=False)
# training in epochsforepochinrange(0, num_epochs):
# Training ================# start the training loop progress barprogress.start_task(train_task)
# progress.reset(train_task, start=True, completed=0)forbatch, ... intrain_dataloader:
...
# update the progressbar for the training loopprogress.advance(train_task, advance=train_dataloader.batch_size)
# stop the training loop progress bar# progress.update(train_task, completed=num_train_data)progress.stop_task(train_task)
# Validation ================# start the validation loop progress barprogress.start_task(valid_task)
# progress.reset(valid_task, start=True, completed=0)forbatch, ... invalid_dataloader:
...
# update the progressbar for the validation loopprogress.advance(valid_task, advance=valid_dataloader.batch_size)
# stop the validation loop progress bar# progress.update(valid_task, completed=num_valid_data)progress.stop_task(valid_task)
...
# update the progressbar for the epoch loopprogress.advance(epoch_task, advance=1)
progress.reset(train_task, start=False)
progress.reset(valid_task, start=False)
The commented code-lines were tests I did, but they did not help to resolve the issue.
It might boil down to this line, as I do reset(task, start=False) and with start=False the start_time is None. Is that the intended behavior? If so, would it be possible to also set a flag as an argument as restart_clocl: bool = False, which in my case could be overridden to True?
I'm using the progressbar to show the progress of my AI model training. Therefore I have three bars, "epochs", "training", "validation". First the "training" bar is filling up (like a few thousand steps). Then "training" stops and "validation" is filling up. After that is finished, I restart both and advance the "epoch" bar once and want to restart the "training" bar. But thats when the error occurs: The running clock of the previously finished and resetted "training" bar does not start again. The advancement etc works, but the clock timer keeps showing
0:00:00
.This is how it looks in the terminal. You can see, the yellow clock of "training" remains, while the loop clearly started and was also advanced many times. Same happens in the "validation" task.
Here is my code snippet:
The commented code-lines were tests I did, but they did not help to resolve the issue.
Output of
rich.diagnose
:The text was updated successfully, but these errors were encountered: