You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently generate the environment.csv for a benchmark run in the same job as the actual benchmark run. However, it uses duplicate code because we were trying to get up and running.
We should work to de-duplicating this, which includes:
producing it one place
uploading it one place, which may involve moving artifacts around
Because there was such a rush from the teams to get this up and running and there were oddities with getting this information within the running job, I made the decision to make some duplicate code. We generate a separate ci/cd environment CSV in a separate post-processing job as well. So that's the duplication.
It's not code you wrote, but we believe models team should own everything generated within the demos pipeline, as the original proposal for this information came from model's team.
It was my understanding that the generation of the run and measurement CSVs would be owned by the models team while the generation of the environment CSV (which is intentionally separate due to its information being specific to the CI environment) would be owned by the infra team. @uaydonat is this correct?
For the duplicate code, the csv saving function used for run/measurements here can be re-used, but besides that the duplication seems CI specific.
Sounds good, but from now on anything to do with benchmarking schemas or extra data generation for benchmarking is to be owned by models team, including any additions to CI information.
We currently generate the
environment.csv
for a benchmark run in the same job as the actual benchmark run. However, it uses duplicate code because we were trying to get up and running.We should work to de-duplicating this, which includes:
cc: @TT-billteng @skhorasganiTT @uaydonat
The text was updated successfully, but these errors were encountered: