Job output transform #548
Job output transform #548
Conversation
Direct Links in job output + makefile + tests
Testing transformer engine added
Remove requirements-trfm Add to main requirements
Why not pandas
Update requirements
Ajout Multi frame Images
github-actions
bot
added
ci/doc
There was a problem hiding this comment.
There's a few linting issues that can be resolved automatically using the make fix[...] targets.
Validate as well using make check[...] targets any other flagged issues that might need some manual adjustments. I didn't comment on all of them.
I did not go in depth into the actual implementation of the transform functions.
I think there is a few tests missing to validate that the actual results from each transformation is valid (eg: pixel values from one format to another are properly rendered or expected text from one format to another is properly structured).
| # noqa: E241,E501 | ||
| (uri_provider_jobs, Visibility.PUBLIC, self.user_admin_id, filter_service(public_jobs)), # noqa: E241,E501 | ||
|
|
||
| ] # type: List[Tuple[str, str, Union[None, int], List[Job]]] |
There was a problem hiding this comment.
Revert auto-formating by your IDE here and above.
tests/transform/test_tranform.py
Outdated
| except Exception as e: | ||
| print(cmt + " -> " + wmt + " failed") | ||
| assert False, os.path.splitext(f)[1] + " -> " + f + " " + str(e) | ||
| pass |
weaver/formats.py
Outdated
|
|
||
|
|
||
|
|
weaver/wps_restapi/jobs/jobs.py
Outdated
| Retrieve the possible formats of an output job. | ||
| """ | ||
| job = get_job(request) | ||
| return get_job_possible_output_formats(job) |
There was a problem hiding this comment.
Should do some kind of sd.JobTranformerSchema().deserialize(<body>) check before returning the contents to ensure they match the format of weaver/wps_restapi/examples/job_transformer.json
weaver/tranform/tiff.py
Outdated
There was a problem hiding this comment.
Add missing docstrings and typings.
weaver/tranform/transform.py
Outdated
| self.file_path = file_path | ||
| self.cmt = current_media_type.lower() | ||
| self.wmt = wanted_media_type.lower() | ||
| self.output_path = self.file_path |
There was a problem hiding this comment.
The output path needs to be modifiable (either from init arg, or modified after before the process/get calls.
More importantly, no additional files should be added under the original WPS outputs sub-directories, because their contents could be critical to the packing/detection of other apps in a workflow chain.
We need to consider the creation of some "sibling" directory next to the WPS-output-ID dir to avoid modifying the output checksums.
github-actions
bot
added
feature/job
github-actions
bot
added
the
feature/cli
fixes #18
fixes #100
Pour les conversions image, si GeoTiff, distinguo fait dans class Tiff