-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
My YAML file appears to not get processed properly #259
Comments
Hi @Chris-Schnaufer thanks for reporting this! Seems there is indeed a bug in the configuration file validation logic,
I think the missing |
Hi @Chris-Schnaufer, this is addressed in v0.0.33 (out tonight), looks like the I did notice that in the |
Thanks @Chris-Schnaufer, found the first problem (the 'Selected' alert doesn't handle the case for directory inputs, only file/files). Will patch shortly. Looking into the MIAPPE project binding too. Really appreciate your help revealing all these issues- I'm not able to reliably test the whole UI surface alone. |
Hi @Chris-Schnaufer, the input selection issue should now be resolved (still working on the project binding fix). Apologies for the delay. Please let me know if you are still unable to submit jobs. I just tested the pipeline and although the submission is successful, the job fails due to a missing
|
Hello @w-bonelli. I was able to get back to testing this and I am still having problems. I changed the docker image so that it's pointing to a test version that has the I'm not sure why it's reposting that it can't find the docker image. I've tried uploading the docker image again and there's no change in the run result. Here's two screen shots showing the step before running the Task, and the Task result. I'm running from the On another note, I appear to have two projects with the same GitHub path: |
Thanks @Chris-Schnaufer looking into this now |
Hi @Chris-Schnaufer, I believe the root cause of the latest issue was the docker image attribute was not parsed properly from I was able to submit |
@w-bonelli That repository has the incorrect Docker image listed. The one at |
Ok, got it. I just did a test run with
Looks like the root issue is that Singularity makes the filesystem read-only by default. So when Would it be possible to alter the way the agdrone workflow accepts configuration info? E.g., allowing the location of Another option might be to use the |
Hello @w-bonelli, I have looked at this and have some comments (I'm also not knowledgable about Singularity):
There are dependencies built into the container that would require additional writes to the file system to allow this. So I don't think it would work.
What are the writable folders on the system? In other words, how do generated files get saved in the container and exported from the container when it's done? Also, the apps that run expect the Any help on this is appreciated! |
Hi @Chris-Schnaufer,
Singularity automatically mounts the current directory on the host into the container (as well as
I think bind mounts might work as an indirect way of making I will try this tomorrow or Fri to check that it works as expected. I wish we could provide more straightforward support for writing arbitrary locations but I think it is a pretty fundamental Singularity limitation. |
The system works by checking out the repo and then building the docker image. The built solution is what's run. |
Trying this now. |
Hi @Chris-Schnaufer my apologies again for the delay. I think I have this working now. See the diff here for the changes. What I did:
The workflow seems to run successfully. The job log includes the following output:
And the
I'm not sure what output files are expected, so I'm not sure how to validate results. You may need to specify exact names of expected output files in the Hope this helps. Please let me know if I can do anything else and thanks again — has been a really valuable edge case to explore and figure out how to support. |
Wow! Thanks @w-bonelli! This looks great so far and I will look into it further. Regarding the mounts, is there a reason that the folders under /scif/data/* are separate mounts versus mounting only |
@Chris-Schnaufer No problem! The workflows refresh every 5 minutes — you may need to reload the page to see the changes reflected. It would be nice to be able to manually refresh particular workflows though. I'll add that to the roadmap. I also have 4, it looks like 2 branches under De-duplication is planned for branches of the same repo (tracked here) but I have not gotten to it yet. In the meantime, we can add one of the workflows to the |
@w-bonelli thanks for the quick response. I'm not ready to have the workflow featured yet but I will let you know when I think it's ready 👍 |
Hello @w-bonelli, I am still seeing these issues. Any updates? Thanks |
Hi @Chris-Schnaufer apologies for the delay, which issue are you seeing? The Stampede2 agent is no longer publicly available but there is a Sapelo2 agent you should be able to submit to. |
Hello, please see above comment (the last image shows the Sapelo2 issue) Please ignore the Stampede2 agent since it's no longer available. Comment link: #259 (comment) |
Is it the authentication failed error? Would you mind sharing a more recent task ID? |
Yes, the error reads I tried to reproduce the problem but I am unable to select the |
That's likely because the agdrone workflow requests 8 cores, while the public Sapelo2 agent allows a max of 2. If you update your In the longer term we're changing the way orchestration works to take advantage of GitHub actions, so I'll update this thread as that work progresses |
I think this occurred because this task's output location is your home folder Thanks for bringing this to light, I've updated the site to disallow selecting the top-level home folder as the output location. |
I am now able to run my workflows. Will open a new issue for problems found |
My YAML file shows that the
name
field is missing, although it's there. Also, trying to run reports an error that theimage
configuration key is missing, even though it's there as well. My YAML file: https://github.com/Chris-Schnaufer/drone-makeflow/blob/main/plantit.yamlThe text was updated successfully, but these errors were encountered: