-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ISSUE-21b(the return of the CSV) Islandora 7 migrations! ready to be used (almost?) #29
Conversation
- Better CSV header normalizing. WE get all fields first. The fetch. We keep track of the ones that actually have values per row. Once ready with the CSV we go back, make a diff on the headers and only keep the ones that are REALLY not empty and write a new clean CSV out. - trying to get rid of authority_marcrelator fields (may need more work?) - AMI set gets created and redirected to the Set. Tested with 7000 Objects, took 25 seconds? - Parent Object gets set into ismemberof `@TODO`: - Cleanups, probably more checks? Lots of checks here - Remove temp files/stuff around - Clear form states/steps - Change the message at the end (since we are redirecting we have a different message
Hello @DiegoPino! Testing this now and really appreciating the extra csv-cleaning! 😃 |
Adds providesKeys and providesTypes. That way Solr data that comes back is not used for UI but only the keys we generate. Makes all cleaner and faster. Also some UI improvements. Children mappings are only shown IF collapse is disabled. @alliomeria i could not reproduce the false collapsing? So wonder if there is something else missing in your code? Let's try this tomorrow again
Why do people make compounds and also part of a collection? Well well well
@alliomeria did you find any issues with this plugin? Things that I should fix before merging? |
@DiegoPino --so far all has been going very well with work related to using this plugin. I have a few small things I noticed and a couple questions about the pages for steps 4 & 5.
|
@alliomeria thanks for your feedback. Yes, most of the issues are on how I'm dealing with Form Steps validations/verifications. I think I can deal with that in another pull to make this one smaller (if you are Ok) About your question
The label is used to provide a Label for NODE in case you forget/do not add a "label" key in your template. Its mostly redundancy and/or meant when you are using the "Direct" mode instead of the Twig template processor. |
@DiegoPino thanks to you for all of your coding developments! 🐝 Sounds good about the different pulls--whatever your own working preference of course. Ah, thank you & got it about the ADO Label mapping, makes sense. I've been using all Custom + Twig template modes, so I did not see how/where it was being used. |
@alliomeria will merge this so #14 can be added. Not perfect (will work on the UI issues in that new Pull) but usable and useful. |
Sounds great @DiegoPino! Looking forward to using this soon! 😄 |
What? Islandora 7 importer ready for testing!
See #21 This is the second part, now full fledged with CSV header normalizer, cleaning out empty columns, parents, mapping for Compounds even if we DO not know what we will get and AMI Set creation at the end of the BATCH. Also some memory optimizations (this is heavy!)
Some stuff:
Tested with 7000 Objects, took 25 seconds?
@TODO
:@alliomeria this solves like 95% of the things we discussed today. the CSV is waaaayy less scary 🥧 and more 🌴
There are some cosmetics to do, but already looking better. I show at the end two CSVs. ON the message itself the original (like 700 columns!!) but in the Set itself, the attached one is clean, normalized, beautiful.