-
-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Help needed to bulkload subsamples of subsamples with part attributes #3627
Comments
See #3628 - I just patched the missing tool into production. I did this to create this then this for Note that I didn't take out the second load's If you have barcodes for everything, this should be trivial. If you don't, I'd suggest a new container_type of "virtual" and a new series of "part identifier" barcodes for this sort of thing (some "series" that you're not going to confuse with anything else or print/order). First load the "liver hacked out of a mixed tube" part layer, then load the "stuff magicked out of the previous part" layer, then loan items. You can save some tiny bit of work by getting the attributes sorted out now, or if you're using barcodes its trivial to load part attributes later. If you want to provide your full data in CSV and probably answer some questions I could push the buttons and take screenshots, or everything should be in place for you to do so. |
Yes, I was heading towards " new container_type of "virtual" and a new
series of "part identifier" barcodes for this sort of thing (some "series"
that you're not going to confuse with anything else or print/order)." - I
created a virtual container for the entire loan, as I want all the loan
subsamples in the same virtual container. None of them have any actual
barcodes.
Without using real barcodes, I think I have to load part attributes at the
same time with this file.
I will work on getting you a full csv based on my understanding of the
process - thanks!
…On Wed, Jun 2, 2021 at 7:48 AM dustymc ***@***.***> wrote:
* [EXTERNAL]*
See #3628 <#3628> - I just
patched the missing tool into production.
I did this
[image: Screen Shot 2021-06-02 at 6 22 45 AM]
<https://user-images.githubusercontent.com/5720791/120489669-ac53f600-c36c-11eb-953f-0a4527b90781.png>
to create this
[image: Screen Shot 2021-06-02 at 6 22 07 AM]
<https://user-images.githubusercontent.com/5720791/120489718-b544c780-c36c-11eb-95c6-63ee19b62ddb.png>
then this
[image: Screen Shot 2021-06-02 at 6 28 53 AM]
<https://user-images.githubusercontent.com/5720791/120489829-ca215b00-c36c-11eb-8330-1ef72d3eceeb.png>
for
[image: Screen Shot 2021-06-02 at 6 28 19 AM]
<https://user-images.githubusercontent.com/5720791/120489876-d1e0ff80-c36c-11eb-9e0c-0522293451d4.png>
Note that I didn't take out the second load's container_barcode so adding
another layer would not be able to use barcode - don't follow my example
too closely....
If you have barcodes for everything, this should be trivial. If you don't,
I'd suggest a new container_type of "virtual" and a new series of "part
identifier" barcodes for this sort of thing (some "series" that you're not
going to confuse with anything else or print/order).
First load the "liver hacked out of a mixed tube" part layer, then load
the "stuff magicked out of the previous part" layer, then loan items.
You can save some tiny bit of work by getting the attributes sorted out
now, or if you're using barcodes its trivial to load part attributes later.
If you want to provide your full data in CSV and probably answer some
questions I could push the buttons and take screenshots, or everything
should be in place for you to do so.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBERKD2UETQUKUAZM4TTQYZCHANCNFSM45X4SJ7Q>
.
|
Hu?
Also hu? Picking a specific part out of the pile is a lot of work. Once that happens, you can
Barcodes, real or otherwise, serve nicely as unique identifiers. In this case, giving everything a barcode would mean you can add the attributes later, gives you a super-easy way to eventually add to the loan, and probably provides a pathway to whatever you mean by "loan subsamples in the same virtual container." (I've been wondering if we need directly-attached stable part IDs for a while, and maybe we do - new Issue - but they're not available NOW and barcodes are.) |
Stable part IDs would be great, so I could just grab the susample Id and
say "subsampled from part ID xxxx". We have hundreds of thousands of
unbarcoded samples, thousands of which have subsamples somewhere that have
led to extractions in some research lab. No way to physically barcode, and
my attempting to create and hoc virtual barcodes for things not in my
possession or no longer existing would be very time consuming, difficult,
and error prone.
…On Wed, Jun 2, 2021, 8:51 AM dustymc ***@***.***> wrote:
* [EXTERNAL]*
want all the loan subsamples in the same virtual container
Hu?
Without using real barcodes, I think I have to load part attributes at the
same time with this file.
Also hu?
Picking a specific part out of the pile is a lot of work. Once that
happens, you can
- give it a unique identifier so you don't ever have to deal with any
ambiguity again, or
- don't, and dig it out again the next time you need it
Barcodes, real or otherwise, serve nicely as unique identifiers. In this
case, giving everything a barcode would mean you can add the attributes
later, gives you a super-easy way to eventually add to the loan, and
probably provides a pathway to whatever you mean by "loan subsamples in the
same virtual container."
(I've been wondering if we need directly-attached stable part IDs for a
while, and maybe we do - new Issue - but they're not available NOW and
barcodes are.)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBB3T4PKKZPDMBYFYELTQZAOZANCNFSM45X4SJ7Q>
.
|
All parts have an identifier (big integer), it's stable enough for "local" use but probably not for publication. I'd need to know a lot more about what you have and what you want to do about it to elaborate. I think some/most of the problem comes back to loan procedures. I doubt much can be done about the legacy mess at this point, but going forward you could just decide not to send out things that don't have citable identifiers. ARKs might work well for that, although the size at which they can be printed might be problematic. I don't think the source of those ("part IDs" vs barcodes) would make much/any difference in the work required to attach the ID to the item. |
We'd have to start barcoding subsamples, which I've considered but which is
a substantial cost in time and materials. However, we create the subsample
associated with each loan item. It would be helpful if that part identifier
could be stable in Arctos, and we could easily download a list of these
identifiers for a given set of guids+ parts. Then we have the option of
generating labels or reports from these.
…On Wed, Jun 2, 2021, 10:32 AM dustymc ***@***.***> wrote:
* [EXTERNAL]*
All parts have an identifier (big integer), it's stable enough for "local"
use but probably not for publication. I'd need to know a lot more about
what you have and what you want to do about it to elaborate.
I think some/most of the problem comes back to loan procedures. I doubt
much can be done about the legacy mess at this point, but going forward you
could just decide not to send out things that don't have citable
identifiers. ARKs might work well for that, although the size at which they
can be printed might be problematic. I don't think the source of those
("part IDs" vs barcodes) would make much/any difference in the work
required to attach the ID to the item.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBCQPSMAKHDI5NKFXITTQZMKDANCNFSM45X4SJ7Q>
.
|
That's above my pay grade, but passing on the cost of buying a bajillion preprinted whatever-you-sample-into-anyway containers doesn't seem completely unrealistic at first glance.
That adds some stability - you can't delete those, but there are still plenty of ways in which "stable" requires procedures.
It would be pretty trivial to add specimen_part.collection_object_id to parts download. |
But when that integer is combined with the record url with which it is associated....isn't that also a GUID? It means we would have to open up the view of those numbers, but seems like that would be useful? |
http://test.arctos.database.museum/guid/MSB:Mamm:231731?barcode=MSB1001913 and http://test.arctos.database.museum/guid/MSB:Mamm:231731?pid=29037926 (and any other ?thing=value pair, real or not) are all functional GUIDs, and both of those could do stuff (eg something like http://test.arctos.database.museum/guid/MSB:Mamm:231731?seid=966527 does). That much would be pretty simple to implement. They're not particularly stable - you can delete and re-create (with a new PID) the part, I can add a billion to all part_ids because why not, you can replace the barcode or move the part to a different container, etc., etc., etc. Whether any of that matters or not would need sorted out and built around or documented if it does. |
Gotcha on the "stable" now and that being intentional about a part ID is important because it means you are going to be stuck with that part. Although like anything else we could create a redirect if we realize something is not correct? I can see how a stable part ID would be a very useful tool and also how stability might not matter so much for some applications. However - it isn't 100% clear what http://test.arctos.database.museum/guid/MSB:Mamm:231731?seid=966527 does. I think if the screen scrolled to the highlighted occurrence that would be cool. And if http://test.arctos.database.museum/guid/MSB:Mamm:231731?pid=29037926 highlighted the appropriate part and scrolled to it, that would be cool too. I know - I want it all!!! I'm not sure I have an answer to the problem of stability....probably needs a group effort. |
Yes, they can handle that, but (given things like more sophisticated bulkloaders/unloaders mixed with our terrifying loan/citation intersection data) you can still:
That's probably overly paranoid, but I also think that's the most practical viewpoint at this stage. "Intentional" seems like a decent one-word summary of what might work best.
Issue - should be pretty quick but I don't have the bandwidth at the moment and I'll forget without. |
For lack of a better place for this: I created new part attributes and new units http://test.arctos.database.museum/info/ctDocumentation.cfm?table=ctcount_units#ratio and a new event type then I cloned http://test.arctos.database.museum/guid/UAM:Env:115 (the environmental sample) into two records: http://test.arctos.database.museum/guid/UAM:Env:350 is a woodfrog which was detected. http://test.arctos.database.museum/guid/UAM:Env:351 is a moose which was not detected (see the specimen event). I created http://test.arctos.database.museum/info/ctDocumentation.cfm?table=ctspecimen_part_name#there_is_no_part because I think WHY we think there are no moose is best given as part attributes, in the same way why we think there are frogs is part attribute data. Weird, needs more discussion. Maybe these data are all attributes of the identification, not the parts?! Everything about the absorbance attribute units are strange, hopefully just ignorance on my part. There's no great place for the relationship between the filter paper in http://test.arctos.database.museum/guid/UAM:Env:115 and the DNA in http://test.arctos.database.museum/guid/UAM:Env:350. http://test.arctos.database.museum/guid/UAM:Env:350 should start with filter paper and have the DNA as a child of that - this could be better - but there's currently no cross-cataloged-item part-relationships; we may be missing something from the UI (and VPDs could very quickly make that very complicated - may not be possible without public and stable part IDs????). The sample itself was apparently cataloged with the frog in mind, which is strange - eg is Spangler only sorta-confident that this is water? (Probably not, I think that's in reference to the frog.) And water could be a taxon - some new chemical-focused source (and type), probably. |
Water is a taxon, sort of - https://arctos.database.museum/name/Ice |
Oh my.... |
I can't see any of it because I don't have access to UAM collections in test.... |
No man, it works! water will just be another "species" of Oxides of hydrogen! |
OK logged in as public user and can see 115, but the other two 404 on me. |
You both have uam_env now. public view will be sparse because I don't have a real dev box.
Sure simplify linnean stuff if we just consider it all carbon with some impurities... |
Sounds good, thanks.
…On Mon, Jun 14, 2021 at 9:26 AM Teresa Mayfield-Meyer < ***@***.***> wrote:
* [EXTERNAL]*
I am swamped with calls today, but maybe we can talk about this right
after we finish with the Norway peeps?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBE7SC2UUEY7CF5RZETTSYNTJANCNFSM45X4SJ7Q>
.
|
I did this; I can un-do it if necessary. new barcode series: New container type: Make some containers Use SQL (below) to put the loaned items in containers. This would have been MUCH easier BEFORE they were loaned, and some of the loaned items also have barcodes - the inconsistency complicates this. See eg https://arctos.database.museum/guid/MSB:Mamm:94113 These parts can be bulkloaded, they'll go into virtual containers. It needs review and maybe cleanup, then... .... can be used to add the parts created in the previous step to the loan using the virtual containers. I did not change any part names; assuming that eg https://arctos.database.museum/guid/MSB:Mamm:248819 provided some homogeneous part, you can just update the loaned part (currently heart, kidney, lung, spleen (frozen)) to whatever was actually sent out, and then the extraction will be a child of the appropriate part following the normal and expected pathway to that information. If the mash was sent out then everything's already correct. If you don't know what was sent then the part should be updated to unknown. Going forward, if you assign a barcode (of container type virtual or otherwise) to loan items then this is all very simple; if you don't then it's not. If you want to use this system then you'll need to figure out how to keep track of the containers. I created 500 (linked spreadsheet) and used 230 of those in this. FWIW I'd guess that the cost of buying/printing cheapy barcodes is orders of magnitude less than the cost of dealing with ambiguous parts and trying to figure out what exactly was cited and all that nonsense, and I'd guess your "customers" would be happy enough to pay an extra few cents per sample for that level or organization. And if they use those barcodes - which should be absolutely trivial if they're doing anything that might be described as organized - and you eventually get hold of their LIMS, whatever form it might take, then integrating that is also trivial. Here's the SQL I used to get barcodes attached to the parts.
|
I need some time to look this over and process, but could we add the
ability to subsample and put into a container to the "subsample for loan"
in the loan form -to deal with "this would have been MUCH easier BEFORE
they were loaned"
Need lunch and coffee first - will take a look shortly
…On Mon, Jun 14, 2021 at 10:46 AM dustymc ***@***.***> wrote:
* [EXTERNAL]*
I did this; I can un-do it if necessary.
new barcode series:
[image: Screen Shot 2021-06-14 at 8 24 22 AM]
<https://user-images.githubusercontent.com/5720791/121917418-f54f6700-cce9-11eb-9ff7-d10178345402.png>
New container type:
[image: Screen Shot 2021-06-14 at 8 31 17 AM]
<https://user-images.githubusercontent.com/5720791/121918473-e9b07000-ccea-11eb-8f12-59de14343dbd.png>
Make some containers
https://docs.google.com/spreadsheets/d/19giHcLHx469iwtct54yQsPg87MaRKxgvLaG_U16y_I8/edit#gid=262565804
Use SQL (below) to put the loaned items in containers. This would have
been MUCH easier BEFORE they were loaned, and some of the loaned items also
have barcodes - the inconsistency complicates this. See eg
https://arctos.database.museum/guid/MSB:Mamm:94113
[image: Screen Shot 2021-06-14 at 8 47 57 AM]
<https://user-images.githubusercontent.com/5720791/121920820-43199e80-cced-11eb-9857-84c395458a9c.png>
temp_msbpartstobl.csv.zip
<https://github.com/ArctosDB/arctos/files/6649829/temp_msbpartstobl.csv.zip>
These parts can be bulkloaded, they'll go into virtual containers. It
needs review and maybe cleanup, then...
temp_msbpartloanbulk.csv.zip
<https://github.com/ArctosDB/arctos/files/6649833/temp_msbpartloanbulk.csv.zip>
.... can be used to add the parts created in the previous step to the loan
using the virtual containers.
I did not change any part names; assuming that eg
https://arctos.database.museum/guid/MSB:Mamm:248819
provided some homogeneous part, you can just update the loaned part
(currently heart, kidney, lung, spleen (frozen)) to whatever was actually
sent out, and then the extraction will be a child of the appropriate part
following the normal and expected pathway to that information. If the mash
was sent out then everything's already correct. If you don't know what was
sent then the part should be updated to unknown.
Going forward, if you assign a barcode (of container type virtual or
otherwise) to loan items then this is all very simple; if you don't then
it's not.
If you want to use this system then you'll need to figure out how to keep
track of the containers. I created 500 (linked spreadsheet) and used 230 of
those in this.
FWIW I'd guess that the cost of buying/printing cheapy barcodes is orders
of magnitude less than the cost of dealing with ambiguous parts and trying
to figure out what exactly was cited and all that nonsense, and I'd guess
your "customers" would be happy enough to pay an extra few cents per sample
for that level or organization. And if they use those barcodes - which
should be absolutely trivial if they're doing anything that might be
described as organized - and you eventually get hold of their LIMS,
whatever form it might take, then integrating that is also trivial.
Here's the SQL I used to get barcodes attached to the parts.
create table temp_msbloanmess as
select
loan_item.collection_object_id,
specimen_part.part_name,
c2.barcode
from loan_item
inner join specimen_part on loan_item.collection_object_id=specimen_part.collection_object_id
inner join coll_obj_cont_hist on specimen_part.collection_object_id=coll_obj_cont_hist.collection_object_id
inner join container on coll_obj_cont_hist.container_id=container.container_id
left outer join container c2 on container.parent_container_id=c2.container_id
where transaction_id=21110899;
alter table temp_msbloanmess add add_part_to_barcode varchar;
CREATE OR REPLACE function temp_test() returns void AS $body$
DECLARE
r RECORD;
y int;
BEGIN
y=1;
for r in (select collection_object_id from temp_msbloanmess where barcode is null) loop
update temp_msbloanmess set barcode='MSBPID' || y::text where collection_object_id=r.collection_object_id;
y=y+1;
end loop;
END;
$body$
LANGUAGE PLPGSQL
SECURITY DEFINER
;
select temp_test();
drop function temp_test;
alter table temp_msbloanmess add add_part_to_cid bigint;
update temp_msbloanmess set add_part_to_cid=(select container_id from container where container.barcode=temp_msbloanmess.barcode) where barcode like 'MSBPID%'
;
alter table temp_msbloanmess add pcid bigint;
update temp_msbloanmess set pcid=(select container_id from coll_obj_cont_hist where coll_obj_cont_hist.collection_object_id=temp_msbloanmess.collection_object_id) where barcode like 'MSBPID%'
;
CREATE OR REPLACE function temp_test() returns void AS $body$
DECLARE
r RECORD;
BEGIN
for r in (select * from temp_msbloanmess where barcode like 'MSBPID%') loop
update container set parent_container_id=r.add_part_to_cid where container_id=r.pcid;
end loop;
END;
$body$
LANGUAGE PLPGSQL
SECURITY DEFINER
;
select temp_test();
drop function temp_test;
-- now set up for part bulkloader
alter table temp_msbloanmess add guid varchar;
update temp_msbloanmess set guid=(select guid from flat inner join specimen_part on flat.collection_object_id=specimen_part.derived_from_cat_item where specimen_part.collection_object_id=temp_msbloanmess.collection_object_id);
-- where'd we leave off?
select replace(barcode,'MSBPID','')::int from temp_msbloanmess where barcode like 'MSBPID%' order by replace(barcode,'MSBPID','')::int;
alter table temp_msbloanmess add new_part_barcode varchar;
CREATE OR REPLACE function temp_test() returns void AS $body$
DECLARE
r RECORD;
y int;
BEGIN
y=112;
for r in (select collection_object_id from temp_msbloanmess) loop
update temp_msbloanmess set new_part_barcode='MSBPID' || y::text where collection_object_id=r.collection_object_id;
y=y+1;
end loop;
END;
$body$
LANGUAGE PLPGSQL
SECURITY DEFINER
;
select temp_test();
drop function temp_test;
-- make a part bulkloader download
create table temp_msbpartstobl as select
guid,
'DNA extraction' as part_name,
'unknown' as condition,
'in collection' as disposition,
1 as lot_count,
new_part_barcode as container_barcode,
barcode as parent_part_barcode
from
temp_msbloanmess
;
-- make a loan item bulkloader download
create table temp_msbpartloanbulk as
select
guid as guid_delete_before_uploading,
new_part_barcode as part_barcode,
'fix this' as loan_guid_prefix,
'fix this' as loan_number
from
temp_msbloanmess
;
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBC4PVAYDVWZHTVINGLTSYW53ANCNFSM45X4SJ7Q>
.
|
Documentation!!! |
I'm not sure what "the loan form" is - there are a dozen or so things that might be called that. Given how containers work, I think what I've done is probably going to work best through the bulk tools. Having (or exposing, if that fulfills whatever goals we might decide on) a dedicated "part identifier" would open up more possibilities, but see below. From a blank slate, this procedure should be something like
If you've got more than maybe 10 items, there's no way clicking through forms is going to be faster, and handling 3 or 30,000 parts should be about the same work. In a barcode-centric collection where virtual containers aren't some kind of heresy, it's hard to imagine why you'd care about any other forms (other than to create the loan itself). |
I did this here because it's easy to copy-paste into the handbook, or send me your next loan request (one that doesn't start with Step 47!) and I'll load/document. |
How would I know which PID to assign? I need to create a subsample to
create a DNA extraction now, and I could put in some sort of part barcode,
but having this be manual doesn't make any sense. I need to have a way to
assign a part identifier at the time of subsampling, and have Arctos assign
it automatically. In the current process, I'd have to do two bulkloads for
every loan - one to create subsamples and assign IDs somehow without
duplicating existing, and then a second to add to loan? This doubles the
workload, and adds a ton of potential for error.
…On Mon, Jun 14, 2021, 1:17 PM dustymc ***@***.***> wrote:
* [EXTERNAL]*
Documentation!!!
I did this here because it's easy to copy-paste into the handbook, or send
me your next loan request (one that doesn't start with Step 47!) and I'll
load/document.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBACLAYN7LI7NPOVEF3TSZITXANCNFSM45X4SJ7Q>
.
|
I'm not sure how 3 clicks (one download a two uploads) and a couple CSV manipulations (removing columns unless you need to change part names) could double the workload, but I don't really understand what your normal procedure is so it's possible. In any case this is just one possible approach and you certainly don't have to use it. I can delete those containers/series and remove the new container type if what I did isn't useful, or the offer of me using this for your next loan stands if you want to see more before deciding. I'm not really seeing any great places for error to sneak in either, but maybe that's just a lack of imagination. The process I laid out just involves creating an identifier and then using it from the same place.
They're just containers, you can use them like any other container, see above for where I left off. |
So I would need to keep a registry of which virtual barcodes I have already
used to know which ones to assign next?
…On Tue, Jun 15, 2021 at 4:31 PM dustymc ***@***.***> wrote:
* [EXTERNAL]*
This doubles the workload, and adds a ton of potential for error.
I'm not sure how 3 clicks (one download a two uploads) and a couple CSV
manipulations (removing columns unless you need to change part names) could
double the workload, but I don't really understand what your normal
procedure is so it's possible. In any case this is just one possible
approach and you certainly don't have to use it. I can delete those
containers/series and remove the new container type if what I did isn't
useful, or the offer of me using this for your next loan stands if you want
to see more before deciding.
I'm not really seeing any great places for error to sneak in either, but
maybe that's just a lack of imagination. The process I laid out just
involves creating an identifier and then using it from the same place.
autogenerate a virtual part barcode
They're just containers, you can use them like any other container, see
above for where I left off.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBBDH6UKPSIEX4PH7NDTS7IDZANCNFSM45X4SJ7Q>
.
|
We know how that ends up.
On Tue, Jun 15, 2021 at 4:36 PM Mariel Campbell ***@***.***>
wrote:
… So I would need to keep a registry of which virtual barcodes I have
already used to know which ones to assign next?
On Tue, Jun 15, 2021 at 4:31 PM dustymc ***@***.***> wrote:
> * [EXTERNAL]*
>
> This doubles the workload, and adds a ton of potential for error.
>
> I'm not sure how 3 clicks (one download a two uploads) and a couple CSV
> manipulations (removing columns unless you need to change part names) could
> double the workload, but I don't really understand what your normal
> procedure is so it's possible. In any case this is just one possible
> approach and you certainly don't have to use it. I can delete those
> containers/series and remove the new container type if what I did isn't
> useful, or the offer of me using this for your next loan stands if you want
> to see more before deciding.
>
> I'm not really seeing any great places for error to sneak in either, but
> maybe that's just a lack of imagination. The process I laid out just
> involves creating an identifier and then using it from the same place.
>
> autogenerate a virtual part barcode
>
> They're just containers, you can use them like any other container, see
> above for where I left off.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#3627 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADQ7JBBDH6UKPSIEX4PH7NDTS7IDZANCNFSM45X4SJ7Q>
> .
>
|
Yes there's some level of organization necessary, I assumed that was "normal." I can probably work out SQL to find the last-used of a series (as long as there's an incrementing integer somewhere in it), or I can just clean up my mess if you're not interested in this - please let me know. |
Can any of this be automated? Click "subsample and assign next virtual
barcode in series"?
…On Tue, Jun 15, 2021, 6:32 PM dustymc ***@***.***> wrote:
* [EXTERNAL]*
Yes there's some level of organization necessary, I assumed that was
"normal." I can probably work out SQL to find the last-used of a series (as
long as there's an incrementing integer somewhere in it), or I can just
clean up my mess if you're not interested in this - please let me know.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBA5MFEVZ7CDHI2ALJDTS7PJHANCNFSM45X4SJ7Q>
.
|
I don't think that's completely implausible, but I can't quite understand why you'd want to either. (And that's starting to sound like "partD" or something beyond what we have/do now, maybe.) If you're doing this by clicking each item, then what good is introducing an ID that's sole purpose is to avoid the situation from which you seem to want to assign it? The "click every part" approach might need something too, I just can't understand how we went from a spreadsheet full of barcodes to there. Something isn't being communicated and/or understood. Can you provide some "normal" example (if there is such a thing), or SOMETHING big-picture to help me understand where this is coming from, or what you're trying to accomplish, or whatever it is that I need to understand? |
I was asked for thoughts 12 days ago. I'm now reading through this thread for the second time because at first, I was like - doesn't apply to me. And now here I'm on a Saturday needing to bulkload 100 DNA extractions and want them linked to the subsample of a tissue we loaned. I would have thought I've learned by now to say never. But reading through this, I'm still not clear on how I'm supposed to do this magic. I think I thumbed up Dusty at some point because I don't like the idea of "assign next virtual barcode in series" but I'm so lost on all of this now. |
@Jegelewicz And yes, we have water as a taxon. I don't know about the Ice but for our environmental samples are listed as WATER. Not sure I agree with ICE being a mineral though. |
I'm in same boat not understanding this process. I gave up and did the last
loan of 100 extractions one at a time using edit parts in catalog record to
create the subsamples linked to the loaned tissues. I have another loan I
need to do this week. I can send another file of samples I need subsamples
created for?
…On Sat, Jun 26, 2021, 12:43 PM Kyndall Hildebrandt ***@***.***> wrote:
* [EXTERNAL]*
@Jegelewicz <https://github.com/Jegelewicz> And yes, we have water as a
taxon. I don't know about the Ice but for our environmental samples are
listed as WATER.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBEIFI2UKH2IVJWPO3TTUYNWPANCNFSM45X4SJ7Q>
.
|
Skin (frozen) --> Skin (frozen) subsample --> DNA extraction: how do I place the DNA extraction as a child of skin (frozen) via the bulkloader? DNA extraction method: part attribute or part remark? Currently as an attribute. I also have the citation for the extraction method. I currently have that as a part attribute remark. No attribute type in the code table fits this. |
Can we have the option of virtual barcodes autoassigned in the bulkloader?
Something like #3630 ?
This is the limiting factor, and a workflow deal breaker, for any other
route that requires curatorial virtual barcode tracking:
"If you want to use this system then you'll need to figure out how to keep
track of the containers"
And I can't assign barcodes to DNA extractions for samples that have been
loaned and extracted in a research lab and which are still sitting in that
lab and have not yet been returned to the archive. And I've had two such
loan requests in the last month - partly as a result of my insistence that
the researchers not just had over the samples themselves from their
freezers without a formal loan process. So yes, I'm making more work for
myself, but it is critical we do this to track usage.
…On Sat, Jun 26, 2021 at 3:59 PM Kyndall Hildebrandt < ***@***.***> wrote:
* [EXTERNAL]*
Skin (frozen) --> Skin (frozen) subsample --> DNA extraction: how do I
place the DNA extraction as a child of skin (frozen) via the bulkloader?
DNA extraction method: part attribute or part remark? Currently as an
attribute. I also have the citation for the extraction method. I currently
have that as a part attribute remark. No attribute type in the code table
fits this.
BoldExtractionsPartUpload.xlsx
<https://github.com/ArctosDB/arctos/files/6720746/BoldExtractionsPartUpload.xlsx>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBHDSFBGJH4IS5VEWULTUZEUVANCNFSM45X4SJ7Q>
.
|
That's it, there's nothing new here. I made a special container type and a dedicated series to try to keep this separate (in case @campmlc wants me to un-do what I did, mostly) but that's not really necessary. (Probably a good idea if and only if you can't just barcode everything for reals, which I still think would save vast amounts of money and add a great deal of Research Grade in the long run, but still "details.")
I can't find it - https://arctos.database.museum/taxonomy.cfm?taxon_name=water&taxon_name_type=&taxon_term=&term_type=&source=&common_name=#taxonsearchresults
Sure - I can zoom you though it, or just do it in anticipation of #3630, or whatever.
Ideally: You'd have barcoded Skin (frozen) subsample and that few cents would be paying for itself now because this would take about 5 seconds, and everyone would know precisely what everyone else is talking about because there's a "part ID" involved. Less-ideally: Do something complicated to give the samples barcodes (I used SQL, I can probably do the same for you, MAYBE there could be a UI), then everything's just the normal bulkloaders.
#3629, I need how, I'm fine with what @campmlc suggested (free-text).
Mostly a social issue, but I'm pretty sure the answer would be no - that would be disruptive for many use cases.
I'm not sure what similarities you're talking about?? Certainly anything we do with a new ID would require new tools.
How do you keep track of normal containers?! I build forms on the assumption that something like https://handbook.arctosdb.org/documentation/container.html#object-tracking-in-general is happening - if it's not, I need to know what is.
Reason 8564 to just barcode everything.... And maybe a decent use case for container_type=virtual.
Isn't that already part of the formal loan process???? OK, I looked, it is not. MSB's loan policy is listed as http://www.msb.unm.edu/divisions/mammals/index.html, which basically says nothing about loans. UAM's is https://www.uaf.edu/museum/collections/mammal/policies/ which says "Loans may not be transferred to another institution or researcher without prior written consent from the curator. Loaned material may not be used for research that was not specified in the original loan request without written permission from the curator." Aside: I still think that "we" should figure out how to better share what works and what doesn't and etc. for things like policy development. Having a bunch of diverse users involved makes everything I look at better, I can't imagine why that wouldn't apply to things like loan policies as well.
I completely agree, but can't quite understand why we allow that to stop at the door. If everything just got a normal barcode then the researcher could talk about "SomeUnambiguousPart" instead of "UAM 12" (which might be complicated). The whole idea that we can't afford barcodes seems something like not changing the oil in your car to save money - it just doesn't work that way in the not-so-long run. Back to reality, I tentatively suggest we resolve and implement #3630 rather than attempting to do more here. |
The mineral people think it is - it isn't up to me. |
The problem with "just barcode everything" is that DNA extractions are made
in a research lab from tissues we've loaned - the researchers don't have
barcodes to slap on their extractions, although they absolutely should. But
it is bad enough trying to get museums to implement a barcoding system, and
grad students and research faculty are even more like herding cats -
although I have tried to do so with faculty curator's labs.
This is why I requested a way for collection managers to assign virtual
barcodes to things that we do not physically have in our possession, or to
legacy samples that were sent on loan before we had a barcoding system.
In answer to: How do you keep track of normal containers?!" - I have a
spreadsheet with a list of used numbers, plus we can double check this list
in Arctos. The numbers are used to order (or print, in UAM's case) physical
labels that are guaranteed to be unique (checked against the database). To
assign a barcode, I just peel off the next one in the roll, knowing it will
be unique. The alternative, keeping a spreadsheet of used numbers and
trying to keep track of which ones have already been used, is what we used
to do with NK numbers - it always leads to duplicates and messes. So the
only way I see to use *virtual *barcodes is for Arctos to assign one that
the database has already checked and cleared to be unique.
…On Mon, Jun 28, 2021 at 8:29 AM Teresa Mayfield-Meyer < ***@***.***> wrote:
* [EXTERNAL]*
Not sure I agree with ICE being a mineral though.
The mineral people think it is - it isn't up to me.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3627 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBFZ7XQTAHJ3F5Z5PD3TVCBLRANCNFSM45X4SJ7Q>
.
|
Was hoping to talk about this during @dustymc office hour, but nobody was there.... |
Bah sorry I think I've got my calendar set up right now, I hope. I can do later today if you want, but I'm mostly convinced we should abandon this in favor of #3630, and I just left a manifesto that might result in a quicker solution over there. I believe that @DerekSikes commonly sends barcodes out on loan and gets researchers to do useful things with them. I'm not sure there's a real replacement for machine-readable "part IDs" but there may be a way to make some of the tools that rely on those available without them - at least that's what I think you're asking for here. |
@campmlc can I delete the MSBPID container series and close this? |
@dustymc I need help bulkloading 99 part subsamples with part attributes.
1)The subsamples are DNA extractions derived from tissue loans, which were themselves subsamples of parts.
2) The actual parent samples are legacy subsamples samples of Arctos parts which were cut for loan and stored for the past decade in a research lab. No barcodes.
3) We are not certain exactly which loan subsample produced the extraction because sometimes there are multiple, so the DNA extraction needs to be a loaded as subsample of the original part, not of the subsample which would be it's immediate parent.
4) The parent part has a name such as "liver" or "liver (frozen)"; the child part subsample however has a different name, =" DNA extraction"
5) I've put together a hypothetical template attached here modified from the part subsample bulkloader and the part attribute bulkloader combined.
Can something like this be developed? Or can you load these data for me for the time being if not?
Loading part attribute data depends on #3624
Priority - high - needed for immediate loan processing, and, if it works, for an upcoming presentation on genomic data in Arctos.
The text was updated successfully, but these errors were encountered: