Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

part name cleanup #1131

Closed
dustymc opened this issue May 9, 2017 · 61 comments
Closed

part name cleanup #1131

dustymc opened this issue May 9, 2017 · 61 comments
Assignees
@dustymc
Labels
Function-CodeTables
Milestone
Active Development

Comments

@dustymc
Copy link
Contributor

dustymc commented May 9, 2017 •
edited
Loading

!!! IMPORTANT !!!

This issue is ongoing; most of what's here has been done. Scroll to the bottom for the latest.

!!! IMPORTANT !!!

can we change "fetus" to "whole organism" and add an age class?

can we change "embryo" to "whole organism" and add an age class?

can we change "other" to "unknown" (or vise-versa)?

can we change...

  • video
  • video recording
  • photo observation
  • photo whole organism
  • photograph
  • image

... to "media" (and add a remark???)

Can we change phalanges to phalange?

Can we move "partial ..." to condition and remove it from part name?


UAM@ARCTOS> select distinct part_name from ctspecimen_part_name where part_name like '%partial%';

PART_NAME
------------------------------------------------------------------------------------------------------------------------
partial organism (70% ethanol)
partial skeleton
partial organism (80% ethanol)
partial skeleton (dry)
partial strobila (slide)
partial organism (95% ethanol)

6 rows selected.
@dustymc dustymc added this to the Needs Discussion milestone May 9, 2017
@ekrimmel
Copy link

ekrimmel commented May 9, 2017

no opinions on media, but I like the fetus/embryo/other proposed part name changes

@AJLinn
Copy link

AJLinn commented May 9, 2017

I'm okay with the general term "media" with a remark to indicate the type. I don't know the difference between "photograph" and "image" anyway.

@ccicero
Copy link

ccicero commented May 9, 2017

I'm good with these changes.

@atrox10
Copy link

atrox10 commented May 9, 2017 via email

@campmlc
Copy link

campmlc commented May 9, 2017 via email

@Jegelewicz
Copy link
Member

Sounds good to me

@dustymc dustymc self-assigned this Jun 8, 2017
@dustymc
Copy link
Contributor Author

dustymc commented Jun 9, 2017

The straight vocabulary replace got a little out of control, so I made a spreadsheet. https://docs.google.com/spreadsheets/d/1qI6syTLbWGb7u3MZP9aYcuicSr1WBtCZv4GlfcorPCo/edit?usp=sharing

This is a proposal, not necessarily recommendations; if your users find value in being able to search for "left dentary" instead of just "dentary," (or whatever), we can talk. The intent in all cases would be to update the existing part to the proposed new part and add a comment of "part was given as {old_part_name}." (Other suggestions welcome, of course.)

AWG, can we set a firm timeline for comments?

Goals are twofold:

  1. reduce redundancy, but not at the cost of usability. If "thing" and "thing (modifier)" or "thing" and "other thing" are close enough to the same that most users will want to find them both, merge them. If they're usefully distinct, do not.

  2. Help with sorting.

  • thing
  • thing (modifier)
  • other junk

is easy to work with, while

  • thing
  • other junk
  • modifier thing

is difficult.

Random confusing stuff:

  • what is part "body"? (Somehow different than "whole organism"?)
  • What is "body parts"?
  • What is "carcass"?
  • What is "fossil"? Can that become "other" and a remark? (or better yet "thing (fossilized)" when those data exist)?
  • Is "observation" still necessary/useful under the current specimen-event model?

This was referenced Jul 14, 2017
Closed
Closed
@dustymc
Copy link
Contributor Author

dustymc commented Jul 14, 2017

Media done (-23 part names).

@dustymc
Copy link
Contributor Author

dustymc commented Jul 17, 2017

I deleted this comment above:

Adding Attributes is leading to confusing data when eg, a fetus is cataloged as a part of it's parent, which seems to be most of these. Instead, I'll update part remark of the new "whole organism"/former "fetus" (etc.) part.

I still don't like those as parts, but I think it's less confusing than the proposed update, which would lead to things like "parts=heart, liver, skeleton, skin, whole organism."

Clever ideas for handling this better?

@dustymc
Copy link
Contributor Author

dustymc commented Jul 17, 2017

phalanges

We also have "spine" and "spines" - I propose we eliminate all non-singular part names and document that (eg so they don't get re-introduced). Someone stop me now if there's a reason to not do this....

Can we also get rid of all "...(s)" part names ("ectoparasite**(s)** (ethanol)")? The "(s)" bit can be derived from lot count and does not seem necessary or useful in part name.

@campmlc
Copy link

campmlc commented Jul 17, 2017 via email

@ekrimmel
Copy link

Assorted opinions on above:

  • pro all part names being singular
  • pro "thing (fossilized)" vs. "fossil"
  • pro ditching "observation" as that should be the kind of specimen event
  • pro all of the part clean-up suggestions you have in the spreadsheet. When you make these part clean-up changes you'll check for and add if necessary the age classes, yes?
  • I could see "body" meaning "whole organism minus head" (which I've seen here and there in the collections) but I think that part could better be represented as "whole organism" with a remark that it's missing the head.

@campmlc
Copy link

campmlc commented Jul 18, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

singular

Yay, working on that now.

age classes

No, I've given up on that for the moment - somewhere ^^ up there^^ I said

I still don't like those as parts, but I think it's less confusing than the proposed update, which would lead to things like "parts=heart, liver, skeleton, skin, whole organism."

body

I dislike remarks for "important bits missing" - it's not searchable. (I may ultimately dislike that less than other ideas, but still...)

@campmlc
Copy link

campmlc commented Jul 18, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

catalog record of the host based on parasite data

I maintain that parts are "things to which one can stick barcodes" - they are/should be physical somehow-discrete THINGs.

The RECORDs may be useful for (data) loans. You've got a squirrel-parasite-host from THERE, might be a good place to look for squirrels....

You don't have/know of parts, so just don't enter any.

https://arctos.database.museum/info/ctDocumentation.cfm?table=CTSPECIMEN_PART_NAME "accepted place of collection" is probably appropriate, and part "observation" seems very confusing when mixed with event type observation which explicitly requires "No biological samples were taken."

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

https://docs.google.com/spreadsheets/d/1WZMkrnyZe5hxRCI6kuQT1xagJtxscVwXMFaKZB1FMM8/edit?usp=sharing is an attempt at the singularized part names and definitions; everyone can edit.

Please review if possible, I'll try to update it later today. I don't think there are any significant changes in there, but hopefully it'll make things more readable/sortable.

@campmlc I don't know what "... (ethanol-fixed)" means - help?

@DerekSikes is "cryovial tissues" just "tissue"?

@ccicero is "parasitic eggs" just "egg"?

And "skeletal element(s)" still makes me twitchy for some reason. "Bone"?

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

Slide-containing parts are at https://docs.google.com/spreadsheets/d/1rET-CZ5EMmYFLq0Zl26mD3WkOhBsM5Z3bWJKx3jxVNM/edit?usp=sharing. I don't know what any of that stuff is and need a lot of help - PLEASE edit!

@campmlc
Copy link

campmlc commented Jul 18, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

Please keep observation as a part until we can get more feedback

Sure, no problem, I just don't want it to fall off the radar - it's weird and needs eliminated or documented or something.

verbatim part info

How about part=skull, disposition=missing (or something more explicit and specialized even - "probably exists somewhere, but we don't have it...")?

And maybe we need an "available for loan" generalization of disposition for search, maybe up by the "tissues?" option:

EXCLUDE (on loan, missing, used up, ...)

@campmlc
Copy link

campmlc commented Jul 18, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Jul 18, 2017

We don't want people thinking that our record is that actual skull

I don't think I see a distinction. "Someone says there's a skull [and we never had it, or we've lost it, or it's used up, or it's on loan, or we gave it away] but we don't have it."

recently linked our data record to the Smithsonian catalog number

You haven't linked until their number does something (eg, opens a web page), and you haven't usefully linked until that "something" can support the "canid hosts of tapeworm" things that paired records in Arctos can. I think this is all a compelling reason to have some data in Arctos even if you don't "own" the specimens, but I still don't see what that has to do with weird parts. Maybe part disposition "observation" (=someone says they saw a skull) makes sense????

@jldunnum
Copy link

jldunnum commented Jul 18, 2017 via email •
edited by dustymc
Loading

@dustymc dustymc mentioned this issue Jul 18, 2017
Closed
@KatherineLAnderson
Copy link

In the above list, request to keep:
cone

@dustymc
Copy link
Contributor Author

dustymc commented Jul 19, 2017

cone

Removed from the list - thx

@dustymc
Copy link
Contributor Author

dustymc commented Jul 20, 2017 •
edited
Loading

@KatherineLAnderson I've for dorsal vertebra mapped to vertebra, thoracic - please edit https://docs.google.com/spreadsheets/d/1aTzwOWxEuW8JJdPiQQKpSo7_i-m7sUhdgTz9WOIC52A/edit#gid=1331535064 if no. Also what's "flat rib"?

@KatherineLAnderson
Copy link

I added a definition for dorsal vertebra. "Flat rib" can be mapped to rib.

@dustymc dustymc mentioned this issue Jul 21, 2017
Closed
@dustymc
Copy link
Contributor Author

dustymc commented Jul 24, 2017

https://docs.google.com/spreadsheets/d/1aTzwOWxEuW8JJdPiQQKpSo7_i-m7sUhdgTz9WOIC52A/edit#gid=1331535064 will load to Arctos Tomorrow, 2017-07-25 5PM Pacific.

  1. If two or more existing parts (PART_NAME) column are the same, merge them by making the values in NEWPARTNAME the same.
    1. rib-->rib (no changes)
    2. flat rib --> rib (data will change, "flat rib" will be removed from the code table)
  2. If they are not the same, or we can't tell from this, do not merge them, but change the value in NEWPARTNAME to the format of "{part_name}, modifiers (or modifiers)" when necessary - that is, put the part name first so this sorts better. (We may need to revisit what's parenthetical etc. later.)
  3. Add definitions to the DEF column when possible; this will load to Arctos.
  4. wat?!? column is things that need defined and possibly merged; it's just notes and won't load to Arctos.
  5. This is an iterative process; the spreadsheet does not need to be perfect (just better!), things from here will probably appear in subsequent cleanup efforts.

@dustymc
Copy link
Contributor Author

dustymc commented Jul 24, 2017

TO DELETE: just map PART_NAME to some existing part.

flat rib --> rib (data will change, "flat rib" will be removed from the code table)

@dustymc
Copy link
Contributor Author

dustymc commented Jul 26, 2017

The last spreadsheet is loaded (warts and all...) and it's BEAUTIFUL! https://docs.google.com/spreadsheets/d/1TV69LgIW7KErcYSDTkkxvcmECuLnh1zzdicnvSuak-g/edit?usp=sharing

Well, almost. https://docs.google.com/spreadsheets/d/1QLaF3W0VjJYG4zxTrJlpEhD0OGjBdVcbQBpcJVMVqr4/edit?usp=sharing is some obvious outliers, one mistake from the last round, and parts which contain "other" or "unknown."

Is there a preference for "other" or "unknown"? If not I'll flip a coin or something.

Help is always appreciated, but I think this is straightforward and I'll plan to finish it up and re-load tomorrow unless I hear otherwise.

@DerekSikes
Copy link

DerekSikes commented Jul 26, 2017 via email

@campmlc
Copy link

campmlc commented Jul 26, 2017 via email •
edited by dustymc
Loading

@dustymc
Copy link
Contributor Author

dustymc commented Jul 26, 2017

re: https://docs.google.com/spreadsheets/d/1QLaF3W0VjJYG4zxTrJlpEhD0OGjBdVcbQBpcJVMVqr4/edit#gid=1127423690

@AJLinn can you define "mount"?
@KatherineLAnderson please help with flat metacarpal, %centrum(=partial vertebra??), %neural arch(=partial vertebra??)
@ccicero parasitic egg?
@DerekSikes pinned nest, pinned tissues??

I've eliminated "... (unknown)" - given just "egg" (etc.) I can't have any idea what you've done with it (and we're updating definitions to make that explicit), so I don't see the value in being told that the information which isn't being given isn't being given. Please let me know if I'm seeing that incorrectly.

@dustymc
Copy link
Contributor Author

dustymc commented Jul 26, 2017

https://docs.google.com/spreadsheets/d/1J5tL20-_TTZgoFogq1ejZ6B3M1u1ZLOob9N6eNTo5A4/edit#gid=555921353 is parts which contain "whole" "body" or "carcass."

Despite any implications of the word "whole," can those be merged - eg, is "carcass" == "whole organism (something or another)"??

What are "body parts"? Can those all merge all that with "unknown," or are there also non-body parts we're avoiding with those terms, or ????

@AJLinn
Copy link

AJLinn commented Jul 26, 2017

@AJLinn can you define "mount"?
Done - let me know if it needs further explanation

@KatherineLAnderson
Copy link

I added definitions for the centra and neural arches. Please map "flat metacarpal" to metacarpal.

@dustymc
Copy link
Contributor Author

dustymc commented Aug 2, 2017

Still to-do: https://docs.google.com/spreadsheets/d/1J5tL20-_TTZgoFogq1ejZ6B3M1u1ZLOob9N6eNTo5A4/edit#gid=555921353

Moving on....

These still seem unlikely:

pinned nest
pinned tissues

There are a few "visual outliers" that I keep noticing, but not sure what do to with them. (Maybe they don't need anything.)

"contents" do not sort nicely, and contains..

bill content (dry)
cheek content (dry)
crop content (70% ethanol)
egg contents (frozen)
hindgut content (70% ethanol)
stomach content

There are some random "...bone" things that probably aren't very discoverable

bone
bone (frozen)
bone marrow (frozen)
ear bone
leg bone
leg bone (frozen)
long bone

Ditto "swab"

buccal and cloacal swab (RNAlater)
cloacal swab (RNAlater)
dorsum swab
nasal swab (frozen)
oral swab
oral swab (formalin)
oral swab (frozen)

various plant-bits floating around ungrouped:

cone
seed
fruit
probably other stuff

coal ball

¯\(ツ)/¯

MAYBE it would somehow be useful to group part-parasites (cestode, nemadode, etc.) - "parasite, cestode, bla (stuff)" is the best I can come up with...

Thoughts?

@ekrimmel
Copy link

ekrimmel commented Aug 2, 2017

this is looking good!

@DerekSikes
Copy link

DerekSikes commented Aug 2, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Aug 2, 2017

Thanks @DerekSikes !

"Is not horrible mistake" is about where I am now, so I'm fine with keeping "pinned tissues" or updating it or whatever - let me know what you prefer.

I'll add a definition to pinned nest, and try to keep in mind that not all nests are piles of sticks....

@dustymc
Copy link
Contributor Author

dustymc commented Aug 2, 2017

@ekrimmel I agree, this feels like real progress!

@dustymc
Copy link
Contributor Author

dustymc commented Aug 2, 2017

Still to-do:

  1. https://docs.google.com/spreadsheets/d/1J5tL20-_TTZgoFogq1ejZ6B3M1u1ZLOob9N6eNTo5A4/edit#gid=555921353

  2. "visual outliers"

Moving on again....

In an attempt to NOT say the exact same thing two ways, I added to http://handbook.arctosdb.org/how_to/Code-Tables.html

Do not include both “thing (modifier)” and “thing” in the code table. Avoid creating new “bare” terms (“thing”) in order to avoid this problem in the future. Parts of unknown preservation/preparation should be explicit: “thing (unknown).”

And there is another spreadsheet at https://docs.google.com/spreadsheets/d/1GSY8DM6WzUvUADxWgpad44CLw7yor3Gisr8k-EittA0/edit?usp=sharing

The spreadsheet contains "unmodified" parts (those without comma or parens) and their potential modified equivalents.

It will contain a row with the "unmodified" part in both old and new; if the unmodified part is to be retained, these should be defined and and all other "suggestions" should be deleted. Also do this if there's not enough information in the spreadsheet to tell what the "old" part might be - I'm not asking anyone to dig up specimens etc., I'm just trying to get rid of "obviously the same as" duplicates.

I could be convinced that we should eliminate all bare terms NOW - "skeleton" should become "skeleton (unknown)" or something of the sort - but that would probably break everyone's reports and it's just sort of goofy.

Alternatively, perhaps it's useful to just define all "bare" parts as "...unknown preservation/preparation" and eliminate all "part (unknown)" type values. I can build more spreadsheets if I've got this backwards.

The spreadsheet will contain mostly very bad suggestions.

ectoparasite | ectoparasite (K2Cr2O7)

K2Cr2O7 is not the default state for ectos, so these can be simply DELETED. (I'm working on that - the easy stuff - now.)

The end result should be unique values in OLD (and I won't load this until that's true), something in all remaining DEF_OF_NEW columns (many should already be defined - yay us!), and in NEW either...

  1. a repeat of OLD (eg, "update nothing for this"), or
  2. a value which will replace OLD (because it's the same thing, just more explicit/clear)

@KatherineLAnderson
Copy link

We use "bone" for unidentifiable bone fragments, and "long bone" for limb bones that are not more specifically identifiable. I don't think these terms need to be changed. Same for cone, seed, and coal ball.

"Bone (frozen)" and "bone marrow (frozen)" are only used in our collection for subsamples from one specimen (Blue Babe); the skeletal element the samples are taken from (if known) are included in the remarks. These terms are probably used more extensively in other collections (?) so open to suggestions.

@campmlc
Copy link

campmlc commented Aug 2, 2017 via email

@dustymc
Copy link
Contributor Author

dustymc commented Aug 2, 2017

Thanks @KatherineLAnderson - it's always useful to know a bit more about how these things are used.

I have 2 (minor, I think) objections to "bone":

  1. I don't know what makes it different than "skeletal element."
  2. I don't know how it's preserved/prepared.

(1) will hopefully sort itself out as we add definitions, (2) really only bothers me in the case of having eg, "brain" and "brain (unknown)." I don't think we need two ways of saying "we don't know."

I dislike "long bone" just because it doesn't sort well - users see "bone...500 unrelated things....long bone...500 more things....skeletal element...." - nobody's going to find what they want in that.

I suppose that's also an argument for "bone, rib" instead of just "rib."

Ditto seed and cone - if they're together (eg, because the part name is "plant-bit, ...."), I might realize that some cones have seeds or something; if they're not, I'll probably find one or the other and not (all of) what I'm looking for.

You might consider making those de facto subsamples into actual subsamples. Doing so creates a strong path back to the "donor" part, and also protects the donor (eg, from deletion). Remarks are just ignored....

@campmlc "stom" should find everything containing the substring (if not let me know and I'll fix it). As above, my concern is only discovery (a botanist looking for seeds might want all the "... content" together) and I think that's very minor at this point.

Yes, it should be "thing, pinned" not "pinned thing." Might take me a while to get used to that...

I think "requiring" preservation/preparation makes some sense, but I'd also expect a revolt the next time someone ends up with "skull (unknown)" on their labels.

You can split preservation out now, but I think we need some consensus on #1119 (comment) before further developing tools.

@KatherineLAnderson
Copy link

With regard to "bone":

  1. We prefer "bone" because it is more specific than "skeletal element" (which could include bones+teeth depending on how it is defined). So yes, it will sort out with definitions.
  2. "Bone" doesn't necessarily need a qualifier. It is used in the same sense as "humerus" (or any other specific element) which doesn't have a qualifier and is preserved/prepared in the same way (at least in the UAMES collection).

Can we change "long bone" to "bone, long bone"? (Definition: "limb bone missing epiphyses") Or something similar. I see your point but would like to retain the term if possible.

What do you suggest for the plant bits? Our paleobotany collection is largely undigitized, so I'm on board for changes now so we are consistent/efficient in part name usage later.

@dustymc
Copy link
Contributor Author

dustymc commented Aug 14, 2017

Thanks again @KatherineLAnderson.

I know (roughly, I think!) what a "bone" is and so also prefer it to "skeletal element" - I still have pretty much no idea what that might be. (Bones, teeth - and other crunchy bits????)

I think, as unrealistic as it may turn out to be "in the wild," that I'd like to see qualifiers on everything; not having them is a significant part of what lead us here. E.g., "frozen bone" may turn out to be a good source of DNA, "petrified bone" maybe not so much, and I can't tell them apart if they're both recorded as "bone." (Would UAM:ES do something different for eg, a normal rocky hadrosaur "bone" vs. blue babe?)

at least in the UAMES collection

That is something we need to avoid. Users should not have to guess at our administrative decisions in order to find material. (Think Blue Babe - UAM:ES and UAM:Mamm have cataloged the same thing. One search should find either both or neither. Where something is cataloged has nothing to do with what that something IS.) This ties in to the point about qualifiers - "prepared/preserved in the normal fashion" IS (somewhat, sometimes...) a discipline-specific thing, so I think we should just avoid that by always being specific.

bone, long bone

That sorts nicely, but it's a mouthful! AWG, can we discuss? Is "bone, rib" (and similar) acceptable as well?

plant bits

I have no suggestions, and I think we can safely ignore that for now. Maybe something will emerge as we deal with the more familiar stuff.

"easy stuff" lingering from above:

  • Can we replace "carcass" with "whole organism, skinned" or develop a definition which does not overlap with "whole organism"?

  • What is a "body"? @ccicero ??

@dustymc dustymc mentioned this issue Aug 22, 2017
Closed
@dustymc dustymc mentioned this issue Oct 16, 2017
Closed
@dustymc
Copy link
Contributor Author

dustymc commented Feb 8, 2018

Closing....

@dustymc dustymc closed this as completed Feb 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dustymc dustymc

Labels
Function-CodeTables
Projects
None yet
Milestone
Active Development
Development

No branches or pull requests
10 participants
@ccicero @atrox10 @dustymc @Jegelewicz @DerekSikes @ekrimmel @campmlc @jldunnum @AJLinn @KatherineLAnderson