What reference data to include

[peterdesmet]

@sarahcd so far I have based the mapping for the start and end HumanObservations on fields I have in my datasets. It would be good however to have a mapping based on all fields potentially available in the reference data. Can you check the field where you agree that they are not useful to map (crossed out) and comment on those you think would be useful to include?

• animal-comments: start occurrenceRemarks?
• animal-death-comments: end
• animal-exact-date-of-birth
• animal-id: organismID
• animal-latest-date-born
• animal-life-stage: lifeStage
• animal-mass
• animal-nickname: organismName
• animal-reproductive-condition: reproductiveCondition
• animal-ring-id: include, where?
• animal-sex: sex
• animal-taxon: scientificName
• animal-taxon-detail
• attachment-type: included in start eventRemarks
• behavior-according-to
• data-processing-software
• deploy-off-date: end
• deploy-off-latitude: end
• deploy-off-longitude: end
• deploy-off-person: end
• deploy-on-date: start eventDate
• deploy-on-latitude: start decimalLatitude
• deploy-on-longitude: start decimalLongitude
• deploy-on-person: start recordedBy?
• deployment-comments: included in start eventRemarks
• deployment-end-comments: end
• deployment-end-type: end
• deployment-id
• duty-cycle
• geolocator-calibration
• geolocator-light-threshold
• geolocator-sensor-comments
• geolocator-sun-elevation-angle
• habitat-according-to
• location-accuracy-comments
• manipulation-comments
• manipulation-type: included in start eventRemarks
• study-site
• tag-beacon-frequency
• tag-comments
• tag-failure-comments
• tag-id: part of eventID
• tag-manufacturer-name: included in start eventRemarks
• tag-mass
• tag-model: included in start eventRemarks
• tag-processing-type
• tag-production-date
• tag-readout-method
• tag-serial-no

[peterdesmet]

Note that the start eventRemarks are currently based on a number of fields:

movepub/inst/sql/movebank_dwc_occurrence.sql

Lines 54 to 60 in 33bcd17

	ref."tag-manufacturer-name" || ' tag attached by ' ||
	ref."attachment-type" || ' to ' ||
	CASE
	WHEN ref."manipulation-type" = 'none' THEN 'free-ranging animal'
	WHEN ref."manipulation-type" != 'none' THEN 'manipulated animal'
	END ||
	' | ' || ref."deployment-comments" AS eventRemarks,

Resulting in e.g.:

UvA-BiTS tag attached by harness to free-ranging animal | Found dead in Sint-Maartensdijk (NL) on 2018-08-24, last active day is 2018-08-23.

This will likely be empty if any of the fields are empty.

[sarahcd]

I can look at this next week, but one thing to think about: The deploy start/end times are sometimes based on something other than a human observation. For the deploy-on time, we can assume with current methods that the tag had to be attached by a person, and therefore assume it at least approximates the time of a human observation. For the deploy-off-time, this is regularly used to define the end of the reliable tracking data, e.g., to exclude locations from the track determined to have been recorded after the tag stopped moving or sending reliable data. Even when tags are physically retrieved, this often happens after an animal has died, or after it has been automatically released from the animal, so doesn't necessarily represent an observation of the live animal.

[peterdesmet]

Hmm, interesting point. I'll reference your comment in #10.

[peterdesmet]

Addressed. @sarahcd can you go over the list and check off the ones that you think make sense to exclude from DwC (I indicated them crossed out). We should then go over the remaining ones to see what to do with those.

[sarahcd]

Sorry for the delay. I am breaking these down based on the entity they describe in Movebank, since this could impact how to code transformation to DwC.

Possible mappings added from above have notes in italics. We can discuss whether to implement any additions to the current version, and I could propose specifics in the sql.

Terms describing the animal

• animal-comments: start occurrenceRemarks? SD: or add start organismRemarks
• animal-death-comments: SD: could include in start organismRemarks?
• animal-earliest-date-born: SD: could include in start organismRemarks
• animal-exact-date-of-birth: SD: could include in start organismRemarks
• animal-id: organismID, part of eventID
• animal-latest-date-born: SD: could include in start organismRemarks
• animal-nickname: organismName
• animal-ring-id: include, where? could include in start organismRemarks
  @tucotuco: For a more specific mapping (not necessarily to use here): Could a bird band/ring number be stored using previousIdentifications? These IDs are used by ringing programs and are sometimes stored there with other information, could be associated with related samples, etc. The band is often but not always attached at the same time as the tracking tag, so 'previous' isn't necessarily accurate, but it's close.
• animal-sex: sex
• animal-taxon: scientificName
• animal-taxon-detail SD: could include in start organismRemarks

Terms describing the tag

• tag-beacon-frequency
• tag-comments
• tag-failure-comments end
• tag-id: part of eventID
• tag-manufacturer-name: included in start eventRemarks
• tag-mass: SD: could be included in start eventRemarks
• tag-model: included in start eventRemarks
• tag-processing-type

Terms describing the deployment, including tags and animals at the time of the deployment

• alt-project-id: SD: maybe something from ResourceRelationship could work here?
• animal-life-stage: lifeStage
• animal-mass: SD: could be included in start eventRemarks
• animal-reproductive-condition: reproductiveCondition
• attachment-type: included in start eventRemarks
• behavior-according-to
• data-processing-software
• deploy-off-date: end
• deploy-off-latitude: end
• deploy-off-longitude: end
• deploy-off-person: end
• deploy-on-date: start eventDate
• deploy-on-latitude: start decimalLatitude
• deploy-on-longitude: start decimalLongitude
• deploy-on-person: start recordedBy? SD: agree
• deployment-comments: included in start eventRemarks
• deployment-end-comments: end
• deployment-end-type: end
• deployment-id
• duty-cycle
• geolocator-calibration
• geolocator-light-threshold
• geolocator-sensor-comments
• geolocator-sun-elevation-angle
• habitat-according-to
• location-accuracy-comments
• manipulation-comments
• manipulation-type: included in start eventRemarks
• study-site: SD: could be included in start event as locationRemarks or locality
• tag-readout-method SD: could be included in start eventRemarks
• tag-production-date
• tag-serial-no

[peterdesmet]

@sarahcd do you think including study-site for human observations makes more sense as:

• locality: https://dwc.tdwg.org/terms/#dwc:locality
• locationRemarks: https://dwc.tdwg.org/terms/#dwc:locationRemarks

Note that for GPS observations, the locationRemarks currently state "elevations are altitude above mean sea level"

[peterdesmet]

Overview of things to tackle:

• animal-comments: Additional information about the animal that is not described by other reference data terms.
  • PD: start occurrenceRemarks
  • SD: or add start organismRemarks?
• animal-death-comments: Comments about the death of the animal.
  • SD: could include in start organismRemarks?
• animal-earliest-date-born: The earliest date an animal is thought to have been born or hatched.
  • SD: could include in start organismRemarks?
• animal-exact-date-of-birth: The exact date on which animal was born or hatched.
  • SD: could include in start organismRemarks?
• animal-latest-date-born: The latest date an animal is thought to have been born or hatched.
  • SD: could include in start organismRemarks?
• animal-ring-id: A number or color scheme for a band or ring attached to the animal. Color bands and other markings can be stored in 'animal marker ID'.
  • SD: could include in start organismRemarks
  • @tucotuco: For a more specific mapping (not necessarily to use here): Could a bird band/ring number be stored using previousIdentifications? These IDs are used by ringing programs and are sometimes stored there with other information, could be associated with related samples, etc. The band is often but not always attached at the same time as the tracking tag, so 'previous' isn't necessarily accurate, but it's close.
• animal-mass: The mass of the animal
  • SD: could be included in start eventRemarks
  • PD: or dynamicProperties
• deploy-on-measurements (new field): A list of additional measurements taken during capture of the animal at the start of the deployment. Recommended best practice is to define units and use a key:value encoding schema for a data interchange format such as JSON.
  • PD: could be mapped to dynamicProperties
• animal-taxon-detail: A more specific name and/or reference for the taxon name provided by 'animal taxon'.
  • SD: could include in start organismRemarks?
  • PD: might probably fit better under taxonRemarks, but the use case seems to small for me to add it
• study-site: A location such as the deployment site or colony, or a location-related group such as the herd or pack name.
  • SD: could be included in start event as locationRemarks or locality
  • PD: likely best as locality, but see comment above
• alt-project-id: A name or unique identifier for a project associated with the deployment, for example a monitoring program or another data platform.
  • SD: maybe something from ResourceRelationship could work here
  • PD: the use case seems to small for me to warrant using an extension
• tag-readout-method: The way the data are received from the tag.
  • SD: could be included in start eventRemarks

[albenson-usgs]

For the animal death- I am really really hoping that vitality will be added to Darwin Core so you should be able to use that term for this.

[sarahcd]

@sarahcd do you think including study-site for human observations makes more sense as:

* locality: https://dwc.tdwg.org/terms/#dwc:locality

* locationRemarks: https://dwc.tdwg.org/terms/#dwc:locationRemarks

Note that for GPS observations, the locationRemarks currently state "elevations are altitude above mean sea level"

locality for human observations sounds good to me. Also, FYI, we are in the progress of adding locality as an event field in Movebank (e.g., for GPS records) for cases that segments of data are assigned location names. I think those should work fine together in the translation to DwC.