TG2-VALIDATION_OCCURRENCESTATUS_STANDARD

[godfoder]

TestField	Value
GUID	7af25f1e-a4e2-4ff4-b161-d1f25a5c3e47
Label	VALIDATION_OCCURRENCESTATUS_STANDARD
Description	Does the value of dwc:occurrenceStatus occur in the bdq:sourceAuthority?
TestType	Validation
Darwin Core Class	dwc:Occurrence
Information Elements ActedUpon	dwc:occurrenceStatus
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:occurrenceStatus is bdq:Empty; COMPLIANT if the value of dwc:occurrenceStatus is resolved in the bdq:sourceAuthority; otherwise NOT_COMPLIANT
Data Quality Dimension	Conformance
Term-Actions	OCCURRENCESTATUS_STANDARD
Parameter(s)	bdq:sourceAuthority
Source Authority	bdq:sourceAuthority default = "Regex present/absent" {["^(present|absent)$"]}
Specification Last Updated	2025-02-18
Examples	[dwc:occurrenceStatus="present": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:occurrenceStatus matches a term in the bdq:sourceAuthority"]
	[dwc:occurrenceStatus="presence": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:occurrenceStatus does not match a term in the bdq:sourceAuthority"]
Source	ALA
References
Example Implementations (Mechanisms)	Kurator/FilteredPush rec_occur_qc Library DOI: 10.5281/zenodo.14968501
Link to Specification Source Code	https://github.com/FilteredPush/rec_occur_qc/blob/v1.0.1/src/main/java/org/filteredpush/qc/metadata/DwCMetadataDQ.java#L418
Notes	The recommended controlled vocabulary for this term consists of "present" and "absent", which are the only two appropriate terms for a Darwin Core Occurrence. This is reflected in the dwc:occurrenceStatus vocabulary for this test. There is currently a mismatch between the (authoritative) lower case values at https://dwc.tdwg.org/terms/#dwc:occurrenceStatus and the vocabulary at GBIF (https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts), which capitalized the first character. This test must return NOT_COMPLIANT if there are case differences, leading or trailing whitespace or there are leading or trailing non-printing characters.

[ArthurChapman]

I feel we need a better example in this test - the one given is confusing.

[Tasilee]

@ArthurChapman Better?

[Tasilee]

Added to Notes: "This test will fail if there are leading or trailing white space or non-printing characters."

[Tasilee]

Changed Source Authority from

bdq:sourceAuthority default = "Darwin Core Standard" [https://dwc.tdwg.org/terms/#occurrenceStatus]

to

bdq:sourceAuthority default = {Darwin Core} {Occurrence status [https://dwc.tdwg.org/terms/#occurrenceStatus] } {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml] }

[Tasilee]

Amended Source Authority values to align with @chicoreus syntax

bdq:sourceAuthority default = {Darwin Core} {Occurrence status [https://dwc.tdwg.org/terms/#occurrenceStatus] } {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml] }

to

bdq:sourceAuthority default = "Darwin Core dwc:occurrenceStatus" {[https://dwc.tdwg.org/terms/#occurrenceStatus]} {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml] }

[Tasilee]

Post Zoom 11/7/2023, I have aligned the Source Authority with the suggested syntax:

bdq:sourceAuthority default = "Darwin Core dwc:occurrenceStatus" {[https://dwc.tdwg.org/terms/#occurrenceStatus]} {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml] }

to

bdq:sourceAuthority default = "Darwin Core" {https://dwc.tdwg.org} {dwc:occurrenceStatus" [https://dwc.tdwg.org/terms/#occurrenceStatus]} {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}

[Tasilee]

Due to recent discussions, changed Source Authority from

bdq:sourceAuthority default = "Darwin Core" {https://dwc.tdwg.org} {dwc:occurrenceStatus" [https://dwc.tdwg.org/terms/#occurrenceStatus]} {Controlled Vocabulary Document [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}

to

bdq:sourceAuthority = "Darwin Core occurrenceStatus" {https://dwc.tdwg.org/list/#dwc_occurrenceStatus} {dwc:occurrenceStatus vocabulary [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}

and Parameter(s) from

bdq:sourceAuthority

to

dwc:occurrenceStatus vocabulary

We will need some Notes from @tucotuco

[tucotuco]

Updated the Notes from

"The reference vocabularies of values for occurrence status need to be developed and published. This test will fail if there is leading or trailing whitespace or there are leading or trailing non-printing characters."

to

"The original recommended controlled vocabulary for this term consisted of "present" and "absent", which are the only two appropriate terms for a Darwin Core Occurrence. This is reflected in the suggested dwc:occurrenceStatus vocabulary for this test. Other values for dwc:occurrenceStatus should only arise under circumstances that do not refer to an Occurrence. This test will fail if there is leading or trailing whitespace or there are leading or trailing non-printing characters."

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted".

Also changed "Field" to "TestField", "Output Type" to "TestType" and updated "Specification Last Updated"

[chicoreus]

Updated note from "fail" to more specific "This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. "

[Tasilee]

For Source Authority API for this and #115, do we want to use

dwc:occurrenceStatus vocabulary [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]

or

dwc:occurrenceStatus vocabulary [https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts]

?

[tucotuco]

I think there is value in using the GBIF vocabularies whenever they exist and do not conflict with the semantics of Darwin Core, thus, I would opt for dwc:occurrenceStatus vocabulary [https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts].

[Tasilee]

Changed Source Authority from

bdq:sourceAuthority = "Darwin Core occurrenceStatus" {https://dwc.tdwg.org/list/#dwc_occurrenceStatus} {dwc:occurrenceStatus vocabulary [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}

to

bdq:sourceAuthority default = "GBIF OccurrenceStatus Vocabulary" [https://api.gbif.org/v1/vocabularies/OccurrenceStatus]} {"dwc:occurrenceStatus vocabulary API" [https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts]}

[chicoreus]

@tucotuco I concur.

[Tasilee]

How do you want to handle this given the disparity in capitalization between

https://dwc.tdwg.org/terms/#dwc:occurrenceStatus

and

https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts?

If we wish to conform to the GBIF vocabulary, we need to change the default and test data, or do a @chicoreus and code to accept either case? I've emailed @timrobertson100

[Tasilee]

For now, changed default to "Present" and modified Example, and added to Notes as per #75.

[Tasilee]

Response from Cicilie at GBIF: "May I propose we instead change the English label (label_EN) to lowercase, and you validate your tests with this value?"

Considering the email comments on this: Darwin Core states the lower case is the standard.

@chicoreus:

1. Will this work for this test and TG2-AMENDMENT_OCCURRENCESTATUS_STANDARDIZED #115
2. Will it work as case insensitive from the vocabulary, or will you have to program it as such? Or both....or....?

#75 is ok, as it is just filling bdq:Empty with bdq default - which should be lower case as "present"

[Tasilee]

Whatever way this goes, we must respond to Cecilie with some guidance and possibly add this as a use case in the Supplement document regarding vocabularies.

[Tasilee]

Change Source Authority from

bdq:sourceAuthority default = "GBIF OccurrenceStatus Vocabulary" [https://api.gbif.org/v1/vocabularies/OccurrenceStatus]} {"dwc:occurrenceStatus vocabulary API" [https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts]}

to

bdq:sourceAuthority default = "Regex present/absent" ["^(present|absent)$"]} {"dwc:occurrenceStatus vocabulary API" [https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts]}

and updated the Notes.

[chicoreus]

Removed the GBIF api reference from the source authority as, having incorrect case, it is not of utility for this validation (though it has some utility for the corresponding amendment #115)

[tucotuco]

The Notes as currently written have two errors. I would also omit the statement "Other values for dwc:occurrenceStatus should only arise under circumstances that do not refer to an Occurrence" because dwc:occurrenceStatus should never refer to anything other than an Occurrence. Here is the recommended fix with the changes highlighted. The second one is hard to see, as it is the comma "following /concepts)".

The recommended controlled vocabulary for this term consists of "present" and "absent", which are the only two appropriate terms for a Darwin Core Occurrence. This is reflected in theth dwc:occurrenceStatus vocabulary for this test. Other values for dwc:occurrenceStatus should only arise under circumstances that do not refer to an Occurrence. There is currently a mismatch between the (authoritative) lower case values at https://dwc.tdwg.org/terms/#dwc:occurrenceStatus and the vocabulary at GBIF (https://api.gbif.org/v1/vocabularies/OccurrenceStatus/concepts), which capitalized the first character. This test must return NOT_COMPLIANT if there are case differences, leading or trailing whitespace or there are leading or trailing non-printing characters.

[chicoreus]

@tucotuco changes applied to the notes.