TG2-VALIDATION_GEODETICDATUM_STANDARD

[iDigBioBot]

TestField	Value
GUID	7e0c0418-fe16-4a39-98bd-80e19d95b9d1
Label	VALIDATION_GEODETICDATUM_STANDARD
Description	Does the value of dwc:geodeticDatum occur as a valid geographic CRS, geodetic Datum or ellipsoid in bdq:sourceAuthority?
TestType	Validation
Darwin Core Class	dcterms:Location
Information Elements ActedUpon	dwc:geodeticDatum
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, or ellipsoid, or for a CRS appropriate for a 2D geographic coordinate in degrees, or is the value "not recorded"; otherwise NOT_COMPLIANT
Data Quality Dimension	Conformance
Term-Actions	GEODETICDATUM_STANDARD
Parameter(s)
Source Authority	bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index]}
Specification Last Updated	2025-03-03
Examples	[dwc:geodeticDatum="EPSG:4326": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:geodeticDatum matches an unambiguous alphanumeric CRS or datum code value in the bdq:sourceAuthority"]
	[dwc:geodeticDatum="7030": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:geodeticDatum doesn't match values in the bdq:sourceAuthority, 7030 is a bare number without an authority.]
Source	ALA, GBIF
References	Maptiler (2019) EPSG.io. https://epsg.io/ EPSG (2024) About the EPSG Dataset. https://epsg.org/ Spatial Reference (2024) What is SpatialReference.org. https://spatialreference.org/ Geomatic Solutions (2018) Georepository. Version 9.0.0.1062. https://georepository.com/ Chapman AD and Wieczorek JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat. https://doi.org/10.15468/doc-gg7h-s853 Wieczorek C and Wieczorek J (2021) Georeferencing Calculator. http://georeferencing.org/georefcalculator/gc.html
Example Implementations (Mechanisms)	Kurator/FilteredPush geo_ref_qc Library DOI: 10.5281/zenodo.14064324
Link to Specification Source Code	https://github.com/FilteredPush/geo_ref_qc/blob/v2.0.1/src/main/java/org/filteredpush/qc/georeference/DwCGeoRefDQ.java#L1500
Notes	Darwin Core recommends best practice is to use a controlled vocabulary. This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. Chapman and Wieczorek (2020) recommend best practice is to use EPSG geographic CRS or Datum codes (https://epsg.io/) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". While "not recorded" is not a valid EPSG code, it is a valid value according to Darwin Core. The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "EPSG:4326".

[iDigBioBot]

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet:
200 meters refers to only a few datums (Australia?) - In NAD27-WGS84 can be as high as 480 meters (from memory) in Aleutian Islands, and greatest distance is around 3,520 meters with an Indian Datum (again from memory)

[iDigBioBot]

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet:
For NOTES Column

[iDigBioBot]

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet:
Should we change name to GEODETIC_DATUM_AMBIGUOUS and make warning type Ambiguous?

[godfoder]

[chicoreus]

Parameter is not needed for this test, the vocabulary expected by dwc;geodeticDatum is the EPSG vocabulary. Parameter might be needed to specify if the expected values are in the form https://epsg.io/4326, EPSG:4326, or WGS84, but the EPSG vocabulary is the one that everyone converges on, and different user communities are not likely to want different vocabularies for this test. The specification of the vocabulary should go into the specification or into the notes, not a parameter.

[ArthurChapman]

Agreed. Just accept https://epsg.io/ as the bdq:sourceAuthority and a value is valid if it is in that vocabulary. Perhaps we should format it as being a valid EPSG Code (as opposed to a Datum Code)

[Tasilee]

Agreed, but again, "https://epsg.io/" in References?

[ArthurChapman]

I have added "epsg:4326" to the examples, but shouldn't we delete WGS:84 and GD66 as valid examples when in #60 we suggest an amendment of WGS84 to epsg:4326?

[tucotuco]

This one is a tough one in that epsg isn't actually a standard either. But it is the closest we have and it is what Darwin Core recommends to use, if possible. Nevertheless, the EXPECTED_RESPONSE says, "or an unambiguous alphanumeric CRS or datum code", which makes all the examples in the Darwin Core definition valid, namely: EPSG:4326, WGS84, NAD27, Campo Inchauspe, European 1950, Clarke 1866, unknown. I think that all of those should trigger a COMPLIANT response. I would use all those examples and add, epsg:4326, and 4326.

[Tasilee]

I've added "4326"

[Tasilee]

Added to Notes: "This test will fail if there are leading or trailing white space or non-printing characters."

[Tasilee]

This is a 'test' where we reference bdq:sourceAuthority under INTERNAL_PREREQUISITES_NOT_MET but then use "is a valid EPSG CRS Code (with or without the "epsg" namespace prepended), or an unambiguous alphanumeric CRS or datum code;" subsequently. Should we reword it to something like

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG CRS Code (with or without the "epsg" namespace prepended) in bdq:sourceAuthority, or an unambiguous alphanumeric CRS or datum code in bdq:sourceAuthority; otherwise NOT_COMPLIANT

or is that overkill?

[tucotuco]

@Tasilee Reading the rewrite struck me as odd. Made me pause. It doesn't seem wrong on inspection, but I am not sure it adds to the clarity of the test.

[chicoreus]

@Tasilee We can get around MUST by asserting MUST NOT.

That is reasonable, as long as we are explicit.

[chicoreus]

The phrasing we have right now in the note explicitly brings "not recorded" in as a valid value for the controlled vocabulary that we MUST use. We can avoid the problems that arise from that by asserting that "not recorded" must not be considered a compliant value.

[chicoreus]

Proposal: Add the following text to the end of the note:

For the purposes of this test "not recorded" is NOT_COMPLIANT.

[Tasilee]

OK...I think we have agreement. I'll change

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is (1) "not recorded" or (2) a valid geographic EPSG code for a CRS, Datum, or ellipsoid in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid geographic EPSG code for a CRS, Datum, or ellipsoid in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

and I'll amend the Notes from

". If none of these is known, use the explicit value "not recorded".

..." If none of these is known, use the explicit value "not recorded", but as this is not a valid EPSG code, the Test will return NOT_COMPLIANT".

OK?

[chicoreus]

Probably safer as a separate sentence at the end. We sound like we are paraphrasing the best pratice guide, and we don't want to conflate what it says with what we are asserting for this test.

…

On Mon, 11 Nov 2024 18:04:27 -0800 Lee Belbin ***@***.***> wrote: and I'll amend the Notes from ". If none of these is known, use the explicit value "not recorded". ..." If none of these is known, use the explicit value "not recorded", but as this is not a valid EPSG code, the Test will return NOT_COMPLIANT". OK?

[tucotuco]

Please hold on action on this one. I have a strong opinion that I do not have time to share at this moment.

[tucotuco]

The description says, "Does the value of dwc:geodeticDatum occur as a valid geographic CRS, geodetic Datum or ellipsoid in bdq:sourceAuthority?" It follows from the definition that "unknown" and "not recorded" aren't standard.

What is the consequence? All values "unknown" and "not recorded" get flagged. A further consequence of flagging is that these values would likely be set to EMPTY to avoid getting the flag and thereby "improving" data quality. Except it would degrade data quality. The value "not recorded" is specifically recommended by best practices, and hopefully will also be accepted as a non-normative change to the comments and examples for dwc:geodeticDatum. This has tangible (potentially drastic) consequences on dwc:coordinateUncertaintyInMeters. The unknown value (along with the dwc:georeferenceProtocol) tells a consumer that the georeference takes into account this source of uncertainty rather than making the (extremely common, but unwarranted) assumption that the geodeticDatum is WGS84 (epsg:4326).

Thus, the test has much more value if it can detect that the model upon which the coordinates are based is not known. The test, in my opinion, should be for standard according to Darwin Core rather than for standard for EPSG.

[chicoreus]

@tucotuco I like that position.

That would be:

Expected Response:
EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is (1) "not recorded" or (2) a valid geographic EPSG code for a CRS, Datum, or ellipsoid in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

Comment text back to ". If none of these is known, use the explicit value "not recorded"

Source authority:
bdq:sourceAuthority = "EPSG plus 'not recorded'" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}

#60 will need a corresponding change in the source authority and the note. Probably worth adding some of your comment above to the note in both cases.

[Tasilee]

Playing devils' advocate again: I'm still uncomfortable about this exception, mainly the 'COMPLIANT' response to "not recorded". I'd feel more comfortable if "not recorded" triggered "INTERNAL_PREREQUISITES_NOT_MET". At least then we would not attributing compliancy and it would make sense in that there isn't a value to check against EPSG.

How would we feel abut the use of a "not recorded" value for other Darwin Core terms?

[ArthurChapman]

I am happy to go either way as I can see both sides of this argument. I do tend to agree with Lee's last post that we can get around the problem without affecting the quality by adding "not recorded" into the INTERNAL_PREREQUISITES_NOTMET

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty or if dwc:geodeticDatum="not recorded"; COMPLIANT if the value of dwc:geodeticDatum is a valid geographic EPSG code for a CRS, Datum, or ellipsoid in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[chicoreus]

On Tue, 12 Nov 2024 20:00:18 -0800 Lee Belbin ***@***.***> wrote: Playing devils' advocate again: I'm still uncomfortable about this exception, mainly the 'COMPLIANT' response to "not recorded". I'd feel more comfortable if "not recorded" triggered "INTERNAL_PREREQUISITES_NOT_MET". At least then we would not attributing compliancy and it would make sense in that there isn't a value to check against EPSG.

That is a complete misuse of INTERNAL_PREREQUISITES_NOT_MET. Not a good idea. @tucotuco is correct, we need to revert this to consider "not recorded" as a valid value for the controlled vocabulary. We really need to go back through all the tests and remove the use of INTERNAL_PREREQUISITES_NOT_MET for cases that are not actually prerequsites for running the tests. We strayed too far from the design and now is the point to correct that. This suggestion indicates that we have made a severe error that we need to correct as it leads to misunderstanding of what prerequsites are.

[ymgan]

Quick question - can we follow the description of similar test for taxonRank #162 ?

Does the value of dwc:geodeticDatum occur in the bdq:sourceAuthority?

It is frustrating because this one single field is doing so many things:

• some value of geodetic datum
• empty
• why it is not a value of a geodetic datum (not recorded)

Does it make sense to have a bdq:sourceAuthority to be the union of

• "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}
• a list of NITS (Nothing Interesting To Say) vocabulary: unknown, not recorded etc ... (please see comments in Recommendations on missing/unknown/not recorded data in Darwin Core dwc#437)?

I don't like this solution because if I am a user of data, I will not expect finding "not recorded" or "unknown" as a valid value for dwc:geodeticDatum. However, since this is a recommendation from BPG and Darwin Core QRG, I respect it.

[Tasilee]

Changed the Description from "Does the value of dwc:geodeticDatum occur as a valid geographic CRS, geodetic Datum or ellipsoid in bdq:sourceAuthority?" to "Is the value of dwc:geodeticDatum valid according to the bdq:sourceAuthority?"

[Tasilee]

Changed Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid geographic EPSG code for a CRS, Datum, or ellipsoid in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid according to the bdq:sourceAuthority; otherwise NOT_COMPLIANT"

[Tasilee]

Changed Source Authority from

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}

to

bdq:sourceAuthority = "GBIF GeodeticDatum Vocabulary" {[https://registry.gbif.org/vocabulary/GeodeticDatum/concepts]}

@chicoreus : Can you follow up on an API?

[Tasilee]

Replace sentence in Notes from

"not recorded" is however not a valid EPSG code, so the Test will return NOT_COMPLIANT

to

While "not recorded" is not a valid EPSG code, it is a valid value according to Darwin Core.

[chicoreus]

We can't use the GBIF geodetic datum vocabulary as the source authority here https://registry.gbif.org/vocabulary/GeodeticDatum/concepts the test is for dwc:geodeticDatum, which is the geodetic datum that applies to dwc:decimalLatitude and dwc:decimalLongitude, and thus is restricted in valid values to EPSG codes that apply to geographic coordinates. The GBIF geodetic datum vocabulary includes values such as EPSG:32736, which is for UTM coordinates, and would be an explicitly incorrect value for dwc:geodeticDatum https://registry.gbif.org/vocabulary/GeodeticDatum/concept/32736 (in addition, the vocabulary only contains bare numbers, without the expected EPSG: pseudo-namespace, except in hidden labels) https://registry.gbif.org/vocabulary/GeodeticDatum/concept/32736/hiddenLabels The GBIF geodetic datum vocabulary is appropriate for dwc:verbatimSRS, as the verbatim coordinates may be in other than geographic coordinate systems, but that isn't the term we are testing here. We need to change the source authority back the EPSG vocabulary (and restore the explicit statement about applicability to geographic coordinates).

[chicoreus]

"not recorded" needs to be explicit in either the source authority or the specification, we can't relegate that to just the notes.

[chicoreus]

Following on offline discussion, recommend we change the expected response from:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid according to the bdq:sourceAuthority; otherwise NOT_COMPLIANT

To:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty or if dwc:geodeticDatum="not recorded"; COMPLIANT if the value of dwc:geodeticDatum is a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a CRS, Datum, or ellipsoid appropriate for a 2D geographic coordinate in degrees; otherwise NOT_COMPLIANT

And the source authority to:

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index]}

Phrasing that way, we don't need "not recorded" in the source authority, or the restriction on 2D geographic CRS codes in the source authority. Some spatial users could switch out for ESRI codes instead of EPSG codes. Notes do need to specify Authority:number as the form, and need to be explicit about the case (EPSG:4326 or epsg:4326).

Notes should mention more information on obtaining the EPSG dataset, particularly https://docs.geotools.org/latest/userguide/library/referencing/epsg.html

[chicoreus]

Rephrasing, per discussion with @tucotuco of intended broad scope of dwc:geodeticDatum, and to match #60

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, or ellipsoid, or for a CRS appropriate for a 2D geographic coordinate in degrees, or is the value "not recorded"; otherwise NOT_COMPLIANT

[Tasilee]

Changed Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid according to the bdq:sourceAuthority; otherwise NOT_COMPLIANT

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, or ellipsoid, or for a CRS appropriate for a 2D geographic coordinate in degrees, or is the value "not recorded"; otherwise NOT_COMPLIANT

and Source Authority from

bdq:sourceAuthority = "GBIF GeodeticDatum Vocabulary" {[https://registry.gbif.org/vocabulary/GeodeticDatum/concepts]}

to

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index]}

and updated Specification Last Updated

[chicoreus]

We might want to add to the notes that EPSG codes are not unique across entity types (e.g. EPSG:6283 is a valid code for both a datum (GDA94) and a transformation), but this test is only concerned with whether the value is "consistent with being a valid value for dwc:geodeticDatum" (to quote @tucotuco from an email thread), rather than interpreting the meaning of the EPSG code.

Correcting the second example, from:

"[dwc:geodeticDatum="7030": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:geodeticDatum doesn't match values in the bdq:sourceAuthority, 1730 (EPSG:1730) is an ellipsoid not a datum"]"

To

[dwc:geodeticDatum="7030": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:geodeticDatum doesn't match values in the bdq:sourceAuthority, 7030 is a bare number without an authority.]

Note that EPSG:7030 would be compliant, as it is a valid EPSG code for an ellipsoid, even though it is not a datum.

[chicoreus]

Restoring the description to "Does the value of dwc:geodeticDatum occur as a valid geographic CRS, geodetic Datum or ellipsoid in bdq:sourceAuthority?" which better reflects the scope (not any EPSG code, but only those that fit the dwc:geodeticDatum definition.

[Tasilee]

Noted. As all Examples are generated from the Test Data, I have amended the comment on dataID 383