TG2-AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT

[iDigBioBot]

TestField	Value
GUID	7498ca76-c4d4-42e2-8103-acacccbdffa7
Label	AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT
Description	Proposes an amendment to fill in dwc:geodeticDatum using a parameterized value if the dwc:geodeticDatum is empty.
TestType	Amendment
Darwin Core Class	dcterms:Location
Information Elements ActedUpon	dwc:geodeticDatum
	dwc:coordinateUncertaintyInMeters
Information Elements Consulted	dwc:decimalLatitude
	dwc:decimalLongitude
Expected Response	If dwc:geodeticDatum is bdq:Empty, fill in dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.
Data Quality Dimension	Completeness
Term-Actions	GEODETICDATUM_ASSUMEDDEFAULT
Parameter(s)	bdq:defaultGeodeticDatum
Source Authority	bdq:defaultGeodeticDatum default = "EPSG:4326"
Specification Last Updated	2024-11-12
Examples	[dwc:geodeticDatum="[null]", dwc:decimalLatitude="-30.00", dwc:decimalLongitude="130.00", dwc:coordinateUncertaintyInMeters="50": Response.status=AMENDED, Response.result=dwc:geodeticDatum="EPSG:4326", dwc:coordinateUncertaintyInMeters="2836", Response.comment="dwc:godeticDatum is bdq:Empty so filled in with default and dwc:coordinateUncertaintyInMeters amended to maximum possible value"]
	[dwc:geodeticDatum="WGS84", dwc:decimalLatitude="", dwc:decimalLongitude="", dwc:coordinateUncertaintyInMeters="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:geodeticDatum contains a value"]
Source	ALA, GBIF
References	Maptiler (2019) EPSG.io. https://epsg.io/ EPSG (2024) About the EPSG Dataset. https://epsg.org/ Spatial Reference (2024) What is SpatialReference.org. https://spatialreference.org/ Geomatic Solutions (2018) Georepository. Version 9.0.0.1062. https://georepository.com/ Chapman AD and Wieczorek JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat. https://doi.org/10.15468/doc-gg7h-s853 Wieczorek C and Wieczorek J (2021) Georeferencing Calculator. http://georeferencing.org/georefcalculator/gc.html
Example Implementations (Mechanisms)	Kurator/FilteredPush geo_ref_qc Library DOI: 10.5281/zenodo.14064324
Link to Specification Source Code	https://github.com/FilteredPush/geo_ref_qc/blob/v2.0.1/src/main/java/org/filteredpush/qc/georeference/DwCGeoRefDQ.java#L2349
Notes	The value of dwc:geodeticDatum applies to dwc:decimalLatitude and dwc:decimalLongitude, thus EPSG:4326 https://epsg.org/crs_4326/WGS-84.html is the appropriate EPSG code as it applies to the WGS84 datum used with a geographic coordinate system. If the dwc:coordinateUncertaintyInMeters is bdq:Empty, not interpretable, or not valid, this amendment should not provide a dwc:coordinateUncertaintyInMeters. If the dwc:coordinateUncertaintyInMeters is bdqNotEmpty and is valid, this amendment should add to the dwc:coordinateUncertaintyInMeters the uncertainty contributed by the maximum datum shift at the given coordinates. Since different systems have differing requirements for what the default datum should be, it is left unspecified, but should match whatever the target datum is in AMENDMENT_COORDINATES_CONVERTED (620749b9-7d9c-4890-97d2-be3d1cde6da8). After the amendment is performed, the dwc:geodeticDatum field should be the assumed default datum as parameterized. An example implementation to determine the uncertainty added by asserting a default datum (datum shift) where a known datum is not declared can be found in datumshiftproj.py in the source code for the Georeferencing Calculator (Wieczorek & Wieczorek 2021). Included in the source code is a 5-degree grid of datum shifts from an unknown datum to WGS84.

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
See 7e0c0418-fe16-4a39-98bd-80e19d95b9d1 GEODETIC_DATUM_INVALID (VALIDATION_GEODETICDATUM_NOTSTANDARD) for the Validation method and MEASURE_GEODETICDATUM_SINGLE_COMPLETENESS for the measurement method. Potential ammendment ordering dependency, should run after an amendment that proposes a value for coordinateUncertantyInMeters

[chicoreus]

Parameter, as @ArthurChapman pointed out in #178 is which geodetic datum to use, not which vocabulary of geodetic datums to use.

bdq:sourceAuthority (default = http://epsg.io/)

Can only be interpreted as dwc:geodeticDatum is assumed to be "http://epsg.io" when not specified. This is clearly nonsensical, the default value should be WGS84, or the appropriate EPSG code, probably EPSG:4326.

Parameter is definitely needed but should be:

bdq:defaultGeodeticDatum (default=EPSG:4326)

or

bdq:defaultGeodeticDatum (default=https://epsg.io/4326)

With the notes specifying that the geodetic datum is taken from the http://epsg.io/ vocabulary....

[ArthurChapman]

@chicoreus This is the default sourceAuthority not the default value - so the value should be a value within that default source authority - whether it is ESPG:4326 (WGS84), EPSG:4674 (SIRGAS2000) or EPSG:7844 (GDA2020), etc. What you are testing for is a value in the field that is consistent with the a value in http://epsg.io/ - the same as you look for a country code in the ISO standard. On thinking about it - probably doesn't need to be Paramaterized and just accept that authority as THE authority.

[Tasilee]

@ArthurChapman - you raise a nice distinction: A source (in this case again unique), but then the default value, of which there can be plenty. I can see therefore two scenarios for Parameterised?

1. A choice of "specified source authority" and/or
2. A choice of a default value from a "specified source authority"

[ArthurChapman]

@Tasilee This is the case for all tests (especially NOTSTANDARD and STANDARDIZED tests) that use an external Standard - ISO, DCMI, in this case EPSG, of any Vocabulary. The vocabulary, standard, etc. is the bdq:sourceAuthority and you are checking to see if the value in the record is a valid record in the bdq:sourceAuthority (in the case of Validations) or can be amended to conform with a value in the bdq:sourceAuthority (in the case of Amendments). In nearly all cases, there is only one sourceAuthority (except as @chicoreus mentions with Taxon names), so there is no choice of sourceAuthority needed, only the choice of a value from that sourceAuthority. Those few cases where there is a choice of sourceAuthority (taxon names) brings in both your 1) and 2) above. Thus, I agree with @chicoreus that we don't need as many Paramaterized tests as we have previously so tagged. Unless @tucotuco has justifications for them that we have not thought of.

[chicoreus]

Phrasing of Amended text is unclear. Don't know how to interpret the comma:

AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable, the value of dwc:geodeticDatum was set to a predefined default value;

Perhaps

AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable and the value of dwc:geodeticDatum was set to a predefined default value;

[Tasilee]

I commented in @chicoreus email of September 1. We do need to standardize the phrasing.

[ArthurChapman]

I just checked the editing history - back in August last year the wording had a "therefore" after the comma
AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable, therefore the value of dwc:geodeticDatum was set to a predefined default value;

[Tasilee]

Sounds good to me. I've edited the ER as issues to discuss/fix are building.

[ArthurChapman]

Needs discussing in conjunction with #60. I have not altered to conform with #59 as it is complicated. Parameterization should be removed I think in line with #59.

[Tasilee]

@ArthurChapman: I think this IS parameterised as it requires a default value for dwc:geodeticDatum? The Parameter isn't EPSG as a sourceAuthority, but which EPSG code is the default?

The Example should have the assumption, e.g., "dwc:geodeticDatum is NULL and has been assumed EPSG:4326"?

[Tasilee]

In line with recent discussions, there is ONE dq:sourceAuthority for the GDs, epsg.io, therefore that isn't a Parameter. In this case, why are we even including "specified source authority" other than than in References and possibly Notes to recommend use of EPSG codes? There is no need to access the source authority in the Expected response, simply to look up the Parameter value, e.g., EPSG:4326 as this is implementation dependent. There is no need for "EXTERNAL_PREREQUISITES_NOT_MET".

I would also suggest amending the example to "dwc:geodeticDatum is NULL, so amended to dwc:geodetciDatum EPSG:4326".

[chicoreus]

@Tasilee Parameterized as different users might want different defaults. This one is appropriate to have a parameter.

[ArthurChapman]

No, @Tasilee - you still have to go to an external source to look up the EPSG Code - so if the EPSG Server is down, then the EXTERNAL_PREREQUISITES_NOT_MET because the bdq:sourceAuthority (i.e. epsg.io) could not be found.

@chicoreus - it is not Paramaterized, because there is only one bdq:sourceAuthority, therefore there is no other Parameter that one can chose - we are only using the EPSG codes

[Tasilee]

Removed reference to bdq:sourceAuthority from Source Authority

[chicoreus]

Corrected typo in information elements s/dwc:decimatitude/dwc:decimalLatitude/

[Tasilee]

After @chicoreus email July 25 2023, I changed the Positive example from FILLED_IN to AMENDED.

[ArthurChapman]

Don't we use AMENDED where a value has been altered, and FILLED_IN where the field was empty and we added something - see definitions in #152. I believe for consistency this should be FILLED_IN - not sure I saw @chicoreus email.

[chicoreus]

@ArthurChapman This one is more complex than usual, and almost calls for a richer response structure. When only an empty geodeticDatum is filled in, then response.status is FILLED_IN, but if a value is present in coordinateUncertaintyInMeters and an empty geodeticDatum is filled in then the coordinateUncertaintyInMeters is ammended to add the uncertainty between any datum and the specified default at the specified latitude/longitude, and the response.status becomes AMENDED.

[ArthurChapman]

Thanks @chicoreus - understood.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted".

Also changed "Field" to "TestField", "Output Type" to "TestType" and updated "Specification Last Updated"

[Tasilee]

Changed Expected response from

If dwc:geodeticDatum is EMPTY, fill in the value of dwc:geodeticDatum with the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are NOT_EMPTY, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is not EMPTY; FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are NOT_EMPTY, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

[Tasilee]

Changed dwc:decimalLatitude and dwc:decimaLongitude from Information Elements ActedUpon to Information Elements Consulted.

[chicoreus]

Issue seen with validation failures on updated test implementation not meeting "NOT_AMENDED" expectations in @Tasilee validation sheet v94: This and #75 are "AssumedDefault" tests for which non-empty values aren't preventing execution of the test, that should probably both have the internal prerequisites clause removed and be able to reach the NOT_AMENDED clause:

FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

[ArthurChapman]

As discussed under #59, I think that this test could be improved by adding "not recorded" and "unknown" as equivalents to Empty and thus filled in with the default

INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:NotEmpty or included "not recorded" or "unknown"; (1) FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, (2) if the value in dwc:geodeticDatum is either "not recorded" or "unknown", amend the value of dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report AMENDED and, (3) if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

[Tasilee]

Given my comments on #59, if dwc:geodeticDatum contains values such as "unknown" or "not recorded", then it is effectively bdq:Empty so we have a case to trigger "AMENDED..." I'd suggest

If ((1) bdq:geodeticDatum is bdq:Empty and dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN or (2) if the value in dwc:geodeticDatum is either "not recorded" or "unknown", amend the value of dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report AMENDED) and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and report AMENDED; otherwise NOT_AMENDED.

[chicoreus]

@Tasilee it is absolutely not bdq:Empty, it contains a value with meaning, not some form of empty value. The concepts are totaly different.

…

On Mon, 11 Nov 2024 13:21:48 -0800 Lee Belbin ***@***.***> wrote: Given my comments on #59, if dwc:geodeticDatum contains values such as "unknown" or "not recorded", then it is effectively bdq:Empty so we have a case to trigger "AMENDED..." I'd suggest

[chicoreus]

Proposed change to the expected response is straightforward:

FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

This gives a path to the logical conclusion NOT_AMENDED if there is a value in dwc:geodeticDatum.

Concerns about "not recorded" go elsewhere.

[Tasilee]

Thanks @chicoreus but we are missing the need to check for bdq:Empty BEFORE FILLED_IN. I suggest

If dwc:geodeticDatum is bdq:Empty, FILLED_IN dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

[chicoreus]

@Tasilee yes, something wasn't quite right in the original wording...

I don't think we need the two FILLED_IN in the same clause, how about:

If dwc:geodeticDatum is bdq:Empty, fill in dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.

Wording still seems a bit awkward.

[Tasilee]

Your rendition looks ok to me, even I can understand it. I'll apply it now.

[chicoreus]

Corrected typo in description.