From 1bfe6c3b0da6db9adbb73949ab31eae59b026121 Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Mon, 6 May 2024 18:53:10 -0700 Subject: [PATCH 1/7] Explain allowReserved in RFC6570 terms This aligns allowReserved with style by similarly correlating it with RFC6570 operators. This will make it easier to write a more in-depth explanation of the process in an appendix. --- versions/3.0.4.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 12b2375a79..198f177c4c 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -1068,7 +1068,7 @@ Field Name | Type | Description ---|:---:|--- style | `string` | Describes how the parameter value will be serialized depending on the type of the parameter value. Default values (based on value of `in`): for `query` - `form`; for `path` - `simple`; for `header` - `simple`; for `cookie` - `form`. explode | `boolean` | When this is true, parameter values of type `array` or `object` generate separate parameters for each value of the array or key-value pair of the map. For other types of parameters this property has no effect. When [`style`](#parameterStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`. -allowReserved | `boolean` | Determines whether the parameter value SHOULD allow reserved characters, as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-2.2) `:/?#[]@!$&'()*+,;=` to be included without percent-encoding. This property only applies to parameters with an `in` value of `query`. The default value is `false`. +allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are not allowed in the query string, or have a meaning other than their literal value (e.g. `+` as an escape for the space character in `application/x-www-form-urlencoded` query strings). This property only applies to parameters with an `in` value of `query`. The default value is `false`. schema | [Schema Object](#schemaObject) \| [Reference Object](#referenceObject) | The schema defining the type used for the parameter. example | Any | Example of the parameter's potential value. The example SHOULD match the specified schema and encoding properties if present. The `example` field is mutually exclusive of the `examples` field. Furthermore, if referencing a `schema` that contains an example, the `example` value SHALL _override_ the example provided by the schema. To represent examples of media types that cannot naturally be represented in JSON or YAML, a string value can contain the example with escaping where necessary. examples | Map[ `string`, [Example Object](#exampleObject) \| [Reference Object](#referenceObject)] | Examples of the parameter's potential value. Each example SHOULD contain a value in the correct format as specified in the parameter encoding. The `examples` field is mutually exclusive of the `example` field. Furthermore, if referencing a `schema` that contains an example, the `examples` value SHALL _override_ the example provided by the schema. @@ -1618,7 +1618,7 @@ Field Name | Type | Description headers | Map[`string`, [Header Object](#headerObject) \| [Reference Object](#referenceObject)] | A map allowing additional information to be provided as headers, for example `Content-Disposition`. `Content-Type` is described separately and SHALL be ignored in this section. This property SHALL be ignored if the request body media type is not a `multipart`. style | `string` | Describes how a specific property value will be serialized depending on its type. See [Parameter Object](#parameterObject) for details on the [`style`](#parameterStyle) property. The behavior follows the same values as `query` parameters, including default values. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. explode | `boolean` | When this is true, property values of type `array` or `object` generate separate parameters for each value of the array, or key-value-pair of the map. For other types of properties this property has no effect. When [`style`](#encodingStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. -allowReserved | `boolean` | Determines whether the parameter value SHOULD allow reserved characters, as defined by [RFC3986](https://tools.ietf.org/html/rfc3986#section-2.2) `:/?#[]@!$&'()*+,;=` to be included without percent-encoding. The default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. +allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are not allowed in the query string, or have a meaning other than their literal value (e.g. `+` as an escape for the space character in `application/x-www-form-urlencoded` query strings). The default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. This object MAY be extended with [Specification Extensions](#specificationExtensions). From 52d3f6c0c21abf3e4a1ea92ebfe75cd4fb9dc7ca Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Mon, 13 May 2024 14:28:01 -0700 Subject: [PATCH 2/7] Appendix on RFC6570-derived behavior This is the one of several appendixes to be added to clarify the most obscure details of Parameter Object and Encoding Object serialization. This clarifies the correspondence between OAS fields and RFC6570 operators, and acknowledges that some field values and combinations do not have analogues. It provides further guidance for how to use RFC6570 implementations to support these configurations. This includes a SHOULD directive regarding using RFC6570 expansion with the non-RFC6570 styles, as the use of "explode" and "allowReserved" does not otherwise make any sense. It perhaps could be a MUST. Examples are included to show both typical usage, and how to work around the lack of exact RFC6570 equivalences for certain configurations. --- versions/3.0.4.md | 212 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 198f177c4c..71325313a4 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -1073,6 +1073,8 @@ Field Name | Type | Description example | Any | Example of the parameter's potential value. The example SHOULD match the specified schema and encoding properties if present. The `example` field is mutually exclusive of the `examples` field. Furthermore, if referencing a `schema` that contains an example, the `example` value SHALL _override_ the example provided by the schema. To represent examples of media types that cannot naturally be represented in JSON or YAML, a string value can contain the example with escaping where necessary. examples | Map[ `string`, [Example Object](#exampleObject) \| [Reference Object](#referenceObject)] | Examples of the parameter's potential value. Each example SHOULD contain a value in the correct format as specified in the parameter encoding. The `examples` field is mutually exclusive of the `example` field. Furthermore, if referencing a `schema` that contains an example, the `examples` value SHALL _override_ the example provided by the schema. +See also [Appendix C: Using RFC6570 Implementations](#usingRFC6570Implementations) for additional guidance. + ###### Fixed Fields and considerations for use with `content` For more complex scenarios, the [`content`](#parameterContent) property can define the media type and schema of the parameter, as well as give examples of its use. @@ -1622,6 +1624,8 @@ Field Name | Type | Description This object MAY be extended with [Specification Extensions](#specificationExtensions). +See also [Appendix C: Using RFC6570 Implementations](#usingRFC6570Implementations) for additional guidance. + ##### Encoding Object Example `multipart/form-data` allows for binary parts: @@ -3523,3 +3527,211 @@ Version | Date | Notes 1.2 | 2014-03-14 | Initial release of the formal document. 1.1 | 2012-08-22 | Release of Swagger 1.1 1.0 | 2011-08-10 | First release of the Swagger Specification + +## Appendix C: Using RFC6570 Implementations + +Serialization is defined in terms of RFC6570 URI Templates in two scenarios: + +Object | Condition +------ | --------- +[Parameter Object](#parameterObject) | When `schema` is present +[Encoding Object](#encodingObject) | When encoding for `application/x-www-form-urlencoded` and any of `style`, `explode`, or `allowReserved` are used + +Implementations of this specification MAY use an implementation of RFC6570 to perform variable expansion, however, some caveats apply. + +### Equivalences Between Fields and RFC6570 Operators + +Certain field values translate to RFC6570 operators (or lack thereof): + +field | value | equivalent +----- | ----- | ---------- +style | simple | _n/a_ +style | matrix | `;` prefix operator +style | label | `.` prefix operator +style | form | `?` prefix operator +allowReserved | `false` | _n/a_ +allowReserved | `true` | `+` prefix operator +explode | `false` | _n/a_ +explode | `true` | `*` modifier suffix + +Multiple `style: form` parameters are equivalent to a single RFC6570 [variable list](https://www.rfc-editor.org/rfc/rfc6570#section-2.2) using the `?` prefix operator: + +```YAML +parameters: +- name: foo + in: query + schema: + type: object + explode: true +- name: bar + in: query + schema: + type: string +``` + +This example is equivalent to RFC6570's `{?foo*,bar}`, and ***NOT*** `{?foo*}{&bar}`, which is problematic because if `foo` is not defined, the result will be an invalid URI. +The `&` prefix operator has no equivalent in the Parameter Object. + +### Non-RFC6570 Field Values and Combinations + +Configurations with no direct RFC6570 equivalent SHOULD also be handled according to RFC6570. +Implementations MAY create a properly delimited URI Template with variables for individual names and values using RFC6570 regular or reserved expansion (based on `allowReserved`). + +This includes: + * the styles `pipeDelimited`, `spaceDelimited`, and `deepObject`, which have no equivalents at all + * the combination of the style `form` with `allowReserved: true`, which is not allowed because only one prefix operator can be used at a time + * any parameter name that is not a legal RFC6570 variable name (alphanumerics, `_`, and percent-encoded characters) + +The Parameter Object's `name` field has a much more permissive syntax than [RFC6570 variable name syntax](https://www.rfc-editor.org/rfc/rfc6570#section-2.3). +A parameter name that includes characters outside of the allowed RFC6570 variable character set MUST be percent-encoded before it can be used in a URI Template, and MUST set `allowReserved: true` to avoid double percent-encoding. + +### Examples + +Let's say we want to use the following data in a form query string, where `formulas` is exploded, and `words` is not: + +```YAML +formulas: + a: x+y + b: x/y + c: x^y +words: +- math +- is +- fun +``` + +#### RFC6570-Equivalent Expansion + +This array of parameter objects uses regular `style: form` expansion, fully supported by RFC6570: + +```YAML +parameters: +- name: formulas + in: query + schema: + type: object + additionalProperties: + type: string + explode: true +- name: words + in: query + schema: + type: array + items: + type: string +``` + +This translates to the following URI Template: + +```urlencoded +{?formulas*,words} +``` + +when expanded with the data given earlier, we get: + +```urlencoded +?a=x%2By&b=x%2Fy&c=x%5Ey&words=math,is,fun +``` + +#### Expansion With Non-RFC6570-Supported Options + +But now let's say that (for some reason), we really want that `/` in the `b` formula to show up as-is in the query string, and we want our words to be space-separated like in a written phrase. +To do that, we'll add `allowReserved: true` to `formulas`, and change to `style: spaceDelimited` for `words`: + +```YAML +parameters: +- name: formulas + in: query + schema: + type: object + additionalProperties: + type: string + explode: true + allowReserved: true +- name: words + in: query + style: spaceDelimited + schema: + type: array + items: + type: string +``` + +We can't combine the `?` and `+` RFC6570 prefixes, and there's no way with RFC6570 to replace the `,` separator with a space character. +So we need to restructure the data to fit a manually constructed URI Template that passes all of the pieces through the right sort of expansion. + +Here is one such template. +The exact variable names are unimportant, but in this example "f" and "w" are meant to suggest "formulas" and "words", while "n" and "v" suggest "name" and "value": + +```urlencoded +?{+fn0}={+fv0}&{+fn1}={+fv1}&{+fn2}={+fv2}&{wn}={wv0} {wv1} {wv2} +``` + +We'll also need to pre-process the names and values for `formaulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string. + +Setting `allowReserved: true` does _not_ make reserved characters that are not allowed in URIs allowed, it just allows them to be _passed through expansion unchanged._ +Therefore, any tooling still needs to percent-encode those characters because reserved expansion will not do it, but it _will_ leave the percent-encoded triples unchanged. +See also Appendix D for further guidance on percent-encoding and form media types, including guidance on handling the delimiter characters for `spaceDelimited`, `pipeDelimited`, and `deepObject` in parameter names and values. + +So here is our data structure that arranges the names and values to suit the template above, where names and values for `formulas` have `[]#&=+` pre-percent encoded (although only `+` appears in this example): + +```YAML +fn0: a +fv0: x%2By +fn1: b +fv1: x/y +fn2: c +fv2: x^y +wn: words +wv0: math +wv1: is +wv2: fun +``` + +Expanding our manually assembled template with our restructured data yields the following query string: + +```urlencoded +?a=x%2By&b=x/y&c=x%5Ey&words=math%20is%20fun + +``` +The `/` and the pre-percent-encoded `%2B` have been left alone, but the disallowed `^` character (inside a value) and space characters (in the template but outside of the expanded variables) were percent-encoded. + +#### Undefined Values and Manual URI Template Construction + +Care must be taken when manually constructing templates to handle the values that [RFC6570 considers to be _undefined_](https://datatracker.ietf.org/doc/html/rfc6570#section-2.3) correctly: + +```YAML +formulas: {} +words: +- hello +- world +``` + +Using this data with our original RFC6570-friendly URI Template, `{?formulas*,words}`, produces the following: + + +```urlencoded +?words=hello,world +``` + +This means that the manually constructed URI Template and restructured data need to leave out the `formulas` object entirely so that the `words` parameter is the first and only parameter in the query string. + +Restructured data: + +```YAML +wn: words +wv0: hello +wv1: world +``` + +Manually constructed URI Template: + +```urlencoded +?{wn}={wv0} {wv1} +``` + +Result: + +```urlencoded +?words=hello%20world +``` From b25f83cf56af928acc4813e56b2103aa0aaae267 Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Sun, 26 May 2024 14:44:53 -0700 Subject: [PATCH 3/7] Note RFC6570 and message body adjustment --- versions/3.0.4.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 71325313a4..14647eec13 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -3539,6 +3539,8 @@ Object | Condition Implementations of this specification MAY use an implementation of RFC6570 to perform variable expansion, however, some caveats apply. +Note that when using `style: form` RFC6570 expansion to produce an `application/x-www-form-urlencoded` HTTP message body, it is necessary to remove the `?` prefix that is produced to satisfy the URI query string syntax. + ### Equivalences Between Fields and RFC6570 Operators Certain field values translate to RFC6570 operators (or lack thereof): From f5914c76e227533afaeae79bbc852254a57b0a79 Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Sun, 26 May 2024 18:19:54 -0700 Subject: [PATCH 4/7] Arrays and objects have limited support in parameters RFC6570-based serialization will not go beyond what is specified to unpack arrays and objects. --- versions/3.0.4.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 14647eec13..42510fe85f 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -3574,6 +3574,8 @@ parameters: This example is equivalent to RFC6570's `{?foo*,bar}`, and ***NOT*** `{?foo*}{&bar}`, which is problematic because if `foo` is not defined, the result will be an invalid URI. The `&` prefix operator has no equivalent in the Parameter Object. +Note that RFC6570 does not specify behavior for compound values beyond the single level addressed by `explode`. The results of using objects or arrays where no behavior is clearly specified for them is implementation-defined. + ### Non-RFC6570 Field Values and Combinations Configurations with no direct RFC6570 equivalent SHOULD also be handled according to RFC6570. From ad97a43b6a3e3da4ac38fb8cdd396fe752386e09 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Mon, 27 May 2024 08:50:56 -0700 Subject: [PATCH 5/7] Fix typeo (review feedback) Co-authored-by: Darrel --- versions/3.0.4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 42510fe85f..19cf5ebd48 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -3671,7 +3671,7 @@ The exact variable names are unimportant, but in this example "f" and "w" are me ?{+fn0}={+fv0}&{+fn1}={+fv1}&{+fn2}={+fv2}&{wn}={wv0} {wv1} {wv2} ``` -We'll also need to pre-process the names and values for `formaulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string. +We'll also need to pre-process the names and values for `formulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string. Setting `allowReserved: true` does _not_ make reserved characters that are not allowed in URIs allowed, it just allows them to be _passed through expansion unchanged._ Therefore, any tooling still needs to percent-encode those characters because reserved expansion will not do it, but it _will_ leave the percent-encoded triples unchanged. From ed7a6d8dc79d262b97d8359c4d9735f597eab541 Mon Sep 17 00:00:00 2001 From: "Henry H. Andrews" Date: Mon, 27 May 2024 12:34:15 -0700 Subject: [PATCH 6/7] Fix handling of parameter names, clarify table entry * Paramter names don't need to be modeled as template variables. * Include an example of a parameter name that needs encoding. * Add more details to the fixed field entries for allowReserved --- versions/3.0.4.md | 77 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 53 insertions(+), 24 deletions(-) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index 19cf5ebd48..b8deb70ad2 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -1068,7 +1068,7 @@ Field Name | Type | Description ---|:---:|--- style | `string` | Describes how the parameter value will be serialized depending on the type of the parameter value. Default values (based on value of `in`): for `query` - `form`; for `path` - `simple`; for `header` - `simple`; for `cookie` - `form`. explode | `boolean` | When this is true, parameter values of type `array` or `object` generate separate parameters for each value of the array or key-value pair of the map. For other types of parameters this property has no effect. When [`style`](#parameterStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`. -allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are not allowed in the query string, or have a meaning other than their literal value (e.g. `+` as an escape for the space character in `application/x-www-form-urlencoded` query strings). This property only applies to parameters with an `in` value of `query`. The default value is `false`. +allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are [not allowed in the query string](https://datatracker.ietf.org/doc/html/rfc3986#autoid-24) (`[`, `]`, `#`), or have a special meaning in `application/x-www-form-urlencoded` (`-`, `&`, `+`); see Appendices [C](#usingRFC6570Implementations) and [E](#percentEncodingAndFormMediaTypes) for details. This property only applies to parameters with an `in` value of `query`. The default value is `false`. schema | [Schema Object](#schemaObject) \| [Reference Object](#referenceObject) | The schema defining the type used for the parameter. example | Any | Example of the parameter's potential value. The example SHOULD match the specified schema and encoding properties if present. The `example` field is mutually exclusive of the `examples` field. Furthermore, if referencing a `schema` that contains an example, the `example` value SHALL _override_ the example provided by the schema. To represent examples of media types that cannot naturally be represented in JSON or YAML, a string value can contain the example with escaping where necessary. examples | Map[ `string`, [Example Object](#exampleObject) \| [Reference Object](#referenceObject)] | Examples of the parameter's potential value. Each example SHOULD contain a value in the correct format as specified in the parameter encoding. The `examples` field is mutually exclusive of the `example` field. Furthermore, if referencing a `schema` that contains an example, the `examples` value SHALL _override_ the example provided by the schema. @@ -1620,7 +1620,7 @@ Field Name | Type | Description headers | Map[`string`, [Header Object](#headerObject) \| [Reference Object](#referenceObject)] | A map allowing additional information to be provided as headers, for example `Content-Disposition`. `Content-Type` is described separately and SHALL be ignored in this section. This property SHALL be ignored if the request body media type is not a `multipart`. style | `string` | Describes how a specific property value will be serialized depending on its type. See [Parameter Object](#parameterObject) for details on the [`style`](#parameterStyle) property. The behavior follows the same values as `query` parameters, including default values. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. explode | `boolean` | When this is true, property values of type `array` or `object` generate separate parameters for each value of the array, or key-value-pair of the map. For other types of properties this property has no effect. When [`style`](#encodingStyle) is `form`, the default value is `true`. For all other styles, the default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. -allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are not allowed in the query string, or have a meaning other than their literal value (e.g. `+` as an escape for the space character in `application/x-www-form-urlencoded` query strings). The default value is `false`. This property SHALL be ignored if the request body media type is not `application/x-www-form-urlencoded`. +allowReserved | `boolean` | When this is true, parameter values are serialized using reserved expansion, as defined by [RFC6570](https://datatracker.ietf.org/doc/html/rfc6570#autoid-20), which allows [RFC3986's reserved character set](https://datatracker.ietf.org/doc/html/rfc3986#autoid-13), as well as percent-encoded triples, to pass through unchanged, while still percent-encoding all other disallowed characters (including `%` outside of percent-encoded triples). Applications are still responsible for percent-encoding reserved characters that are [not allowed in the query string](https://datatracker.ietf.org/doc/html/rfc3986#autoid-24) (`[`, `]`, `#`), or have a special meaning in `application/x-www-form-urlencoded` (`-`, `&`, `+`); see Appendices [C](#usingRFC6570Implementations) and [E](#percentEncodingAndFormMediaTypes) for details. This property only applies to parameters with an `in` value of `query`. The default value is `false`. This object MAY be extended with [Specification Extensions](#specificationExtensions). @@ -3541,6 +3541,9 @@ Implementations of this specification MAY use an implementation of RFC6570 to pe Note that when using `style: form` RFC6570 expansion to produce an `application/x-www-form-urlencoded` HTTP message body, it is necessary to remove the `?` prefix that is produced to satisfy the URI query string syntax. +Note also that not all RFC6570 implementations support all four levels of operators, all of which are needed to fully support the OpenAPI Specification's usage. +Using an implementation with a lower level of support will require additional manual construction of URI Templates to work around the limitations. + ### Equivalences Between Fields and RFC6570 Operators Certain field values translate to RFC6570 operators (or lack thereof): @@ -3587,7 +3590,7 @@ This includes: * any parameter name that is not a legal RFC6570 variable name (alphanumerics, `_`, and percent-encoded characters) The Parameter Object's `name` field has a much more permissive syntax than [RFC6570 variable name syntax](https://www.rfc-editor.org/rfc/rfc6570#section-2.3). -A parameter name that includes characters outside of the allowed RFC6570 variable character set MUST be percent-encoded before it can be used in a URI Template, and MUST set `allowReserved: true` to avoid double percent-encoding. +A parameter name that includes characters outside of the allowed RFC6570 variable character set MUST be percent-encoded before it can be used in a URI Template. ### Examples @@ -3664,39 +3667,36 @@ parameters: We can't combine the `?` and `+` RFC6570 prefixes, and there's no way with RFC6570 to replace the `,` separator with a space character. So we need to restructure the data to fit a manually constructed URI Template that passes all of the pieces through the right sort of expansion. -Here is one such template. -The exact variable names are unimportant, but in this example "f" and "w" are meant to suggest "formulas" and "words", while "n" and "v" suggest "name" and "value": +Here is one such template, using a made-up convention of `words.0` for the first entry in the words value, `words.1` for the second. ```urlencoded -?{+fn0}={+fv0}&{+fn1}={+fv1}&{+fn2}={+fv2}&{wn}={wv0} {wv1} {wv2} +?a={+a}&b={+b}&c={+c}&words={words.0} {words.1} {words.2} ``` -We'll also need to pre-process the names and values for `formulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string. +RFC6570 [mentions](https://www.rfc-editor.org/rfc/rfc6570.html#section-2.4.2) the use of `.` "to indicate name hierarchy in substructures," but does not define any specific naming convention or behavior for it. +Since the `.` usage is not automatic, we'll need to construct an appropriate input structure for this new template. + +We'll also need to pre-process the values for `formulas` because while `/` and most other reserved characters are allowed in the query string by RFC3986, `[`, `]`, and `#` [are not](https://datatracker.ietf.org/doc/html/rfc3986#appendix-A), and `&`, `=`, and `+` all have [special behavior](https://www.rfc-editor.org/rfc/rfc1866#section-8.2.1) in the `application/x-www-form-urlencoded` format, which is what we are using in the query string. Setting `allowReserved: true` does _not_ make reserved characters that are not allowed in URIs allowed, it just allows them to be _passed through expansion unchanged._ Therefore, any tooling still needs to percent-encode those characters because reserved expansion will not do it, but it _will_ leave the percent-encoded triples unchanged. -See also Appendix D for further guidance on percent-encoding and form media types, including guidance on handling the delimiter characters for `spaceDelimited`, `pipeDelimited`, and `deepObject` in parameter names and values. +See also [Appendix E](#to indicate name hierarchy in substructures) for further guidance on percent-encoding and form media types, including guidance on handling the delimiter characters for `spaceDelimited`, `pipeDelimited`, and `deepObject` in parameter names and values. -So here is our data structure that arranges the names and values to suit the template above, where names and values for `formulas` have `[]#&=+` pre-percent encoded (although only `+` appears in this example): +So here is our data structure that arranges the names and values to suit the template above, where values for `formulas` have `[]#&=+` pre-percent encoded (although only `+` appears in this example): ```YAML -fn0: a -fv0: x%2By -fn1: b -fv1: x/y -fn2: c -fv2: x^y -wn: words -wv0: math -wv1: is -wv2: fun +a: x%2By +b: x/y +c: x^y +words.0: math +words.1: is +words.2: fun ``` Expanding our manually assembled template with our restructured data yields the following query string: ```urlencoded ?a=x%2By&b=x/y&c=x%5Ey&words=math%20is%20fun - ``` The `/` and the pre-percent-encoded `%2B` have been left alone, but the disallowed `^` character (inside a value) and space characters (in the template but outside of the expanded variables) were percent-encoded. @@ -3723,15 +3723,14 @@ This means that the manually constructed URI Template and restructured data need Restructured data: ```YAML -wn: words -wv0: hello -wv1: world +words.0: hello +words.1: world ``` Manually constructed URI Template: ```urlencoded -?{wn}={wv0} {wv1} +?words={words.0} {words.1} ``` Result: @@ -3739,3 +3738,33 @@ Result: ```urlencoded ?words=hello%20world ``` + +#### Illegal Variable Names as Parameter Names +In this example, the heart emoji is not legal in URI Template names (or URIs): + +```YAML +parameters: +- name: ❤️ + in: query + schema: + type: string +``` + +We can't just pass `❤️: love!` to an RFC6570 implementation. +Instead, we have to pre-percent-encode the name (which is a six-octet UTF-8 sequence) in both the data and the URI Template: + +```YAML +"%E2%9D%A4%EF%B8%8F": love! +``` + +```urlencoded +{?%E2%9D%A4%EF%B8%8F} +``` + +This will expand to the result: + +```urlencoded +?%E2%9D%A4%EF%B8%8F=love%21 +``` + +## Appendix E: Percent-Encoding and Form Media Types From b5d222914ceb4618b641db0683b480111929da01 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Tue, 28 May 2024 14:45:44 -0700 Subject: [PATCH 7/7] Review feedback on RFC quotations vs linking Co-authored-by: Ralf Handl --- versions/3.0.4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versions/3.0.4.md b/versions/3.0.4.md index b8deb70ad2..96025bdac1 100644 --- a/versions/3.0.4.md +++ b/versions/3.0.4.md @@ -3587,7 +3587,7 @@ Implementations MAY create a properly delimited URI Template with variables for This includes: * the styles `pipeDelimited`, `spaceDelimited`, and `deepObject`, which have no equivalents at all * the combination of the style `form` with `allowReserved: true`, which is not allowed because only one prefix operator can be used at a time - * any parameter name that is not a legal RFC6570 variable name (alphanumerics, `_`, and percent-encoded characters) + * any parameter name that is not a legal RFC6570 variable name The Parameter Object's `name` field has a much more permissive syntax than [RFC6570 variable name syntax](https://www.rfc-editor.org/rfc/rfc6570#section-2.3). A parameter name that includes characters outside of the allowed RFC6570 variable character set MUST be percent-encoded before it can be used in a URI Template.