Source test suite should use something more powerful than JSON pointer

[euclio]

I wrote this originally as a proof of concept. What we really need is a way to query the JSON. For example, the current rustdoc uses XPath, which is a very powerful query language. The current tests are fragile because every directive depends on the exact order of the JSON. If there's more than one def kind or multiple defs in the same file, things can get messy fast.

I'm not too familiar with this domain, but there are solutions like JPATH and json-query. I'm even less familiar with the options available to Rust.

We should address this before we get too many tests :)

[steveklabnik]

Marking this as a quest issue; I don't know the best solution either!

[euclio]

Some options:

• JSONpath
• JMESpath (seems to have a well-tested crate!)

[steveklabnik]

Marking this as hard since there's a lot of work to do; first to come up with the tech we should use, second to implement it.

[euclio]

I have a proof of concept using JMESPath instead of JSON pointer. Here's what I'm thinking design-wise:

The @has directive takes one or two arguments. The first argument is a JMESPath, and the second argument is a JSON data type. If the two arguments evaluate to the same value, the test passes.

This means that a simple check could be

// @has included[0].type '"module"'

(I'm not super happy about having to duplicate the quotes, but I think it's necessary to resolve ambiguity)

If the second argument is not supplied, it's assumed to be true. To check that a string contains another:

// @has 'data.attributes.docs | contains(@, "crate")'
// or
// @has 'contains(data.attributes.docs, "crate")'

You can also provide a JSON object on the right side. This can be used to match more complex queries.

// @has data.relationships.modules.data[0] '{ "type": "module", "id": "modules::module" }'

While we no longer would have the ability to Regex match on the right side, JMESPath is extremely powerful, and I believe that contains would serve most of the use case. If it became a problem we could customize the JMESPath runtime with a regex_match function.
Thoughts, @steveklabnik?

[steveklabnik]

My thoughts are mostly, "I've never heard of JMESPath before and it looks interesting, but that kinda gives me pause."

Hm.

[euclio]

I hadn't heard of it either, but it seems like we are constrained by which crates are available. The jmespath.rs crate at least fully passes the JMESPath conformance test suite.

…

On Wed, Aug 9, 2017, 5:41 PM Steve Klabnik ***@***.***> wrote: My thoughts are mostly, "I've never heard of JMESPath before and it looks interesting, but that kinda gives me pause." Hm. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABTxFnG24m1nM6k49IOE_6gYqafvGNOBks5sWieagaJpZM4OqF2p> .

[euclio]

We could consider using jq as a library. That's a bit more tried and true, but we'd be introducing a C dependency.

[euclio]

Not to sound like a broken record, but I really think that JMESPath is the right choice here.

• Well tested
• Maintained
• Simple syntax for simple cases, but very powerful for more advanced cases

If I get the go-ahead, I'll rewrite that PR.