x/tools/go/analysis/analysistest: buggy // want behavior with block comments

[Groxx]

Go version

go version go1.21.1 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/groxx/.cache/go-build'
GOENV='/home/groxx/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/groxx/code/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/groxx/code/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go-1.21'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go-1.21/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/groxx/code/go/src/github.com/Groxx/analysisbug/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2410812203=/tmp/go-build -gno-record-gcc-switches'

golang.org/x/tools version

require golang.org/x/tools v0.16.1

What did you do?

I wrote an analysistest testdata containing a file like this:

package whatever

/*
text // want "anything"

a comment // want "comment"
*/
func x(){}

with a very simple analysistest.Run like this:

func TestIt(t *testing.T) {
	analysistest.Run(t,
		analysistest.TestData(),
		&analysis.Analyzer{
			Name: "test",
			Doc:  "test",
			Run: func(pass *analysis.Pass) (interface{}, error) {
				return nil, nil
			},
		},
		"analysis")
}

What did you see happen?

That test fails with an error like this:

analysistest.go:415: analysis/main.go:3: in 'want' comment: got Ident after a, want ':'

while also missing the lack of report on the other // want "comment" further down.

Which is very simply reproduced here: Groxx/analysisbug@4178723

I'm not sure how to build that into go.dev/play, but I'd be willing to try if someone can point me to a general strategy / an example analysistest with a testdata folder (or writing one during the test).

---

Other block-comment structures seem to produce a variety of behaviors like this:

1. unexpected diagnostic: anything where // want "anything" exists
2. failure to detect the // want "comment" / not noticing the missing report
3. diagnostic "anything" does not match pattern "anything" where // want "anything" exists
4. in 'want' comment: got Ident after [the next word in the comment], want ':' either on the comment's line or sometimes a different nearby line

The single-line // comment variants of all of these work as expected, as do all single-line /* comment // want "something" */ comments. It seems to be just multi-line blocks causing issues.

2 and 4 are reproduced in my sample repository at this (initial) commit: Groxx/analysisbug@4178723

The other two I see in a larger project where I'm less confident about what's my flaw and what's a // want issue, and I have not yet managed to reproduce them in a small test.

I kinda suspect they'll be fixed in the process of finding and fixing 2 and 4 though so I haven't sunk much energy into that yet. I can push the (extremely messy) state of that repository if it seems actually useful though.

What did you expect to see?

// want comments should work the same in both multi-line /* comments */ and // comment lines.

[dmitshur]

CC @timothy-king, @zpavlinovic.

[timothy-king]

This is at the moment WAI and not a bug.

Multiline expectations in Go code are currently unsupported by analysistest. In particular, there is an implementation limitation about which line one is putting the expectation on. What is the line number for // want "anything" and // want "comment" in the example? The line c.Pos() (the /*) is on? The line the want is on? The line x() is on. At the moment these are attached to /*. So only the first want line can make sense at the moment.

But perhaps we want to be able to attach to multiple points in a comment block. Theses lines are all in the middle of a comment block. So what can they match? They can match within the comment I guess. So comment Diagnostics. Not clear that we have a user need to support this. The users of analysistest are limited.

If you have a use for this, please submit a CL to iterate over c.Text line by line and include a unit test (this will require a test analyzer that creates expectations on comments). I can review this. Alternatively, you can open a new issue for a feature request.

I am going to close this issue as I don't think there is a bug.

analysistest.go:415: analysis/main.go:3: in 'want' comment: got Ident after a, want ':'

While quite obscure this complaining that it is seeing something roughly equivalent to the following:

// text // want "anything" a comment // want "comment"

This expects a ":" after a as in a:"expectation". If you wish to contribute a better error message, I can also review this.