Java idempotency issue with single-line comments in the middle of a variable declaration

[greg-at-moderne]

What version of OpenRewrite are you using?

Current main = 22df20c

What is the smallest, simplest way to reproduce the problem?

    @Test
    void trailingEmptySingleLineInTheMiddle() {
        rewriteRun(
          java(
            """
              public class Test {
                private//
                int i = 436;
              }
              """
          )
        );
    }

Inspired by real-life: https://github.com/spring-projects/spring-data-jpa/blob/1d4e301195fedb573a9647b6856dd8bb34d4d5b2/spring-data-jpa/src/test/java/org/springframework/data/jpa/domain/sample/EmbeddedIdExampleEmployee.java#L34

What did you expect to see?

No change.

What did you see instead?

  public class Test {
-   private//
    int i = 436;
  }

[greg-at-moderne]

I haven't looked into this one, but I think there might be one challenge with fully supporting cases like this.
The comments might appear between various modifiers (e.g. between private and static), etc. And our LST model doesn't seem to support this.

[traceyyoshima]

I have a personal Java parser that parses the entire language spec idempotent, but it’s modeled closely to the complier tree to enable analysis that OR can’t cover.

I’ve seen this issue before. IIRC, it’s caused by the modifier parsing conditions that detect the end of a modifier.

I don’t have time now, but I can dig up the changes and you can modify them as necessary.

[traceyyoshima]

IIRC, the issue is here, and the modifier won't be matched.

I don't recall what I modified from the OR parser, but it should be close.

Parsing:

    private List<Java.Modifier> mapModifiersAndAnnotations(Map<Integer, JCTree.JCAnnotation> annotationPosTable) {
        List<RightPadded<Java.Annotation>> currentAnnotations = null;
        List<Java.Modifier> sortedModifiers = null;

        boolean inComment = false;
        boolean inMultilineComment = false;
        int afterLastModifierPosition = cursor;
        int lastAnnotationPosition = cursor;

        int keywordStartIdx = -1;
        for (int i = cursor; i < source.length(); i++) {
            if (annotationPosTable.containsKey(i)) {
                Java.Annotation annotation = convert(annotationPosTable.get(i));
                if (currentAnnotations == null) {
                    currentAnnotations = new ArrayList<>(2);
                }
                currentAnnotations.add(padRight(annotation, whitespace()));
                i = cursor -1;
                lastAnnotationPosition = cursor;
                continue;
            }

            char c = source.charAt(i);
            if (c == '/' && source.length() > i + 1) {
                char next = source.charAt(i + 1);
                if (next == '*') {
                    inMultilineComment = true;
                } else if (next == '/' && !inMultilineComment) {
                    inComment = true;
                }
            }

            if (inMultilineComment && c == '/' && source.charAt(i - 1) == '*') {
                inMultilineComment = false;
            } else if (inComment && c == '\n' || c == '\r') {
                inComment = false;
            } else if (!inMultilineComment && !inComment) {
                if (Character.isWhitespace(c)) {
                    if (keywordStartIdx != -1) {
                        Modifier matching = MODIFIER_BY_KEYWORD.get(source.substring(keywordStartIdx, i));
                        keywordStartIdx = -1;

                        if (matching == null) {
                            this.cursor = afterLastModifierPosition;
                            break;
                        } else {
                            if (sortedModifiers == null) {
                                sortedModifiers = new ArrayList<>(2);
                            }

                            sortedModifiers.add(mapModifier(matching, currentAnnotations));
                            currentAnnotations = null;
                            afterLastModifierPosition = cursor;
                        }
                    }
                } else if (keywordStartIdx == -1) {
                    keywordStartIdx = i;
                } else {
                    if (i + 1 < source.length()) {
                        char next = source.charAt(i + 1);
                        if (next == '[' || next == '@' || next == '/' || next == '<') {
                            Modifier matching = MODIFIER_BY_KEYWORD.get(source.substring(keywordStartIdx, i+1));
                            keywordStartIdx = -1;

                            if (matching == null) {
                                this.cursor = afterLastModifierPosition;
                                break;
                            } else {
                                if (sortedModifiers == null) {
                                    sortedModifiers = new ArrayList<>(2);
                                }

                                sortedModifiers.add(mapModifier(matching, currentAnnotations));
                                currentAnnotations = null;
                                afterLastModifierPosition = cursor;
                            }
                        }
                    }
                }
            }
        }
        if (ListUtils.isNullOrEmpty(sortedModifiers)) {
            cursor = lastAnnotationPosition;
        }
        return ListUtils.isNotEmpty(sortedModifiers) ? sortedModifiers : emptyList();
    }

Test:

    @Test
    void modifiers() {
        parse("""
            class Test {
                public/**/final/**/int/**/foo = 0;
            }
            """
        );
    }