Test is passing without running when putting ".only" on "it"

[benbracha]

Hey,

Since mocha 2.4.1 it seems like test code doesn't wait for page to sync anymore.
Meaning, this was working in the past:

it.only('mocha test', function() {
        var addAppBtn = $('[debugId="create-new-app-btn"]');
        addAppBtn.click();      

        console.log('HERE'); 
 });

But since mocha 2.4.1 the click doesn't happen, and test is passed (console log is shown).

Since mocha 2.4.1 we must use callbacks on the promises, which make tests writing a lot harder and complicated:

it.only('mocha test', function(done) {
      var addAppBtn = $('[debugId="create-new-app-btn"]');
      addAppBtn.click().then(function() {
           // Do something
           done();
       }
});

I opened an issue for mocha, but I think it should get some help / attention:
mochajs/mocha#2157

Thanks

Bug report

• Node Version: 0.10.29
• Protractor Version: 1.5.0
• Browser(s): Chrome 49
• Operating System and Version OS X 10.11.1

[sjelin]

Hey, I downloaded your testapp but I'm not reproducing the problem. Could you clone a fresh one and try it again?

[benbracha]

It happens only when putting .only on the " it", and then running the test.
Note that no other .only should be there.
Did you try this one ?
On Mar 21, 2016 23:02, "Sammy Jelin" notifications@github.com wrote:

Hey, I downloaded your testapp but I'm not reproducing the problem. Could
you clone a fresh one and try it again?

—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
#3045 (comment)

[sjelin]

Yes I added only to the it and still no problem. It fails where it should

[benbracha]

Got it.
I'll check it tomorrow and will update.
Please share your protractor, web driver and node versions just in case.
On Mar 21, 2016 23:12, "Sammy Jelin" notifications@github.com wrote:

Yes I added only to the it and still no problem. It fails where it should

—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
#3045 (comment)

[sjelin]

I'm on 3.1.1, selenium standalone version 2.52.0, chrome version 2.21

[benbracha]

Tried on a clean copy - it is reproduced.
I compared the versions, I used protractor 2.1.15.
I updated protractor to 3.1.1 and selenium standalone to 2.51 (chrome 2.21).
But now I cannot run the test, I get the following exception:

[launcher] Process exited with error code 1

/usr/local/lib/node_modules/protractor/node_modules/q/q.js:155
throw e;
^
SyntaxError: Unexpected token )
at goog.loadModuleFromSource_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:1123:19)
at Object.goog.loadModule (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:1085:46)
at /usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1:6
at Object.Context.closure.goog.retrieveAndExecModule_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/base.js:135:8)
at :1:6
at Context.closure.closure.vm.createContext.CLOSURE_IMPORT_SCRIPT (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/base.js:104:12)
at Object.goog.importScript (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:879:9)
at Object.goog.importModule (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:900:14)
at Object.goog.writeScripts_ (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:1323:16)
at Object.goog.require (/usr/local/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:663:14)

Any idea?

[benbracha]

Ok - it seems an issue with node (I'm using 0.10.29), I'll upgrade it.
But I'm not sure we can move to protractor 3, there are breaking changes I think since 2.
Can you check it up with protractor 2?

[benbracha]

I upgraded node (5.5.0) and protractor (3.1.1) and it still happens (running on mac, BTW).

[benbracha]

After doing a re-upgrade of all elements, the issue stopped happening.
But then I came to back to step one: works with mocha 2.0.0 but does't work (test passed without running) with mocha 2.4.1.

Now that we are with the same (latest) of node, protractor, webdriver and chrome - try to change mocha version on the package.json file and see if it reproduced for you.

[sjelin]

Ok, I'm gonna switch to OSX and see what happens

[sjelin]

Still cannot reproduce on OSX, mocha 2.4.1, node 5, etc

[NicholasTuck]

I also am having this same issues (Windows 10). It "runs" the test, but never fails the assertion. Using mocha, chai, dirty chai, and chai as promised.

Node version: 4.1.0
npm version: 2.14.3

"protractor": "^3.1.1",
"chai": "^3.5.0",
"chai-as-promised": "^5.2.0",
"dirty-chai": "^1.2.2",
"mocha": "^2.4.5",

[sjelin]

@cnishina Can you reproduce this issue? @benbracha's test app is at: https://github.com/benbracha/protractor-test-app

[benbracha]

Hello,
Any progress (@cnishina, @sjelin ?)

[cnishina]

I was able to get this into a weird state where webdriver-manager logs were quitting the session...

16:05:33.171 INFO - Done: [delete session: 215d3d25-8832-4f3f-880d-7b778c1b65a6]
16:05:34.090 WARN - Exception thrown
org.openqa.selenium.remote.SessionNotFoundException: Session ID is null. Using WebDriver after calling quit()?

but now that I am trying again, I can no longer reproduce this. @benbracha when this error is happening, what is going on with webdriver-manager logs?

[benbracha]

@cnishina , thanks for checking this up.
I'll need to re-setup and get back to you.
Can you specify the different versions you are using? (protractor, webdriver-manager, mocha..)

[benbracha]

@cnishina I can still reproduce it, and the webdriver-manager logs doesn't shout anything strange..
Here it is:

12:11:11.473 INFO - Launching a standalone server
Setting system property webdriver.chrome.driver to /usr/local/lib/node_modules/webdriver-manager/selenium/chromedriver
12:11:11.636 INFO - Java: Oracle Corporation 25.65-b01
12:11:11.637 INFO - OS: Mac OS X 10.11.1 x86_64
12:11:11.709 INFO - v2.45.0, with Core v2.45.0. Built from revision 5017cb8
12:11:12.032 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{ensureCleanSession=true, browserName=internet explorer, version=, platform=WINDOWS}] does not match with current platform: MAC
12:11:12.230 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
12:11:12.235 INFO - Version Jetty/5.1.x
12:11:12.250 INFO - Started HttpContext[/selenium-server,/selenium-server]
12:11:12.420 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@16c0663d
12:11:12.420 INFO - Started HttpContext[/wd,/wd]
12:11:12.420 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
12:11:12.421 INFO - Started HttpContext[/,/]
12:11:12.460 INFO - Started SocketListener on 0.0.0.0:4444
12:11:12.460 INFO - Started org.openqa.jetty.jetty.Server@30dae81
12:11:20.707 INFO - Executing: [new session: Capabilities [{count=1, browserName=chrome, platform=MAC}]])
12:11:20.722 INFO - Creating a new session for Capabilities [{count=1, browserName=chrome, platform=MAC}]
Starting ChromeDriver 2.15.322455 (ae8db840dac8d0c453355d3d922c91adfb61df8f) on port 7375
Only local connections are allowed.
12:11:23.476 INFO - Done: [new session: Capabilities [{count=1, browserName=chrome, platform=MAC}]]
12:11:23.495 INFO - Executing: [set script timeoutt: 11000])
12:11:23.503 INFO - Done: [set script timeoutt: 11000]
12:11:23.712 INFO - Executing: [get: data:text/html,<html></html>])
12:11:23.718 INFO - Executing: [delete session: 68841838-9f23-4371-9d81-79cca0aee6f8])
12:11:23.753 INFO - Done: [get: data:text/html,<html></html>]
12:11:24.396 INFO - Done: [delete session: 68841838-9f23-4371-9d81-79cca0aee6f8]

[noisette44]

Hi,

I have same issues with it.only since mocha 2.4.1.

Protractor version : 3.3.0
Node version: 5.5.0
npm version: 3.5.3

I updated original mocha issue : mochajs/mocha#2157

[noisette44]

Before mocha 2.4.1 "it.only" function used "global.it" (protractor/lib/frameworks/mocha.js), so wrapping "global.it.only" was not necessary.

But since mocha 2.4.1, "it.only" function no longer use "global.it" but the original "it" function (lib/interfaces/bdd.js - commit with change), so now wrapping "global.it.only" is necessary .

[benbracha]

Thanks @noisette44 for the followup. I hope protractor guys will pick that soon..

[ScottFreeCode]

I've been able to put together a minimal test case and fix for this: gist. Note that this does not require only to be added (it's already there), and will fail if the test incorrectly does not wait for the asynchronous browser code and pass if it correctly does wait (i.e. passing of this test indicates the problem has been resolved, failure of this test indicates the problem still exists, instead of the other way around).

The basic idea for the fix is to remove the line that sets global.it.only to the original only, instead letting Selenium's wrapped version stay there and do its thing -- Selenium's wrapped version works correctly, whereas the original version bypasses the wrapped it since mochajs/mocha#1964 / mochajs/mocha#1965 .

The test should be able to be adapted as a regression test, but will have to be run separately from other tests because it has to call only in order to test it (and would thus stop all other tests from running).

I also tried downgrading Mocha to 1.x.x and running with the fixed version, and the double-wrapping inherent in using the wrapped it.only with the old Mocha it.only implementation (that delegates to the global it) didn't seem to cause any issues, contrary to the comment in the Protractor source.

[heathkit]

This kind of relates to that other thing I just did with Jasmine, so I'm going to take this up.

Thanks for the detailed repro and test! We'll fix this for the next release.