False Positive from Hobbes Test | Community
Skip to main content
A
Anonymous
October 16, 2015
Solved

False Positive from Hobbes Test

  • October 16, 2015
  • 6 replies
  • 1438 views

Was looking at the sample Hobbes tests that come with the example project from new AEM6 eClipse tooling.

A couple of the tests navigate to a location, assert that location was reached, and finally that a component is visible on that page. That test PASSED E.g. :-

.addTestCase(new hobs.TestCase("Navigate to french page") .navigateTo("/content/aem6test29/fr.html") .asserts.location("/content/aem6test29/fr.html", true) .asserts.visible(".servicecomponent", true) );

What I then did was delete the fr.html page and re-ran the test. Sure enough it failed BUT, only because the assert relating to the servicecomponent failed and NOT the assert.location.

Curious, I removed the servicecomponent assertion so I just have this :-

.addTestCase(new hobs.TestCase("Navigate to french page") .navigateTo("/content/aem6test29/fr.html") .asserts.location("/content/aem6test29/fr.html", true) )

This test PASSED even though in the preview pane it clearly shows a 404 ??

Is the expected model for writing Hobbes tests that use assert.location always to include another assertion to verify that it has in fact been found, or is there another way of detecting the fact that assert.location itself was the problem.

Like a lot of you guys I'm sure, I write a lot of unit tests, and dealing with false positives (and negatives) is a really important aspect.

Appreciate your help

Kind Regards

Fraser.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by gcarlino

.asserts.location verifies the URL in the browser, so if the web server doesn't redirect 404s to an error URL, then .asserts.location is not the good test method.

In such a case, to confirm that you're on the page you want, another page parameter should be tested, for example the page title, or the availability of a specific DOM element.

6 replies

O
Ove_Lindström
Level 6
October 16, 2015

Technically, if the server returns 404, the location is still "/content/aem6test29/fr.html". If I was to validate a structure I think I would use the exists assertion on the <html> element in this case. But I do agree that the logical it hte node does not exist, that location would return false. Maybe the options should have an "acceptable return codes" so that if you try to validate a forwarding function, you could assert.status(30x).

/Ove

gcarlino
Adobe Employee
gcarlinoAdobe EmployeeAccepted solution
Adobe Employee
October 16, 2015

.asserts.location verifies the URL in the browser, so if the web server doesn't redirect 404s to an error URL, then .asserts.location is not the good test method.

In such a case, to confirm that you're on the page you want, another page parameter should be tested, for example the page title, or the availability of a specific DOM element.

A
Anonymous
October 16, 2015

Ove Lindström wrote...

Technically, if the server returns 404, the location is still "/content/aem6test29/fr.html". If I was to validate a structure I think I would use the exists assertion on the <html> element in this case. But I do agree that the logical it hte node does not exist, that location would return false. Maybe the options should have an "acceptable return codes" so that if you try to validate a forwarding function, you could assert.status(30x).

/Ove

 

Thanks for your reply. You are of course correct, but nonetheless, it would be nice if I could assert my intention directly rather than indirectly based on something which may or may not be present (I have always found testing for the absence of something rather weaker than something that is explicitly present .. but that's just me).

Yes, I was looking through Hobbes.js hoping to find either an option for the assert.location function I could use to assert the expected response status code, but alas none. Perhaps the guys at Adobe would be prepared to add this ?

Be nice if Hobbes had some better documentation. There are a few overview articles around but other than that it appears you are reduced to having to look through the code itself.

A
Anonymous
October 16, 2015

gcarlino44 wrote...

.asserts.location verifies the URL in the browser, so if the web server doesn't redirect 404s to an error URL, then .asserts.location is not the good test method.

In such a case, to confirm that you're on the page you want, another page parameter should be tested, for example the page title, or the availability of a specific DOM element.

 


Thanks. As Ove mentioned above, it would be nice to be able to pass an option to assert.location for the expected status response code. Testing for the absence of something I find a bit naff, but YMMV of course ... and as it stands I appear to have little choice here.

gcarlino
Adobe Employee
gcarlinoAdobe Employee
Adobe Employee
October 16, 2015

goffinf wrote...

Yes, I was looking through Hobbes.js hoping to find either an option for the assert.location function I could use to assert the expected response status code, but alas none. Perhaps the guys at Adobe would be prepared to add this ?

This is currently under development, but there is no official ETA.

goffinf wrote...

Be nice if Hobbes had some better documentation. There are a few overview articles around but other than that it appears you are reduced to having to look through the code itself.

Have you noticed these fairly new pages?

New content is still being added.

A
Anonymous
October 16, 2015

gcarlino44 wrote...

goffinf wrote...

Yes, I was looking through Hobbes.js hoping to find either an option for the assert.location function I could use to assert the expected response status code, but alas none. Perhaps the guys at Adobe would be prepared to add this ?

This is currently under development, but there is no official ETA.

goffinf wrote...

Be nice if Hobbes had some better documentation. There are a few overview articles around but other than that it appears you are reduced to having to look through the code itself.

Have you noticed these fairly new pages?

New content is still being added.

 

 

Good to know that Adobe is considering an update to Hobbes that may include the suggestion above.

Thanks for the links. I was aware of the second but not the first, which was the one I was really looking for :-)

Regards

Fraser

Terms & ConditionsAccessibility statement

Loading footer...