The Aurelia Core Team is Looking for an e2e Testing Lead

1

Greetings Aurelia Community!

Hopefully you’ve been able to stay up to date on some of the amazing progress we’ve been making towards the next major version of Aurelia. We’re pretty excited about how things are turning out. One of the quality goals we have for vNext is to make it the most stable, tested, and verified framework available today. To that end, we’re focusing a lot on our testing. There are a few different aspects to testing that we’re tracking: unit and integration tests, e2e tests, and performance benchmarks

For unit and integration tests, we’re very proud to say that Aurelia vNext currently has 92% test coverage and over 92,000 unit and integration tests. Wow!!! We’re doing pretty well on this front and we expect to continue in the same direction. Huge thanks to @fkleuver for casting a vision and making it happen!

For e2e tests, we’ve got a few basic pieces in place. However, we need to do much more. To do this right, we’d like to add a new core team member who is passionate about e2e testing and can focus on helping us take our e2e testing to the next level. Here’s what we’d like to accomplish:

  • Full test coverage on any API surface that deals with user interaction (so from-view and two-way binding with all kinds of form elements, event handling, etc.)
  • Full test coverage on the various combinations of navigation scenarios for the router
  • Some coverage on all other resources, renderers and binding commands
  • Preferably (but this is not an urgent goal right now) have these tests run in all major browsers, including IE11 (with polyfills where appropriate)

This is an important task for vNext, but fortunately it’s one that can be done very incrementally, by adding a few new tests each week. The value here is in the critical mass that builds over time from consistent contribution in this area.

If you have some expertise and passion in this area, please drop a message here and let’s start discussing how you can get involved. We’re open to a variety of solutions to accomplish this.

The third area of testing is benchmarking. Fortunately, the industry now has a number of standard benchmarks that we can use, including: db-monster, krausest and uibench. We’re looking for someone to implement these benchmarks for Aurelia vNext and vCurrent, and help us get them integrated into our CI process so that we can track performance improvements and regressions over time.

If you’d like to help Aurelia vNext be the fastest framework today, this is a great way to contribute, even if you don’t have a ton of experience doing performance optimization itself. Having these benchmarks in place will help us market the excellent performance of Aurelia vNext and provide the concrete data that other Aurelia team members need to ensure Aurelia stays at the top.

8 Likes
3

Hello,

I’ll be happy to help i already use Aurelia in two biggest application, but i don’t know if i have (can) the knowledge for e2e testing.

so if you think I can help I’m ready.

thank you

4 Likes
4

From what I see, you don’t have to do all the things, you can help just by throwing in the kitchen sink app features of Aurelia you would like to be tested, it’s like everyone writing an application together to help ensure Aurelia runs smoothly.

5

As @bigopon pointed out, anyone can help simply by providing test cases. The weirder, unusual and complex usages of the APIs, the better.
Of course, there needs to be a place for those samples/cases to be submitted first.

We are currently using BrowserStack for cross-browser testing, combined with WDIO, and also WDIO standalone for some simple spike tests with the different bundlers. There’s also an older setup with some basic benchmarks built on selenium-webdriver.

None of these necessarily need to be used if someone has a better suggestion or is simply more knowledgeable about a different tool.

The main thing you’d need to be familiar with is the Page Object Pattern. That in itself would go a long way.

2 Likes
6

@mabouchalkha I think I speak on behalf of everyone when I say any help is good help and greatly appreciated. I have put my hands up to help with the testing front and if you wanted to jump in and help out, many hands make light work.

Even if you don’t have much e2e knowledge, you’ll find once you have a few tests you can copy as a starting point, you’ll pick it up pretty quickly anyways. Admittedly, I am still learning e2e tests the more I write them as well.

2 Likes
7

I don’t see myself being a lead in this but I definitely could help out (creating a test/tests).

Does vnext already include a e2e testing framework? If not I would suggest looking at testcafe, in contrary to most other tooling out there it does not rely on selenium. It’s node based and supports multiple browsers (I have to say I like Cypress better due to debugging info but it’s limited to chrome only so not an option for vnext I guess).

If you would like to enable people to easily pitch in writing scenario’s only it could be a good idea to have a look at a cucumber implementation to enable writing features in gherkin format.

1 Like
8

So far it looks like the team have decided on Cypress which also doesn’t rely on selenium. I do like the way TestCafe has been designed to work via WebSockets, its just disappointing about the syntax. It would have been nice if they’d made it a drop in replacement for webdriver. It certainly would have helped adoption!

9

We had a lot of people with positive experience with Cyprus, so we decided to go with that as a long-term solution, even though they only support Chrome. We checked into their product plan and it seems they are currently working on other browser support, which we’ll leverage as soon as it’s available.

We’d love to get you guys involved in helping with the writing of tests. @dwaynecharrington to facilitate others getting involved in the e2e effort can you author a markdown file for our engineering documentation that explains how to get setup, run, and write tests, where to put tests, etc.? Here’s the place to put it: https://github.com/aurelia/aurelia/tree/master/docs/engineering There’s already a testing strategy doc, but I don’t wan to overload that. So, maybe write a new doc and then link to it from a small section on e2e testing in the overall strategy doc.

1 Like