[resolved] Help with a Custom Element - Jest flaky test (fails without cache)

I have problem with a Jest unit test in my Open Source Aurelia-Slickgrid library (a datagrid lib), it’s a flaky test only when starting fresh or using jest --clearCache (basically no cache it always fails). When I run the 2nd pass, basicaly as soon as I have cache and I’m not clearing the cache, it always pass but I would like to make it work at all time and I need it to always pass with CircleCI that I just implemented). Note that it’s a big Custom Element, it’s a datagrid library. My main goal is to get it passing on CircleCI which always start without cache .

The error I get is
Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

Here’s a quick look at my flaky unit test (see the full test code here). Also note that I made the same library with Angular (code is ~75% the same between Aurelia & Angular) and the test is not flaky in Angular)

    import { StageComponent } from 'aurelia-testing';
    import { bootstrap } from 'aurelia-bootstrapper';
    import { PLATFORM } from 'aurelia-pal';

    describe('Aurelia-Slickgrid Custom Component', () => {
      let component;
      const view = `<aurelia-slickgrid

      beforeEach(() => {
        component = StageComponent
          .boundTo({ gridId: 'grid1', columnDefinitions: [], dataset: [], gridOptions: {}, aureliaGridReady });

        component.bootstrap((aurelia, callback) => {
          // aurelia.container.registerInstance(SlickgridConfig, new SlickgridConfig());

      // this passes fine
      it('should make sure Aurelia-Slickgrid is defined', () => {

      // this always FAIL with empty cache but works fine on 2nd run after cache is filled
      it('should create a grid and expect multiple Event Aggregator being called', async () => {
        await component.create(bootstrap);
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(1, 'onBeforeGridCreate', true);
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(2, 'onGridCreated', expect.any(Object));
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(3, 'onDataviewCreated', expect.any(Object));

        expect(eventAggregator.publish).toHaveBeenNthCalledWith(4, 'onBeforeGridDestroy', expect.any(Object));
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(5, 'onAfterGridDestroyed', true);

I also have an index.ts to load and register certain things but I don’t think it affects the flaky test… or does it?

index.ts (here is the file on GitHub)

import { PLATFORM } from 'aurelia-pal';
import { AureliaSlickgridCustomElement } from './aurelia-slickgrid';
import { SlickPaginationCustomElement } from './slick-pagination';
import { SlickgridConfig } from './slickgrid-config';
import { Filters } from './filters/index';

// expose all public classes
// aggregators, editors, formatters, services...
export * from './models/index';
export * from './formatters/index';
export * from './grouping-formatters/index';
export * from './sorters/index';

export * from './aggregators/index';
export * from './editors/index';
export * from './filter-conditions/index';
export * from './filters/index';
export * from './services/index';

export function configure(aurelia: any, callback: any) {

  // must register a transient so the container will get a new instance everytime

  const config = new SlickgridConfig();

  aurelia.container.registerInstance(SlickgridConfig, config);

  if (typeof callback === 'function') {

export {
1 Like

I guess you are not showing all the code for brevity reasons, but can you help show the relevant part related to eventAggregator, especially how it’s instantiated?

Edit: nvm, saw the tests

As you found out, the link to the full test suite is here

Doing more troubleshooting on this and I wonder if there’s an actual bug in aurelia-testing itself because if I run the debugger, it will only go in the library code only after it times out (it should go in the test code before the component.create() is called) which is a bit strange. I mean considering the tests below

it('should create a grid and expect multiple Event Aggregator being called', async () => {
    await component.create(bootstrap);

the debugger will only step in the test code only after the timeout of the async reaches 5000ms… why doesn’t it go in the test code right from the start? Also why does it always work (100% of the time) when I have cache??

Feel free to clone the repo and try it out on your side. It’s totally open source and certainly welcome any help with PRs.

Here’s a print screen of the error

1 Like

I also tried to create a new EventAggregator separately (instead of the complete lib mock) but that has the same effect, it times out on 1st run but always pass after I have cache.

  let ea = new EventAggregator();
  let component;

  beforeEach(() => {
    component = StageComponent

    component.bootstrap((aurelia, callback) => {
      aurelia.container.registerInstance(EventAggregator, ea);

  it('should create a grid and expect multiple Event Aggregator being called', async () => {
    const eaSpy = jest.spyOn(ea, 'publish');
    await component.create(bootstrap);

It times out somewhere but I can’t figure out where. The debugger doesn’t help either, I’m not sure if it’s in the Depency Injection (DI). I guess I can’t see it in the debugger because it times out within the aurelia-testing. Perhaps my lib is too complex to test, it has many DI and uses few old 3rd party libs (jQuery, jQuery-UI, …)

I’m not sure what else to try

1 Like

@ghiscoding - did you ever find any solution or remedy to this? We’re hitting this a lot now that we’re north of 1000 tests, running with --no-cache reliably reproduces the errors.

1 Like

Unfortunately no, I couldn’t find the issue and I moved my focus to other Jest Unit Tests which don’t require mounting the View/ViewModel like Services which I manually do the injection of every Service it might require, I’m out of trouble this way but I can’t test the plugin component in itself. I am north of 1000 test cases, however this issue was there from the start if I remember correctly. Does it actually works for you if you go below 1000 tests?

Even if it’s annoying, not just for me, I’m actually glad to know that I’m not the only one having issues.

1 Like

I found a workaround today:

// patient-bootstrap.ts
import { bootstrap } from 'aurelia-bootstrapper';

// tslint:disable-next-line:ban-types
export async function patientBootstrap(configure: Function) {
  jest.setTimeout(30000); // 30 second timeout

  await bootstrap(configure);


Then update all usages to:

await component.create(patientBootstrap);

The trick is just that sometimes creating the component without a cache can be slow (10-15 seconds in a few of my tests), so this will just wait for an exceptionally long time, but that only applies to the next async call, after that, it returns to 5 seconds for the rest of your test case.

1 Like

That’s a good finding, I’ll have to try it out. On a side note, there is this Jest issue that might shed some light in our problem… and while we’re on GitHub issue, there’s also this GitHub issue for Angular-Preset-Jest (not by choice but yeah my work uses Angular so…) but I wonder if this last one which mentions that it’s worst with Jest 24 might have an influence in our problem

1 Like

We’re still on 23, but I’d imagine the problem gets worse then if it’s slower!

1 Like

I just saw this post in Jest-Preset-Angular which might help

tests in my project were super slow after upgrade too, but i enabled isolatedModules in ts-jest options and they were back to normal. See this link. It might help you, but it has its costs.

if we go the ts-jest link mentioned, it says the following:

By default ts-jest uses TypeScript compiler in the context of a project (yours), with full type-checking and features. But it can also be used to compile each file separately, what TypeScript calls an ‘isolated module’. That’s what the isolatedModules option (which defaults to false ) does.
You’ll lose type-checking ability and some features such as const enum , but in the case you plan on using Jest with the cache disabled ( jest --no-cache ), your tests will then run much faster.

Since this is what we are using jest --no-cache, this might be helpful with our issue… worth the try I guess, it’s just 1 flag to enable in your Jest config

1 Like

OMG this actually works, setting isolatedModules: true in my Jest config does the trick and it finally works in CircleCI (that gave me a big boost in code coverage). I spent so many days on this problem. However as it was pointed out in that issue, which I referenced in previous post, it does have an effect on Jest version 24.x (not sure about 23.x)

Does that make any sense to you? I know you’ve commented on the issue at first, so just wondering if that rings any bell.

1 Like