Help with a Custom Element - Jest flaky test (fails without cache)

#1

I have problem with a Jest unit test in my Open Source Aurelia-Slickgrid library (a datagrid lib), it’s a flaky test only when starting fresh or using jest --clearCache (basically no cache it always fails). When I run the 2nd pass, basicaly as soon as I have cache and I’m not clearing the cache, it always pass but I would like to make it work at all time and I need it to always pass with CircleCI that I just implemented). Note that it’s a big Custom Element, it’s a datagrid library. My main goal is to get it passing on CircleCI which always start without cache .

The error I get is
Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

Here’s a quick look at my flaky unit test (see the full test code here). Also note that I made the same library with Angular (code is ~75% the same between Aurelia & Angular) and the test is not flaky in Angular)

    import { StageComponent } from 'aurelia-testing';
    import { bootstrap } from 'aurelia-bootstrapper';
    import { PLATFORM } from 'aurelia-pal';

    describe('Aurelia-Slickgrid Custom Component', () => {
      let component;
      const view = `<aurelia-slickgrid
        grid-id="grid1"
        column-definitions.bind="columnDefinitions"
        grid-options.bind="gridOptions"
        dataset.bind="dataset">
      </aurelia-slickgrid>`;

      beforeEach(() => {
        component = StageComponent
          .withResources([
            PLATFORM.moduleName('./aurelia-slickgrid'),
            PLATFORM.moduleName('./slick-pagination'),
            PLATFORM.moduleName('./value-converters/asgNumber')
          ])
          .inView(view)
          .boundTo({ gridId: 'grid1', columnDefinitions: [], dataset: [], gridOptions: {}, aureliaGridReady });

        component.bootstrap((aurelia, callback) => {
          aurelia.use.standardConfiguration();
          // aurelia.container.registerInstance(SlickgridConfig, new SlickgridConfig());
        });
      });

      // this passes fine
      it('should make sure Aurelia-Slickgrid is defined', () => {
        expect(component).toBeTruthy();
        expect(component.constructor).toBeDefined();
      });

      // this always FAIL with empty cache but works fine on 2nd run after cache is filled
      it('should create a grid and expect multiple Event Aggregator being called', async () => {
        await component.create(bootstrap);
        expect(eventAggregator.publish).toHaveBeenCalledTimes(3);
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(1, 'onBeforeGridCreate', true);
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(2, 'onGridCreated', expect.any(Object));
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(3, 'onDataviewCreated', expect.any(Object));

        component.dispose();
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(4, 'onBeforeGridDestroy', expect.any(Object));
        expect(eventAggregator.publish).toHaveBeenNthCalledWith(5, 'onAfterGridDestroyed', true);
      });
    });

I also have an index.ts to load and register certain things but I don’t think it affects the flaky test… or does it?

index.ts (here is the file on GitHub)

import { PLATFORM } from 'aurelia-pal';
import { AureliaSlickgridCustomElement } from './aurelia-slickgrid';
import { SlickPaginationCustomElement } from './slick-pagination';
import { SlickgridConfig } from './slickgrid-config';
import { Filters } from './filters/index';

// expose all public classes
// aggregators, editors, formatters, services...
export * from './models/index';
export * from './formatters/index';
export * from './grouping-formatters/index';
export * from './sorters/index';

export * from './aggregators/index';
export * from './editors/index';
export * from './filter-conditions/index';
export * from './filters/index';
export * from './services/index';

export function configure(aurelia: any, callback: any) {
  aurelia.globalResources(PLATFORM.moduleName('./aurelia-slickgrid'));
  aurelia.globalResources(PLATFORM.moduleName('./slick-pagination'));
  aurelia.globalResources(PLATFORM.moduleName('./value-converters/asgNumber'));

  // must register a transient so the container will get a new instance everytime
  aurelia.container.registerTransient(Filters.autoComplete);
  aurelia.container.registerTransient(Filters.compoundDate);
  aurelia.container.registerTransient(Filters.compoundInput);
  aurelia.container.registerTransient(Filters.compoundSlider);
  aurelia.container.registerTransient(Filters.input);
  aurelia.container.registerTransient(Filters.multipleSelect);
  aurelia.container.registerTransient(Filters.singleSelect);
  aurelia.container.registerTransient(Filters.select);
  aurelia.container.registerTransient(Filters.slider);

  const config = new SlickgridConfig();

  aurelia.container.registerInstance(SlickgridConfig, config);

  if (typeof callback === 'function') {
    callback(config);
  }
}

export {
  AureliaSlickgridCustomElement,
  SlickPaginationCustomElement,
  SlickgridConfig
};
1 Like
#2

I guess you are not showing all the code for brevity reasons, but can you help show the relevant part related to eventAggregator, especially how it’s instantiated?

Edit: nvm, saw the tests

#3

As you found out, the link to the full test suite is here

Doing more troubleshooting on this and I wonder if there’s an actual bug in aurelia-testing itself because if I run the debugger, it will only go in the library code only after it times out (it should go in the test code before the component.create() is called) which is a bit strange. I mean considering the tests below

it('should create a grid and expect multiple Event Aggregator being called', async () => {
    await component.create(bootstrap);
    expect(eventAggregator.publish).toHaveBeenCalledTimes(3);
});

the debugger will only step in the test code only after the timeout of the async reaches 5000ms… why doesn’t it go in the test code right from the start? Also why does it always work (100% of the time) when I have cache??

Feel free to clone the repo and try it out on your side. It’s totally open source and certainly welcome any help with PRs.

Here’s a print screen of the error

#4

I also tried to create a new EventAggregator separately (instead of the complete lib mock) but that has the same effect, it times out on 1st run but always pass after I have cache.

  let ea = new EventAggregator();
  let component;

  beforeEach(() => {
    component = StageComponent
      ...

    component.bootstrap((aurelia, callback) => {
      aurelia.use.standardConfiguration();
      aurelia.container.registerInstance(EventAggregator, ea);
    });
  });

  it('should create a grid and expect multiple Event Aggregator being called', async () => {
    const eaSpy = jest.spyOn(ea, 'publish');
    await component.create(bootstrap);
    expect(eaSpy).toHaveBeenCalledTimes(3);
  });

It times out somewhere but I can’t figure out where. The debugger doesn’t help either, I’m not sure if it’s in the Depency Injection (DI). I guess I can’t see it in the debugger because it times out within the aurelia-testing. Perhaps my lib is too complex to test, it has many DI and uses few old 3rd party libs (jQuery, jQuery-UI, …)

I’m not sure what else to try