How do you setup your development environment for core modules?

rluba · June 1, 2018, 7:06pm

I’ve started testing Aurelia by porting on one of our existing apps last week and really like it. I’ve submitted my first two PRs to core modules today. But developing them was a PITA.

I forked & cloned the two core repositories, ran npm install and gulp build, and npm linked them into my aurelia project. So far, so good.

But running my Aurelia app failed while these modules were symlinked because it changed how some of the modules got resolved – loading from the module’s local node_modules instead of my main project.

I had to:

Change something in one of the modules
Run gulp build
Delete (or move) the module’s node_modules to force loading from the main project.
Test change in the main project.
Re-run npm i (or re-rename node_modules)
Repeat

I doubt that’s the way core developers work. How do you set up your development environment when working on core modules to have a reasonable dev cycle?

MaximBalaganskiy · June 1, 2018, 11:57pm

Sometimes it’s just easier to plain copy files to the node_modules

EisenbergEffect · June 2, 2018, 5:36am

There’s no hiding it…it is a huge pain. Back when we started Aurelia, we thought the best way to break things apart was to have each library in a separate repo. There wasn’t really anyone doing “monorepos” at the time or any tooling to help with that. The separate repos helped in shipping the libraries independently but made it very painful to developer together, especially if there were dependencies. We are hoping to move to a monorepo for vNext at least, either using Lerna or perhaps the upcoming TypeScript project system.

In the mean time, I use a variety of custom copy solutions myself. Yes, even sometimes copying to node_modules directly. We recognize that this is a barrier to contribution, particularly in non-trivial situations where there are dependencies. We do hope to get it addressed.

Working with the router, which is where I think your contributions were mostly, is one of the more painful pieces, because it’s higher up in the stack and can often depend on changes to templating and many other pieces. I don’t end up doing a lot of cross-dependent work myself (recently). For normal work, the unit test suites that are present provide the bulk of the validation.

@bigopon Has been doing a lot of work in dependent repos and testing demos. I bet he has some great ideas on how he’s been working with this more recently.

bigopon · June 2, 2018, 6:00am

Webpack tracks not only changes from project code but also code in node_modules. I normally base my workflow on this, by putting framework repos and test app created from thencli together in same level inside a folder, then i can adjust build script of framework repos to put it in node modules of the test app. Planning some POC sometimes is simpler than that, just edit code directly inside node modules of the test app.

huochunpeng · June 2, 2018, 12:25pm

I have a trick I just started using in few days.

With cli+requirejs or cli+system.js, it supports so called “custom package” which allows you to use an arbitrary folder as npm package. The feature was not designed for this usage, but it’s flexible enough to support this.

For instance, to use my local aurelia-task-queue,

{
  "name": "aurelia-task-queue",
  // the relative path starts at my-app/src
  // use .../../repo to reference sibling folder repo/
  "path": "../../task-queue/dist/amd",
  // you need to specify packageRoot for custom package
  "packageRoot": "../../task-queue/dist/amd",
  "main": "aurelia-task-queue"
},

This trick totally bypasses local node_modules, npm, or yarn. It doesn’t matter how npm/yarn installs dependencies. You can even reference a package which is not in your explicit dependencies in package.json.

If you need to test aurelia-router with a special version of aurelia-history-browser and aurelia-templating-router, just do the same thing for all the three packages.

This trick is much reliable and easier than npm link or manual copy to node_modules folder. You can keep developing your local aurelia-core-modules as you normally do.

The only trouble I had so far is to enable “–watch”, I only get to a point to let the compiled ../task-queue/dist/amd/**.*js triggers app build. It re-builds, but cli doesn’t replace the existing bundled task-queue with new code. I probably need to enhance cli itself in order to support this usage.

fkleuver · June 2, 2018, 12:51pm

Planning some POC sometimes is simpler than that, just edit code directly inside node modules of the test app.

This is what I ended up doing as well when I was trying things out in templating-resources or templating-router. Trying to link things up doesn’t feel like a very fruitful way to spend time when you can edit the same code in node_modules. It feels hacky, but honestly it’s very simple and effective. Of course this only applies when the transpiled output is very similar to the original source.

After “POC mode” when it’s time to put the code in the original source and run some tests on it, I run npm pack and install the local tarball to see if it still plays nicely with the rest.

I’m not particularly fond of Lerna. It’s very difficult to get right, especially with TypeScript. Looking forward to the TypeScript project system…

rluba · June 2, 2018, 6:23pm

Hacking in compiled output files is definitely not how I’d like to develop anything. Everything else sounds sub-optimal as well.

It’s probably not healthy for Aurelia (as an ecosystem) if there’s no builtin, simple, and fast “edit-save-try” cycle for developing and fixing core modules.

Let’s see if I can cook up a better solution…

timfish · June 2, 2018, 7:40pm

We’ve got a TypeScript app in a mono-repository and have the build running as per the dependency tree. None of solutions I’ve found appear to fix these issues:

watch mode doesn’t correctly rebuild dependencies
Go to definition across modules takes you to d.ts. definitions rather than the actual source
Renaming is not project wide
Import auto completion resolves to the actual source file rather than the module ('../../../../packages/@scope/etc/some-file' rather than '@scope/etc')

The TypeScript project system that Rob refers to should hopefully fix all of the above!

rluba · June 2, 2018, 8:45pm

I’ve created a pull-request for aurelia-router that demonstrates what I’d suggest:

Run Webpack in watch mode on your project and open it in any browser.
Open another shell tab, go to your aurelia-router directory, and run gulp dev --target <path_to_your_project>
Make changes to aurelia-router
Refresh the browser.
Test your changes.
Go to 3.

Should work with any number of Aurelia modules in parallel and enables a very simple and fast “edit-save-try” development cycle.

Let me know what you think!

ben-girardet · June 4, 2018, 10:04pm

This article has helped me setup an environment for developing non-core modules (haven’t tried with core modules):

My exemple features a plugin I’m working on which is called aurelia-resources. My folders are like this:

.../aurelia-resources <- with my plugin
.../a-webpack-aurelia-app <- with an app created with CLI

From the webpack-aurelia-app I do:

npm link ../aurelia-resources.

Out of the box this doesn’t work. But following the advices in the mentioned article, I’ve edited the webpack.config.jsfile like so:

module.exports = ({production, server, extractCss, coverage, analyze} = {}) => ({
  resolve: {
    extensions: ['.ts', '.js'],
    modules: [srcDir, 'node_modules'],
    symlinks: false,
    alias: {
      '@aurelia-ux/core': path.resolve('./node_modules/@aurelia-ux/core'),
      'aurelia-binding': path.resolve('./node_modules/aurelia-binding'),
      'aurelia-bootstraper': path.resolve('./node_modules/aurelia-bootstraper'),
      'aurelia-dependency-injection': path.resolve('./node_modules/aurelia-dependency-injection'),
      'aurelia-event-aggregator': path.resolve('./node_modules/aurelia-event-aggregator'),
      'aurelia-fetch-client': path.resolve('./node_modules/aurelia-fetch-client'),
      'aurelia-framework': path.resolve('./node_modules/aurelia-framework'),
      'aurelia-history': path.resolve('./node_modules/aurelia-history'),
      'aurelia-history-browser': path.resolve('./node_modules/aurelia-history-browser'),
      'aurelia-loader': path.resolve('./node_modules/aurelia-loader'),
      'aurelia-loader-default': path.resolve('./node_modules/aurelia-loader-default'),
      'aurelia-loader-nodejs': path.resolve('./node_modules/aurelia-loader-nodejs'),
      'aurelia-logging': path.resolve('./node_modules/aurelia-logging'),
      'aurelia-logging-console': path.resolve('./node_modules/aurelia-logging-console'),
      'aurelia-metadata': path.resolve('./node_modules/aurelia-metadata'),
      'aurelia-pal': path.resolve('./node_modules/aurelia-pal'),
      'aurelia-pal-browser': path.resolve('./node_modules/aurelia-pal-browser'),
      'aurelia-pal-nodejs': path.resolve('./node_modules/aurelia-pal-nodejs'),
      'aurelia-path': path.resolve('./node_modules/aurelia-path'),
      'aurelia-polyfills': path.resolve('./node_modules/aurelia-polyfills'),
      'aurelia-route-recognizer': path.resolve('./node_modules/aurelia-route-recognizer'),
      'aurelia-router': path.resolve('./node_modules/aurelia-router'),
      'aurelia-task-queue': path.resolve('./node_modules/aurelia-task-queue'),
      'aurelia-templating': path.resolve('./node_modules/aurelia-templating'),
      'aurelia-templating-binding': path.resolve('./node_modules/aurelia-templating-binding'),
      'aurelia-templating-resources': path.resolve('./node_modules/aurelia-templating-resources'),
      'aurelia-testing': path.resolve('./node_modules/aurelia-testing')
    }
  },

with the symlinks: false that ensure the link with npm link and alias: {...} properties which helps webpack resolve the concurrent dependencies from the right directory.

The pain is to write all the correct aliases. But once this is done, any changes in the plugin is reflected (in watch mode) inside the app and development becomes a lot more enjoyable.

rluba · June 5, 2018, 12:40pm

Thanks for the info, @ben-girardet. It sounds like a pain to set up, but might be worth it if you develop your own module/plugin over a longer period.

I think the proposed gulp build --target <dir> works better for replacing a few modules with local versions whenever you need to develop features or fixes for those modules. It’s easy to set up and doesn’t require any changes to the projects.

huochunpeng · June 6, 2018, 11:59pm

Guys, introduced by @graycrow in Aurelia-cli (RequireJS): move node_modules to upper directory

It looks like yarn workspaces is a great fit for core modules development.

Keep core modules repos and dev app repo under same workspaces, it will ensure the dev app uses your local core modules code.

This works if your dev app uses webpack. cli+requirejs has issue to support parent folder node_modules currently (this issue is now fixed in auto-tracing version of cli) .

rluba · June 13, 2018, 11:42am

yarn workspaces sound very interesting for more permanent setups, i.e., regularly working on a set of modules. I’ll keep it in mind!

But I’d prefer my --target suggestion for “there’s something wrong with this or that module module and I wanna test my changes before submitting a PR”. The biggest drawback is that it requires support by the module (becaues it must offer the ---target option. Maybe there’s a better solution…

Topic		Replies	Views
Library module in Aurelia? Framework Knowledge	5	959	July 31, 2019
Lernajs/yarn workspaces with an aurelia application	49	3083	October 9, 2020
Aurelia-cli (RequireJS): move node_modules to upper directory Help Requests	6	1008	June 12, 2018
Add modules to aurelia-core.js Help Requests	1	637	April 17, 2018
Webpack - where to begin? Help Requests	2	806	November 15, 2018

How do you setup your development environment for core modules?

Related Topics