Au 2.0: How to add metadata to validation properties (not rules, but properties)

[estaylorco]

Hello,

I’ve decided in migrating to 2.0 to abandon my validator in favor of Aurelia’s, largely because of the state-based validation implementation, which I hadn’t implemented yet on my own validator.

But I need to be able to add a group property at the validation property level for presentational purposes (not to rules, which I know I can accomplish with tag, but to properties). Consider this code snippet from my Au 1.0 app, focusing on startGroup() and endGroup()in my fluent API:

_defineValidationRules() {
    this.validationRules = new TsiValidator()
      .virtualProperty('sufficientJobsites')
      .virtualProperty('validJobsite', 'Invalid jobsite')

      .property('templateId')
      .required()

      .done()
      .startGroup('titling', 'project titling')  // <-- Here

      .property('name')
      .required('Project name is required')
      .size(3, 100)

      .endGroup()  // <-- And here

      ...

      .done()
      .startGroup('singleJobsite', 'jobsite')  // <-- Here

      .property('description')
      .size(3, 50)

      .property('streetAddress')
      .callback((entity, value, args) => {
        return !entity.PAS.isAmbiguous;
      }, 'Address format must not be ambiguous')
       
      ...

      .endGroup() // <-- And here
}

Every validation rule defined between startGroup and endGroup picks up a “group” meta property, which is both functional and presentational. Since you already provide the functional part through tagging, I’m focusing on the presentational part in this post.

The group property allows me to group the display of the validation errors in the validation panel at the right edge of the screen, which nicely corresponds to panels out in the source view:

Is this possible with Aurelia? I haven’t found such a scenario in the documentation.

[Sayan751]

I think that you are looking for .ensureGroup.

Excerpt from the docs:

The .ensureGroup method allows you to define a validation rule that depends on the values of multiple properties at once. This is useful for cross-field validation scenarios, such as ensuring consistency between related fields.
…

validationRules
 .on(flight)
 .ensure('direction')
   .required()
 .ensureGroup(
   ['direction', 'departureDate', 'returnDate'],
   (direction: 'one-way' | 'round-trip', departureDate?: Date, returnDate?: Date) => {
     // if the direction is not yet specified, we don't have to validate anything
     if (!direction) return true;
     const $departureDate = departureDate ? new Date(departureDate) : undefined;
     const $returnDate = returnDate ? new Date(returnDate) : undefined;
     switch (direction) {
       case 'round-trip':
         return $departureDate > $returnDate
           ? { property: 'departureDate', message: 'Not possible to go back in time' }
           : true;
       case 'one-way':
         if ($departureDate < currentDate) return { property: 'departureDate', message: 'No time travel possible' };
         if ($returnDate) return { property: 'returnDate', message: 'One way flight has no return' };
         return true;
       default:
         return { property: 'direction', message: 'Invalid flight direction' };
     }
 });

For more details refer to the docs (look for .ensureGroup).

Does this satisfy your use-case?

[estaylorco]

Hello @Sayan751 ,

EDIT #1

As further explanation of my use case in terms of Aurelia’s validation package, what I’m looking for would be equivalent to this:

validationRules
  .on(project)
  .ensure('projectName')
  .tag('project titling')
...

Aurelia allows objects and rules to be tagged, but not properties.

ORIGINAL

I saw .ensureGroup in the docs, and that provides the functional aspect of grouping, which, if you take a look at my own fluent API, is covered by the first argument of .startGroup(…). I, too, with my own validator, can group (I can even cross-group and add properties to more than one group). I believe Aurelia uses the term ruleSet.

In this thread, I’m focusing on the the second argument of my .startGroup(…)method, which is a presentational group heading. I stopped on a breakpoint below to show you what my externalized validation object looks like:

As you can see, groupHeading is property-level data, not associated with messages (which are the product of applying rules).

I handle validation results a little bit differently from Aurelia. The code looks like this on my model:

validate() {
  this.validationRules?.validateAll(this);
  this.validation = this.validationRules?.externalize() ?? {};

  return this.isValid;
}

I take internal validation results and externalize them for use in the view.

[Sayan751]

Let me first validate my understanding of the problem. As far as I understand, you want to group a few properties, including all the rules that are defined for these properties, and then execute all those grouped rules at once, filtered by the group name.

For example, you have these three properties:

• ProjectName: is required and should match some pattern
• ShortName: is optional, but when specified should match some pattern and should not be more than 20 characters.
• StoreNumber: is optional and can only be numeric.

Now, you want to put these 3 properties in a group named ‘ProjectNaming’ and when you do something like controller.validate(new ValidateInstruction(project, undefined, undefined, 'ProjectNaming')) then you want all the rules defined for all the properties defined under the said group to be executed.

Is this description of the problem correct?

If my understanding is wrong, then disregard the following. However, if the problem is correctly understood, then you have two alternatives with Au2.

Firstly, the most economic option might be to still use .ensureGroup. There you can put as many properties as you like and write your own custom rules and return errors for whatever properties, in the group, you like. IMO this gives you maximum freedom in terms of defining rules, etc. When one of the properties from the group gets changed, this rule will be called.

If that does not fit your use case, then the next alternative is to roll out your own validator.

• At the first step, take a look at how the standard validator is implemented. You can copy that implementation and simply change how the rules are filtered (currently the propertyTag has no meaning if the propertyName is missing; most possibly you need to change that).
• Next, you just need to register your validator with the Au2 validation configuration. The configuration allows setting a custom ValidatorType.

And, you should be good to go.

Let here know how you proceed; I am just interested.

[estaylorco]

EDIT #1

I just realized that your documentation discusses “tags” in terms of validation, not presentation. Perhaps that’s what’s causing confusion.

I guess I don’t know how else to convey to you what I’m talking about. I think the information for the use case is in all of the posts in this thread.

ORIGINAL

So the use case under discussion in this thread is not a functional one. I’m not talking about being able to validate by group. That’s already there in the API. I’m talking about simply being able to add metadata at the property level of validation results to facilitate presentation in the view.

The groupHeading property in my previous post is simply used to influence presentation, not validation. We have to be able to group properties on objects in some presentable way.

[Sayan751]

I am assuming that you mean displaying validation errors, associated with the properties, when you talk about presentation.
Based on that assumption, I would like to point out, that if you use the @aurelia/validation-html package then displaying the errors are already taken care of. Have a look at this mini StackBlitz project that demonstrate the usage of .ensureGroup: au2-grouped-validation - StackBlitz.

[estaylorco]

Thank you for the link, @Sayan751 , but that isn’t my use case. I’m working now to prepare one.

[estaylorco]

Hello @Sayan751 ,

OK. Please see the GIF below using my own validation package:

In the VALIDATION ERRORS panel at the right, you can see the group headings grouping the errors together. “Project Titling” in the validation panel corresponds to a panel with the same heading in the form. “Jobsite,” with the “Jobsite” heading in the form, etc.

I’m pulling the groupHeading right off my validation results (${result.groupHeading}). This is the markup I use to render the validation results:

<div if.bind="model?.validationAsList?.length" class="tsi-bid-navigator-side-panel">
  <div class="tsi-validation-side-panel-heading">validation errors</div>
  <div repeat.for="result of model.validationAsList; key.bind: result.id">
    <div class="tsi-validation-error-group-heading" if.bind="result.groupHeading !== $previous?.groupHeading">
      <div class="tsi-validation-error-group-text">
        <div class="tsi-validation-bullet"></div>
        <div>${result.groupHeading}</div>
      </div>
    </div>
    <div class="tsi-validation-error-prop" if.bind="result.property !== $previous?.property">
        ${result.propertyAlias}
    </div>
    <div class="tsi-validation-error-messages">
      <div class="tsi-validation-severity-icon">
        <icon-validation-error></icon-validation-error>
      </div>
      <div class="tsi-validation-error-message" repeat.for="message of result.messages; key.bind: message.id">
        ${message.text}
      </div>
    </div>
  </div>
</div>

The validationAsList property on my model facilitates list-based rendering of validation error in the view. Also, I’m leveraging here the new $previous contextual (@dwaynecharrington ) feature made available in RC to facilitate rendering groups.

[Sayan751]

Thanks @estaylorco for the gif. It was helpful. I tried to do something similar here in the quick and dirty SB project: au2-grouped-validation-errors - StackBlitz.

Have a look.

[estaylorco]

Thank you for quick-and-dirty, @Sayan751 . It’s useful because now I think we can bridge our two understandings of this use case.

The SB project highlights the two limitations I’m facing with Au’s validation package:

• There’s no way to achieve a higher level of organization of the validation results than that of the property (we can’t associate metadata with the property itself, only its rules or the object that it’s a part of);
• Every rule has to be tagged.

With respect to first bulleted point above, if you take a look at my GIF again, the VALIDATION ERRORS panel, you’ll see the grouping bands (those light bands). The information comes directly from my validation results (groupHeading is metadata on each result).

Your SB project shows a level of grouping no higher than that of property because there’s no way for you to leverage additional information in the validation results—or metadata, if you will.

As a first pass at bridging our understandings, looking at your project, imagine if we could do this:

resolve(IValidationRules)
  .on(this.person)
  .ensure('firstName')
  .tag(this.tags[0])  // <-- Here, I'm tagging the property (unavailable right now)
  .required()  
  .matches(/[a-z]+/)

  .ensure('lastName')
  .tag(this.tags[0])
  .required()
  .matches(/[a-z]+/)

...

Can you see the subtle difference above? I’m tagging right after the .ensure and before any of the rules. Now, let’s extend this to my use case, but staying within the bounds of Aurelia’s conception of validation, to see if we can make a second pass at bridging our understandings:

resolve(IValidationRules)
  .on(this.project)
  .ensure('name')
  .tag('project titling')  // <-- not DRY, but better (not available right now)
  .required()  
  .matches(/[a-z]+/)
  .ensure('storeNumber')
  .tag('project titling')  // <-- not DRY, but better (not available right now)
  .required()
  .matches(/[a-z]+/)

  .ensure('state')
  .tag('jobsite')
  .required()
  .matches(/[a-z]+/)

...

We’re still not quite DRY above, though, since .tag has to be called for each property. So, now, let me bring in my own conception of validation using my fluent API (and a small test harness to support what you see in the GIF) as a third pass:

this.validationRules = new TsiValidator()
   .startGroupHeading('project-titling')  // <-- DRY: Called once for all properties up to .endGroupHeading() (from my validator)
      .property('name')      
      .required()
      .size(3, 100)      

      .property('storeNumber')
      .size(3, 30)
   .endGroupHeading()

   .startGroupHeading('jobsite')  // <-- DRY: Called once for all properties up to .endGroupHeading() (from my validator)
      .property('state')
      .required()
      .size(2, 2)
   .endGroupHeading()

   .done()

Does this make sense?

FEATURE DISCUSSION

If we could tag properties, not just rules and objects, we would be mostly there, but as I’ve illustrated, still not quite DRY. The startGroupHeading/endGroupHeading API’s on my validator allow me to stay DRY by grouping properties. It’s also semantically clear what I’m doing and it has parity with the display of the validation errors.

I’ve been studying my own validator, and I see where I could refactor startGroupHeading/endGroupHeading into a more general accumulation API (that’s all startGroupHeading/endGroupHeading are as a pair, an accumulator against an internal object that’s then processed by .done()).

BOTTOM LINE

We need to be able to associate metadata (tags, if you wish), with not only the object and rules, but also properties—data that allows us to organize properties, not just rules.

[bigopon]

If the desire is to add metadata/more information to a group of properties automatically, I think it’s a good addition. We probably only need to decide on the API. One thing is the current API doesn’t have end/stop, whenever we call start, we indicates an end automatically.

So what about

resolve(IValidationRules)
  .on(this.project)
  .startTag('core info')
    .ensure('name').required()
    .esnure('dob').required()
  .startTag('address')
    .ensure('email').matchs(/mighty_email_regex/)
    .ensure('line1').matches(/a-zA-Z\d/)

[estaylorco]

Last night I put together a fuller example of the use case. Over the last few days, I’ve been refactoring my validator. In the exchange I had with @Sayan751 , I noticed that I could move from purpose-built fluent API methods to a more general-purpose metadata API.

I have two projects running simultaneously: a) migration target and b) migration test target. The latter is where I am trying out both new Au 2.0 features and enhancements I’ve been wanting to make to the Au 1.0 app.

From the test target, here’s a static image of the use case with a code overlay:

and a GIF of the use case at work:

I’m using metadata to drive the following:

• Structure (view headings, group headings, etc.)
• One-off pieces of information to further clarify an error
• Dynamic visibility updates leveraging the IntersectionObserver API (so the user knows when the input associated with an error is hidden, either by collapse or by scrolling)

The Metadata API comprises the following:

• startMeta()
• endMeta()
• meta()
• setMeta()
• getMeata()

The first two control visitation on each property between them, attaching the metadata to each upon visit. The third—meta()—attaches metadata to a specific rule, which is then made available on any error messages produced by applying that rule. The last two are utility methods.

I would appreciate some feedback on this, and would love to see this API on Aurelia’s validator. If you think this is a good feature, and I get to parity with my own validator, I would love to move to Aurelia’s validator.

One of my overarching goals in moving from Au 1.0 to Au 2.0 is to embrace as much of the framework and its plugins as possible.

EDIT #1

I thought I would also show how I’m interacting with metadata in the context of intersection handling:

...

inputsIntersectionHandler(entries) {
    entries.forEach((_entry) => {
      const validationEntry = Object.values(this.model.validation).find(
        (_ve) => _ve && _ve.id === _entry.target.dataset.validationId,
      );

      if (validationEntry) {
        validationEntry.meta.isVisible = _entry.isVisible;
      }
    });
  }

_observeForValidationErrors() {
    if (!this.inputsObserver) return;
    this.inputsObserver?.disconnect();

    const $inputs = this.$el.querySelectorAll('.tsi-input-wrapper');
    Array.from($inputs).forEach((_$input) => {
      this.inputsObserver.observe(_$input);
    });
  }

...

[Sayan751]

My personal preference is to stay away from .startGroup wording to avoid confusion, since there is already a .ensureGroup. For that reason, I like @bigopon’s idea. Just like the presence of the next .ensure terminates the ruleset of the previous property, the next .startTag also does the same thing.

However, that does not work for the example of the metadata nesting. The wordings .startMetadata and .endMetadata sound odd to me, since my mental model about metadata is to attach a piece of information to a target, and there is no “start” or “end” of it. Since that example looks similar to C# using, I thought about .startScope but in the context of binding, scope has a different meaning, so that seems suboptimal as well. Can we go for

• .startMetadataScope,
• .endMetadataScope,
• .metadata?

@estaylorco My suggestion would be to open an RFC issue in GH and let the community members weigh in on the suggested API, etc., to iron out the details.

[estaylorco]

I’m not sure if your post is in response to me or to @bigopon . I say that because I never used the terms startGroup/endGroup. I did have startGroupHeading/endGroupHeading, but as I indicated in my most recent post, I saw an opportunity to refactor to a general-purpose API: a Metadata API. So I have refactored in the following way:

...
.startGroupHeading('Some Group') --> .startMeta({groupHeading: 'Some Group'})
...

As for tag, it is already used for functional purposes as a designation of a validation group. It might cause confusion to repurpose it or give it multiple meanings.

Also, I never used the terms startMetadata and endMetadata. Nor did I use the term metadata. I think those are too pedantic. meta should be sufficient.

I don’t think there is any similarity to C#’s using as that is also a functional term.

In fluent API’s, start*/end*or begin*/end* are not uncommon. I wish I could say I invented the idea, but I didn’t.

The wordings .startMetadata and .endMetadata sound odd to me, since my mental model about metadata is to attach a piece of information to a target, and there is no “start” or “end” of it

In fact, they do specify the attaching of pieces of information to targets: in this case, the properties that are the targets of validation that occur between the start and end, and the number of targets is plural.

Support for nesting is essential.

As for the term metadata, well, I mean it is what it is. If you take a look at the code overlay of my most recent post, it has an unambiguous semantic relationship to what’s manifesting in the UI. Many times I have seen other end-developers like myself, as well as library and framework designers, use the term meta or metadata for user-defined, bespoke data.

@estaylorco My suggestion would be to open an RFC issue in GH and let the community members weigh in on the suggested API, etc., to iron out the details.

I would have to leave that to the team in this case. I’m looking for feedback, yes, but not comment in the sense of determining what the terminology should be, etc., or whether metadata support is necessary. For sure it is. But however the team chooses to implement metadata (and whatever terms are used) is good enough for me so long as the API is rich enough.

The point of metadata is to allow end-developers the ability to leverage it in whatever way supports their use case. My use case is but one example, but I don’t think it’s an exotic one.