feat(core): create the Weir language

[elijah-potter]

Description

The Weir programming language is a DSL for defining linters for Harper. This is a draft PR because it will take time for me to finalize everything and write up all the associated documentation. I also want to embody my own policy.

See my recent blog posts for more deteils:

• https://elijahpotter.dev/articles/building-the-weir-language
• https://elijahpotter.dev/articles/updates-on-the-weir-language
• https://elijahpotter.dev/articles/generating-weir-code-with-LLMs

Specifically, I'd like to ping @hippietrail and @ccoVeille. I suspect they'll have thoughts on syntax and direction.

How Has this Been Tested?

Extensive unit and integration testing.

Checklist

• I have performed a self-review of my own code
• I have added tests to cover my changes

[hippietrail]

You may or may not already be aware but I better point out just in case that the $ operator will result in both false positives and false negatives.

Languages have two kinds of affixes: Inflectional and derivational. Harper currently does not distinguish. So using $, like the DerivedFrom pattern will include all regular inflections of all parts of speech that apply to the word, but also none of the irregular ones.

So using it for click would cause it not just to match clicked, clicks, and clicking, but also click's and, hypothetically at least, words like clicker andreclick. Some words have prefixes and suffixes which build upon each other too. Some of these won't be correct when part of a compound or other type of expression.

And since irregular forms are not generated by affixation, if you used $ for something like give it wouldn't include gave or given.

We could of course just have more experienced contributors fix up any made by new contributors by turning them into full linters. But we should probably document it.

Now we could make this part of the affix system. By adding new annotation flags to distinguish inflectional suffixes (-s, -'s, -ing, -ed, -er when applied to an adjective, -est) from derivational suffixes (-ation, -er when applied to a verb). But this wouldn't address irregular verbs or irregular plurals.

That may be achievable via the irregular noun and verb modules, possibly with some extra functionality added, or by additions to the dictionary and annotation format, in which case we'd have to go beyond single character annotation flags as far as I can see.

Thoughts?

[elijah-potter]

There are two important things here.

First: The DerivedFrom expression does not properly match all inflections of the word. Specifically, it does not include the irregular forms. I believe your recent work makes this a solvable problem, right?

As for the right-click example. I believe that all of those derived from are still valid. Don't they all need to be hyphenated?

[hippietrail]

There are two important things here.

First: The DerivedFrom expression does not properly match all inflections of the word. Specifically, it does not include the irregular forms. I believe your recent work makes this a solvable problem, right?

You could try adding lookups into the irregular tables every time DerivedFrom is used. It sounds expensive to me but might not actually be. We could add an annotation flag for words with irregular inflection and only do the lookups when it's true. Or we could add one for verbs and one for plurals.

By the way there are other times inflected forms of words are split across multiple entries in the dictionary. They could be regular but have an ending-change pattern that's not handled by the affix code. The pattern where the last consonant is doubled is the most obvious but not the only one.

Another reason for splitting an entry is when an annotation flag didn't take into account the POS because the orthographic change is the same: "I bike->he bikes" / "one bike / two bikes". But this doesn't always hold: "one knife->two knives" / "this killer knifes his victims". So to prevent false entries in the FST dictionary we split the entry. This means not all will be connected by the 'derived' flag.

You can probably write a script to compare the full expanded FST dictionary to a curated wordlist that includes inflected forms.

As for the right-click example. I believe that all of those derived from are still valid. Don't they all need to be hyphenated?

That was a made up example but I don't think there's an accepted term "right-clicker". "right-clickable" seems sound but what about "right-unclickable"? That might not happen for "click" but is bound to happen for other words.

[hippietrail]

English copyedit.

[elijah-potter]

That was a made up example but I don't think there's an accepted term "right-clicker". "right-clickable" seems sound but what about "right-unclickable"? That might not happen for "click" but is bound to happen for other words.

You're right that those phrases are incorrect. However, the rule only requires that they are hyphenated. Since I don't think it's necessarily wrong to hyphenate those phrases (even if we should mark them as wrong), I think we can overlook it.