QA: "cascade" enabling and flake module exports

[shimeoki]

hello. first of all, i am pretty new in nix(os), so my understanding is a bit shaky sometimes. and sorry for the longread in advance.

i found your project while searching for ways to automate the process of importing modules. and your project looks amazing! it's like everything i wanted in my head.

before i found it, i even wanted to start implementing something like this by myself to comply with DRY principle, because modular configuration with options becomes boilerplate'y. and now i see no reason to. thank you.

so, i started to migrate my configuration. right now i have a structure for apps only: no "feature" sets like "cli", "gui", "development", etc..

aside from that, i have a single pattern: every module's enable option is dependent on parent's enable option. if shimeoki is enabled, then all shimeoki.XXX modules are enabled, if shimeoki.nixvim is enabled, then shimeoki.nixvim.plugins or shimeoki.nixvim.languages are enabled and so on.

i planned for my configuration to be used internally and externally. for my own usage, i can write

shimeoki = {
    enable = true;
    waybar.enable = false;
};

to enable everything, except for waybar. and other users can write

shimeoki = {
    waybar.enable = true;
};

to enable waybar only.

i think it's pretty convenient. so, is this pattern possible in denix?

i tried to write

options = { myconfig, ... }: delib.singleEnableOption myconfig.enable;

but it errors, because "an option declaration has type lambda". i think that's because singleEnableOption is a function itself, so it becomes a closure.

of course, a declaration like this works:

options =
    { myconfig, ... }:
    {
        zoxide = with delib; {
            enable = boolOption myconfig.enable;
        };
    };

but it's pretty verbose and not very convenient. something like this

options.zoxide =
    { myconfig, ... }:
    with delib;
    {
        enable = boolOption myconfig.enable;
    };

doesn't work, because the value assigned to options needs to be a lambda, and not to any of the values "below" options.

and besides from the verbosity, you need to write myconfig with full path to the current module. in case of a refactoring, this becomes a nightmare.

i think something like "relative paths", but for the modules would be amazing. my idea is to add a parent variable to the closure of options, which equals to "parent attribute set" of the current module name.

for example, the example above with more nested module would look like this:

options =
    { parent, ... }:
    {
        nixvim.plugins.lualine = with delib; {
            enable = boolOption parent.enable; # parent equals to nixvim.plugins
        };
    };

and to go even further, this could be translated into the wrapper function like singleEnableOption:

options = delib.singleCascadeEnableOption { };

or singleEnableCascadeOption. naming is debatable. it shouldn't have an argument, but as i know all functions in nix should have one argument, and i don't know if this could be a non-function to "hide" the argument.

the issue is, that i don't know is this even implementable (and if so, i would like to help), and is this against your philosophy or not. maybe, this can be done as an extension.

that was the first problem. it was just about providing a convenient interface to functionality that's already available, but i don't know about the second problem in this regard.

as mentioned above, i want to allow other users to import my modules into their configuration to use them. so, i do something like this in my flake:

nixosModules.shimeoki = ./shimeoki/nixos.nix;
nixosModules.default = self.nixosModules.shimeoki;

homeModules.shimeoki = ./shimeoki/hm.nix;
homeModules.default = self.homeModules.shimeoki;

where ./shimeoki/nixos.nix imports all <app>/nixos modules. these modules contain only nixos options. ./shimeoki/hm.nix works the same way, but declares home-manager options. the code can be viewed in the repository.

and, as i think it should work, the user adds my flake to their flake in the inputs, and then writes something like this:

{ inputs, ... }:
{
    imports = [
        inputs.nixconfig.nixosModules.shimeoki
    ];

    shimeoki.enable = true; # enable all nixos options from my modules
}

so, is this possible in denix? as i got it, it uses the "monomodule" approach, where delib.module contains the configuration for all platforms at once, and activates it based on the platform. if the platform is not nixos, then nixos is skipped, and the same applies to darwin and home. so, modules are not available to me, because they are "generated" dynamically based on the platform. is that right?

with this approach, is this possible to achieve similar functionality to mine? that the user can import modules for the selected platform from my flake and use them, i mean.

i haven't looked into the code yet, but if denix.lib.configurations uses "module generation based on platform" logic inside and then passes it to nixpkgs.lib.nixosSystem in modules, then the api can look something like this:

with denix.lib; {
    nixosModules.shimeoki = modules "nixos";
    homeModules.shimeoki = modules "home";
};

the issue with this approach that the nixos module in this case would export the home-manager modules as well (because if the platform is nixos, home platform is not disabled, but config is just declared in a different way), but i personally don't think it's a problem. it's probably weird to just import the nixos module without the home configuration, and many flakes do this as well (import home-manager module is nixos module is imported and home-manager is present).

[yunfachi]

Hello!

Regarding your first issue, the main thing I want to point out is that you used delib.singleEnableOption incorrectly. The reason is that delib.singleEnableOption is a function with 2 arguments: the first one is the default value, and the second is the function arguments, which are automatically provided to the values of delib.module :: options:

options = { myconfig, ... } @ args: delib.singleEnableOption myconfig.enable args;

However, of course, this doesn't look very clean and feels too boilerplate like, so the better solution would be to add a new template function similar to singleEnableOption. What do you think of such a solution?

options = singleCascadeEnableOption;

As for your second issue, it's true that Denix currently doesn't have a convenient way to create nixosModules, homeModules, darwinModules, and so on. However, I'm already working on a rewrite of Denix where this will be implemented as one of the key features.

[shimeoki]

oh, i understand now. using delib.singleEnableOption this way works. thanks.

and it's a pretty long message once again, so i tried to divide it into sections.

while thinking about this, i got many ideas and i described them here. if you like them and approve them, i would be happy to contribute to the project. i think most of them are not that hard to implement (at least, i hope so).

singleCascadeEnableOption

as for singleCascadeEnableOption, it looks perfect? if you like this naming, i don't have any objections. i don't know if this can be any more minimal.

i forgot that the argument to the function is not required to be provided at the user level and just could be passed, so i included the { } :)

parent argument

but, to address my initial idea, i think it's better to include both approaches: to provide a wrapper and a parent argument to the options (or other fiels such as nixos and home) as well.

naming is debatable, but i think it's pretty short, concise and self-explanatory.

it allows for manual granual control. one use case that first came to my mind is a "networking" module example.

# networking/default.nix
{ delib, ... }:
delib.module {
	name = "networking";

	options.networking = with delib; {
		# define names of the devices to allow connections to
		# (arbitrary example)
		devices = listOfOption str [ ];
	};
}

# networking/wifi.nix
{ delib, ... }:
delib.module {
	name = "networking.wifi";

	options = { parent, ... }:
	{
		networking.wifi = with delib; {
			# use devices from the parent as the default
			devices = listOfOption str parent.devices;
		};
	};
}

if the wrapper would parse the name of the module already, it could be provided to the user on more "upper" level, and the wrapper would just use this variable to make the concrete implementation of the wrapper simpler.

module name with dots

also, as a side question, is using dots for the module name is supported? using as in "raw values for the names and not as a nesting indicator".

example:

{ delib, ... }:
delib.module {
	name = ''wallpapers."swww.d"'';
}

where swww.d is not nested swww and then d, but as a single name.

even if it's not, i don't think it's crucial. it's the reverse: i think this should be discouraged and not allowed.

just thought it's still worth to mention it. if parent argument could be implemented, it's probably will be done via splitting the name with dots. if dots are supported, that is much more difficult.

delib.options wrapper

example for parent argument also shows some problems with the current approach.

as shown above, everything is perfect if the options declaration is not a lambda: the "path" can be defined as options.networking.wifi. but if arguments are needed (such as myconfig, cfg or proposed parent), then it's required to assign a lambda to options specifically. then you type the name of the module once again and with an additional indentation write the options' themselves.

that feels wrong to me. i think you almost never want to define options not of the module itself, so a helper function like delib.options (naming is debatable as well), which assigns options under the attribute set as the first argument to the name of the module automatically, would be amazing. usage example:

{ delib, ... }:
delib.module {
	name = "pretty.long.module.name";

	options = with delib; options {
		# "path" is "${myconfigName}.pretty.long.module.name.enable"
		enable = boolOption true;
	};
}

like this:

• it's more intuitive: you just assign options to the current module
• module name is defined only in one place and reused automatically later
• backwards compatible, because it's an optional helper; user still has full control over options with custom attribute set

and for the example above, because this delib.options is assigned to options of the delib.module, custom lambda for the arguments can be provided with no drawbacks:

# networking/default.nix
{ delib, ... }:
delib.module {
	name = "networking";

	options = with delib; options {
		# define names of the devices to allow connections to
		# (arbitrary example)
		devices = listOfOption str [ ];
	};
}

# networking/wifi.nix
{ delib, ... }:
delib.module {
	name = "networking.wifi";

	options = with delib; { parent, ... }@args: options {
		# use devices from the parent as the default
		devices = listOfOption str parent.devices;
	} args;
}

it's a more hybrid approach. i feel like right now it's either "full minimal" (delib.singleEnableOption wrapper usage) or "full manual" (defining options field manually). with this, the user can choose from three levels:

1. minimal - just one wrapper for a single option
2. hybrid - delib.options to define options for the current (or nested under the current) module only
3. manual - define options manually if you need to make declarations for the parent module for some reason

composables

this is an extreme form of reducing the boilerplate and is hybrid^2 approach.

what if the library will provide not only the values for the options, but keys as well, like some parts that compose the options of a module?

example:

{ delib, ... }:
delib.module {
	name = "the_module";

	options = with delib; compose [
		(enableOption true)
		(nameOption "default")
	];
}

where the implementation is something like this:

delib.compose = components: args:
	lib.mkMerge (lib.map (component: component args) components);

delib.enableOption = default: args:
	delib.singleEnableOption default args;

delib.nameOption = default: { name, ... }:
	delib.attrset.setAttrByStrPath "${name}.name" (strOption default);

i can be incorrect in my code, but the general idea is that components are options with predefined keys, and they can be merged together.

naming

one of the issues is the naming. essentially, delib.singleEnableOption right now is a component by itself, but single in the name assumes that this option is the only one for the options. so i defined enableOption as the alias.

also, the namespace would be cluttered very fast if many components are created. i think, if that's going to be implemented, that should be an extension like delib.components.

if this extension is enabled, then delib.compose and delib.components are exposed, and the user can write something like this:

{ delib, ... }:
delib.module {
	name = "the_module";

	options = with delib.components; delib.compose [
		(enable true)
		(name "default")
	];
}

but in this case i really don't like how many times delib is written, so maybe this is better:

{ delib, ... }:
delib.module {
	name = "the_module";

	# the function is delib.components.compose
	options = with delib.components; compose [
		(enable true)
		(name "default")
	];
}

user components

also, it would be amazing to define custom components. either by assigning to delib.components.XXX or providing a custom api.

for the first idea, i don't know is this even possible. i don't know is this declaration vs definition situation and would it break or not if the user did this.

as for the second, something like

{ delib, ... }:
delib.component {
	name = "enable";
	option = delib.boolOption;
}

could be provided. the example should mirror the delib.singleEnableOption behaviour.

the option can be nested, so something complex like defining a big delib.component.host with many nested options is possible to then reuse this component anywhere.

this should partly resolve the second problem from the first example with devices: the type which was nested is written twice, so change in the type would result in a hellish refactoring, depending on the nesting.

with the components, this can look like this:

# components/devices.nix
{ delib, ... }:
delib.component {
	name = "devices";
	option = with delib; listOfOption str;
}

# networking/default.nix
{ delib, ... }:
delib.module {
	name = "networking";

	options = with delib.components; compose [
		# define names of the devices to allow connections to
		# (arbitrary example)
		(devices [ ])
	[;
}

# networking/wifi.nix
{ delib, ... }:
delib.module {
	name = "networking.wifi";

	options = with delib.components; { parent, ... }@args: compose [
		# use devices from the parent as the default
		(devices parent.devices)
	] args;
}

cascade wrapper

also, as the cherry on top, the logic for cascade options can be elevated to another level:

# networking/wifi.nix
{ delib, ... }:
delib.module {
	name = "networking.wifi";

	options = with delib.components; compose [
		# use devices from the parent as the default
		(cascade devices)
	];
}

the delib.components.cascade wrapper would pass the option value of the same name from the parent as the default for the component.

or this can be done as a string like (cascade "devices"). in any case, it should inherit the option definition as well as the default, so it will be type safe and convenient.

and because delib.components.compose is just merging the options, using my structure of "simple cascade enabled modules" becomes trivial:

{ delib, ... }:
delib.module {
	# or nested name like "nixvim.plugins.yazi",
	# the options don't change at all!
	name = "kitty";

	options = with delib.components; cascade enable;

	home.ifEnabled = { 
		# ...
	};
}

total

• delib.singleCascadeEnableOption would be great
• parent argument to the lambdas of the delib.module, delib.host etc. would be very useful
• delib.options to define options via an attribute set on the current cfg level would be amazing
• delib.components will provide an infinite amount of reusability for options, but this should probably be done as an extension

thanks for reading, i hope it was useful. i know that every new feature is an additional work and responsibility, but i felt like this will perfectly fit into the library. and of course you are not obliged to approve it (i just came here and dropped a huge wall of text with ideas). but i think it's better to develop an already existing project like yours than to create an another one (otherwise js frameworks moment all over again).

also, is the planned refactoring would be backwards compatible or not? as in "am i safe to migrate to denix right now?". and can i help with the refactoring in some way?

[yunfachi]

Thanks for the ideas!

delib.singleCascadeEnableOption would be great

If you're unsure about the name, there's also the option singleInheretedEnableOption.

delib.options to define options via an attribute set on the current cfg level would be amazing

I agree, manually writing out the full option path is not very convenient; this will be implemented in the Denix rewrite.

delib.components will provide an infinite amount of reusability for options, but this should probably be done as an extension

Interesting idea! And I think yes, it could be implemented as an extension, but even in the core lib it wouldn't hurt, since it would be in a separate file just adding new functions.

also, is the planned refactoring would be backwards compatible or not? as in "am i safe to migrate to denix right now?".

Well, the new Denix will be a modular system where modules and everything else are defined through options, not functions. However, the old wrapper approach for modules (delib.module, delib.host, delib.rice) will most likely remain as an alternative due to its design. The ideas will all stay the same, only the implementation will be cleaner, so I think yes, you are safe to migrate to Denix right now.

and can i help with the refactoring in some way?

Hmm, if you've gotten familiar with the codebase, I'd be grateful if you could implement the components extension, delib.singleCascadeEnableOption, or the parent argument. I'll try to make a WIP commit soon in a new branch with what I've already managed to do in the rewrite; you could also propose ideas for new features, like generating nixosModules, homeModules, etc.

[shimeoki]

Hmm, if you've gotten familiar with the codebase, I'd be grateful if you could implement the components extension, delib.singleCascadeEnableOption, or the parent argument.

in the works. what about the delib.options wrapper idea? should this be implemented?

i think that's a good middle ground between fully OOTB solution like proposed components and fully manual control via config paths. also, i think it's easier to implement, because it's just a single wrapper function and not a new "system" like components.

current plans look like this (in order):

• parent parameter (feat: parent parameter #53)
• delib.singleCascadeEnableOption (lib/options: singleCascadeEnableOption #57
• delib.moduleOptions (lib/options: add moduleOptions #59)
• components extension
• exportable modules

the last task is probably delayed until the refactoring.

as for the other features, i would be willing to "migrate" these features to the new structure.

but i am a bit hesitant to start components development, because

the new Denix will be a modular system where modules and everything else are defined through options, not functions.

and if the approach for the components would look something like this

{ delib, ... }:
delib.component {
    # ...
}

then it's not "dead on arrival", but probably still not very smart.

[yunfachi]

in the works. what about the delib.options wrapper idea? should this be implemented?

I think the current extensions API allows this to be implemented. You can initially implement the delib.options wrapper as an extension.

the last task is probably delayed until the refactoring.

Yes.

as for the other features, i would be willing to "migrate" these features to the new structure.

I've already made a WIP commit at https://github.com/yunfachi/denix/tree/rewrite , but the configurations themselves, hosts, and rices are not implemented yet. Only the options, the module system, and the compilation of the module system into nixosModules, homeModules, etc.