Tabular Deep Learning parameters

[edgararuiz]

Closes #452

1. Adds 14 new parameters for deep learning models:

• SAINT (via brulee):

  • attention_type() - values: "column", "row", "both"
  • dropout_hidden() - range: 0, 0.5
  • dropout_last() - range: 0, 0.5

• tdl package (brulee engine):

  • bottleneck_units() - range: 2L, 25L
  • dropout_attn() - range: 0, 0.5
  • dropout_embedding() - range: 0, 0.5
  • num_attn_blocks() - range: 1L, 6L
  • num_attn_feat() - range: 8L, 64L
  • num_attn_heads() - range: 1L, 8L
  • num_embedding() - range: 8L, 64L
  • penalty_average() - range: -15, -5 (log10 scale)
  • penalty_type() - values: "L1", "L2"
  • resid_at() - range: 2L, unknown()
  • step_rate() - range: 0, 8 (log10 scale)

2. All new parameters are added to the pkgdown reference index.

3. Several existing .Rd files have incidental changes from roxygen2 dropping \docType{data} and \format{} blocks for values_* vectors.

Example:

library(dials)

attention_type()
#> Attention Type (qualitative)
#> 3 possible values include:
#> 'column', 'row', and 'both'

dropout_hidden()
#> Hidden Dropout Rate (quantitative)
#> Range: [0, 0.5]

dropout_last()
#> Final Layer Dropout Rate (quantitative)
#> Range: [0, 0.5]

saint_params <- parameters(attention_type(), dropout_hidden(), dropout_last())
grid_random(saint_params, size = 5)
#> # A tibble: 5 × 3
#>   attention_type dropout_hidden dropout_last
#>   <chr>                   <dbl>        <dbl>
#> 1 column                 0.286        0.447 
#> 2 row                    0.497        0.0539
#> 3 both                   0.0125       0.0425
#> 4 column                 0.331        0.305 
#> 5 both                   0.0550       0.285 

[topepo]

I guess it is time to make a call on the new package name: bartab tabular.

I'll update this and start the renaming process.

[topepo]

Looks good. I changed the package name from tdl

[hfrick]

I usually do a separate PR for updating the roxygen2 version so that it's easier to check for diffs due to that vs diffs of other nature. I've updated this PR from main with those changes so that we now only have the changes related to the new parameters on here.

Are all these parameters ones you expect to reuse across different engines or models? If not, could you please group them in a meaningful way? So far we've grouped by engine, that might also be right choice here. Grouping helps with navigating the options/docs and the code base. The rest is probably very straightforward and I'll take a look once we've established if they can be grouped. 🙌

[edgararuiz]

Hi @hfrick!

Thanks! So the new groupings will be:

Group	File	Doc topic	Parameters
Neural network (generic)	R/param_network.R	?dropout	dropout, epochs, hidden_units, hidden_units_2, batch_size, dropout_hidden, dropout_last, num_embedding, dropout_embedding
Attention-based tabular models	R/param_attention.R	?attention-param	attention_type, dropout_attn, num_attn_heads, num_attn_blocks, num_attn_feat
Regularization learning networks	R/param_rln.R	?rln-param	penalty_average, step_rate, penalty_type
Residual networks	R/param_resnet.R	?resnet-param	bottleneck_units, resid_at

[hfrick]

Thanks!

[topepo]

I have one more parameter to add. I'll put it in this PR today.