-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathpublications.html
More file actions
559 lines (433 loc) · 41.2 KB
/
publications.html
File metadata and controls
559 lines (433 loc) · 41.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="description" content="Homepage of KleinLab">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>KleinLab - Homepage</title>
<link href="./libs/bootstrap-3.4.1/css/bootstrap.css" rel="stylesheet">
<link href="./libs/fontawesome-6.2.0/css/all.min.css" rel="stylesheet">
<link href="./style.css" rel="stylesheet">
<link href='https://fonts.googleapis.com/css?family=Didact Gothic' rel='stylesheet'>
<style>
body {
font-family: 'Didact Gothic';
}
</style>
<link rel="icon" href="images/logo_light_wo_frame.png">
</head>
<script async src="https://www.googletagmanager.com/gtag/js?id=G-E2B3ZSQM8N"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-E2B3ZSQM8N');
</script>
<body data-spy="scroll" data-target="#myScrollspy" data-offset="80">
<a href="#" id="scrollbutton" class="hidden-md hidden-lg hidden-xl" style="display: none;"><span></span></a>
<div class="container">
<div style="margin-top: 15px; margin-bottom: 15px;">
<img class="filter-img" src="images/lettering_dark.png" alt="Group logo" style="height: 40px;">
<h2 style="color:#4c8a90; display: inline; vertical-align:middle; margin-left: 10px; "> - Methods for Big Data</h2>
</div>
<div id="navbar"></div>
<script>
fetch('/navbar.html')
.then(response => response.text())
.then(data => {
document.getElementById('navbar').innerHTML = data;
// Highlight current page
const path = window.location.pathname;
const navLinks = document.querySelectorAll('.nav-link');
navLinks.forEach(link => {
const href = link.getAttribute('href');
if (href === path || (path === '/' && href === '/index.html')) {
link.classList.add('active');
}
});
// Highlight parent for subpages
if (path.startsWith('/subpages_pubs/')) {
document.querySelector('a[href="/publications.html"]').classList.add('active');
}
if (path.startsWith('/subpages_research/')) {
document.querySelector('a[href="/research.html"]').classList.add('active');
}
});
</script>
<!-- ------------------------------------------------------------------------------------------------------------------------------------ -->
<!-- ------------------------------------------------------------------------------------------------------------------------------------ -->
<!-- ------------------------------------------------------------------------------------------------------------------------------------ -->
<!-- ------------------------------------------------------------------------------------------------------------------------------------ -->
<!-- Publications -->
<section id="publications" style="font-size:medium;">
<h2>Publications <span class="fa fa-book" aria-hidden="true" style="float:left;padding-right:15px;padding-top:6px;font-size: 75%;"></span></h2>
<p>
See the <a href="https://scholar.google.de/citations?user=upS2UTIAAAAJ" target="_blank">Google Scholar page of Prof. Klein</a> for a complete and up-to-date list, below some key publications are listed.
For a complete list of publications, please refer to the following subpages:
</p>
<div class="row" style="margin-left: 15px;">
<div class="column">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/methodology.html" style="font-size: large;">Peer-Reviewed Journals (Methodology)</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/applications.html" style="font-size: large;">Peer-Reviewed Journals (Applications)</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/books.html" style="font-size: large;">Books & Book Chapters</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/editorials.html" style="font-size: large;">Editorials</a>
</div>
</div>
</div>
<div class="column">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/conferences.html" style="font-size: large;">Peer-Reviewed Conferences & Workshops</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/supplementary.html" style="font-size: large;">Supplementary Materials</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/software.html" style="font-size: large;">Software</a>
</div>
</div>
<br>
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa fa-folder-open icon"></span>
</div>
<div class="column2" style="margin-left: 15px;">
<a href="subpages_pubs/working_papers.html" style="font-size: large;">Working Papers</a>
</div>
</div>
</div>
</div>
<br>
<h3>Selected Publications</h3>
<ul class="fa-ul list-group" style="margin-left:25px">
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>bamlss: Bayesian Additive Models for Location, Scale, and Shape (and beyond) <span class="label label-info"><a href="https://dx.doi.org/10.18637/jss.v100.i04" target="_blank">pdf</a></span></h4>
N. Umlauf, N. Klein, T. Simon and A. Zeileis<br>
<i>Journal of Statistical Software, 100(4), 1–53</i>, 2021
</div>
<p style="text-align: justify"><u>Abstract:</u> Over the last decades, the challenges in applied regression and in predictive modeling have been changing considerably: (1) More flexible regression model specifications are needed as data sizes and available information are steadily increasing, consequently demanding for more powerful computing infrastructure. (2) Full probabilistic models by means of distributional regression - rather than predicting only some underlying individual quantities from the distributions such as means or expectations - is crucial in many applications. (3) Availability of Bayesian inference has gained in importance both as an appealing framework for regularizing or penalizing complex models and estimation therein as well as a natural alternative to classical frequentist inference. However, while there has been a lot of research on all three challenges and the development of corresponding software packages, a modular software implementation that allows to easily combine all three aspects has not yet been available for the general framework of distributional regression. To fill this gap, the R package bamlss is introduced for Bayesian additive models for location, scale, and shape (and beyond) - with the name reflecting the most important distributional quantities (among others) that can be modeled with the software. At the core of the package are algorithms for highly-efficient Bayesian estimation and inference that can be applied to generalized additive models or generalized additive models for location, scale, and shape, or more general distributional regression models. However, its building blocks are designed as "Lego bricks" encompassing various distributions (exponential family, Cox, joint models, etc.), regression terms (linear, splines, random effects, tensor products, spatial fields, etc.), and estimators (MCMC, backfitting, gradient boosting, lasso, etc.). It is demonstrated how these can be easily combined to make classical models more flexible or to create new custom models for specific modeling challenges.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/bamlss.png" alt="bamlss" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Bayesian conditional transformation models <span class="label label-info"><a href="https://www.tandfonline.com/doi/abs/10.1080/01621459.2023.2191820" target="_blank">pdf</a></span></h4>
M. Carlan, T. Kneib and N. Klein<br>
<i>Journal of the American Statistical Association, 119(546), 1360-1373</i>, 2024
</div>
<p style="text-align: justify"><u>Abstract:</u> Recent developments in statistical regression methodology shift away from pure mean regression toward distributional regression models. One important strand thereof is that of conditional transformation models (CTMs). CTMs infer the entire conditional distribution directly by applying a transformation function to the response conditionally on a set of covariates toward a simple log-concave reference distribution. Thereby, CTMs allow not only variance, kurtosis or skewness but the complete conditional distribution to depend on the explanatory variables. We propose a Bayesian notion of conditional transformation models (BCTMs) focusing on exactly observed continuous responses, but also incorporating extensions to randomly censored and discrete responses. Rather than relying on Bernstein polynomials that have been considered in likelihood-based CTMs, we implement a spline-based parameterization for monotonic effects that are supplemented with smoothness priors. Furthermore, we are able to benefit from the Bayesian paradigm via easily obtainable credible intervals and other quantities without relying on large sample approximations. A simulation study demonstrates the competitiveness of our approach against its likelihood-based counterpart but also Bayesian additive models of location, scale and shape and Bayesian quantile regression. Two applications illustrate the versatility of BCTMs in problems involving real world data, again including the comparison with various types of competitors. Supplementary materials for this article are available online.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/BCTM.png" alt="BCTM" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Bayesian Generalized Additive Models for Location, Scale and Shape for Zero-Inflated and Overdispersed Count Data <span class="label label-info"><a href="https://www.tandfonline.com/doi/abs/10.1080/01621459.2014.912955" target="_blank">pdf</a></span></h4>
N. Klein, T. Kneib and S. Lang<br>
<i>Journal of the American Statistical Association, 110(509), 405-419</i>, 2015
</div>
<p style="text-align: justify"><u>Abstract:</u> Frequent problems in applied research preventing the application of the classical Poisson log-linear model for analyzing count data include overdispersion, an excess of zeros compared to the Poisson distribution, correlated responses, as well as complex predictor structures comprising nonlinear effects of continuous covariates, interactions or spatial effects. We propose a general class of Bayesian generalized additive models for zero-inflated and overdispersed count data within the framework of generalized additive models for location, scale, and shape where semiparametric predictors can be specified for several parameters of a count data distribution. As standard options for applied work we consider the zero-inflated Poisson, the negative binomial and the zero-inflated negative binomial distribution. The additive predictor specifications rely on basis function approximations for the different types of effects in combination with Gaussian smoothness priors. We develop Bayesian inference based on Markov chain Monte Carlo simulation techniques where suitable proposal densities are constructed based on iteratively weighted least squares approximations to the full conditionals. To ensure practicability of the inference, we consider theoretical properties like the involved question whether the joint posterior is proper. The proposed approach is evaluated in simulation studies and applied to count data arising from patent citations and claim frequencies in car insurances. For the comparison of models with respect to the distribution, we consider quantile residuals as an effective graphical device and scoring rules that allow us to quantify the predictive ability of the models. The deviance information criterion is used to select appropriate predictor specifications once a response distribution has been chosen. Supplementary materials for this article are available online.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/BayesAdditiveScaleShape.png" alt="Bayesian Generalized Additive Models for Location, Scale and Shape for Zero-Inflated and Overdispersed Count Data" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Bayesian Inference for Regression Copulas <span class="label label-info"><a href="https://www.tandfonline.com/doi/abs/10.1080/07350015.2020.1721295" target="_blank">pdf</a></span></h4>
M. S. Smith and N. Klein<br>
<i>Journal of Business and Economic Statistics, 39(3), 712-728</i>, 2021
</div>
<p style="text-align: justify"><u>Abstract:</u> We propose a new semi-parametric distributional regression smoother that is based on a copula decomposition of the joint distribution of the vector of response values. The copula is high-dimensional and constructed by inversion of a pseudo regression, where the conditional mean and variance are semi-parametric functions of covariates modeled using regularized basis functions. By integrating out the basis coefficients, an implicit copula process on the covariate space is obtained, which we call a `regression copula'. We combine this with a non-parametric margin to define a copula model, where the entire distribution - including the mean and variance - of the response is a smooth semi-parametric function of the covariates. The copula is estimated using both Hamiltonian Monte Carlo and variational Bayes; the latter of which is scalable to high dimensions. Using real data examples and a simulation study we illustrate the efficacy of these estimators and the copula model. In a substantive example, we estimate the distribution of half-hourly electricity spot prices as a function of demand and two time covariates using radial bases and horseshoe regularization. The copula model produces distributional estimates that are locally adaptive with respect to the covariates, and predictions that are more accurate than those from benchmark models.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/BayesInfRegressionCopulas.png" alt="Bayesian Inference for Regression Copulas" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Boosting distributional copula regression <span class="label label-info"><a href="https://onlinelibrary.wiley.com/doi/full/10.1111/biom.13765" target="_blank">pdf</a></span></h4>
N. Hans, N. Klein, F. Faschingbauer, M. Schneider and A. Mayr<br>
<i>Biometrics, 79(3), 2298–2310</i>, 2023
</div>
<p style="text-align: justify"><u>Abstract:</u> Capturing complex dependence structures between outcome variables (e.g., study endpoints) is of high relevance in contemporary biomedical data problems and medical research. Distributional copula regression provides a flexible tool to model the joint distribution of multiple outcome variables by disentangling the marginal response distributions and their dependence structure. In a regression setup, each parameter of the copula model, that is, the marginal distribution parameters and the copula dependence parameters, can be related to covariates via structured additive predictors. We propose a framework to fit distributional copula regression via model-based boosting, which is a modern estimation technique that incorporates useful features like an intrinsic variable selection mechanism, parameter shrinkage and the capability to fit regression models in high-dimensional data setting, that is, situations with more covariates than observations. Thus, model-based boosting does not only complement existing Bayesian and maximum-likelihood based estimation frameworks for this model class but rather enables unique intrinsic mechanisms that can be helpful in many applied problems. The performance of our boosting algorithm for copula regression models with continuous margins is evaluated in simulation studies that cover low- and high-dimensional data settings and situations with and without dependence between the responses. Moreover, distributional copula boosting is used to jointly analyze and predict the length and the weight of newborns conditional on sonographic measurements of the fetus before delivery together with other clinical variables.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/BoostingDistrCopula.png" alt="Boosting distributional copula regression" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices <span class="label label-info"><a href="https://onlinelibrary.wiley.com/doi/full/10.1002/jae.2959" target="_blank">pdf</a></span></h4>
N. Klein, M. S. Smith and D. J. Nott<br>
<i>Journal of Applied Econometrics, 38(4), 493-511</i>, 2023
</div>
<p style="text-align: justify"><u>Abstract:</u> Recurrent neural networks (RNNs) with rich feature vectors of past values can provide accurate point forecasts for series that exhibit complex serial dependence. We propose two approaches to constructing deep time series probabilistic models based on a variant of RNN called an echo state network (ESN). The first is where the output layer of the ESN has stochastic disturbances and a Bayesian prior for regularization. The second employs the implicit copula of an ESN with Gaussian disturbances, which is a Gaussian copula process on the feature space. Combining this copula process with a nonparametrically estimated marginal distribution produces a distributional time series model. The resulting probabilistic forecasts are deep functions of the feature vector and marginally calibrated. In both approaches, Markov chain Monte Carlo methods are used to estimate the models and compute forecasts. The proposed models are suitable for the complex task of forecasting intraday electricity prices. Using data from the Australian market, we show that our deep time series models provide accurate short-term probabilistic price forecasts, with the copula model dominating. Moreover, the models provide a flexible framework for incorporating probabilistic forecasts of electricity demand, which increases upper tail forecast accuracy from the copula model significantly.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/DeepDistributionalTimeSeries.png" alt="Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Distributional Adaptive Soft Regression Trees <span class="label label-info"><a href="https://arxiv.org/abs/2210.10389" target="_blank">pdf</a></span></h4>
N. Umlauf and N. Klein, 2022<br>
<i>Submitted.</i>
</div>
<p style="text-align: justify"><u>Abstract:</u> Random forests are an ensemble method relevant for many problems, such as regression or classification. They are popular due to their good predictive performance (compared to, e.g., decision trees) requiring only minimal tuning of hyperparameters. They are built via aggregation of multiple regression trees during training and are usually calculated recursively using hard splitting rules. Recently regression forests have been incorporated into the framework of distributional regression, a nowadays popular regression approach aiming at estimating complete conditional distributions rather than relating the mean of an output variable to input features only - as done classically. This article proposes a new type of a distributional regression tree using a multivariate soft split rule. One great advantage of the soft split is that smooth high-dimensional functions can be estimated with only one tree while the complexity of the function is controlled adaptive by information criteria. Moreover, the search for the optimal split variable is obsolete. We show by means of extensive simulation studies that the algorithm has excellent properties and outperforms various benchmark methods, especially in the presence of complex non-linear feature interactions. Finally, we illustrate the usefulness of our approach with an example on probabilistic forecasts for the Sun's activity.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/DistrAdaptiveTrees.png" alt="Distributional Adaptive Soft Regression Trees" style="width: 90%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Informed Priors for Knowledge Integration in Trajectory Prediction <span class="label label-info"><a href="https://www.springerprofessional.de/informed-priors-for-knowledge-integration-in-trajectory-predicti/26052870" target="_blank">pdf</a></span></h4>
C. Schlauch, N. Klein and C. Wirth<br>
<i>ECML/PKDD 2023</i>, 2023
</div>
<p style="text-align: justify"><u>Abstract:</u> Informed learning approaches explicitly integrate prior knowledge into learning systems, which can reduce data needs and increase robustness. However, existing work typically aims to integrate formal scientific knowledge by directly pruning the problem space, which is infeasible for more intuitive world and expert knowledge, or requires specific architecture changes and knowledge representations. We propose a probabilistic informed learning approach to integrate prior world and expert knowledge without these requirements. Our approach repurposes continual learning methods to operationalize Baye’s rule for informed learning and to enable probabilistic and multi-modal predictions. We exemplify our proposal in an application to two state-of-the-art trajectory predictors for autonomous driving. This safety-critical domain is subject to an overwhelming variety of rare scenarios requiring robust and accurate predictions. We evaluate our models on a public benchmark dataset and demonstrate that our approach outperforms non-informed and informed learning baselines. Notably, we can compete with a conventional baseline, even using only half as many observations of the training dataset.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/InformedPriors.png" alt="Informed priors for knowledge integration in trajectory prediction" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Marginally Calibrated Deep Distributional Regression <span class="label label-info"><a href="https://www.tandfonline.com/doi/abs/10.1080/10618600.2020.1807996" target="_blank">pdf</a></span></h4>
N. Klein, D. J. Nott and M. S. Smith<br>
<i>Journal of Computational and Graphical Statistics, 30(2), 467-483</i>, 2021
</div>
<p style="text-align: justify"><u>Abstract:</u> Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are `marginally calibrated'. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer weights, from which an implicit copula process on the feature space is extracted. This copula process is combined with a non-parametrically estimated marginal distribution for the response. The end result is a scalable distributional DNN regression method with marginally calibrated predictions, and our work complements existing methods for probability calibration. The approach is first illustrated using two applications of dense layer feed-forward neural networks. However, our main motivating applications are in likelihood-free inference, where distributional deep regression is used to estimate marginal posterior distributions. In two complex ecological time series examples we employ the implicit copulas of convolutional networks, and show that marginal calibration results in improved uncertainty quantification. Our approach also avoids the need for manual specification of summary statistics, a requirement that is burdensome for users and typical of competing likelihood-free inference methods.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/MargCalibDistrRegression.png" alt="Distributional Adaptive Soft Regression Trees" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Marginally calibrated response distributions for end-to-end learning in autonomous driving <span class="label label-info"><a href="https://projecteuclid.org/journals/annals-of-applied-statistics/volume-17/issue-2/Marginally-calibrated-response-distributions-for-end-to-end-learning-in/10.1214/22-AOAS1693.short" target="_blank">pdf</a></span></h4>
C. Hoffmann and N. Klein<br>
<i>Annals of Applied Statistics, 17(2), 1740-1763</i>, 2023
</div>
<p style="text-align: justify"><u>Abstract:</u> End-to-end learners for autonomous driving are deep neural networks that predict the instantaneous steering angle directly from images of the street ahead. These learners must provide reliable uncertainty estimates for their predictions in order to meet safety requirements and to initiate a switch to manual control in areas of high uncertainty. However, end-to-end learners typically only deliver point predictions, since distributional predictions are associated with large increases in training time or additional computational resources during prediction. To address this shortcoming, we investigate efficient and scalable approximate inference for the deep distributional model of Klein, Nott and Smith (J. Comput. Graph. Statist. 30 (2021) 467–483) in order to quantify uncertainty for the predictions of end-to-end learners. A special merit of this model, which we refer to as implicit copula neural linear model (IC-NLM), is that it produces densities for the steering angle that are marginally calibrated, that is, the average of the estimated densities equals the empirical distribution of steering angles. To ensure the scalability to large n regimes, we develop efficient estimation based on variational inference as a fast alternative to computationally intensive, exact inference via Hamiltonian Monte Carlo. We demonstrate the accuracy and speed of the variational approach on two end-to-end learners trained for highway driving using the comma2k19 dataset. The IC-NLM is competitive with other established uncertainty quantification methods for end-to-end learning in terms of nonprobabilistic predictive performance and outperforms them in terms of marginal calibration for in-distribution prediction. Our proposed approach also allows the identification of overconfident learners and contributes to the explainability of black-box end-to-end learners by using the predictive densities to understand which steering actions the learner sees as valid.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/EndToEndAutonomousDriving.png" alt="Marginally calibrated response distributions for end-to-end learning in autonomous driving" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Modelling Regional Patterns of Inefficiency: A Bayesian Approach to Geoadditive Panel Stochastic Frontier Analysis with an Application to Cereal Production in England and Wales <span class="label label-info"><a href="https://www.sciencedirect.com/science/article/abs/pii/S0304407619301587" target="_blank">pdf</a></span></h4>
N. Klein, H. Herwartz and T. Kneib<br>
<i>Journal of Econometrics, 214(2), 513-539</i>, 2020
</div>
<p style="text-align: justify"><u>Abstract:</u> We propose a flexible Bayesian approach to inefficiency modelling that accounts for regional patterns of local performance. The model allows for a separated treatment of individual heterogeneity and determinants of inefficiency. Regional dependence structures and location-specific unobserved spatial heterogeneity are modelled via geoadditive predictors in the inefficiency term of the stochastic frontier model. Inference becomes feasible through Markov chain Monte Carlo simulation techniques. In an empirical illustration we find that regional patterns of inefficiency characterize cereal production in England and Wales. Neglecting common performance patterns of farms located in the same region induces systematic biases to inefficiency estimates.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/GeoadditivePanelAnalysis.png" alt="Modelling Regional Patterns of Inefficiency: A Bayesian Approach to Geoadditive Panel Stochastic Frontier Analysis with an Application to Cereal Production in England and Wales" style="width: 70%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Semi-Structured Distributional Regression <span class="label label-info"><a href="https://www.tandfonline.com/doi/abs/10.1080/00031305.2022.2164054?journalCode=utas20" target="_blank">pdf</a></span></h4>
D. Rügamer, C. Kolb and N. Klein<br>
<i>To appear in The American Statistician</i>, 2023
</div>
<p style="text-align: justify"><u>Abstract:</u> Combining additive models and neural networks allows to broaden the scope of statistical regression and extend deep learning-based approaches by interpretable structured additive predictors at the same time. Existing attempts uniting the two modeling approaches are, however, limited to very specific combinations and, more importantly, involve an identifiability issue. As a consequence, interpretability and stable estimation are typically lost. We propose a general framework to combine structured regression models and deep neural networks into a unifying network architecture. To overcome the inherent identifiability issues between different model parts, we construct an orthogonalization cell that projects the deep neural network into the orthogonal complement of the statistical model predictor. This enables proper estimation of structured model parts and thereby interpretability. We demonstrate the framework’s efficacy in numerical experiments and illustrate its special merits in benchmarks and real-world applications.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/SemiStructuredDistributionalRegression.png" alt="Semi-Structured Distributional Regression" style="width: 70%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
<br>
<li class="list-group-item">
<div class="row">
<div class="column2" style="width: 2%;">
<span class="fa-li fa fa-check-square icon"></span>
</div>
<div class="column2" style="width: 50%;">
<div class="content">
<h4>Variational inference and sparsity in high dimensional deep Gaussian mixture models <span class="label label-info"><a href="https://doi.org/10.1007/s11222-022-10132-z" target="_blank">pdf</a></span></h4>
L. Kock, N. Klein and D. J. Nott<br>
<i>Statistics and Computing, 32(5)</i>, 2022
</div>
<p style="text-align: justify"><u>Abstract:</u> Gaussian mixture models are a popular tool for model-based clustering, and mixtures of factor analyzers are Gaussian mixture models having parsimonious factor covariance structure for mixture components. There are several recent extensions of mixture of factor analyzers to deep mixtures, where the Gaussian model for the latent factors is replaced by a mixture of factor analyzers. This construction can be iterated to obtain a model with many layers. These deep models are challenging to fit, and we consider Bayesian inference using sparsity priors to further regularize the estimation. A scalable natural gradient variational inference algorithm is developed for fitting the model, and we suggest computationally efficient approaches to the architecture choice using overfitted mixtures where unnecessary components drop out in the estimation. In a number of simulated and two real examples, we demonstrate the versatility of our approach for high-dimensional problems, and demonstrate that the use of sparsity inducing priors can be helpful for obtaining improved clustering results.</p>
</div>
<div class="column2" style="width: 48%; margin-left: 10px; display: flex; align-items: center; justify-content: center;">
<img src="images/key_pubs/VI_DeepGaussianMixture.png" alt="Variational inference and sparsity in high dimensional deep Gaussian mixture models" style="width: 100%; display: block; margin-left: auto; margin-right: auto;">
</div>
</div>
</li>
</ul>
</div>
</section>
<!-- Imprint and data protection -->
<div style="display: inline-block; text-align: right; width: 100%; margin-top: 15px;">
<a href="https://www.scc.kit.edu/en/legals.php">Imprint</a> / <a href="https://www.scc.kit.edu/en/datenschutz.php">Privacy Policy</a>
</div>
</div>
</div>
</div>
</div>
<!-- jquery -->
<script src="./libs/jquery-3.6.1.min.js"></script>
<!-- boostrap -->
<script src="./libs/bootstrap-3.4.1/js/bootstrap.min.js"></script>
<!-- Functionality for popover of buttons -->
<script>
$(document).ready(function(){
$('[data-toggle="popover"]').popover();
});
</script>
<!-- Functionality for scroll-up button -->
<script>
$(document).ready(function(){
if ($(this).scrollTop() > 100) {
$('#scrollbutton').fadeIn();
}
$(window).scroll(function(){
if ($(this).scrollTop() > 100) {
$('#scrollbutton').fadeIn();
} else {
$('#scrollbutton').fadeOut();
}
});
$('#scrollbutton').click(function(){
$("html, body").animate({ scrollTop: 0 }, 300);
return false;
});
});
</script>
</body>
</html>