Neural networks are susceptible to minor distortions in their input, which can lead to errors they would not otherwise make. This susceptibility, termed as the network’s robustness, is a crucial...Show moreNeural networks are susceptible to minor distortions in their input, which can lead to errors they would not otherwise make. This susceptibility, termed as the network’s robustness, is a crucial aspect to evaluate. While several methods exist for measuring robustness, they usually suffer from interpretability issues and do not provide a statistical guarantee. In this work, we propose a novel robustness measure that addresses these short- comings by modeling the robustness as a probability distribution and mea- suring its 0.05 quantile. Additionally, previous work suggests the poten- tial modeling of robustness through a log-normal distribution. To eval- uate this hypothesis and its computational benefits, we introduce an es- timator that assumes the distribution is log-normal. A comparison with the standard parameter-free estimator reveals significantly improved com- putational efficiency with the parametrized approach. However, the log- normal assumption requires further research. The assumption is too strong and needs to be relaxed before the parametrized estimator can reliably be utilized.Show less
Neural networks have been an active field of research for years, but relatively little is understood of how they work. Specific types of Neural Networks have a layer structure with decreasing width...Show moreNeural networks have been an active field of research for years, but relatively little is understood of how they work. Specific types of Neural Networks have a layer structure with decreasing width which acts like coarse-graining, reminiscent of the renormalization group (RG). We examine the Restricted Boltzmann Machine (RBM) and discuss it’s possible relation to RG. The RBM is trained on the 1D and 2D Ising model, as well as the MNIST dataset. In particular for the 2D Ising model showing a flow towards the critical point Tc ≈ 2.27, opposite to the RG-flow. Examining the behaviour of the RBM on the MNIST dataset shows that sparse datasets can allow multiple fixed points which can be removed by artificially creating new samples. We conclude that this RBM-flow exists due to the multiple relevant length scales at the critical point and we briefly discuss why.Show less
