Robustness Measure for Deep Neural Networks

Neural networks are susceptible to minor distortions in their input, which can lead to errors they would not otherwise make. This susceptibility, termed as the network’s robustness, is a crucial aspect to evaluate. While several methods exist for measuring robustness, they usually suffer from interpretability issues and do not provide a statistical guarantee. In this work, we propose a novel robustness measure that addresses these short- comings by modeling the robustness as a probability distribution and mea- suring its 0.05 quantile. Additionally, previous work suggests the poten- tial modeling of robustness through a log-normal distribution. To eval- uate this hypothesis and its computational benefits, we introduce an es- timator that assumes the distribution is log-normal. A comparison with the standard parameter-free estimator reveals significantly improved com- putational efficiency with the parametrized approach. However, the log- normal assumption requires further research....Show moreNeural networks are susceptible to minor distortions in their input, which can lead to errors they would not otherwise make. This susceptibility, termed as the network’s robustness, is a crucial aspect to evaluate. While several methods exist for measuring robustness, they usually suffer from interpretability issues and do not provide a statistical guarantee. In this work, we propose a novel robustness measure that addresses these short- comings by modeling the robustness as a probability distribution and mea- suring its 0.05 quantile. Additionally, previous work suggests the poten- tial modeling of robustness through a log-normal distribution. To eval- uate this hypothesis and its computational benefits, we introduce an es- timator that assumes the distribution is log-normal. A comparison with the standard parameter-free estimator reveals significantly improved com- putational efficiency with the parametrized approach. However, the log- normal assumption requires further research. The assumption is too strong and needs to be relaxed before the parametrized estimator can reliably be utilized.Show less

Leiden University Student Repository

Documents

In Collections

Robustness Measure for Deep Neural Networks