Navegar por los elementos (1 total)
- Resumen es exacto "Artificial intelligence advances by leaps and bounds. However, this trend could become stagnant if significant progress is not achieved in certain issues, two of which will be studied in this thesis. On the one hand, statistical learning seeks to develop algorithms that not only correctly classify a set of known data, but also that this behavior is generalized to new samples. The ability to generalize algorithms is often addressed with mere disturbances in the learning stage. An intelligent use of such disturbance methods could result in a considerable improvement of the algorithms. On the other hand, information theory seeks to create representations as compressed as possible, without affecting its subsequent use. The amount of data to be stored grows exponentially day by day and efficient methods of information compression become essential in the advancement of these kind of technologies. This thesis addresses the problem of information bottleneck, focusing on the links between statistical learning and information theory. The search for accurate and generalizable representations of statistical learning, such as the extraction of information concentrated on few data from information theory, seem to be two sides of the same coin: the bottleneck between generating relevant representations with a low complexity level
The main contributions of this thesis can be separated into four categories, which will be studied in chapters 5, 6, 7 and 8. The first is based on the development of an algorithm capable of applying both the problem of multitasking learning and to finding the fundamental limits of a information theory communication scheme with side information. Different theoretical and practical analyzes on it are developed in depth. The second is based on a theoretical link that relates the bottleneck effect with the generalization problem of statistical learning through a level of confidence about the deviation of empirical risk. This result is complemented by an experimental study, showing that it can be extremely relevant when the distribution of training and testing data is subtly different. The third is based on the study of the decoupling when cross entropy is the training cost function to minimize the error probability. This decoupling also seems to be linked to information theory metrics. Finally, the fourth is the study of the information bottleneck problem in distributed schemes that are relevant in practice, but in a theoretical way. An in-depth analysis demonstrates how cooperation between different nodes where information is stored can be advantageous"
Ordenar por: