Course Syllabus

Export

Lo studente apprenderà le tecniche di Machine Learning più efficaci, comprendendo i fondamenti teorici di ogni tecnica e acquisendo il know-how per poterle applicare con successo alla risoluzione di problemi pratici. Sarà inoltre fornita una panoramica sulle più innovative soluzioni per l’identificazione del miglior algoritmo di Machine Learning e della sua configurazione ottimale (Automated Machine Learning – AutoML), dato un dataset. Lo strumento di riferimento per il corso sarà Python, in particolare la suite "scikit-learn", ma verranno anche presentate alcune soluzioni equivalenti in R (ad esempio mlr e mlrbbo) e Java (ad esempio WEKA, KNIME, SMAC3).

Concetti basi del Machine Learning: tipologie di dati, istanze, features, tasks e scenarios, parametri e iper-parametri, misure di performance
Concetti di similarità e distanza (tra dati puntuali, insiemi di dati, tra distribuzioni di probabilità)
Tecniche di apprendimento non-supervisionato (basate su distanze)
Tecniche di apprendimento supervisionato (basate su distanze): classificazione e regressione
Modellare non-linearità nei dati: tecniche basate sul concetto di kernel
Approccio connessionista: le reti neurali
Automated Machine Learning: configurazione automatica di un modello di Machine Learning
Utilizzo di librerie Python per l'esecuzione di task di Machine Learning

Introduzione

  • Machine Learning scenarios & tasks, notazioni utili
  • Tipi di dati e problemi: tabular, streams, text, time-series, sequences, spatial, graph, web, social, immagini, distribuzioni

Similarità e distanza

  • Distanze tra dati puntuali
    - in spazi continui: Minkowski di ordine p (e casi particolari: Manhattan, Euclidea, Chebyshev)
    - in spazi discreti: Hamming e Jaccard
  • DIstanze tra insiemi di dati
    - "linkage" tra insiemi di dati
    - il problema di Monge (introduzione)
  • Distanze tra distribuzioni di probabilità
    - Optimal Transport theory: distanza tra due distribuzioni continue, due discrete, o una discreta e una continua
    - Insiemi di dati come distribuzioni empiriche: il problema di Monge ed il problema di Kantorovitch
    - Distanza di Wasserstein e differenza con le misure di divergenza
    - Distanza tra insiemi di dati in spazi differenti: Gromov-Wasserstein

Unsupervised Learning

  • Clustering: approcci deterministici vs probabilistici; flat vs gerarchici; basati su distanza/similarità vs densità
  • Outlier and anomaly detection

Supervised Learning

  • I fondamenti del "learning": classificazione binaria, processo di generazione dei dati, concetto vs ipotesi, errore empirico vs errore di generalizzazione
  • Classificazione e regressione: metriche e tecniche di validazione (hold-out, k fold-cross, leave-one-out)
  • Approcci model-free/instance-based, un semplice algoritmo: k-nearest neighbors (KNN)
  • Approcci model-based: Support Vector Machine (lineari)

Supervised Learning per dati non-lineari

  • Non-linearità, VC dimensions, kernel-trick
  • Un richiamo a Decision Tree e Random Forest
  • Kernel-based learning: kernel-SVM e Gaussian Processes per classificazione e regressione
  • Dimensionlity reduction: Principal Component Analysis (PCA) e kernel-based PCA (kPCA)

L’approccio connessionista

  • Artificial Neural Networks: perceptron, shallow networks e deep networks (Recurrent Neural Networks, Long-Short Term Memory, Gated Recurrent Unit)
  • Modelli neurali Generativi: Auto-Encoder (AE) e Variational-AE (VAE), Generative Adversarial Network (GAN) e Wasserstein-GAN (WGAN), Transformers

Automated Machine Learning (AutoML)

  • Ottimizzazione degli iperparametri di un algoritmo di Machine Learning
  • Selezione del miglior algoritmo di Machine Learning e (simultanea) ottimizzazione dei suoi iperparametri attraverso Bayesian Optimization

Esercizi ed esempi pratici

Si consiglia la conoscenza di elementi di base di informatica, matematica applicata, probabilità e statistica

L'intera attività formativa viene svolta attraverso lezioni in presenza. Le lezioni riguarderanno sia aspetti teorici che applicazioni pratiche, specificatamente l'utilizzo di librerie software e dati open.

  • Testo di riferimento: Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar (2018). Foundations of Machine Learning.
  • Slides e materiale didattico fornito dal docente

Altri tesi suggeriti:

  • Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for machine learning. Cambridge University Press.
  • Charu C. Aggarwal (2023). Neural Networks and Deep Learning – A Textbook
  • Bishop, C. M., & Bishop, H. (2023). Deep learning: Foundations and concepts. Springer Nature.
  • Robert B. Gramacy (2020). Surrogates – Gaussian Processes Modeling, Design, and Optimization for the Applied Statistics.
  • Charu C. Aggarwal (2015). Data Mining – the Textbook

Primo semestre - primo periodo

La modalità di verifica prevede le seguenti 2 prove:

  • lo svolgimento di un progetto con associata redazione di un rapporto tecnico, stile articolo scientifico,
  • un esame orale (individuale e obbligatorio) finalizzato a verificare il grado di comprensione degli argomenti trattati.

Il progetto può essere svolto in team (max 3 studenti per gruppo) ed i dataset oggetto delle attività saranno concordati con il docente a partire da piattaforme open quali OpenML, Kaggle o UCI Repository.
La qualità del progetto è stabilita sulla base del corretto utilizzo degli algoritmi di ML e all'analisi critiva dei risultati. L'esame orale è finalizzato alla verifica della comprensione di aspetti teorici e metodologici del ML.
Il progetto contribuisce al 60% della valutazione finale, la prova orale al restante 40%.

Non sono previste prove intermedie.

Su appuntamento

ISTRUZIONE DI QUALITÁ
Export

The student will learn the most effective Machine Learning techniques, understanding the theoretical foundations of each technique and acquiring the know-how to successfully apply them to solving practical problems. An overview of the most innovative solutions for the identification of the best Machine Learning algorithm and its optimal configuration, given a dataset (Automated Machine Learning - AutoML), will also be provided. The reference tool for the course will be python, specifically the suite "scikit-learn", but some equivalent solutions will also be presented in R (for example mlr and mlrmbo) and Java (for example WEKA, KNIME, SMAC3).

Machine Learning basics: types of data, instances, features, tasks and scenarios, parameters and hyper-parameters, performance measures
Similarity and distance (between data points, between data sets, between probability distributions)
Unsupervised learning techniques (based on distances)
Supervised learning techniques (based on distances): classification and regression
Modeling non-linearity in data: kernel-based techniques
The connectionists approach: artificial neural networks
Automated Machine Learning: automatic configuration of a Machine Learning model
Usage of Python libraries for implementing Machine Learning tasks

Introduction

  • Machine Learning scenarios & tasks, useful notations
  • Types of data and problems: tabular, streams, text, time-series, sequences, spatial, graph, web, social, immagini, distribuzioni

Similarity and distance

  • Distance between data points
    - over continuous spaces: p-order Minkowski (and special cases: Manhattan, Euclidea, Chebyshev)
    - over discrete spaces: Hamming and Jaccard
  • DIstance between datasets
    - "linkage" between sets of data points
    - the Monge's problem
  • Distance between probability distributions
    - Optimal Transport theory: distance between two continuous, two discrete, or one continuous and one discrete probability distributions
    - Datasets as empirical discrete probability distributions: Monge's and Kantorovitch problem
    - Wasserstein distance and differences with divergences
    - Distance between datasets laying into two different spaces: Gromov-Wasserstein

Unsupervised Learning

  • Clustering: deterministic vs probabilistic approaches; flat vs hierarchical; distance/similarity vs density -based
  • Outlier and anomaly detection

Supervised Learning

  • Foundations of "learning": binary classification, data generation process, concept vs hypothesis, empirical vs generalization error
  • Classification and regression: metrics and validation techniques (hold-out, k fold-cross, leave-one-out)
  • Model-free/instance-based approaches, a simple algorithm: the k-nearest neighbors (KNN)
  • Model-based approaches: Support Vector Machine (linear)

Supervised Learning for non-linear data

  • Non-linearity, VC dimensions, kernel-trick
  • A brief recall on Decision Tree and Random Forest
  • Kernel-based learning: kernel-SVM and Gaussian Processes, for classification and regression
  • Dimensionlity reduction: Principal Component Analysis (PCA) and kernel-based PCA (kPCA)

The connectionist approach

  • Artificial Neural Networks: perceptron, shallow networks e deep networks (Recurrent Neural Networks, Long-Short Term Memory, Gated Recurrent Unit)
  • Generative neural models: Auto-Encoder (AE) and Variational-AE (VAE), Generative Adversarial Network (GAN) and Wasserstein-GAN (WGAN), Transformer

Automated Machine Learning (AutoML): an overview

  • Hyeprparameter optimization of a Machine Learning algorithm
  • Algorithm selection and (simultaneous) hyperparameter optimization of a Machine Learning algorithm via Bayesian Optimization

Exercises and examples

Basic knowledge on computer science, applied math, probability calculus and statistics

Teaching is provided in-person. Lectures will address both theory and hands-on, specifically the adoption of open data and software libraries.

  • Reference textbook: Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar (2018). Foundations of Machine Learning.
  • Slides and materials provided by the lecturer

Other suggested texts:

  • Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for machine learning. Cambridge University Press.
  • Charu C. Aggarwal (2023). Neural Networks and Deep Learning – A Textbook
  • Bishop, C. M., & Bishop, H. (2023). Deep learning: Foundations and concepts. Springer Nature.
  • Robert B. Gramacy (2020). Surrogates – Gaussian Processes Modeling, Design, and Optimization for the Applied Statistics.
  • Charu C. Aggarwal (2015). Data Mining – the Textbook

First semester - First period

Assessment is organized on two tests:

  • the development of a project along with the preparation of an associated technical report, similar to a scientific paper,
  • an oral examination (individual and mandatory) aimed at assessing the degree of understanding of the course's topics.

The project can be performed by working in team (max 3 students per group) and the datasets to adopt, in agreement with the lecturer, will be selected among those available on open platforms such as OpenML, Kaggle or UCI Repository.
The quality of the project will be assessed according to the correct adoption of ML algorithms and the analysis of the results. Oral examination is devoted to assess the understanding of theoretical and methodological aspects of ML.
The project amounts for 60% of the final mark, while the oral examination amounts for the remaining 40%.

There are no mid-term review(s)

On appointment

QUALITY EDUCATION
Enter

Key information

Field of research
INF/01
ECTS
6
Term
First semester
Activity type
Mandatory to be chosen
Course Length (Hours)
42
Degree Course Type
2-year Master Degree
Language
English

Staff

    Teacher

  • Antonio Candelieri
    Antonio Candelieri

Students' opinion

View previous A.Y. opinion

Bibliography

Find the books for this course in the Library

Enrolment methods

Manual enrolments

Sustainable Development Goals

QUALITY EDUCATION - Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all