Table of Contents
Tematy projektów WSHOP -- zima 2023/2024
[KKT] FACE APIs comparison - follow-up study
- Student: Michał Przysucha
- Namespace in the wiki: faceapis-follow
- The goal of the project: Comparison of the effectiveness of off-the-shelf APIs and pre-trained models for emotion recognition in non-trivial images
- Technology: Python, data analysis
- Description: The facial expression recognition tools are trained and evaluated on benchmark datasets that contain many expressions generated 'at the request' of the expressor and photographed en face. This does not match the reality, where expressions are not so strong and where the face is not always facing the camera. The project is the continuation of the previous research published by our team and extended during previous WSHOP. During the project, one should: (a) explore previous results, (b) integrate them together, (c) perform consistent experiments in a unified environment on a unified dataset (it may be necessary to find/add elements to the dataset!), (d) summarise the results by indicating the strengths and weaknesses (supported situations) for each API.
- Links:
- AffectNet: https://paperswithcode.com/dataset/affectnet (a benchmark dataset)
- Our previous work is briefly summarized in the paper: Evaluation of Selected APIs for Emotion Recognition from Facial Expressions
[KKT] Affect changes as probability blobs (with RegFlow) in Valence x Arousal space
- Student: Konrad Micek
- Namespace in the wiki: regflow
- The goal of the project: Adapt RegFlow method to 2-D emotion prediction task
- Technology: Python, data analysis
- Description: The research shows that emotion prediction models do not have high accuracy in 2-D space (Valence x Arousal). So, we want to accept this fact, and think about this as some kind of blob of probability somewhere in this 2-D space. This blob will move through time (e.g., when the subject is playing a game, the emotions will change during the subsequent phases of the game). In this project, we will use existing RegFlow method and try to adapt it to data from BIRAFFE2 experiment to show how emotion-blobs moved through the time of the game.
- Links:
- RegFlow: description, repo
[KKT] Emotion prediction in artificial space (then mapped to actual emotion space)
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Create a prediction model that maps features into artificial X-dimensional space. Then, map this space into actual emotions/labels space
- Technology: Python, machine learning, data analysis
- Description: Emotions in computational models are often represented as 2-dimensional Valence x Arousal space (sometimes it is a 3-D space with Dominance). When it comes to train prediction models, one try to map features space (features of physiological signals) into this 2-D space. A lot of research shows that this does not work. So, we want to try another approach: create a classifier that maps features into X-dimensional artificial space (with X » 2), and then create some mapping from this space into the Valence x Arousal space. As X will be higher than 2 it should catch more sophisticated relations in the data. A mapping from X-dimensional space into 2-D space should also provide some information about accuracy of the prediction/mapping.
You can start with the DEAP dataset, but ultimately, the project should be done with BIRAFFE2 dataset.
There should be some “classical” model trained/downloaded as a baseline (to show that the new model is better).
Transformers should be useful to generate the artificial X-D space. - Links:
- DEAP (the dataset can be obtained from KKT)
[KKT] Support in BIRAFFE3 experiment
- Student: Honorata Zych
- Namespace in the wiki: bir3support
- The goal of the project: Support in BIRAFFE3 experiment preparation (pilot data analysis) and then collaboration with actual experiment
- Technology: Python, data analysis
- Description: The project has two phases. During the first phase, one will take part in work conducted in BIRAFFE3 team focused on pilot data analysis: verification of the collected data (Are there no gaps? Can the data be combined? Do we have everything that can be harvested?) and simple analysis. Then, in the second part, one will help with conducting the actual experiment (scope of work to be determined). If the work is done with appropriate involvement, one could also become a co-author of a publication on BIRAFFE3 in Nature Scientific Data.
- Links:
- BIRAFFE2 in Nature Scientific Data – at the end of a day, we want something similar for the BIRAFFE3 experiment
[KKT] Emotion recognition for everyday life - evaluation of the state-of-the-art
- Student: Anastasiya Yurenia
- Namespace in the wiki: emognition
- The goal of the project: Replicate and evaluate methods and tools proposed for emotion recognition by Emognition team from PWr
- Technology: reading :), Python, data analysis, machine learning
- Description: Our colleagues from Emognition team deal with the same tasks as we in BIRAFFE series of experiments. In this project, we want to explore if their results are really as good as they say by following their papers/tools/methods (starting point is indicated by the Links below) and trying to replicate their results. In the subsequent projects (or Master Thesis) there will be possibility to use these methods and tools to other datasets like BIRAFFE2/BIRAFFE3.
- Links:
- Emognition Dataset: https://www.nature.com/articles/s41597-022-01262-0
- How to calculate features? Almost 900 features are described in: https://ieeexplore.ieee.org/document/9431143
- Various models evaluated in: https://ieeexplore.ieee.org/document/9767233
- Personalization of models discussed in: https://ieeexplore.ieee.org/document/9767502
[KKT] Loki on a triplestore
- Student: Dominik Tyszownicki
- Namespace in the wiki: loki
- The goal of the project: Get out of the SWI-Prolog from the Loki. Review current graph bases engines (triplestores), select the most promising one and move the whole knowledge to the selected triplestore.
- Technology: PHP, Semantic Web
- Description: Semantic wiki Loki is a DokuWiki system with a set of plugins for putting knowledge within wiki pages. It can be then easily extracted/processed. Now, the whole knowledge processing is done via plain files and SWI-Prolog, which is interesting approach, but not the optimal one, as there are many triplestores (dedicated graph base engines) available.
- Links:
[KKT] Ontology for handwritten manuscripts in Jagiellonian Library
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Preparation and evaluation of the ontology
- Technology: Semantic Web
- Description: Prepare a preliminary ontology that will form the basis of a knowledge base describing the manuscripts. It is necessary to take into account existing standards (to maintain the possibility of integration with them: import and export of data) and the specific needs of the project conducted at the Jagiellonian Library (e.g., specific data collected in the process of digitization).
- Links:
[KKT] Ontology for the SOLARIS synchrotron
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Preparation and evaluation of the ontology
- Technology: Semantic Web
- Description: Work in the close collaboration with the SOLARIS team to create and ontology from the perspective of training and maintenance.
- Links:
[SBK] OpenML dataset creation script for Meta-Learning
- Student:
- Namespace in the wiki: openmlds
- The goal of the project: Prepare a script that will build meta-learnign dataset out of OpenML logs
- Technology: Python, OpenML API
- Description: The main goal of the project is to create a script that will fetch all of the runs/pipelines and dataset from OpenML platform and create a dataset out of it. The challenge is to transform pipeline definitions which are code snippets into logical components of machine-learning pipeline (including deep neural networks). Such a dataset will serve as a learn-to-learn dataset for meta-learning solutions.
- Links:
[SBK] Wizualizacja (interaktywna) drzew decyzyjnych
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Implementacja warstwy wizualizacji drzew decyzyjnych, pozwalajacych na umieszczenia na nich większej ilości infromacji niż wyłącznie dane o nazwach zmiennych i wartościach
- Technology: Python, Streamlit
- Description: Celem projektu jest swtorzenie interaktywnej wizualizacji dla drzew decyzyjnych generowanych algorytmem pyUID3 w ramach wyjaśniacza LUX. Chodzi między innymi o uwzględenienie w wizualizacji drzewa: granicy decyzyjnej wyznaczanej przez warunek na krawędzi (w formie wykresu dwuwymiarowego); informacji o istotności danej zmiennej zgodnie z ich wagą oikreśloną przez SHAP, oraz niepewneością wartości; modyfikowanie drzewa;
- Links:
[SBK] Explainable Domain Adaptation for Streaming data
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Use Domain adaptation and XAI mechanisms to distinguish between concept drits, novelty and anomaly.
- Technology: Python, DeepLearnign
- Description: The goal is to see if it is possible to obtain reliable explanations of a phenomena that may appear in time-series data and distinguish between them
- Links:
[SBK] Time-series transformer for anomaly detection
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Build Transformer-based model for anomaly detection in data streams.
- Technology: Python, PyTorch, Hugginface
- Description:
Take part inWin the competition: Kaggle competitions - Links:
[SBK] Packageing and test writting for LUX
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Add tests and make the LUX software ready for packagein; publish it in SoftwareX journal.
- Technology: Python
- Description: As abowe
- Links:
[SBK] Self-explainable deep neural networks
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: The goal is to reproduce results from Self-Explainable DNN architectures over selected datasets
- Technology: Python
- Description: As above
- Links:
[SBK] Time-SHAP implementation
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: The goal is to improve the SHAP XAI algorithm to work more efficiently on time-seris data
- Technology: Python, Keras/PyTorch
- Description: The goal is to imploement upgraded version of WindowShap that is based on changepoint detection algorithm to improve efficiency/quality of explanations.
- Links:
[SBK] Improvement of Classifier accuracy with simulated domain adaptation
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: The goal is to create architecture that improves classification accuracy with DA algorithms
- Technology: Python, Keras
- Description: The goal is to exploit the fact that classifier trained on the full datasets with more than one subpopulations can be weaker than two classifiers where one is trained on majority subpopulation and the other is fine-tuned for the minority subpopulation. This is a more realistic case of Simpson Paradox.
- Links:
[JKO] Universal speech recognition with differentiable allophone graphs
- Student:
- Namespace in the wiki: UPhoRMPS
- The goal of the project: Implementing in k2 framework a model that learns phone-to-phoneme mappings.
- Technology: Python, k2 (Automatic Speech Recognition)
- Description: The ultimate goal (larger than this project) is to create a speech recognition (SR) model for low-resource languages. In this project, we would like to implement a method for learning phonetic structures of language (allowed cooccurrence of variants of speech sounds). In speech processing, such constraints were commonly encoded via Weighted finite-state transducers (WFSTs). Recently, people realised that WFSTs can be treated as differentiable and so can be incorporated in neural SR models – k2 serves exactly that purpose. The goal is to learn this ASR framework, reimplement in it a model by Yan et al., and start experimenting with it on various datasets (like accented English, Polish dialects or languages you haven't even heard of) that we have access to.
- Links:
[JKO] Domain Adversarial Training for speech model tuning
- Student:
- Namespace in the wiki: UPhoRMPS
- The goal of the project: Implementing Domain Adversarial Training (DAT) in k2 framework.
- Technology: Python, k2 (for Automatic Speech Processing)
- Description: The ultimate goal (larger than this project) is to create a speech recognition (SR) model for low-resource languages. k2 is a novel that reformulates training of neural SR models with Weighted finite-state transducers (WFSTs). In this project, we would like to incorporate into this framework DAT (a neural network training scheme that learns to disregard differences between some predefined domains) in order to make SR models more robust to highly differing source and target data (e.g., accents). The implementation should be further exhaustively tested on the Speech Accent Archive dataset that we have obtained.
- Links:
[JKO] Physics informed neural networks (PINN) for object tracking in sports
- Student:
- Namespace in the wiki: FIXME
- The goal of the project: Paper review and adjusting a chosen PINN for single object traking.
- Technology: Python, Julia
- Description: The ultimate goal (larger than this project) is to create a model to infer the movement of table tennis players to analyse their gameplay from single camera videos. One has to merge (a) pose estimation models (including a detailed hand position estimation) with (b) ball trajectory tracking and informing (a) with the physical parameters inferred from (b). The challenge is low sampling (only a couple of frames per one ball shot), blurr, camera angles make depth estimation hard, etc. There are a number of existing implementations of neural networks that explicitly incorporate physical equations/quantities. The goal it to find a suitable one, scrap a small amount of data (we can actually record high quality data ourselves), and try it out. Possibly there are Julia and Python alternatives to be considered.
- Links (as a starting point):
[FIXME] Template
- Student:
- Namespace in the wiki: FIXME
- The goal of the project:
- Technology:
- Description:
- Links: