Programming Assignment I

Select one of the assignments below. Assignments marked with can be extended as a continution in Programming Assgnment II. Projects marked with can be (with some additional work and depending on the results) published as scientific papers.

Main goal is to use the tobii-pytracker software and MVTec dataset to design and conduct pilot study on how XAI methods and humans “analyze” images in the task of anomaly detection. Note: you can use eyetracking device emulaiton that is implemented in the tobii-pytracker software, but also try a real hardware if you wqant.

In particular this assumes:

1. Obtain an MVTec datset and plug it in to the tobii-pytracker
2. Train anomaly-detecot using patchcore algorithm: see how to do this – its a one-liner.
3. Get human-eyetracking results and compare with heatmaps that can be obtained from anomalib. Calculate similarities.

Additional extensions:

1. Use more models than just patchore and XAI methods like SHAP which are model-agnostic
2. Implement customModel for tobii-pytracker, that will create bounding boxes, cor convex hulls for the detected anomalies at runtime (See CustomModel section).

The goal is to take the ACFX software and test it both from the perspective of online app: Streamplit app and SDK.

Any issue, bug or inconsistency with documentaiton should be reported to the issue tracker.

Possible extensions to a paper/master thesis: how to use thes emethod to other than tabular cases/models, for instance: Graph neural networks, or images?

Testing WinCLIP model for explainable anomaly detection. See if the model can be used to obtain textual explanations of a detrected anomalies.