Researcher explores visual media through the lens of machine vision

Large visual collections, such as paintings, photographs, drawings, and other forms of visual media, offer valuable insights into historical events, social life, and artistic expression. These collections are key to understanding how societies produce and use images to shape cultural meaning over time. Yet they remain difficult to study due to their sheer size, often consisting of hundreds of thousands of items, and their intrinsic complexity, including diverse visual features, contents, contexts, and metadata structures.

In his doctoral thesis, Tallinn University School of Digital Technologies student Tillmann Ohm proposes a new way to explore large visual collections: through the lens of machine vision, focusing not on what an image is but on what it resembles. Instead of sorting images into fixed categories, the approach arranges them in maps based on visual similarity as perceived by algorithms.

This results in similarity spaces, generated from mathematical representations of images, where the distance between two points reflects how similar the images appear. Understanding how visual similarity is modeled, perceived, and interpreted by both humans and machines forms a central research question of the dissertation.

A core contribution of the thesis is the Collection Space Navigator, a browser-based interface that allows researchers and curators to explore visual collection data. Two-dimensional similarity maps enable open-ended, interpretive inquiry by revealing patterns, clusters, and visual relationships that are usually hard to detect using traditional metadata or keyword-based methods.

The interface supports interactive navigation, filtering, and comparison of different models and methods, making it adaptable to a wide range of research questions and collection types. By facilitating visual exploration at scale, it bridges computational analysis with human expertise in cultural interpretation.

The Collection Space Navigator has been applied to a range of cultural heritage collections and integrated into interdisciplinary research workflows. A key case study analyzed more than 200,000 frames from Soviet newsreels, using the tool to uncover long-term visual patterns in propaganda film.

Clusters of similar images revealed recurring motifs, such as staged leadership scenes in front of the Lenin Mausoleum, while others shifted in tone over time, such as formal negotiation scenes at long tables, reflecting changing propaganda strategies. These evolving and persistent visual narratives are difficult to detect through manual viewing and became immediately apparent through similarity-based exploration.

Ohm’s research contributes to the growing field of cultural data analysis. It enables museums, scholars, and the public to gain fresh insights into visual culture by challenging both institutional and algorithmic authority, while fostering cross-disciplinary collaboration and deepening our understanding of cultural narratives through innovative computational tools.

Ohm defended his doctoral dissertation “Designing Processes and Tools to Research Similarity Spaces of Visual Collections” on 11 June. Thesis supervisor was Maximilian Günther Schich, Professor at Tallinn University. Opponents were Lauren Tilton, Professor at the University of Richmond and Iyad Rahwan, Professor at the Max Planck Institute for Human Development.

Provided by
Estonian Research Council