Detecting FIMI: A methodological framework for testing OSINT tools in TTPs detection.

BACKGROUND: Foreign Information Manipulation and Interference (FIMI) represents a growing hybrid threat targeting democratic processes, social cohesion, and diaspora communities. Detecting it requires identifying underlying tactics, techniques, and procedures (TTPs) described in frameworks such as DISARM, AMITT, and MITRE ATT&CK. Although many open-source intelligence (OSINT) tools exist, their evaluation remains fragmented and lacks methodological standardisation. This article presents a framework for testing OSINT tools for TTP detection, developed within the Horizon Europe project RESONANT.

METHODS: The methodology integrates three components: a structured multi-source collection of OSINT tools informed by academic, operational, and grey literature; a controlled, mixed-source evaluation corpus of project-prepared examples, public reference material, and selected third-party datasets across text, multimedia, network and infrastructure analysis; and controlled testing by law enforcement partners from Greece and Ukraine. Tools were classified into eight categories and evaluated using predefined indicators of accuracy, usability, interoperability, coverage of adversarial behaviours, and sustainability.

RESULTS: Several content-verification and multimedia-forensics tools performed well on the retained material, and infrastructure-level tools gave useful indicators; these are time-bound descriptive observations, not general rankings. Exact case-level replication is limited, as the package does not preserve the full CT-A/CT-B mapping, run-level outputs, denominators, or configurations. Social media analytics and bot-detection tools were least stable under platform API restrictions. Shortcomings recurred in multilingual processing, interoperability, explainability, and integration into threat-intelligence formats.

CONCLUSIONS: The study offers a transferable methodology for evaluating OSINT tools in the context of FIMI TTP detection. Combining a mixed-source corpus, structured testing, practitioner validation, and publication of the available material, it advances methodological transparency while clarifying the limits of both automated outputs and the retained replication record. The proposed tool suite is a layered decision-support baseline, not a verdict-producing system. Future work should expand multilingual and out-of-distribution testing, operationalise bias and explainability metrics, preserve run-level records, and pursue lawful access to platform data.