A Scottish consortium of geospatial and environmental data experts has developed new standards to assess the reliability of Artificial Intelligence (AI) and Machine Learning (ML) algorithms used in environmental mapping.
Funded by Innovate UK, the Trustable AI in Mapping (TAiM) initiative aims to establish clear evaluation standards for AI-driven environmental mapping, ensuring its credibility and effectiveness in real-world applications.
AI and ML technologies have the potential to revolutionise environmental mapping by automating feature identification and analysis from remotely sensed data. They offer greater efficiency over traditional methods like manual digitisation and Geospatial Information System (GIS)-based operations. However, concerns over accuracy, reliability, and the “black box” nature of AI models have slowed adoption.
“Without standardised assessment methods, users struggle to fully trust AI-driven mapping solutions,” said Douglas McNeil, Managing Director of EOLAS Insight Ltd. “By developing ‘trustable AI,’ we can help users understand the strengths and limitations of these technologies, fostering confidence in their application.”
To tackle these challenges, the TAiM consortium—which is led by EOLAS Insight Ltd and includes Agrimetrics, Highlands Rewilding, the James Hutton Institute and Omanos Analytics—is creating a standardised framework to assess AI model performance for land cover mapping. This involves developing example AI algorithms to test the system and generating high-quality validation datasets. The resulting frameworks will be openly published, enabling organisations to assess and refine their own AI models with greater confidence.
“Setting industry standards and encouraging open methodologies will accelerate AI adoption in geospatial applications,” Douglas added. “By building trust through transparency, we can support data-driven decision-making that benefits the entire environmental sector.”
The key outcome of this year-long project, concluding in March 2025, is a website featuring a map-accuracy data portal and a knowledge base. The data portal enables map producers and users to assess AI-derived maps by comparing them with high-quality field data, including detailed habitat maps, sub-metre forest canopy height and surveys of specific ecosystems like salt marshes, grasslands and peatlands. Meanwhile, the knowledge base offers articles on best practices for understanding and testing AI map accuracy, along with case studies demonstrating how the portal and its data can support effective map evaluation.
Data Scientist Pierre Stratonovitch from Agrimetrics said: “Agrimetrics developed new Earth Observation land cover mapping models to identify a range of natural habitats, woodlands, and crop species. It is always difficult to obtain high-quality validation datasets and so we’re delighted that the TAiM project generated an impressive dataset offering a wide range of observations. The assessment frameworks developed by the consortium will set the industry standard on how Earth Observation models should be assessed.”
Cathy Atkinson, Chief Data Scientist at Highlands Rewilding said: “Highlands Rewilding has carried out detailed ground surveys across a range of habitats, including woodland, grassland, peatland and saltmarsh. These ecological surveys provide robust data against which outputs from AI algorithms can be assessed.This data also demonstrates how ground surveys can be used together with AI to reduce errors and uncertainties in environmental monitoring, vital for the natural capital industry.”
Dr Hannah Rudman from the James Hutton Institute, who worked on the project with her colleague Dr Matt Aitkenhead, said: “The use of AI for mapping Scotland’s landscape is making the process faster and reducing costs. However, field data will always be needed to drive the modelling, and the data products are interpreted differently from ground survey mapping. The TAiM project is helping to resolve challenges and increase trustability in this new kind of knowledge.”
Clare Rumsey, Technical Director at Omanos Analytics said: “At Omanos Analytics, we focused on understanding ground-truthing needs for Earth Observation-based, AI-driven mapping. To measure AI mapping accuracy, we supported fieldwork that surveyed areas using a classification system suitable for satellite-derived imagery. We also developed ‘QuickBeam’, a new smartphone app designed for simple, user-friendly ground-truthing of AI mapping products. By allowing users to add extra information like condition and biodiversity, Quickbeam brings together the power of EO and on-the-ground expertise to mapping products, increasing confidence and trust for non-experts.
The TAiM website and data portal can be accessed at (NB. this website will be live by 10:00 AM Monday 31st April: https://www.trustable-mapping.xyz
