Corporate CDMS and Climate Change AI: ML integration for advanced hydrometeorological data management
CDMS Aziendale e Climate Change AI: integrazione ML per la gestione avanzata dei dati idrometeoclimatici
The research and development line dedicated to the Climate Data Management System represents one of the most solid and ongoing initiatives carried out by Smart Shaped in recent years. After the first experiments in 2022, focused on the management and analysis of hydrometeorological data from measurement stations and observational platforms, the work continued in 2023 with operational tests on an IoT architecture for the intelligent collection of climate data in the Dominican Republic. In 2024, the path took a further step forward, shifting the focus to the integration between the corporate CDMS based on SURFACE and a Machine Learning module dedicated to advanced weather forecasting, with the aim of further enhancing the information assets managed by the platform.
The activities carried out in 2023–2024 are part of a project supported by the ERDF funds of the Abruzzo Region, which has recognized climate data management and modelling as a high-impact area for innovation and competitiveness in the region. The combination of CDMS infrastructure, predictive models and a dedicated data fusion layer now represents a key building block for enabling intelligent applications in sectors such as environmental monitoring, energy planning and climate risk management.
/ Growing market and climate data
CDMSs are becoming essential components of national and international infrastructures, also thanks to the standards promoted by the World Meteorological Organization, which certify the quality, interoperability and correctness of information flows.
/ AI for climate prediction
Alongside the evolution of climate technologies, Artificial Intelligence is also evolving, transforming the ability to learn from data and generate accurate and scalable predictive models. Machine Learning platforms are seeing significant increases in industrial adoption, while solutions based on generative models and automatic learning techniques are advancing, capable of overcoming the limits of traditional numerical methods.
/ Integration of SURFACE, GenCast and Data Fusion Layer
The project integrates three components into a single architecture: SURFACE CDMS, the GenCast model and a new Data Fusion Layer. The integration of these three blocks takes place via an FTP server that acts as an exchange node, keeping the pipeline simple and allowing future evolution of the ML model without changing the architecture.
/ Integration goal
The main goal of the project was to integrate three key elements into a single functional architecture: SURFACE CDMS, the GenCast model and the new Data Fusion Layer designed for heterogeneous management of data sources. SURFACE continues to represent a reliable foundation for the management, validation and consultation of climate data, thanks to its compliance with WMO standards and its ability to manage the entire data life cycle, from collection to permanent storage.
Alongside the CDMS, GenCast introduces a completely new approach to weather forecasting. It is a probabilistic model based on conditional diffusion techniques, capable of generating global 15‑day ensembles with a resolution of up to 0.25 degrees, offering better performance than operational ensemble systems based on traditional NWP. Its ability to realistically model atmospheric trajectories, capturing the spatio‑temporal variability of events, makes it a particularly promising tool for applications such as risk analysis, renewable energy and environmental monitoring.
Connecting these two elements is the Data Fusion Layer, a component developed to acquire hydrometeorological data from external sources that may be very different from each other, normalize them consistently and make them available both to the CDMS and to the ML model. The layer implements mechanisms for download, validation, pre‑processing and distributed storage, ensuring a uniform and reusable data pipeline. The modular design also makes it possible to extend the system with new providers or new machine learning models without changing the overall architecture.
Operational integration takes place through an FTP server that acts as a point of exchange between CDMS, GenCast and Data Fusion Layer. This approach keeps the pipeline simple, while ensuring a high degree of flexibility and the possibility of replacing the ML model in the future with similar solutions without significant impacts on the infrastructure.
/ Challenges and solutions
The integration work brought to light some critical issues, mainly related to the management of very large, heterogeneous and highly variable datasets, which require complex normalization processes to ensure consistency and quality of results. A further challenge concerns the computing capacity required to run GenCast at its native resolution: a powerful and accurate model, but demanding in terms of resources, especially in research and development contexts.
To overcome this limitation, experiments were carried out using the 1‑degree‑resolution version, which is lighter and more suitable for the infrastructures currently available. Solutions based on scalable cloud infrastructures and HPC resources were also evaluated, which could in the future allow systematic adoption of the 0.25‑degree version. From the point of view of the pipeline, the modularity of the Data Fusion Layer and the CDMS is a key element for maintaining robustness, management simplicity and expansion possibilities.
/ Current status and future evolution
The ERDF project concluded with the creation of a complete operational demo at TRL 7, in which SURFACE, GenCast and Data Fusion Layer work in a coordinated way to generate weather forecasts based on real and historical data collected by the CDMS. This first integration already makes it possible to observe the potential of the AI‑driven approach applied to the climate domain, paving the way for more advanced future applications.
This line of work is now one of Smart Shaped’s most promising assets in the field of Climate Change AI, confirming the company’s commitment to developing intelligent systems for understanding, forecasting and managing climate phenomena in an increasingly data‑driven world.
