Summary
The strategic agreement between Ferrovial and NTT DATA in 2023 established the Center of Excellence for Digital Infrastructure Solutions. Building on this collaboration, NTT DATA developed a multimodal AI solution (VLM/MLLM) that analyzes images and video to enable continuous monitoring and detect predefined situations.
Business need
A smarter approach to infrastructure monitoring
Large infrastructure operators need to monitor and control their assets with the utmost precision to maintain safety and continuity. Traditional methods are slow, rely on manual inspections and can’t fully leverage the large amount of visual data generated. In this context, GenAI is a key ally in anticipating risks and enabling immediate responses.
Ferrovial needed a centralized technology solution capable of monitoring its infrastructure in an agile way to accelerate processes, minimize delays and costs, and provide real-time visibility to improve decision-making. It required a scalable and flexible platform that could be deployed across multiple locations without significant additional investment.
Ferrovial wanted to assess the potential of multimodal AI for video analysis in specific construction and highway use cases.
The speed at which data is interpreted determines how quickly immediate action can be taken, helping to prevent larger disasters. Early intervention is essential. That’s what we want to achieve with this project: improve incident management and make our infrastructure safer. NTT DATA’s work has been key in this regard.”
Solution
Transforming video into operational intelligence
To address this need, and as part of the Center of Excellence for innovation and technology solutions for infrastructure, NTT DATA developed a proof of concept (PoC) that uses data from real-world cameras to evaluate the potential of generative AI for video analysis. The PoC is built on an innovative technology platform powered by GenAI multimodal models.
The solution was first used to analyze recorded footage in controlled environments, enabling AI to interpret scenes with high levels of accuracy. Its use was then extended to live video, enabling a continuous monitoring system capable of detecting patterns, generating alerts and delivering insights instantly.
This innovative solution integrates visual data to monitor infrastructure in real time. This enables the identification of anomalies, the prediction of risks and the generation of automated recommendations, while adapting to the growth and diversity of managed assets. The solution is expected to incorporate additional data sources and be deployed in live production environments in the future.
This new GenAI-based tool offers the following capabilities:
- Zero-shot capability and flexibility allows the tool to be applied across different scenarios by simply adjusting prompts, without training or fine-tuning — unlike traditional computer-vision solutions that require targeted adaptation
- Scalability supports the processing of large volumes of video, live streaming and simultaneous deployment across multiple locations
- Operational efficiency supports more agile and automated processes
- Data-driven management provides objective, real-time information for project management and infrastructure operations
- Cross-industry applicability extends beyond construction and highways to sectors such as energy, transport and heavy industry
- Real-time visibility optimizes decision-making
- Instant responses
This solution can be applied to multiple use cases, including tracking project progress, monitoring sites, analyzing people, vehicle and machinery flows, monitoring and managing during operations and rapidly responding to emergencies.
Thanks to its scalable architecture, Ferrovial benefits from a robust, flexible system designed to evolve alongside new operational needs.
This joint project for Ferrovial demonstrates that large vision models — as an advanced form of GenAI applied to vision — together with real-time analysis and advanced automation, represent a new paradigm in infrastructure monitoring and management.”
Outcomes
Achieving real-world impact with AI
The large-vision-model platform delivers:
- A versatile solution that supports multiple use cases across different sectors with minimal changes and no training required
- Strong performance in real-world scenarios, achieving 70% accuracy in conducted tests
- Immediate generation of detailed reports for complex scenarios
- Rapid identification of alerts for specific situations within seconds