PANGEO Boosts Geospatial Analysis with EGI Services at GEO-OPEN-HACK and IEEE IGARSS

With contributions by Anne Fouilloux, Simula Research Laboratory, and Tina Odaka, IFREMER.

PANGEO has recently spearheaded two dynamic training events, GEO-OPEN-HACK 2024 in Austria and the 2024 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) in Greece, also exploiting the quality of EGI services for geospatial analysis and Earth observation.

Held 24-28 June under the aegis of the International Institute of Applied Systems Analysis (IIASA) and Spatial Ecology, GEO-OPEN-HACK 2024 was a comprehensive event aimed at advanced geo-coders. This hackathon focused on exploring open tools and advanced techniques for processing large geospatial datasets using High-Performance Computing (HPC) infrastructure. The event provided hands-on experience with HPC resources, enabling participants to tackle tasks such as geospatial data preprocessing, spatial modelling, and visualisation.

The tutorial began with an introduction to Pangeo, emphasising scalable Earth Science workflows and data-intensive research. It then explored Xarray for managing, manipulating, and visualising multi-dimensional datasets, highlighting chunking and using Dask for lazy computations on large data arrays. 

One of the main highlights was showcasing the synergy between cloud and HPC technologies via the Pangeo software stack. This demonstrated Pangeo's scalability with Dask, from personal laptops to HPCs. The Pangeo training provided deep insights into managing and processing large geospatial datasets, enhancing participants' skills and equipping them with new techniques.

Starting on the Copernicus data space ecosystem and moving to Pangeo@EOSC, we demonstrated how Pangeo with Dask can be easily leveraged on HPC infrastructure, enabled by SURF. This illustrated how substantial computing resources could be utilised for effective geospatial analysis. The Pangeo environment, with its cloud-native approach, eased data management and streamlined the preprocessing phase.

Additionally, the Pangeo tutorial enabled seamless scalability in visualisation, allowing participants to transition smoothly from local data to larger global datasets, creating interactive and highly informative visualisations.

Access the contents

"Delivering the training at GEO-OPEN-HACK 2024 was an incredibly rewarding experience. We have learned a lot from the geo community.  Using Pangeo-EOSC and EGI services, we provided participants with invaluable hands-on experience on how to scale from their laptops to the Cloud and High-Performance Computing resources. We hope this training significantly enhanced their ability to process and analyse large geospatial datasets."  Tina Odaka, IFREMER, France

At the IEEE IGARSS 2024, held on 7-12 July in Athens, Greece, PANGEO conducted a tutorial titled “Earthly Marvels Revealed: Pangeo, AI, and Copernicus in Action.” This session was designed to introduce participants to scalable Earth Science workflows within the Pangeo ecosystem, with a particular focus on utilising Artificial Intelligence (AI) for Earth Observation (EO) data analysis, especially from the Copernicus program.

Participants engaged in hands-on sessions that demonstrated how to use AI techniques on Copernicus datasets, employing tools like Xarray, Zarr and FOSCAT. The tutorial also included an introduction to computer vision models hosted on the EO4EU platform, enabling participants to perform tasks such as segmentation and object detection.

Access the contents

"Conducting the tutorial at IEEE IGARSS 2024 was an exciting opportunity to share our expertise with participants. Utilising Pangeo, EOSC and EGI services, we provided hands-on training on scalable Earth Science workflows and AI techniques for Earth Observation data analysis, focusing on Copernicus datasets. The training covered advanced tools like Xarray, Zarr, Dask, and FOSCAT for gap-filling. Despite limited time and internet connection, the extra work sessions ensured participants could retry at home. The knowledge gained will be instrumental in advancing their Earth Observation projects." Tina Odaka, IFREMER, France

The integration of EGI services across both events was instrumental in their success. The Cloud Container Compute service provided by CESNET via Masaryk University enabled participants to manage and scale their computational tasks efficiently, while the Online Storage service ensured seamless data management. The Check-In service by GRNET provided secure and hassle-free access to all required resources, enhancing the overall participant experience.

Through these training events, PANGEO not only empowered participants with advanced knowledge and skills but also demonstrated the key role of EGI services in fostering geospatial analysis and Earth observation. The successful implementation of these services highlights their potential to support future initiatives aimed at addressing global challenges through innovative technological solutions.

For more information on EGI services and their applications in geospatial analysis, visit the links below.