EISCAT is a non-profit scientific organisation that operates four radar antenna sites to enable research on the lower, middle and upper atmosphere and ionosphere using the incoherent scatter radar technique. Its research facilities are located in three main countries: Finland, Norway and Sweden, North of the Arctic circle.
Environmental Science
Supporting the Development of Infrastructures to study the Earth’s Atmosphere with EISCAT_3D
About
The EISCAT_3D radar system will provide a thirty-year update to EISCAT’s existing mainland facilities, exploiting the capabilities of state-of-the-art electronics, networking, storage and computing. In doing so, it will outstrip the capabilities of any such radar operating in the world and become the centrepiece of the international network of instruments monitoring the Earth’s upper atmosphere and space environment in the first half of the twenty-first century.
The challenge
The new system will open up new opportunities for physicists to explore a variety of research fields, but it comes with significant challenges in handling large-scale experimental data. The incoming data rates from all 60000 receivers amounting to 100Tb/s in total need to be reduced to a few PB/year to be within funding limits.
The EISCAT_3D research infrastructure has to be set up to make this data browsable and analysable for researchers, as well as archiving it for the long-term (it is planned to archive around 2PB per year). This primary mission is achieved by procuring sufficiently big storage and network capacities at a few data/compute centres (for back-up or load distribution). The data will be transferred to these locations after initial filtering and calibration at the data source(s). The data/compute centres are responsible for data archival, data curation, generation of derived data and for sharing the data with scientists. Data must be shared with scientists via a data access portal and programming APIs.
The EISCAT Data Access Portal should offer the following main features:
Authentication and Authorization: Any researcher should be able to access the portal and browse metadata. The portal grants/denies access to data and processing based on affiliation.
Data browser: Authorised researchers should be able to browse reference applications in the portal, select an application for use, feed their data in, visualise or download the analysis result.
Data download: Authorised researchers should be able to select the EISCAT_3D data they are interested in for download.
Online computing: Authorised researchers should be able to analyse data without downloading them, using cloud resources, reference applications or user’s own software.
The Solution
To support the development of a working prototype of the EISCAT Data Access Portal, the following EGI services have been used/investigated:
The EGI Workload Manager, is customized for the community requirements that provides searching, retrieval and reprocessing (visualisation, analysis) of data products generated by EISCAT radars.
The EGI Cloud Compute and the cloud-based EGI Online Storage to distribute the computational task to a scalable compute platform and to store intermediate results from the user jobs.
The EGI Check-In and Perun are integrated to enable user registration, authenticated and authorised access to the EISCAT Data Access portal and to the underlying distributed compute infrastructure.
Services Provided by the EGI Federation
Impact
12 Members
6 associated member states, 6 affiliated member institutes, 3 continents
10TB of Data per Year
produced by three EISCAT radars
3-level Datasets
accessed by thousands users every year from all continents
+50 Publications
per year, in refereed journals, with multi-national contributions
Highlights
Publications
The list of publications submitted by the EISCAT Scientific Association can be found in the EISCAT repository. A total number of 69 publications were published during the period 2020-2021.
Tutorial Materials and Courses
User documentation was made available and training has been delivered during EISCAT user forums. Training materials are available in the EISCAT repo.
Participating Partners
