Service

DataHub

Access key scientific datasets in a scalable way

DataHub allows you to enable simple and scalable access to distributed data for computation, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites.

Overview

DataHub

DataHub allows you to bring data close to computing to exploit it efficiently, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites.
DataHub is based on the Onedata technology.

Main features

With EGI DataHub, you’ll enjoy a range of unique benefits:

Discover data with ease: Our central portal makes it simple to see the datasets that are available for exploitation and request access to it, saving you time and effort.
Customisable access: Whether you require unauthenticated open access, user registration, or restricted access for members of a Virtual Organisation, we’ve got you covered.
Multiple access options: Access data via GUI, POSIX or REST API, depending on your needs.
Resilient replication: Our platform is built to ensure maximum uptime and availability by replicating data from providers on-demand or automatically.
Authentication and Authorisation Infrastructure Integration (AAI): The integration with EGI Checkin seamlessly links EGI DataHub with other EGI components and with user communities existing infrastructure to ensure secure and convenient access to data.
Metadata and Shares Management: EGI DataHub simplifies the process of managing metadata and shares, saving you time and effort.
Data Import and Caching: Our platform uses file popularity to cache data, ensuring quick and easy access for users.
Multiple Backend Support: With support for CEPH, S3, GlusterFS, POSIX, and more, EGI DataHub is truly versatile and convenient.

TRL 8: Actual system proven in operational environment.

See the DataHub privacy policy to learn how personal data are processed when using this service.

LAGO: Unleashing astroparticle research in Latin America with EGI

LAGO is a collaborative organisation of IberoAmerican institutions operating the Latin American Giant Observatory. The observatory is a network of astroparticle detectors used for high-energy astrophysics, and space weather phenomena research across 10 Latin American countries. Facing data management and resource sharing challenges, EGI stepped in with a combination of services, including the EGI DataHub that empowered LAGO researchers to store, publish, and access crucial data and metadata. The service enabled them to access a 15TB cloud-stored library containing the expected flux of signals produced by more than 1013 cosmic rays showers. This service has been crucial in ensuring the continuity of research programmes in Latin America.