EGI
EGI
  • 143
  • 34 003
EGI DataHub: an innovative platform for the ingestion, management and publishing of distributed data
00:00 - Yin Chen (EGI) - Welcome and introduction
04:48 - Łukasz Opioła (Onedata, CYFRONET) - EGI DataHub: an innovative platform for ingestion, management and publishing of distributed scientific data
12:53 - Chapter I - Ingestion
23:15 - Chapter II - Data management
35:25 - Q&A 1
39:05 - Galaxy + Onedata = workflows that understand and embrace data distribution
41:30 - The EUreka3D Project
47:35 - Chapter III - Publishing and Archiving
58:20 - Q&A 2
1:04:10 - Demo for publishing and archiving
1:11:40 - Q&A 3
Key topics:
Comprehensive data management flow; from ingestion, through processing and annotation, to publishing and archiving.
Enhanced Open Data publishing; among numerous improvements, the platform now supports the Europeana Data Model metadata. We will showcase the EUreka3D project, a real-world application relying heavily on DataHub.
New data access interfaces, including S3 and a lightweight PyFilesystem (Python) plugin, to enhance flexibility and accessibility for various use cases.
Evolved GUI; particularly the file browser has undergone significant improvements, making data management more intuitive and efficient.
This webinar is designed for a diverse audience, including:
Researchers and data scientists seeking efficient access to distributed data.
Developers interested in integrating with the Onedata system or creating middleware solutions.
Administrators and data providers looking to leverage the latest platform features.
Speaker
Łukasz Opioła - Researcher, Onedata team leader, PhD candidate
Łukasz Opioła is a researcher at Academic Computer Centre CYFRONET, AGH University of Krakow, involved in multiple Polish and European research projects. Developer and team leader of the Onedata project - a distributed data management platform for science. Research areas: virtual filesystems, distributed and decentralized architectures, data access control, and AAIs.
Find the slides and more information on the EGI Secrets Store on our website: go.egi.eu/tCvmW
Переглядів: 49

Відео

EGI: Advanced Computing for a Data-Driven Future
Переглядів 17 тис.Місяць тому
In today's digital world, data is everything. Data guides our decisions, facilitates learning, cracks challenges, and fuels the innovation of new technologies. Data is the foundation of groundbreaking research helping to solve the problems our society is facing. But data alone is not enough. Researchers need access to essential tools and services that are key to unlocking the insights hidden wi...
Webinar: Managing Secrets in the EGI Infrastructure with EGI Secrets Store
Переглядів 222 місяці тому
00:00 - Yin Chen (EGI) - Welcome and introduction 04:15 - Viet Tran (IISAS) - Managing Secrets in the EGI Infrastructure with EGI Secrets Store 12:31 - Demonstration: GUI and CLI 19:34 - Real use case: Federated learning server (example from AI4EOSC) 24:44 - Advanced features of EGI Secrets Store 37:00 - Q&A In this session, we will explore the EGI Secrets Store, a powerful solution designed to...
Discover EGI2024 in Lecce, Italy! | 30 Sep - 04 Oct 2024 | Secure Your Early Bird Tickets!
Переглядів 762 місяці тому
Be part of #EGI2024 in Lecce, Italy, from 30 Sep to 04 Oct 2024! Connect with global experts in data-intensive research and innovation. Hosted at the Hilton Garden Inn and in collaboration with CMCC Foundation, a leading research institution. Explore key topics such as Scientific Computing, EOSC, Platforms and Gateways, Open Science, AI and ML, Data Innovations, Environmental Informatics, Trust...
Webinar: TOSCA-based cloud orchestration and deployment with Infrastructure Manager (IM)
Переглядів 403 місяці тому
00:15 - Miguel Caballer (UPV) - TOSCA-based cloud orchestration and deployment with Infrastructure Manager (IM) 35:20 - Q&A The Infrastructure Manager (IM) is a framework that eases the access and usability of IaaS clouds by automating the deployment, configuration, software installation, monitoring, and update of Virtual Appliances. It supports APIs from a large number of virtual platforms, ma...
Harnessing Advanced Research Infrastructures for Earth & Environmental Data Management and Analysis
Переглядів 903 місяці тому
00:30 - Gianni Della Torre (EGI Foundation) - Welcome 04:40 - Kostas Leptokaropoulos (Matlab) - Harnessing Advanced Research Infrastructures for Earth and Environmental Data Management and Analysis: A MATLAB Perspective 41:15 - Q&A The increasing volume and complexity of Earth and environmental data requires efficient interdisciplinary collaboration and management. The use of advanced research ...
Webinar: Reproducible Open Science with EGI Replay
Переглядів 444 місяці тому
April 17 2024 In this webinar we showcase EGI Replay, a managed service for the EGI Community based on Binder. EGI Replay offers users the ability to effortlessly reproduce intricate calculations, simulations, and visualizations by importing Jupyter Notebooks and their respective runtime environments. With a single link, users can easily share their work with collaborators, making Replay an inv...
Webinar: Introduction to the new generation EGI container execution platform
Переглядів 1937 місяців тому
The new generation EGI container execution platform enhances the well known and widely used EGI Cloud Container Compute service into a system that delivers fully managed Kubernetes clusters for end users. The event introduces how to build container images, how to store and share them through the Harbor repository, and how to run them on fully managed Kubernetes clusters with the Rancher service...
20230604_Webinar: I-NERGY, European AI-on demand platform
Переглядів 41Рік тому
Description: I-NERGY aims to support and develop novel AI-based energy services as part of the enrichment of European AI-on demand platform. This webinar will present the objectives and scope of the project, its requirements in terms of resources and the successful utilisation of EGI infrastructure. The webinar will conclude with a demo of I-NERGY services. Target audience Domain scientists and...
20230628_Webinar: Access to the EISCAT tools with help of EGI checkin
Переглядів 54Рік тому
Description: The present era of rapid technological advances creates a challenge for data providers and scientists to create and maintain FAIR data and services not just for future operations but also for historical data gathered and analysed with technologies that are slowly phasing out of their usage. GUISDAP is an open-source software package, written in MATLAB, C and Fortran and provided an...
20230614_Leveraging the Onedata Platform for Long-Term Data Archiving and Processing
Переглядів 48Рік тому
In this presentation, we will discuss the latest advancements in the Onedata platform, focusing on its new features for long-term data archiving and processing. We will demonstrate how the platform has been optimized to meet the Open Archival Information System (OAIS) standards, ensuring the reliable preservation and accessibility of archived information over time. Furthermore, we will explore ...
Unveiling the Impact of the EGI-ACE Project: A Journey of Innovation and Success
Переглядів 106Рік тому
Unveiling the Impact of the EGI-ACE Project: A Journey of Innovation and Success
20230329_Webinar: Distributed Deep Learning with Horovod
Переглядів 143Рік тому
20230329_Webinar: Distributed Deep Learning with Horovod
20221212: How to use accelerated computing resources in EGI for supporting Ukraine researchers
Переглядів 34Рік тому
20221212: How to use accelerated computing resources in EGI for supporting Ukraine researchers
Recording
Переглядів 44Рік тому
Recording
The EGI-ACE webODV - Online extraction, analysis and visualisation of SeaDataNet and Argo data
Переглядів 257Рік тому
The EGI-ACE webODV - Online extraction, analysis and visualisation of SeaDataNet and Argo data
EGI-ACE - Free services supporting data- and compute-intensive research
Переглядів 1352 роки тому
EGI-ACE - Free services supporting data- and compute-intensive research
20220427_Webinar: Supercomputing in Brazil and the LNCC
Переглядів 512 роки тому
20220427_Webinar: Supercomputing in Brazil and the LNCC
20220406_Webinar: FAIR EVA (Evaluator, Validator & Advisor)
Переглядів 1792 роки тому
20220406_Webinar: FAIR EVA (Evaluator, Validator & Advisor)
20220330_Webinar How Green Is My Infrastructure?
Переглядів 302 роки тому
20220330_Webinar How Green Is My Infrastructure?
20220323_Webinar: The Virtual Imaging Platform: Scientific Applications as a Service and Beyond
Переглядів 1482 роки тому
20220323_Webinar: The Virtual Imaging Platform: Scientific Applications as a Service and Beyond
20220309_ENES Data Space Service Webinar
Переглядів 882 роки тому
20220309_ENES Data Space Service Webinar
EGI at ENVRIweek2022
Переглядів 682 роки тому
EGI at ENVRIweek2022
20220112_Webinar: openRDM by Priyasma Bhoumik (ETH Zurich)
Переглядів 1872 роки тому
20220112_Webinar: openRDM by Priyasma Bhoumik (ETH Zurich)
20211019_EGI2021 Keynote Ivan Marić : 50 years of SRCE
Переглядів 312 роки тому
20211019_EGI2021 Keynote Ivan Marić : 50 years of SRCE
20211019_Keynote Pierre Etienne Macchi (IN2P3) 50 years of IN2P3 -
Переглядів 292 роки тому
20211019_Keynote Pierre Etienne Macchi (IN2P3) 50 years of IN2P3 -
20211019_EGI2021 Tiziana Ferrari Scientific Computing with EGI: Current Status & Future Outlook
Переглядів 482 роки тому
20211019_EGI2021 Tiziana Ferrari Scientific Computing with EGI: Current Status & Future Outlook
20201029_EGI Operations and responsibilities of an NGI
Переглядів 362 роки тому
20201029_EGI Operations and responsibilities of an NGI
20211201_How to train your AI models in EOSC
Переглядів 2342 роки тому
20211201_How to train your AI models in EOSC
20211124_Webinar: The Role of Research in Data Spaces and Data Ecosystems
Переглядів 952 роки тому
20211124_Webinar: The Role of Research in Data Spaces and Data Ecosystems

КОМЕНТАРІ

  • @EOSCBeyond
    @EOSCBeyond Місяць тому

    Looking forward to further developing EOSC Core Services with EGI!

  • @envricommunity
    @envricommunity Місяць тому

    Great video, EGI! The ENVRI community is excited to partner with you on developing the next generation of ENVRI-Hub through the ENVRI-Hub NEXT project (enabling open science!)

  • @MarcosLlopisTorregrosa
    @MarcosLlopisTorregrosa Місяць тому

    🚀🚀🚀🚀

  • @yurpol
    @yurpol Місяць тому

    Amazing video!

  • @JockeUni
    @JockeUni 3 місяці тому

    Why don't you use podman to build your containers?

  • @bassome3000ify
    @bassome3000ify Рік тому

    How much

  • @tizianaferrari470
    @tizianaferrari470 2 роки тому

    Thanks to the European Commission for the trust and funding received to support this project, and most importantly to the EGI members and the EGI Foundation that make this possible by uniting European data centres to support international scientific collaborations!

  • @satyajitghosh6384
    @satyajitghosh6384 2 роки тому

    Thank you for uploading. 🙏

  • @books4493
    @books4493 2 роки тому

    GOOD JOB

  • @shubochakrabarti
    @shubochakrabarti 3 роки тому

    ua-cam.com/video/Hbf1yg32sso/v-deo.html

  • @shubochakrabarti
    @shubochakrabarti 3 роки тому

    ua-cam.com/video/zT9aW1xHCJU/v-deo.html

  • @nikhilnick8886
    @nikhilnick8886 3 роки тому

    Thanks for sharing

  • @landondelilah1031
    @landondelilah1031 3 роки тому

    Awesome vids! You deserve more views. I think you should google smzeus . c o m!!! Loads of UA-cam channels use the site to promote their videos!!

  • @ushanair513
    @ushanair513 8 років тому

    does he have to catch a train?

  • @ushanair513
    @ushanair513 8 років тому

    why is the host speaking so fast? i dont like that.

  • @petrosvouris7864
    @petrosvouris7864 9 років тому

    This might as well be as exclusive as being "in a museum" as you say as It seems impossible to find any links to the Epigonion program/sounds. I have looked everywhere! Can you please prove me wrong, share this with the world as this video suggests, and share with us a link to this program. Thank you.

  • @ironDsteele
    @ironDsteele 9 років тому

    In a dimension where Top quarks could exist without annihilating you have to wonder what kinds of atoms, particles and substances they could produce. It's logical to assume they are compatible with Bottom quarks, but we may never find out. Great video.

  • @EGI_eInfra
    @EGI_eInfra 11 років тому

    The private key was created temporarily for the demo and has been destroyed. The proxy that was used has also expired.

  • @agiantalienforce
    @agiantalienforce 11 років тому

    Are you sure you want to show your SSH private key on youtube?

  • @Jazzman8100
    @Jazzman8100 12 років тому

    where could one download the virtual instrument to be played on a midi keyboard?

  • @EGI_eInfra
    @EGI_eInfra 12 років тому

    Thanks! Glad you enjoyed it. Check out the other 2 short films from our 'Stories from the grid' series, with more episodes to come in the future.

  • @DaRealFiberOptix
    @DaRealFiberOptix 12 років тому

    very well presented i actually understood most of that

  • @SimonLeinen
    @SimonLeinen 12 років тому

    In this case, the generic music is especially annoying, but I guess there's a law that says all videos of this kind HAVE to be covered in it.