Beyond DeltaFS - Designing Storage Systems to Support HPC and AI Workloads (SDC 2019)

Skip ad
114 Views

  • Thanks! Share it with your friends!

    You disliked this video. Thanks for the feedback!

caroline
Published
To meet the needs of physicists interested in tracking only a handful of particles within a population of trillions of particles, Los Alamos National Laboratory and Carnegie Mellon partnered in the creation of DeltaFS. DeltaFS provides a scalable metadata plane that has been demonstrated to support the creation of billions of files per second on HPC platforms. Building upon the lessons learned during the development of DeltaFS, in this talk explore how the techniques underlying DeltaFS can be extended beyond the I/O requirements of scientific simulation and high-performance computing. Specifically, we examine the I/O requirements common to both scientific simulations and artificial intelligence workloads to begin the development of a converged storage system for both HPC and AI workloads. Finally, we discuss our progress in building a storage system that supports both HPC and AI workloads.

Presented by Bradley Settlemyer, Sr. Scientist, Los Alamos National Laboratory
Category
Network Storage
Be the first to comment