Building File System Semantics for an Exabyte Scale Object Storage System (SDC 2019)

Skip ad
Current Time 0:00
Duration -:-
Loaded: 0%
Stream Type LIVE
Remaining Time -:-
 
1x
127 Views

  • Thanks! Share it with your friends!

    You disliked this video. Thanks for the feedback!

caroline
Published
Azure Blob Storage is Azure's exabyte scale object storage service. Big Data and Analytics workloads have emerged as a key workload on public cloud object storage services. However, a flat namespace object storage system is ill suited to the file system semantics of commonly used file systems like HDFS including directory structure, file level acls, etc. Azure Blob Storage has built a novel solution to this problem though a server side hierarchical namespace service that provides file system semantics while retaining the inherent scalability and cost efficiency of object storage. This is publicly available since February 2019 as the "Azure Data Lake Storage Gen 2" capability for Blob Storage that allows blob storage to be used as a very efficient HDFS compliant store for analytics scenarios. In this talk, we present the architecture of our solution, engineering challenges and solutions and demos and perf analyses of the benefits for Big Data and Analytics.

Presented by Shane Mainali, Group Engineering Manager, Microsoft and Vamshidhar Kommineni, Principal PM Manager, Microsoft
Category
Network Storage
Show more
Be the first to comment