Quobyte®
Inc., a leading developer of modern storage system software, today
announced that the Quobyte Data Center File System is the first
distributed file system to offer a TensorFlow plug-in, providing
increased throughput performance and linear scalability for ML-powered
applications to enable faster training across larger data sets while
achieving higher-accuracy results.
TensorFlow is an open source library for numerical computation and
large-scale machine learning used across industries such as autonomous
vehicles, robotics, financial services, healthcare, government,
aerospace, defense, and many others. Using Quobyte storage with
TensorFlow helps to simplify and streamline the operation of machine
learning.
Quobyte’s TensorFlow File System Plug-in allows TensorFlow applications
to talk directly to Quobyte, bypassing the operating system kernel to
significantly reduce kernel mode context switches and lower CPU usage.
While Quobyte storage can be used with all stages of ML, the resulting
increased GPU utilization from the TensorFlow plug-in speeds up model
training of ML workflows.
Quobyte provides users the flexibility to train anywhere and seamlessly
move models into production to better support ML workloads from the data
center to the cloud to the edge. The TensorFlow plug-in can be used to
train models locally on sample data sets and use the Google Cloud
Platform for training at scale because Quobyte runs on-prem and in the
cloud. Additionally, because it bypasses the kernel entirely, Quobyte’s
TensorFlow plug-in works with both current and older versions of Linux,
providing a full range of flexible deployment options for use in ML.
Using the Quobyte TensorFlow plug-in is seamless since there are no
application modifications required.
“As more and more businesses look to leverage ML to increase innovation,
achieve a faster time to market and provide a more positive customer
experience, there is an increasing need for storage infrastructures that
offer higher performance and increased flexibility that these workloads
need,” said Frederic Van Haren, Lead Analyst HPC and AI Systems of
analyst firm Evaluator Group. “Vendors, like Quobyte, that offer high
performance, broad platform support and flexibility of deployment
options are well positioned to help companies handle bigger data sets,
achieve more accurate results and run ML workloads in any environment.”
With Quobyte, there is no need for specialized storage systems to get
the most out of ML. Quobyte is a single storage system that addresses
many different performance profiles, including the high-throughput,
low-latency requirement of ML’s model training stage, as well as large
block sequential, small block random or mixed general workloads. Quobyte
supports the broadest set of access protocols and clients, such as S3,
Linux, Hadoop, Windows and NFS for greater platform flexibility and more
complete data ingest and preparation. Data is readily available at any
stage all within a single global namespace and all managed through
Quobyte’s intuitive management console.
Additional benefits of Quobyte’s TensorFlow File System Plug-in include:
-
The ability to leverage HDD and SSD to get the best price-performance
ratio without cumbersome tiering -
Prefetching of training data can deliver substantial performance
improvement. Much machine-generated data uses a sequential naming
convention that makes it ideal for prefetching. -
Infinite scalability that allows users to grow storage in terms of
throughput and capacity when they need it. As ML project requirements
change – oftentimes more quickly than anticipated – the Quobyte
installation will adapt. Disks or servers can be quickly and easily
added when needed to provide more capacity or performance without any
interruption to applications or services. -
Multi-tenancy that provides additional security by allowing users to
define isolated namespaces and physical separation of data/workloads
inside the same cluster. Administrators can further isolate tenants by
controlling to which physical hardware they have access in order to
ensure performance and that data is not accessible to any unauthorized
users on the network.
“By providing the first distributed file system with a TensorFlow
plug-in, we are ensuring as much as a 30 percent faster throughput
performance improvement for ML training workflows, helping companies
better meet their business objectives through improved operational
efficiency,” said Bjorn Kolbeck, Quobyte CEO. “With the higher accuracy
of results, scalability to handle bigger data sets and flexibility to
run on-prem to the cloud, and edge, we believe we are providing an
optimal experience that allows customers to fully leverage the value of
their Machine Learning infrastructure investments.”
Follow Quobyte
https://www.twitter.com/quobyte
https://www.linkedin.com/company/quobyte
https://www.facebook.com/quobyte
About Quobyte
Building on a decade of research and experience with the open-source
distributed file system XtreemFS and from working on Google’s
infrastructure, Quobyte delivers on the promise of software-defined
storage for the world’s most demanding application environments
including High Performance Computing (HPC), Machine Learning (ML), Media
& Entertainment (M&E), Life Sciences, Financial Services, and Electronic
Design Automation (EDA). Quobyte uniquely leverages hyperscaler parallel
distributed file system technologies to unify file, block, and object
storage. This allows customers to easily replace storage silos with a
single, scalable storage system — significantly saving manpower, money,
and time spent on storage management. Quobyte allows companies to scale
storage capacity and performance linearly on commodity hardware while
eliminating the need to expand administrative staff through the
software’s ability to self-monitor, self-maintain, and self-heal. Please
visit www.quobyte.com
for more information.
View source version on businesswire.com: https://www.businesswire.com/news/home/20190416005100/en/