Object Storage Use Cases
- Use Case: Disaster recovery — Backup and archiving
- Use Case: Static website hosting
- Use Case: Document Store and file sharing
- Use Case: Big Data analytics
This document reviews the reasons for using Interoute Object Storage and explains the most frequent use cases.
Traditional network-based storage technologies, such as Network-Attached Storage (NAS) and Storage Area Networks (SANs), suffer from major problems of scalability. NAS lacks the flexibility to operate at the petabyte scale which is demanded by current IT environments, while SANs are challenging and costly to scale out, due to complex architectures which are usually proprietary and expensive to maintain.
Traditionally, a computer operating system handles the task of accessing storage by means of block storage or network storage protocol layers (Figure 1). These protocols manage the storage allocation, access and file attributes. Scalability is inherently problematic as a result of these chained layers of abstraction.
Object Storage uses a different way of storing, organizing and accessing data on disk. It does not use a file system, in which file metadata are stored separately from file data, rather the file data and metadata are stored as a single 'object'. This allows for massive scaling as there is no underlying file system or database structure needed to keep track of attributes like file names, types, permissions, creation/access times, and so on. Combining data and metadata into objects allows for autonomy and flexibility in management across a network of resources, which can be geographically-distributed, and extended to global scales. All objects are built up from collections of data blocks that are labelled by a unique identifier. The storage architecture and every object are accessible by software applications over the Internet, or a private network, using a REST API and Secure HTTP communication protocol.
Added to this, the Interoute Object Storage platform automatically maintains multiple copies of objects in geographically-separated storage locations, thus providing redundancy and resilience for data storage.
In Interoute Object Storage your applications directly interface with the storage platform through the Internet using HTTP(S). This eliminates the complexities associated with traditional hosting environments and allows for rapid deployment and ease of use.
Object Storage provides a number of key benefits:
Geographic resilience with user control over data sovereignty
Accessibility via Internet using HTTP/HTTPS protocols
Uses an industry standard (S3 compatible) REST API
You only pay for data storage, per GB of the object content. There are no additional charges for the multiple copies of objects, and no charges for data transfers or inbound/outbound Internet bandwidth.
Objects are fixed and can’t be modified 'in place' after creation. Instead objects are updated by creating a completely new version of an object which either replaces the current version, or (if the versioning function is enabled) all of the uploaded versions are stored. Applications that demand in-place updating or that that use extensive locking and file-sharing are best served using the block storage offerings in Interoute VDC. For high I/O storage requirements, the SSD-based block storage in VDC is recommended.
Common use cases for Object Storage:
Disaster recovery — Backup and Archiving
Static web site hosting and static content distribution
Document Store and file sharing
Big Data analytics
Each of these cases is presented in detail in the rest of this document.
A typical enterprise runs a variety of critical IT workloads that need to adhere to business continuity management plans. To mitigate the effects of disruptive incidents and assure system recovery with the least amount of data loss within an acceptable timeframe, disaster recovery solutions are essential. Local redundancy is typically arranged to mitigate risk of losing data in case of component failure. To deal with disaster scenarios that disrupt the entire data centre, a secondary data centre must be setup and maintained. Besides the requirements for system availability, backup and replication of data is another critical element of disaster recovery, locally and in secondary locations. Disaster recovery solutions for redundancy and replication are expensive to set up, manage and maintain. Object storage is an ideal alternative solution for data replication, given its easily-deployed network-based accessibility and its built-in geo-resilience.
Consider the below IT scenario (Figure 3):
vSphere ESX cluster hosting an estate of various virtual machines. All VM datadisks are backed up to an enterprise backup platform as well as file based and agent based backups for structured data applications.
Physical database cluster (based on Oracle, SAP etc.) running an enterprise database backend
Primary datacentre backup solution using a disk-based storage array and tape-based backup for longer term archiving.
Expensive dedicated network interconnect between the primary and secondary datacentre locations to allow for replication of the backup environment.
Secondary backup platform in the Disaster Recovery site used to replicate backed-up data.
Interoute Object Storage is an ideal solution to lower the cost and increase the resiliency of this solution by replacing one or both of the backup platforms.
Figure 5. Interoute Object Storage as a replacement for both the primary and secondary backup platforms
Interoute Object Storage is a secure, high performance and very cost-effective solution because it is accessed via Interoute's global backbone network that offers low latency and high throughput. Interoute does not charge for data transfers within its backbone network, or inbound/outbound Internet bandwidth.
One of the key advantages of Object Storage is the ability to distribute your data objects directly to the Internet using HTTP. This eliminates the need to build or provide your own webservers to host static web content because Interoute Object Storage can be simply configured to behave as a static website for stored objects.
In the case of dynamic websites (such as for e-commerce or online services), considerable performance and cost benefits can be achieved by outsourcing static content to Object Storage. By serving data directly from Interoute's cost effective Object Storage platform (via Interoute's backbone network into the Internet) this will ease the load on your production webservers, and thus reduce the amount of resources required to run your website.
If you run your ecommerce or enterprise website using server-side logic processed (partly) on a pool of application servers, chances are that a major part of the content served is static files like PDF documents, zip archives, and video files. All these files consume costly storage space on your server estate, and costly bandwidth for content delivery to users. Object Storage is an ideal solution to offload static content to a resilient and always-on platform that can serve your files directly to your customers, via free Internet bandwidth (that is, you only pay for the cost of storage in the platform). Interoute Object Storage servers run in multiple locations across Europe (and soon in global locations) which provide content delivery to the Internet at points which are closer to your customers and end users.
Object Storage utilises strong in-transit encryption and granular security controls on buckets and objects. For Linux and Microsoft Windows operating systems file system drivers are available to mount one of more buckets in the operating system directly. Many document management system (DMS) applications provide a function to directly store and retrieve documents from an Object Storage service, via an S3-compatible API interface.
Consider the below scenario where a company exchanges many documents with clients and employees globally (for example, marketing information, legal documents, media files):
Maintaining a server estate and storage platform for a document management system is cumbersome. Transforming the storage backend entirely to an Object Storage platform is a logical step to lower the administrative burden. Interoute Object Storage, an inherently secure storage platform with compliancy and data sovereignty as part of its design, is ideally suited to take on this task.
Replace the file servers and storage backend with Interoute Object Storage in this setup:
'Big Data' consists of high-volume, high-velocity and high-variety information assets that require efficient forms of information processing to support insight and decision making.
Some examples of Big Data information sources include:
Shopping information from an e-commerce environment
Social media analytics
IoT sensors in cars, domestic appliances, industrial machinery, healthcare etc
What complicates the task for companies is not the data itself but the management of metadata which adds information and permits a more sophisticated approach to analytics, thus making the data itself a more valuable asset. Properly tagged data is searchable and becomes meaningful and actionable content.
Interoute Object Storage supports the creation of user-defined object metadata (key-value pairs) together with a variety of system-defined metadata that are stored by default. Metadata are controllable via the API which means that they can be manipulated by Big Data management and analysis software.
As an example, consider a medical application storing a collection of X-ray images. Traditionally this type of dataset is managed in closed database applications with access only for a select group of medical specialists. Anonymised upload of the X-ray images to Object Storage could expose valuable information to various medical research groups. By allowing research teams to enrich the data with metadata, targeted searches and analytics can be built up to reveal previously-undiscovered trends.