During this week's 62nd Edition of The IT Press Tour in San Francisco, cloud storage management company Lucidity announced Lumen, a new solution that promises to simplify disk tiering for enterprise environments. The announcement comes at a time when organizations struggle with complex cloud storage decisions that often lead to wasted spending and potential downtime risks.

The Growing Cloud Storage Challenge

Let's be honest - cloud storage management isn't exactly the most exciting topic in tech. But here's the thing: it's becoming a massive problem for enterprises. During the ITPT briefing, Vatsal Rastogi, co-founder of Lucidity, shared some eye-opening numbers about the cloud storage market's expected growth - from 134.5 billion in 2024 to 1.1 trillion by 2035. 

"Storage on the cloud is complex. There are multiple cloud providers, especially the segment that we target, fortune 2000 customers tend to have at least two in order to have a multi cloud strategy," explained Rastogi during the briefing.

The problems organizations face are pretty stark:

• 50-70% of disk space gets wasted
• Applications run about 30% slower than they should
• IT teams waste hundreds of hours on manual optimization
• Businesses face constant downtime risks

AutoScaler: The Foundation of Lucidity's Approach

Before diving into the new Lumen announcement, it's worth understanding Lucidity's existing AutoScaler technology, which has been in the market for about four years. The company, backed by $31 million in funding, has built what they call a "no-ops" approach to storage management.

"What does no-ops mean? No ops essentially is a completely hands free onboarding and management experience, which essentially ensures that once you are onboarded onto the Lucidity platform, you spend zero effort in capacity management," Rastogi explained.

The results speak for themselves. One of their customers, a major US airline, automated over 7,500 provisioning activities and saved $88,000 per month - a 77% return on investment. Their disk utilization jumped from 21% to 82%.

Enter Lumen: Reimagining Disk Tiering

The new Lumen platform, announced today, targets another thorny problem in cloud storage: disk tiering. According to Lucidity's research, about 50% of disks are on the wrong tier, and organizations typically waste 25% or more of their spending on unnecessary premium tiers.

"Cloud storage deserves the same level of automation and cost optimization that cloud compute has gotten," says Nitin Bhaudaria, Cofounder at Lucidity. "Getting the most out of your cloud storage is complex-leading to a reality of overprovisioned capacity, premium-by-default disk tiers, and idle and orphaned disks, just to name a few."

Key Capabilities

Lumen brings three main capabilities to the table:

1. Unified Visibility: A consolidated dashboard showing real-time tiering efficiency and potential savings
2. Data-Driven Recommendations: Analysis based on historical usage patterns
3. One-Click Actions: The ability to change disk tiers without causing downtime

Technical Implementation

For the tech-savvy VMblog readers wondering about the implementation details, let's dig into how Lumen actually works alongside Lucidity's established AutoScaler technology.

Architecture Overview

Lucidity deploys a lightweight agent that runs on each VM. The agent works alongside the kernel layer and actively monitors storage metrics, communicating with Lucidity's autoscaler service. When asked about potential downtime risks, Rastogi addressed concerns head-on: "Even if Lucidity were to not exist or were to go down for, let's say, an hour... your IOs still continue to remain unimpacted, because we are not in the IO path."

Platform Specifics

During the briefing, Philippe Nicolas asked about the underlying technology, particularly regarding file systems. Rastogi explained that Lucidity has different solutions for Windows and Linux environments:

• For Linux: The solution leverages Btrfs and its sub-volume model for space management
• For Windows: Works directly with native NTFS, requiring no file system changes
• Both implementations achieve the same results without disrupting existing operations

Deployment Options

Lucidity offers three distinct hosting mechanisms to meet various enterprise requirements:

1. Community Edition: The default option where requests route through Lucidity's centrally hosted cluster
2. Private Link Mechanism: Dedicated clusters for specific cloud service providers, ensuring no data transfers over the public internet
3. Self-Hosting Option: For large enterprises, Lucidity can deploy their entire infrastructure within the customer's cloud account, providing complete isolation and enhanced security

How Lumen Makes Decisions

The platform's recommendation engine analyzes several key metrics to optimize disk tiering:

• Historical IOPS patterns
• Throughput requirements
• Latency sensitivity
• Cost implications across different tiers
• Seasonal usage patterns
• Application-specific requirements

"Performance needs are unpredictable," Rastogi noted during the briefing. "Disk IOPS and throughput requirements can vary for seasonality or other spikes. Predicting application performance needs is a guessing game."

Integration Points

Lumen works seamlessly across various enterprise environments, supporting:

• Databases (including MySQL, Kafka, and 10+ popular DBs)
• Application server disks
• Root volumes and data disks
• Kubernetes environments
• Cloud-native applications

Security and Compliance Considerations

A key point emphasized during the briefing was Lucidity's position in the data stack. "We are not in the data path," Rastogi stressed. "We do not intercept any read write operations. We, in fact, don't even have access to the data that is residing on the partitions."

This architecture choice helps address common security and compliance concerns, particularly for enterprises handling sensitive data.

Real-time Operations

The platform's ability to perform tier changes without downtime represents a significant technical achievement. Traditional tier modifications typically require taking VMs offline and rebooting - a process that many organizations avoid due to operational impact. Lumen's approach eliminates this constraint, allowing for:

• Live tier conversions
• Zero-downtime optimization
• Continuous performance monitoring
• Automatic adjustment based on actual usage patterns

Market Position and Availability

Lumen is now generally available for Microsoft Azure, with AWS and GCP support coming in future releases. The pricing follows Lucidity's established model of charging based on data managed rather than capacity provisioned.

For IT teams looking to test drive the technology, Lucidity offers a free assessment tool that can quickly scan cloud environments and provide detailed ROI projections.

Looking Ahead

The introduction of Lumen marks an important step in Lucidity's vision of comprehensive cloud storage management. While AutoScaler addressed the capacity management challenge, Lumen tackles the complexity of disk tiering - and there are hints of more to come.

For enterprise IT teams tired of manual storage management and unexpected cloud bills, Lumen offers an interesting proposition: data-driven automation that promises both cost savings and performance improvements. The early customer results suggest they might be onto something.

Those interested in seeing Lumen in action can visit Lucidity at booth S27 at FinOps X in San Diego, CA, or schedule a demo through their website.