Stateful Workloads on Kubernetes are on the Rise

A recent survey of 1,000 IT professionals found that most organizations are now running stateful workloads on Kubernetes. The benefits are clear: increased agility, scalability, and resilience. But to reap those benefits, organizations need to get a handle on managing all the resources first. And that includes data.

A possible solution for that could lie in adopting one of the key elements of Kubernetes – its declarative nature – to the data layer. In other words: to increase the adoption of Kubernetes for stateful workloads, data has to become declarative, just like the application layer.

However, it’s easier said than done.

 

The Problem with Data

When it was first released, Kubernetes quickly gained popularity thanks to its ability to effortlessly handle stateless workloads. On the other hand, stateful applications — those that require access to persistently stored data — have been left out in the cold.

The Kubernetes engine tried to remediate this issue by providing APIs for interacting with databases and other persistent data sources.

The problem with that approach is that stateful applications usually require considerably more coordination and management than stateless ones. And with Kubernetes, that responsibility falls on the shoulders of the application developer.

Data lacks the mobility you need to respond quickly to changing circumstances. Yes, you can set up a Kubernetes cluster and fire up all of your containers rather quickly anywhere. But this new cluster is a clean slate and won’t have the application state your business needs to function. In a way, stateful applications have data that acts as an anchor and creates a gravitational pull that renders your “theoretically mobile” Kubernetes applications linked forever with a certain location.

This brings us to the problem of inconsistent configurations between different locations, the solution for which is data resources that are declarative, in the same way your clusters are.

Data agility is a problem that still needs to be solved. If your data were as agile and as portable as your Kubernetes clusters, you could respond to new operational requirements quickly. 

This can be a difficult task, as evident by the number of open-source projects aimed at helping developers manage stateful applications on Kubernetes. The Kubernetes community is still trying to develop a simpler approach.

According to the most recent Data on Kubernetes (DoK) report, 70% of businesses are running stateful applications in production. While there are many “solutions” for stateful workloads on Kubernetes today, more standardization across tools is required to enable large-scale production usage.

 

What are the Benefits of Standardization in Kubernetes?

There are many benefits to standardizing on Kubernetes, such as shorter release cycles and incremental changes to the application.

Because Kubernetes takes away a lot of operational activities, it allows developers to focus on business-critical features. Developers now may spend less time setting up and maintaining the infrastructure that has no direct impact on the company’s bottom line.

This means that businesses can shift their technical resources from operational maintenance to quickly responding to changes in customer behavior and market shifts and adapting their technology offering to fit market changes.

For dev teams, it means eliminating unnecessary tasks in order to focus on the actual software they’re developing instead of the infrastructure hassles.

Another essential idea is “skills portability” arising from using standardized operating models and toolchains. Large organizations demand that developers use industry standards to limit training expenses and remove roadblocks for employees switching from project to project. When software is deployed using the same core set of cloud-native tools, it also makes finding and training personnel easier and quicker.

Perhaps the most significant advantage of Kubernetes and cloud-native technologies is their ability to make skills easily transferable across organizations, which results in a substantial performance boost for both employers and employees. It’s one more incentive for businesses to continue investing in Kubernetes.

In other words, what people want from Kubernetes is a standardized approach to infrastructure. 

The standardization efforts so far, however, have left the data, or applications’ state out of the picture. 

Today, standardization applies only to stateless Kubernetes resources but not to data, and its replication across regions and clouds. Once stateful applications are involved, the benefits of Kubernetes standardization quickly fade away, as there is no standard approach to stateful coordination and management. Every organization must cobble together bespoke solutions, requiring specialized skills and taking up resources.

 

The Challenge: Standardization is Required for Data As Well

Stateful workloads are not new to Kubernetes; they have been present for years, but running stateful applications in production on Kubernetes is still a challenge.

According to Melissa Logan, the DoKC director, in an interview with The New Stack, today’s most advanced Kubernetes users:

“see these really massive productivity gains, so they want to standardize,” At the same time, “they’re trying to kind of figure out how to make all these things work together.”

According to the Data on Kubernetes (DoKC) report, most organizations (70%) are running stateful workloads, and they plan to run even more. But to do that without tying up all their resources in this task, they need to get a handle on managing all the resources (including the state) first.

To do so, we will need greater integration and interoperability with the existing range of technologies, skilled staff, better Kubernetes operators, and more trusted vendors. In other words, we need a standardized approach to managing stateful apps on Kubernetes.

 

The Solution: A Declarative Approach to Data

The stateful story in Kubernetes has essentially been ‘unsettled’ as the community is still trying to find an approach that could be standardized and accepted across the industry.

A possible solution for that could lie in adopting one of the key elements of Kubernetes – a declarative approach. In other words: to increase the adoption of Kubernetes for stateful workloads, data has to become declarative, just like the application layer.

In the declarative approach, you declare which resources you want to use and how you want to use them, and Kubernetes brings the system to the desired state, such as the desired number of nodes in a cluster, pods of a certain application, or a network configuration.

The next generation of operators should enable users to declare their application state: to specify which data should be used with each application, without going deep into configuring the storage itself.

 

Statehub Turns Data into an Always-Available, Declarative Resource

We need tools to address the problem of state at the infrastructure level. That’s where Statehub comes in. 

Statehub is a managed Kubernetes Data Fabric that assures data availability across public cloud regions, turning data into an always available, declarative resource.

Stateful applications can now enjoy the same user experience formerly only available for stateless containerized apps because of Statehub’s inventive approach to decoupling data from its physical location.

With Statehub’s App Warping™ technology, you can extend your applications’ state across any infrastructure and distance simply by registering your clusters with Statehub with a single command.

Read our Statehub implementation guides for Kafka, PostgreSQL, MongoDB, WordPress, and MySQL for further information.