Dealing With Persistent Storage And Fault Tolerance In Apache Mesos


Why does storage matter?
● MESOS offers great support for stateless services
● But what about data persistence?
● Distributed Databases
● Distributed Filesystems
● Docker Volumes on distributed storage
● Two perspectives:
● Support for Distributed Storage Frameworks
● Support for Frameworks using the Distributed Storage Frameworks

Cloud Architect Musings

redundancy_hours_just_a_few_redundacy_motis-s800x600-58577-580

In part 1 of this series on Apache Mesos, I provided a high level overview of the technology and in part 2, I went into a bit more of a deep dive on the Mesos architecture.  I ended the last post stating I would do a follow-up post on how resource allocation is handled in Mesos.  However, I received some feedback from readers and decided I would first do this post on persistent storage and on fault tolerance before moving on to talk about resource allocation.

Persistent Storage Question

Screen Shot 2015-03-30 at 10.22.52 AM

As my previous posts discussed and as Mesos co-creator Ben Hindman’s diagram above indicates, a key benefit of using Mesos is the ability to run multiples types of applications (scheduled and initiated via frameworks as tasks) on the same set of compute nodes.  These tasks are abstracted from the actual nodes using isolation modules (currently some type of container technology)…

View original post 1,265 more words

Advertisements

Apache Mesos: Open Source Community Done Right


Leveraging Mesos as the Ultimate
Distributed Data Science
Platform

Cloud Architect Musings

open-source-590x273

I’ve been writing recently about Apache Mesos and it’s importance as an operating system kernel for the next generation data center. You can read those posts here:

Part 1: True OS For The SDDC

Part 2: Digging Deeper Into Mesos

Part 3: Dealing with Persistent Storage And Fault Tolerance

Part 4: Resource Allocation

Besides the technology though, I am also excited about the progression of the Mesos project itself.  So I want to take a detour from my technology-focused posts to make some general observation about the project.  As I said on Twitter previously, I’ve been particularly impressed with three characteristics:

Screen Shot 2015-03-22 at 11.26.31 PM

I’ve had the opportunity to speak to a number of people recently about Mesos and found it’s been extremely easy for them to grasp the concept and to understand the value of the technology.  This is very important for a project that is growing and looking to expand its…

View original post 635 more words