Dealing With Persistent Storage And Fault Tolerance In Apache Mesos


Why does storage matter?
● MESOS offers great support for stateless services
● But what about data persistence?
● Distributed Databases
● Distributed Filesystems
● Docker Volumes on distributed storage
● Two perspectives:
● Support for Distributed Storage Frameworks
● Support for Frameworks using the Distributed Storage Frameworks

Cloud Architect Musings

redundancy_hours_just_a_few_redundacy_motis-s800x600-58577-580

In part 1 of this series on Apache Mesos, I provided a high level overview of the technology and in part 2, I went into a bit more of a deep dive on the Mesos architecture.  I ended the last post stating I would do a follow-up post on how resource allocation is handled in Mesos.  However, I received some feedback from readers and decided I would first do this post on persistent storage and on fault tolerance before moving on to talk about resource allocation.

Persistent Storage Question

Screen Shot 2015-03-30 at 10.22.52 AM

As my previous posts discussed and as Mesos co-creator Ben Hindman’s diagram above indicates, a key benefit of using Mesos is the ability to run multiples types of applications (scheduled and initiated via frameworks as tasks) on the same set of compute nodes.  These tasks are abstracted from the actual nodes using isolation modules (currently some type of container technology)…

View original post 1,265 more words

Advertisements

Apache Mesos: Open Source Community Done Right


Leveraging Mesos as the Ultimate
Distributed Data Science
Platform

Cloud Architect Musings

open-source-590x273

I’ve been writing recently about Apache Mesos and it’s importance as an operating system kernel for the next generation data center. You can read those posts here:

Part 1: True OS For The SDDC

Part 2: Digging Deeper Into Mesos

Part 3: Dealing with Persistent Storage And Fault Tolerance

Part 4: Resource Allocation

Besides the technology though, I am also excited about the progression of the Mesos project itself.  So I want to take a detour from my technology-focused posts to make some general observation about the project.  As I said on Twitter previously, I’ve been particularly impressed with three characteristics:

Screen Shot 2015-03-22 at 11.26.31 PM

I’ve had the opportunity to speak to a number of people recently about Mesos and found it’s been extremely easy for them to grasp the concept and to understand the value of the technology.  This is very important for a project that is growing and looking to expand its…

View original post 635 more words

Docker Tutorial Series : Part 6 : Docker Private Registry


iRomin

This is part 6 of the Docker Tutorial Series.

In this part we shall take a look at how you can host a local Docker registry. In an earlier part, we had looked at the Docker Hub, which is a public registry that is hosted by Docker. While the Docker Hub plays an important role in giving public visibility to your Docker images and for you to utilize quality Docker images put up by others, there is a clear need to setup your own private registry too for your team/organization.

View original post 903 more words

CI/CD the Docker Way


Tutum Blog

CI-CD@2x

Tutum has recently announced build & testing capabilities and we are very happy not only about what we think is a huge step forward in providing the best CI/CD tools for developers, but also because of the amazing planning and team collaboration that was required from us to deliver such a big feature.

Tutum CI/CD design goals were to provide a flexible, agnostic, automatable and locally replicable solution for build & tests, which mirror the goals for the Docker project. This is why we based our solution for CI/CD on the open source image tutum/builder.

In a nutshell, tutum/builder performs the following steps:

  • Configure docker credentials using $DOCKERCFG.
  • Clone the repository using $GIT_REPO and checkouts the commit in $GIT_TAG.
  • Build the docker image specified by $DOCKERFILE_PATH.
  • Push the image to the image tag specified in $IMAGE_NAME.

How do we get the right values for…

View original post 652 more words

The First-Ever Go Conference in India – GopherConIndia 2015


Josh Software - Where Programming is an Art!

After the resounding success of GopherCon 2014 in Denver, Co, USA the Go Language Community in India together with the Innovation And Technology Trust (ITT) (a non-profit organization, established to organize and conduct technology conferences in India whose current portfolio includes RubyConf India, GopherCon India and DevOpsDays India) are bringing you the first-ever Go conference in India – GopherConIndia 2015 in Bengaluru (Bangalore) from 19-22 Feb. 2015.

The Indian Go programming community is growing at a dramatic pace. The number of companies utilizing Go, as part of their technology stack, continues to grow steadily.

GopherConIndia 2015 GopherConIndia 2015

Planning and organizing an all India conference is not an easy task but a small team of dedicated volunteers (Ajey Gore, Gautam Rege, Karan Misra, Krishnaprasad Varma, Pravin Mishra, Santosh B Malleshappa, Sathish VJ, Satish Talim – this does not list all of the volunteers, without…

View original post 428 more words

Heroku useful commands


1. Access heroku database

$ heroku console

2. Delete heroku table

$ ModelName.delete_all

3.  Get all heroku releases

$ heroku releases

kl-new Releases
v31   Deploy cb9e22e   mpravin@maisasolutions.com   14m ago
v30   Deploy 0cee730   mpravin@maisasolutions.com   2012/06/22 11:08:36
v29   Deploy 07e6aa5   mpravin@maisasolutions.com   2012/06/22 10:31:06
v28   Deploy 40c13ba   mpravin@maisasolutions.com   2012/06/20 10:42:13
v27   Deploy 2464731   mpravin@maisasolutions.com   2012/06/19 16:19:16
v26   Deploy 99241c5   mpravin@maisasolutions.com   2012/06/19 15:43:15
v25   Deploy 2c7bb6e   mpravin@maisasolutions.com   2012/06/18 13:40:31
v24   Deploy f600d0a   mpravin@maisasolutions.com   2012/06/15 12:57:57
v23   Deploy b02fc6a   mpravin@maisasolutions.com   2012/06/15 12:45:32
v22   Deploy 79edf6d   mpravin@maisasolutions.com   2012/06/15 12:39:27

4. get detailed info on a release:

$ heroku releases:info v24

5. Roll back to the last release

$ heroku rollback

6. Rollback to specific version

$heroku rollback v31