I've been hammering away at learning DC/OS and Mesos for the past week, and it is starting to come together. I've spun up clusters on AWS and also the Google Cloud Platform, and for the most part, it is relatively easy to do, mostly because they have it already templated for you. Then you are just creating services to run on it, which in my case will largely be containers. In a way, it is kind of like building your own abstract cloud (which may or may not exist on top of another cloud) that you can just throw tasks at. Usually these tasks stick around forever, but not always.
On top of that, I added Kubernetes because it is awesome. The folks at Mesosphere have put together a quickstart that will allow you to spin up your own DC/OS cluster and then put Kubernetes on it. Check it out here:
They script it to run on Google Cloud Platform, and if you are free tier like I am for now, you might have to tweak the cluster template from 8-CPU machines to 4-CPU machines so that you stay under your quota, but it definitely works. You run one little command and then you have your own Kubernetes cluster that is already set up (I've actually gone through the process of setting up OpenShift natively, and even using Ansible it actually has a lot of steps). You just start defining resources in YAML and then spin up services and pods using kubectl. I haven't quite figured out how the routing from the outside world works, but I'm guessing there is some integration between Kubernetes and haproxy and then you put a proxy in place that interacts with haproxy and then magic happens. I'm not a network guy, I'm a container guy. Ok, actually, I'm just a guy with limited networking experience. I'm learning.
Platforms like this are helping the drive towards microservices. One thing I learned today is that when you build a service inside of DC/OS, it is given a simple DNS name such as service.marathon.mesos. Any other service that is running in the same cluster can reach that service by that DNS name. Then you can just build up a library of services the same way that you used to build up a library of functions (and actually, if you wanted, you could literally externalize simple functions as services to make them easily reusable). One huge benefit of this is that in order to upgrade parts of your application, you no longer have to regression test everything and then redeploy it all - you can just upgrade the affected services. It's also a lot easier to test because your services are smaller units and therefore have smaller scope of functionality. As a software engineer who believes that we need to make testing more of a priority, this is a great thing.
My attention on Mesos and DC/OS is largely driven by a platform that I'm working on that is based on these technologies, but I don't know yet if I would recommend this platform when there is such explosive growth around Kubernetes. A lot of the companies that are building platforms around Kubernetes give you a lot of the same ability to define applications that are just submitted and you spend less time worrying about the where and how. In our case, we have specific use cases that we are trying to address and a lot of the work was already done, but if I were coming to the table with a brand new application, I would probably want to try a few other things first. I'm starting to be a believer though.