A lot of development organizations use Scrum as their preferred Agile process. Sometimes that’s just what you’ll have to go with. I was part of development teams before Agile was “en vogue” - we still delivered software in useful bits on time and within budget and with minimal preexisting specification. I’m not going to critic Scrum. Instead I will focus on how to change it to make it more pleasant for everybody involved.

I set up this project to demonstrate how to dockerize a Spring Boot app and deploy, configure and scale it on Kubernetes.

I evaluated ways to deploy an Kubernetes cluster on AWS and settled on kops, since it is well supported by the Kubernetes community and can set up HA clusters.

Dirg is an orchestration tool for docker. It reads a yaml file describing services made of docker container definitions and allows to apply a number of commands to these groups of containers.

I wrote a blog post for Smaato on how to approach stabilizing and tuning a Spark Streaming application:

Spark’s accumulators can be used for much more than just adding up numbers. Spark only ships with int/double accumulators, but you can always create your own by implementing AccumulatorParam. Let’s say we’re processing log files with Spark Streaming and we would like to create a running tally of the most used browsers.