Lightbend Activator

Monitoring Akka with Kamon and Statsd

Activator will be EOL-ed on May 24, 2017.

We’re making it easier and simpler for developers to get started with Lightbend technologies. This unfortunately means that future releases of Play, Akka and Scala will no longer include Activator support, and Lightbend’s Activator server will be decommissioned by the end of 2017. Instead of supporting Activator to create and set up development projects, we'll be supporting standard Giter8 templates for sbt users and Maven archetypes for Maven users. So going forward,

To create new Lightbend projects

Instead of using the Activator command, make sure you have sbt 0.13.13 (or higher), and use the “sbt new” command, providing the name of the template. For example, “$ sbt new akka/hello-akka.g8”. You can find a list of templates here.

Also, as a convenience, the Lightbend Project Starter allows you to quickly create a variety of example projects that you just unzip and run.

To create new templates

If you want to create new templates, you can now do that in Giter8.

To migrate templates from Activator to Giter8

If you created Activator templates in the past, please consider migrating them to Giter8 with this simple process.

Monitoring Akka with Kamon and Statsd

muuki88
Source
September 7, 2015
akka scala kamon monitoring statsd

Start a small akka app and a docker container with all statsd related stuff

How to get "Monitoring Akka with Kamon and Statsd" on your computer

There are several ways to get this template.

Option 1: Choose akka-monitoring-kamon-statsd in the Lightbend Activator UI.

Already have Lightbend Activator (get it here)? Launch the UI then search for akka-monitoring-kamon-statsd in the list of templates.

Option 2: Download the akka-monitoring-kamon-statsd project as a zip archive

If you haven't installed Activator, you can get the code by downloading the template bundle for akka-monitoring-kamon-statsd.

  1. Download the Template Bundle for "Monitoring Akka with Kamon and Statsd"
  2. Extract the downloaded zip file to your system
  3. The bundle includes a small bootstrap script that can start Activator. To start Lightbend Activator's UI:

    In your File Explorer, navigate into the directory that the template was extracted to, right-click on the file named "activator.bat", then select "Open", and if prompted with a warning, click to continue:

    Or from a command line:

     C:\Users\typesafe\akka-monitoring-kamon-statsd> activator ui 
    This will start Lightbend Activator and open this template in your browser.

Option 3: Create a akka-monitoring-kamon-statsd project from the command line

If you have Lightbend Activator, use its command line mode to create a new project from this template. Type activator new PROJECTNAME akka-monitoring-kamon-statsd on the command line.

Option 4: View the template source

The creator of this template maintains it at https://github.com/muuki88/activator-akka-kamon#master.

Option 5: Preview the tutorial below

We've included the text of this template's tutorial below, but it may work better if you view it inside Activator on your computer. Activator tutorials are often designed to be interactive.

Preview the tutorial

Akka Monitoring with Kamon

This activator shows you how to monitor your akka applications with kamon. The original article with some more details can be found here. The tutorial has the following structure:

  • Run it
  • The big picture
  • Kamon
  • The Application

Run it

If you just want to play around. Read the next chapter if you want to know more details.

Open two terminals. The first one will start your monitoring backend, which is compressed into one single docker container. If you have a running statsd infrastructure you don't need this, but have to change the configuration.

docker run -v /etc/localtime:/etc/localtime:ro -p 80:80 -p 8125:8125/udp -p 8126:8126 -p 8083:8083 -p 8086:8086 -p 8084:8084 --name kamon-grafana-dashboard muuki88/grafana_graphite:latest
In the second terminal start the example application with
sbt run
Now you have your monitoring running and an application which generates some load. Open localhost and play around with the Grafana Dashboard.

The Big Picture

First of all, when we are done we will have this infrastructure running akka-kamon-flowcharts-single Thanks to docker we don't have to configure anything on the right hand-side to get started.

The application contains two message generators: one for peaks and one for constant load. Two types of actors handle these messages. One creates random numbers and the child actors calculate the prime factors.

Kamon

Starting on the left of the picture. Kamon is a library which uses AspectJ to hook into methods calls made by the ActorSystem and record events of different types. The  Kamon docs have some big gaps, but you can get a feeling of what is possible. I will not make any special configuration and just use the defaults to get started as fast as possible.

Kamon Dependencies and sbt-aspectj

First we add the kamon dependencies via

val kamonVersion = "0.3.4"

libraryDependencies ++= Seq(
  "com.typesafe.akka" %% "akka-actor" % "2.3.5",
  "io.kamon" %% "kamon-core" % kamonVersion,
  "io.kamon" %% "kamon-statsd" % kamonVersion,
  "io.kamon" %% "kamon-log-reporter" % kamonVersion,
  "io.kamon" %% "kamon-system-metrics" % kamonVersion,
  "org.aspectj" % "aspectjweaver" % "1.8.1"
)
    
Next we configure the sbt-aspectj-plugin to weave our code at compile time. First add the plugin to your plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-aspectj" % "0.9.4")

And now we configure it

aspectjSettings

javaOptions <++= AspectjKeys.weaverOptions in Aspectj

// when you call "sbt run" aspectj weaving kicks in
fork in run := true
    

Last step is to configure what should be recorded. Open up your application.conf where your akka configuration resides. Kamon uses the kamon configuration key.

kamon {

  # What should be recorder
  metrics {
    filters = [
      {
        # actors we should be monitored
        actor {
          includes = [ "user/*", "user/worker-*" ] # a list of what should be included
          excludes = [ "system/*" ]                # a list of what should be excluded
        }
      },

      # not sure about this yet. Looks important
      {
        trace {
          includes = [ "*" ]
          excludes = []
        }
      }
    ]
  }
  
  # ~~~~~~ StatsD configuration ~~~~~~~~~~~~~~~~~~~~~~~~

  statsd {
    # Hostname and port in which your StatsD is running. Remember that StatsD packets are sent using UDP and
    # setting unreachable hosts and/or not open ports wont be warned by the Kamon, your data wont go anywhere.
    hostname = "127.0.0.1"
    port = 8125

    # Interval between metrics data flushes to StatsD. It's value must be equal or greater than the
    # kamon.metrics.tick-interval setting.
    flush-interval = 1 second

    # Max packet size for UDP metrics data sent to StatsD.
    max-packet-size = 1024 bytes

    # Subscription patterns used to select which metrics will be pushed to StatsD. Note that first, metrics
    # collection for your desired entities must be activated under the kamon.metrics.filters settings.
    includes {
      actor       = [ "*" ]
      trace       = [ "*" ]
      dispatcher  = [ "*" ]
    }

    simple-metric-key-generator {
      # Application prefix for all metrics pushed to StatsD. The default namespacing scheme for metrics follows
      # this pattern:
      #    application.host.entity.entity-name.metric-name
      application = "yourapp"
    }
  }
}
    
Our app is ready to run. But first, we deploy our monitoring backend.

Monitoring Backend

As we saw in the first picture, we need a lot of stuff running in order to store our log events. The libraries and components used are most likely reasonable and you (or the more Ops than Dev guy) will have to configure it. But for the moment we just fire them up all at once in a simple docker container. I don't put them in detached mode so I see what's going on.
docker run -v /etc/localtime:/etc/localtime:ro -p 80:80 -p 8125:8125/udp -p 8126:8126 -p 8083:8083 -p 8086:8086 -p 8084:8084 --name kamon-grafana-dashboard muuki88/grafana_graphite:latest
My image is based on a fork from the original docker image by kamon.

Run and build the Dashboard

Now go to your running Grafana instance at localhost. You see a default, which we will use to display the average time-in-mailbox. Click on the title of the graph (  First Graph (click title to edit ). Now select the metrics like this: akka-kamon-grafana
×

Welcome to the Lightbend Enterprise Suite


You are excited about Reactive applications. And you want to build, manage and monitor them easily and reliably in your environment.
We get it. So we built Lightbend Enterprise Suite to help you do that, and more, with a range of powerful Application Management, Intelligent Monitoring, Enterprise Integration and Advanced Tooling features.