Last summer we blogged about Takipi, a new tool that gives you visibility into how your Scala code executes in real-time in staging or production. Last month, Takipi wrote their own post about how they Analyzed 30,000 GitHub projects and provided a list of the top 100 libraries in Java, JS and Ruby. This month, they ran the same test with Scala, so we asked them to blog about it. Check out the results below:
As Scala developers working in a language and ecosystem that’s rapidly growing and evolving, we’re faced with a constant dilemma whenever we write new code - go with that hot new Scala framework that everyone’s talking about, or stick with a Java library we know and trust?
When we began building Takipi we wanted to know what are the most common frameworks developers use today, so we could better optimize it for them. Since a large part of Scala applications are commercial or closed-source in nature, it can be hard sometimes to tell the number of projects putting a library to use.
We decided to use a data based approach to get more insight into this by analyzing what Scala developers are actually using on the world’s largest open project repository - GitHub. With a wide variety of projects ranging from small to very large, GH provides us with an extensive data-set, one which is also highly up-to-date.
Much like with the results we saw in Java, there were some pretty big surprises. As both Java and Scala run on the JVM, it was interesting to notice similarities between the frameworks used, and also some stark differences. Overall 42 libraries appear in both the top 100 Java and Scala libraries, helping reaffirm the fact that Scala isn’t just a different language, but it also has its own universe of tools and libraries.
To generate our dataset we queried 10,000 Scala projects, with a bias towards the ones most favorited by the community, as a strong qualifying indicator towards their relative importance.
We searched for dependencies in sbt and Maven which the vast majority of Scala projects on GH use to build their projects. For sbt we analyzed the build.sbt, project/Build.scala and any .scala files that extend them. For Maven projects we scanned the pom.xml dependencies file.
We then analyzed and grouped the results into categories. The results were interesting to say the least -
TDD is big in Scala. JUnit, the classic Java testing framework, is the most popular library with 2513 projects using it. Scalatest comes in at a close second with 2197 entries. TestNG which is fairly popular in Java (ranked 14th in the Java top 100) isn't in the top 100 libraries for Scala.
SPECS2, the framework for writing software specifications is being used by 1331 projects. SPECS V1 which was deprecated in early 2011 still has 312 projects using it.
A new generation of frameworks. Using Scala is not just about the language, but also about a new generation of frameworks. The Play Framework for building web apps is crushing it when it comes to Scala developers, with 18% of the projects using it. The Akka framework is also doing very well with 776 entries (ranked 9th). Lift, another well known framework for building Scala web applications, is only used by 124 projects, which came as something of a surprise to us.
Some frameworks originally built for Java, however, are seeing much greater use in Scala. The lightweight web server Jetty, which was used by 100 projects in Java, has 4.5X that amount of projects using it in Scala, with 447 entries (17th).
Where’s the Java old guard? This comes in contrast to some of Java’s most venerated languages and frameworks seeing considerable less use in Scala.
Logging. Ceki’s SLF4J is leading the pack -
SQL. Big surprises on the Scala DB front -
NoSql sees less traction than in Java. It’s also worthwhile noting that Hadoop, which is seeing a good amount of usage in Java, isn’t on the Scala top 100 board. The only NoSql DB on the Scala list is MongoDB with 97 entries.
Android. While Scala is very much a server-side language aimed at building scalable server applications, we still saw some presence for Android development with 82 projects using the sbt-android-plugin.
Surprised by some of the results? We know we were with some of them. Take a look at the full list of the top 100 Scala libraries on GitHub below, and let us know what you think in the comments section. We’d love to hear your thoughts and questions.
Click here to see the complete top 100 Scala libraries list.