Eugene Yokota

Scala Team
July 31, 2018
| scala

sbt 1.2.0 released

Hi everyone. Lightbend Tooling team is happy to announce sbt 1.2.0. This is the second feature release of sbt 1, a binary compatible release focusing on new features. sbt 1 is released under Semantic Versioning, and the plugins are expected to work for sbt 1.x series.

Warning: We found forward compatibility breakage in 1.2.0, so we recommend everyone to upgrade to sbt 1.2.1.

Note: I will be using this URL to announce the release candidates and final as well.

  • 1.2.0 is now final.
  • If no serious issues are found by **Monday, July 30th 2018**, 1.2.0-RC3 will become 1.2.0 final.
  • If no serious issues are found by **Monday, July 16th 2018**, 1.2.0-RC1 will become 1.2.0 final.

The headline features of sbt 1.2 are cross JDK forking, composite project, and experimental thin clients. But, there are lots of other bug fixes and enhancements that we’ve been accumulating for six months since sbt 1.1.

SbtPlugin for plugin development

SbtPlugin is a plugin to declare a project for sbt plugins. This automatically brings in scripted tests, and sets sbtPlugin := true.

lazy val root = (project in file("."))
	.enablePlugins(SbtPlugin)

Compatibility note: ScriptedPlugin is no longer a triggered plugin.

#3875 by @eed3si9n

Cross JDK forking

For forked run and test, java++ can now switch Java Home.

sbt:helloworld> run
[info] Running (fork) Hello
[info] 1.8.0_171
sbt:helloworld> java++ 10!
[info] Reapplying settings...
sbt:helloworld> run
[info] Running (fork) Hello
[info] 10.0.1

sbt will try to detect Java homes into discoveredJavaHomes setting, supporting shyiko/jabba. This can be augmented by Global / javaHomes:

Global / javaHomes += "6" -> file("/something/java-6")

This feature is intended for testing your library in an older JVM to check compatibility.

#4139 by @2m, @cunei, and @eed3si9n

scalaVersion-filtered aggregation

In 2015 James Roper contributed scalaVersion-filtered aggregation to sbt-doge. This feature is brought back into sbt 1.2 by Rui Gonçalves (@ruippeixotog) in #3698/#3995!

This extends switch command ++ to take an optional <command>:

> ++2.12.6 compile

This will aggregate only the subproject where ++2.12.6 is valid, which is useful when you have a build where some subprojects are 2.11 only etc.

Composite project

sbt 1.2.0 introduces “composite project” trait, which allows plugin authors to generate subprojects, for example for cross building.

trait CompositeProject {
	def componentProjects: Seq[Project]
}

This was contributed by @BennyHill as #4056.

Project matrix

Experimental. As a reference implementation of the CompositeProject I implemented a new DSL called projectMatrix introduced by sbt-projectmatrix plugin.

lazy val core = (projectMatrix in file("core"))
	.scalaVersions("2.12.6", "2.11.12")
	.settings(
		name := "core"
	)
	.jvmPlatform()

lazy val app = (projectMatrix in file("app"))
	.dependsOn(core)
	.scalaVersions("2.12.6")
	.settings(
		name := "app"
	)
	.jvmPlatform()

The aim of the plugin is to support a generic notion of cross building (Scala version, platform, etc) expressed using subprojects. In the above projectMarix will produce three subprojects: coreJVM2_12, coreJVM2_11, and appJVM2_12.

Semantic Version selector API

sbt 1.2.0 introduces Semantic Version selector on VersionNumber() datatype supporting basic match, comparison (<=, <, >=, >), combination (>1.0.0 <2.0.0, ||), ranges (A.B.C - D.E.F), and wildcard (2.12.x).

scala> import sbt.librarymanagement.{ VersionNumber, SemanticSelector }
import sbt.librarymanagement.{VersionNumber, SemanticSelector}

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector(">=2.12"))
res1: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("<2.12"))
res2: Boolean = false

scala> VersionNumber("2.13.0-M4").matchesSemVer(SemanticSelector("2.13"))
res3: Boolean = false

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.12.1 - 2.12.6"))
res4: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.12.x"))
res5: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.11.x || 2.12.x"))
res6: Boolean = true

Note: This has no effect on library management at the moment.

This was contributed by Rikito Taniguchi (@tanishiking) as lm#239.

addPluginSbtFile command

There’s been a request from IntelliJ to safely inject a plugin to a build. sbt 1.2.0 adds -addPluginSbtFile command to do so.

$ cat /tmp/extra.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.7")

$ sbt -addPluginSbtFile=/tmp/extra.sbt
...
sbt:helloworld> plugins
In file:/xxxx/hellotest/
	...
	sbtassembly.AssemblyPlugin: enabled in root

Implmented by @eed3si9n as #4211.

Extensible sbt server

Experimental. sbt server can now be extended via the plugin.

    Global / serverHandlers += ServerHandler({ callback =>
			import callback._
			import sjsonnew.BasicJsonProtocol._
			import sbt.internal.protocol.JsonRpcRequestMessage
			ServerIntent(
				{
					case r: JsonRpcRequestMessage if r.method == "lunar/helo" =>
						jsonRpcNotify("lunar/oleh", "")
						()
				},
				PartialFunction.empty
			)

This feature is still experimental and the API may change in the future.

#3975 by @eed3si9n

Thin client(s)

Experimental. sbt 1.2.0 adds a new mode called -client. When sbt is started with -client command, it no longer to loads the build, and instead tries to connect to an instance of sbt server over JSON-RPC. When the server is not running (portfile is not found), it will fork a new instance of sbt entirely in a new JVM.

This lets you invoke sbt from the terminal shell or from an editor.

$ time sbt -client clean
[info] entering *experimental* thin client - BEEP WHIRR
[info] server was not detected. starting an instance
[info] waiting for the server...
[info] waiting for the server...
[info] waiting for the server...
[info] waiting for the server...
[info] server found
> clean
[success] completed
sbt -client clean  9.23s user 2.33s system 22% cpu 50.558 total

# server stays
$ ps | rg java
21860 ttys015    1:22.43 java -Xms2048M -Xmx2048M -Xss2M -jar /usr/local/Cellar/sbt/1.1.6/libexec/bin/sbt-launch.jar
22014 ttys015    0:00.00 rg java

$ time sbt -client clean
[info] entering *experimental* thin client - BEEP WHIRR
> clean
[info] Updating ...
[info] Done updating.
[success] completed
sbt -client clean  3.39s user 1.75s system 104% cpu 4.898 total

To end the server, call sbt -client shutdown. #4227 by @eed3si9n

In addition, there are also an alternative thin clients cb372/sbt-client and dwijnand/sbtl implemented using Rust.

Changes with compatibility implication

  • Removes deprecated commands -, --, and ---. Use onFailure, sbtClearOnFailure, and resumeFromFailure instead. #4124
  • Makes ++ fail when it doesn’t affect any subprojects #4269 by @eed3si9n

Other bug fixes and improvements

  • Fixes output caching bug. util#169 by @bpholt
  • Fixes “destination file exists” error message. lm#255 by @eed3si9n
  • Reintroduces Command.process(String, State): State. #4023 by @dwijnand
  • Fixes active.json not getting removed on JVM shutdown. #4194 by @veera83372
  • Fixes file permission error (“CreateFile() failed”) while reading the timestamp on Windows. io#134 by @cunei
  • Fixes the linter that detects missing .value. #4090 by @eed3si9n
  • Fixes StringIndexOutOfBoundsException in removeEscapeSequences. util#139 by @dwijnand
  • Fixes OkHttp’s JavaNetAuthenticator with a null check. lm#177 by @eed3si9n
  • Fixes Sonatype timeout issue by extending the default timeout to 1h. lm#246 by @peterneyens
  • Fixes thread thrashing error during the parallel download. lm249 by @OlegYch
  • Fixes JavaDoc warnings logged as errors. zinc#506 by @kaygorodov
  • Fixes class dependency not picking up classOf[A]. zinc#510 by @natansil
  • Fixes class dependency including non-existing objects. zinc422 by @romanowski
  • Fixes link to the documentation of deprecated 0.10/0.12 DSL syntax. #3901 by @colindean
  • Fixes the documentation of skip key. #3926 by @dkim
  • Fixes race condition in non-forked parallel tests. #3985 by @retronym
  • Fixes Ctrl-C handing in forked tests when Global / cancelable is set to true. #4226 by @driquelme
  • Fixes the stacktrace of run. #4232 by @eed3si9n
  • Bumps the version of Giter8 used by sbt new to 0.11.0, fixing various issues #4263 by @eed3si9n
  • Improves Javac error parsing. zinc#557 by @eed3si9n
  • Displays only the eviction warning summary by default, and make it configurable using ThisBuild / evictionWarningOptions. lm211 and #3947 by @exoego
  • Allow varargs in inThisBuild(...), inConfig(C)(...), inTask(t)(...), inScope(scope)(...). #4106 by @dwijnand
  • Adds fgRun and fgRunMain tasks that behaves like sbt 0.13’s run. #4216 by @agaro1121
  • Supports test.script and pending.script as the scripted file name. #4220 by @regadas
  • Supports aliases in inspect command. #4221 by @gpoirier
  • Adds the current project’s id to ~’s watching message. #2038 / #3813 by @dwijnand
  • Changes PathFinder#get to get(). io#104 by @dwijnand
  • Improves the error message when access is denied. lm#203 by @stephennancekivell
  • Improve the warning message “Choosing local” to something more actionable. lm#248 by @khvatov
  • Adds an option to ignore scalac options change. zinc#548 by @lukaszwawrzyk
  • Enable parallel execution of scripted in the plugin. #3891 by @jvican
  • Adds factory methods for Configuration axis scope filters inConfigurationsByKeys and inConfigurationsByRefs. #3994
  • Adds lastGrep, loadFailed, etc commands to replace the kebab-cased commands. #4080 by @naferx, #4159 by @Asamsig, and #4169 by @tiqwab
  • Adds timestamp field to JUnitXML report. 4154 by @timcharper
  • “Loading settings” log messages now show subproject name. #4164 by @alodavi
  • about command sorts and indents plugins list. #4187 by @mcanlas
  • -Dsbt.offline sets offline setting. #4198 by @eed3si9n
  • Selects most recent JDK during cross JDK forking (see below for details) #4245 by @raboof

Internal

Scala Spree NYC

Alongside Scala Days, Scala Spree came to New York City on June 19, 2018 hosted by Tapad and Scala Center (Lightbend provided breakfast and lunch). A Scala Spree is a hackathon event where you work on an OSS Scala project with a representative of that project (usually a maintainer) to send one (or more!) pull request(s) and thus becoming a Scala contributor.

The Lightbend Tooling Team attended the event representing sbt, and Dale and I came in prepared with GitHub issues tagged with “help wanted” and/or the “good first issue” labels to make sure contributors had things to work on.

 

 

There were discussions and bug investigation throughout the day, we had pull requests coming in on the day of and some following up later:

  • Adds fgRun and fgRunMain tasks that behaves like sbt 0.13’s run. #4216 by @agaro1121
  • Supports test.script and pending.script as the scripted file name. #4220 by @regadas
  • Supports aliases in inspect command. #4221 by @gpoirier
  • Fixes Ctrl-C handing in forked tests when Global / cancelable is set to true. #4226 by @driquelme
  • Improve the warning message “Choosing local” to something more actionable. lm#248 by @khvatov
  • Adds doc to CI process. #4218 by @regadas

We also want to thank to friendly Tapad folks were making sure that the event is welcoming.

Update: Check out Scala Spree NYC, a community effort — Open Sourcing Live @ Tapad report by Tapad too.

Participation

The last feature release of sbt 1 was sbt 1.1.0 in January, 2018. Since then, we’ve released six patch releases under sbt 1.1.x for bug fixes, but most of the feature enhancements were merged to 1.x branch.

In February, we published sbt 1.2.0 roadmap that listed three areas of focus:

  • the sbt 1.x build migration
  • making sbt easier to contribute to
  • more sbt server (LSP) enhancements

Lightbend’s Tooling team has been focusing on the idea of open source contribution throughout 2018. This means both in terms of what we can contribute to the community, as well as making it so that tools like sbt and Zinc are welcoming to the contribution from the Scala community at large. Holding meeting every Wednesday (you’re welcome to join), publishing the roadmap, improving the contributor’s guide, addressing flaky CI tests were some of the concrete steps that we’ve taken. In addition, we’ve tried to emphasize that there are many avenues of contribution:

  1. Help adoption
  2. Assist other users at work, or on Stackoverflow etc.
  3. Contribute to documentation
  4. Garden the issue tracker
  5. Help review pull requests
  6. Report issues
  7. Expand ecosystem
  8. Patch the core

Thanks again to everyone who’s helped improve sbt and Zinc 1.

sbt 1.2.0 was brought to you by 60 contributors. Dale Wijnand, Eugene Yokota, Kenji Yoshida (xuwei-k), Yasuhiro Tatsuno (exoego), Łukasz Wawrzyk, Jorge Vicente Cantero (jvican), Alistair Johnson, Antonio Cunei, Jason Zaugg, Rikito Taniguchi (tanishiking), Seiya Mizuno, Tim Harper, Aloisia Davì (alodavi), Arnout Engelen, Ethan Atkins, Johannes Rudolph, Krzysztof Romanowski, Allan Renucci, Brian P. Holt, Filipe Regadas, Hiroshi Ito, Martijn Hoekstra, OlegYch, Seth Tisue, natans, Aaron S. Hawley, Alex Khvatov, Alexander Samsig, Andreas Jim-Hartmann, Andrei Pozolotin, Andrey Kaygorodov, Anthony Garo, Christopher Hunt, Colin Dean, Daniel Riquelme, Deokhwan Kim, Gerard Maas, Guillaume Poirier, Heikki Vesalainen, Jason Pickens, Jonas Fonseca, Julien Jerphanion, Justin Pihony, Kazufumi Nishida, Kyle Goodale, Maksym Fedorov, Mark Canlas, Martynas Mickevičius, Michael Pollmeier, Mike Skells, Nafer Sanabria, Naohisa Murakami (tiqwab), PanAeon, Peter Neyens, Rui Gonçalves, Sean Sullivan, Stephen Nancekivell, Veera Venky, blakkan, ortigali. Thank you!

Share


Author(s)

Eugene Yokota

Scala Team, Lightbend
twitter: @eed3si9n | github: eed3si9n

Eugene is a maintainer of sbt, and a member of Scala Team at Lightbend. He enjoys long walks looking for Döner kebab, and functional programming.


Filter by Tag

all posts akka akka-streams cloudflow cloud-native cloudstate fast-data java kafka kubernetes lagom machine-learning microservices openshift play-framework scala spark streaming telemetry