Eugene Yokota

Scala Team

July 31, 2018

scala

sbt 1.2.0 released

Hi everyone. Lightbend Tooling team is happy to announce sbt 1.2.0. This is the second feature release of sbt 1, a binary compatible release focusing on new features. sbt 1 is released under Semantic Versioning, and the plugins are expected to work for sbt 1.x series.

Warning: We found forward compatibility breakage in 1.2.0, so we recommend everyone to upgrade to sbt 1.2.1.

Note: I will be using this URL to announce the release candidates and final as well.

1.2.0 is now final.
~~If no serious issues are found by **Monday, July 30th 2018**, 1.2.0-RC3 will become 1.2.0 final.~~
~~If no serious issues are found by **Monday, July 16th 2018**, 1.2.0-RC1 will become 1.2.0 final.~~

The headline features of sbt 1.2 are cross JDK forking, composite project, and experimental thin clients. But, there are lots of other bug fixes and enhancements that we’ve been accumulating for six months since sbt 1.1.

SbtPlugin for plugin development

SbtPlugin is a plugin to declare a project for sbt plugins. This automatically brings in scripted tests, and sets sbtPlugin := true.

lazy val root = (project in file("."))
	.enablePlugins(SbtPlugin)

Compatibility note: ScriptedPlugin is no longer a triggered plugin.

#3875 by @eed3si9n

Cross JDK forking

For forked run and test, java++ can now switch Java Home.

sbt:helloworld> run
[info] Running (fork) Hello
[info] 1.8.0_171
sbt:helloworld> java++ 10!
[info] Reapplying settings...
sbt:helloworld> run
[info] Running (fork) Hello
[info] 10.0.1

sbt will try to detect Java homes into discoveredJavaHomes setting, supporting shyiko/jabba. This can be augmented by Global / javaHomes:

Global / javaHomes += "6" -> file("/something/java-6")

This feature is intended for testing your library in an older JVM to check compatibility.

#4139 by @2m, @cunei, and @eed3si9n

scalaVersion-filtered aggregation

In 2015 James Roper contributed scalaVersion-filtered aggregation to sbt-doge. This feature is brought back into sbt 1.2 by Rui Gonçalves (@ruippeixotog) in #3698/#3995!

This extends switch command ++ to take an optional <command>:

> ++2.12.6 compile

This will aggregate only the subproject where ++2.12.6 is valid, which is useful when you have a build where some subprojects are 2.11 only etc.

Composite project

sbt 1.2.0 introduces “composite project” trait, which allows plugin authors to generate subprojects, for example for cross building.

trait CompositeProject {
	def componentProjects: Seq[Project]
}

This was contributed by @BennyHill as #4056.

Project matrix

Experimental. As a reference implementation of the CompositeProject I implemented a new DSL called projectMatrix introduced by sbt-projectmatrix plugin.

lazy val core = (projectMatrix in file("core"))
	.scalaVersions("2.12.6", "2.11.12")
	.settings(
		name := "core"
	)
	.jvmPlatform()

lazy val app = (projectMatrix in file("app"))
	.dependsOn(core)
	.scalaVersions("2.12.6")
	.settings(
		name := "app"
	)
	.jvmPlatform()

The aim of the plugin is to support a generic notion of cross building (Scala version, platform, etc) expressed using subprojects. In the above projectMarix will produce three subprojects: coreJVM2_12, coreJVM2_11, and appJVM2_12.

Semantic Version selector API

sbt 1.2.0 introduces Semantic Version selector on VersionNumber() datatype supporting basic match, comparison (<=, <, >=, >), combination (>1.0.0 <2.0.0, ||), ranges (A.B.C - D.E.F), and wildcard (2.12.x).

scala> import sbt.librarymanagement.{ VersionNumber, SemanticSelector }
import sbt.librarymanagement.{VersionNumber, SemanticSelector}

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector(">=2.12"))
res1: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("<2.12"))
res2: Boolean = false

scala> VersionNumber("2.13.0-M4").matchesSemVer(SemanticSelector("2.13"))
res3: Boolean = false

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.12.1 - 2.12.6"))
res4: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.12.x"))
res5: Boolean = true

scala> VersionNumber("2.12.5").matchesSemVer(SemanticSelector("2.11.x || 2.12.x"))
res6: Boolean = true

Note: This has no effect on library management at the moment.

This was contributed by Rikito Taniguchi (@tanishiking) as lm#239.

addPluginSbtFile command

There’s been a request from IntelliJ to safely inject a plugin to a build. sbt 1.2.0 adds -addPluginSbtFile command to do so.

$ cat /tmp/extra.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.7")

$ sbt -addPluginSbtFile=/tmp/extra.sbt
...
sbt:helloworld> plugins
In file:/xxxx/hellotest/
	...
	sbtassembly.AssemblyPlugin: enabled in root

Implmented by @eed3si9n as #4211.

Extensible sbt server

Experimental. sbt server can now be extended via the plugin.

    Global / serverHandlers += ServerHandler({ callback =>
			import callback._
			import sjsonnew.BasicJsonProtocol._
			import sbt.internal.protocol.JsonRpcRequestMessage
			ServerIntent(
				{
					case r: JsonRpcRequestMessage if r.method == "lunar/helo" =>
						jsonRpcNotify("lunar/oleh", "")
						()
				},
				PartialFunction.empty
			)

This feature is still experimental and the API may change in the future.

#3975 by @eed3si9n

Thin client(s)

Experimental. sbt 1.2.0 adds a new mode called -client. When sbt is started with -client command, it no longer to loads the build, and instead tries to connect to an instance of sbt server over JSON-RPC. When the server is not running (portfile is not found), it will fork a new instance of sbt entirely in a new JVM.

This lets you invoke sbt from the terminal shell or from an editor.

$ time sbt -client clean
[info] entering *experimental* thin client - BEEP WHIRR
[info] server was not detected. starting an instance
[info] waiting for the server...
[info] waiting for the server...
[info] waiting for the server...
[info] waiting for the server...
[info] server found
> clean
[success] completed
sbt -client clean  9.23s user 2.33s system 22% cpu 50.558 total

# server stays
$ ps | rg java
21860 ttys015    1:22.43 java -Xms2048M -Xmx2048M -Xss2M -jar /usr/local/Cellar/sbt/1.1.6/libexec/bin/sbt-launch.jar
22014 ttys015    0:00.00 rg java

$ time sbt -client clean
[info] entering *experimental* thin client - BEEP WHIRR
> clean
[info] Updating ...
[info] Done updating.
[success] completed
sbt -client clean  3.39s user 1.75s system 104% cpu 4.898 total

To end the server, call sbt -client shutdown. #4227 by @eed3si9n

In addition, there are also an alternative thin clients cb372/sbt-client and dwijnand/sbtl implemented using Rust.

Changes with compatibility implication

Removes deprecated commands -, --, and ---. Use onFailure, sbtClearOnFailure, and resumeFromFailure instead. #4124
Makes ++ fail when it doesn’t affect any subprojects #4269 by @eed3si9n

Other bug fixes and improvements

Fixes output caching bug. util#169 by @bpholt
Fixes “destination file exists” error message. lm#255 by @eed3si9n
Reintroduces Command.process(String, State): State. #4023 by @dwijnand
Fixes active.json not getting removed on JVM shutdown. #4194 by @veera83372
Fixes file permission error (“CreateFile() failed”) while reading the timestamp on Windows. io#134 by @cunei
Fixes the linter that detects missing .value. #4090 by @eed3si9n
Fixes StringIndexOutOfBoundsException in removeEscapeSequences. util#139 by @dwijnand
Fixes OkHttp’s JavaNetAuthenticator with a null check. lm#177 by @eed3si9n
Fixes Sonatype timeout issue by extending the default timeout to 1h. lm#246 by @peterneyens
Fixes thread thrashing error during the parallel download. lm249 by @OlegYch
Fixes JavaDoc warnings logged as errors. zinc#506 by @kaygorodov
Fixes class dependency not picking up classOf[A]. zinc#510 by @natansil
Fixes class dependency including non-existing objects. zinc422 by @romanowski
Fixes link to the documentation of deprecated 0.10/0.12 DSL syntax. #3901 by @colindean
Fixes the documentation of skip key. #3926 by @dkim
Fixes race condition in non-forked parallel tests. #3985 by @retronym
Fixes Ctrl-C handing in forked tests when Global / cancelable is set to true. #4226 by @driquelme
Fixes the stacktrace of run. #4232 by @eed3si9n
Bumps the version of Giter8 used by sbt new to 0.11.0, fixing various issues #4263 by @eed3si9n
Improves Javac error parsing. zinc#557 by @eed3si9n
Displays only the eviction warning summary by default, and make it configurable using ThisBuild / evictionWarningOptions. lm211 and #3947 by @exoego
Allow varargs in inThisBuild(...), inConfig(C)(...), inTask(t)(...), inScope(scope)(...). #4106 by @dwijnand
Adds fgRun and fgRunMain tasks that behaves like sbt 0.13’s run. #4216 by @agaro1121
Supports test.script and pending.script as the scripted file name. #4220 by @regadas
Supports aliases in inspect command. #4221 by @gpoirier
Adds the current project’s id to ~’s watching message. #2038 / #3813 by @dwijnand
Changes PathFinder#get to get(). io#104 by @dwijnand
Improves the error message when access is denied. lm#203 by @stephennancekivell
Improve the warning message “Choosing local” to something more actionable. lm#248 by @khvatov
Adds an option to ignore scalac options change. zinc#548 by @lukaszwawrzyk
Enable parallel execution of scripted in the plugin. #3891 by @jvican
Adds factory methods for Configuration axis scope filters inConfigurationsByKeys and inConfigurationsByRefs. #3994
Adds lastGrep, loadFailed, etc commands to replace the kebab-cased commands. #4080 by @naferx, #4159 by @Asamsig, and #4169 by @tiqwab
Adds timestamp field to JUnitXML report. 4154 by @timcharper
“Loading settings” log messages now show subproject name. #4164 by @alodavi
about command sorts and indents plugins list. #4187 by @mcanlas
-Dsbt.offline sets offline setting. #4198 by @eed3si9n
Selects most recent JDK during cross JDK forking (see below for details) #4245 by @raboof

Internal

Removes some compiler warnings. #3087 by @dwijnand
Lots of other refactorings by @dwijnand
Removes some compiler warnings in Zinc. zinc#493 by @exoego
Perf: Prevents creation of useless URI copies in IO.directoryURI. io#132 by @jrudolph
Perf: Avoids reflect universe initialization in initStringCodecs. util#153 by @jrudolph
Perf: Speeds up Parsers.validID. #3952 by @jrudolph
Perf: Optimizes scope delegation by hand rolling for comprehension. #4003 by @jrudolph and @eed3si9n
Use val instead of var in an internal code. #4253 by @xuwei-k

Scala Spree NYC

Alongside Scala Days, Scala Spree came to New York City on June 19, 2018 hosted by Tapad and Scala Center (Lightbend provided breakfast and lunch). A Scala Spree is a hackathon event where you work on an OSS Scala project with a representative of that project (usually a maintainer) to send one (or more!) pull request(s) and thus becoming a Scala contributor.

The Lightbend Tooling Team attended the event representing sbt, and Dale and I came in prepared with GitHub issues tagged with “help wanted” and/or the “good first issue” labels to make sure contributors had things to work on.

Thanks everyone who joined us for Scala Spree NYC today! pic.twitter.com/LPzi1V9Hkf
— sbt (@scala_sbt) June 19, 2018

There were discussions and bug investigation throughout the day, we had pull requests coming in on the day of and some following up later:

Adds fgRun and fgRunMain tasks that behaves like sbt 0.13’s run. #4216 by @agaro1121
Supports test.script and pending.script as the scripted file name. #4220 by @regadas
Supports aliases in inspect command. #4221 by @gpoirier
Fixes Ctrl-C handing in forked tests when Global / cancelable is set to true. #4226 by @driquelme
Improve the warning message “Choosing local” to something more actionable. lm#248 by @khvatov
Adds doc to CI process. #4218 by @regadas

We also want to thank to friendly Tapad folks were making sure that the event is welcoming.

Update: Check out Scala Spree NYC, a community effort — Open Sourcing Live @ Tapad report by Tapad too.

Participation

The last feature release of sbt 1 was sbt 1.1.0 in January, 2018. Since then, we’ve released six patch releases under sbt 1.1.x for bug fixes, but most of the feature enhancements were merged to 1.x branch.

In February, we published sbt 1.2.0 roadmap that listed three areas of focus:

the sbt 1.x build migration
making sbt easier to contribute to
more sbt server (LSP) enhancements

Lightbend’s Tooling team has been focusing on the idea of open source contribution throughout 2018. This means both in terms of what we can contribute to the community, as well as making it so that tools like sbt and Zinc are welcoming to the contribution from the Scala community at large. Holding meeting every Wednesday (you’re welcome to join), publishing the roadmap, improving the contributor’s guide, addressing flaky CI tests were some of the concrete steps that we’ve taken. In addition, we’ve tried to emphasize that there are many avenues of contribution:

Help adoption
Assist other users at work, or on Stackoverflow etc.
Contribute to documentation
Garden the issue tracker
Help review pull requests
Report issues
Expand ecosystem
Patch the core

Thanks again to everyone who’s helped improve sbt and Zinc 1.

sbt 1.2.0 was brought to you by 60 contributors. Dale Wijnand, Eugene Yokota, Kenji Yoshida (xuwei-k), Yasuhiro Tatsuno (exoego), Łukasz Wawrzyk, Jorge Vicente Cantero (jvican), Alistair Johnson, Antonio Cunei, Jason Zaugg, Rikito Taniguchi (tanishiking), Seiya Mizuno, Tim Harper, Aloisia Davì (alodavi), Arnout Engelen, Ethan Atkins, Johannes Rudolph, Krzysztof Romanowski, Allan Renucci, Brian P. Holt, Filipe Regadas, Hiroshi Ito, Martijn Hoekstra, OlegYch, Seth Tisue, natans, Aaron S. Hawley, Alex Khvatov, Alexander Samsig, Andreas Jim-Hartmann, Andrei Pozolotin, Andrey Kaygorodov, Anthony Garo, Christopher Hunt, Colin Dean, Daniel Riquelme, Deokhwan Kim, Gerard Maas, Guillaume Poirier, Heikki Vesalainen, Jason Pickens, Jonas Fonseca, Julien Jerphanion, Justin Pihony, Kazufumi Nishida, Kyle Goodale, Maksym Fedorov, Mark Canlas, Martynas Mickevičius, Michael Pollmeier, Mike Skells, Nafer Sanabria, Naohisa Murakami (tiqwab), PanAeon, Peter Neyens, Rui Gonçalves, Sean Sullivan, Stephen Nancekivell, Veera Venky, blakkan, ortigali. Thank you!

Author(s)

Eugene Yokota

Scala Team, Lightbend

twitter: @eed3si9n | github: eed3si9n

Eugene is a maintainer of sbt, and a member of Scala Team at Lightbend. He enjoys long walks looking for Döner kebab, and functional programming.

Comments

Filter by Tag

all posts akka akka-streams cloudflow cloud-native cloudstate fast-data java kafka kubernetes lagom machine-learning microservices openshift play-framework scala spark streaming telemetry

Eugene Yokota

Scala Team

July 31, 2018

sbt 1.2.0 released

SbtPlugin for plugin development

Cross JDK forking

scalaVersion-filtered aggregation

Composite project

Project matrix

Semantic Version selector API

addPluginSbtFile command

Extensible sbt server

Thin client(s)

Changes with compatibility implication

Other bug fixes and improvements

Internal

Scala Spree NYC

Participation

Share

Author(s)

Eugene Yokota

Scala Team, Lightbend

Comments

Filter by Tag