Recently, the Lightbend team sponsored O'Reilly SACon London (and say hi to us at SACon SF) and met hundreds of architects, executives and developers interested in modernizing with Reactive technologies. In this interview with Mike Henderson, VP of Content at O'Reilly, Lightbend's Duncan DeVore, co-author of the Manning book Reactive Application Development, talks about the differences between Reactive Programming (one component of Reactive systems) and Reactive System Design, which looks at a much broader whole. Watch this 12-minute interview and check out the transcript below!
"We look at Reactive Programming as one of the methodologies or pieces of the puzzle for Reactive [Systems] as a broader term."
Mike Henderson (MH): Hi, this is Mike Henderson from Software Architecture in London. I'm here with Duncan from Lightbend. Duncan, how are you doing?
Duncan Devore (DD): I'm doing well, thank you.
MH: So you are with Lightbend. Lightbend has been known and associated with reactive a lot. Can you unpack all the different components of reactive?
DD: Yeah, sure. Reactive is probably the keyword that we're associated with. And one of the challenges, I think, behind the word reactive is that it's a fairly broad term and it can mean different things to different people. So, from our perspective, reactive has a distinct meaning specifically allocated towards reactive architectures or reactive design. I think the other meaning that folks associate with it is the notion of something called "Reactive Programming".
So reactive programming, as opposed to something like imperative programming–imperative programming, just as an example, you would have maybe an equation like A = B + C. And as time progresses, B and C change, but A does not. In a reactive programming model, A would change. As B and C were updated, that update would be propagated to A.
And that's an important piece of the puzzle. So we look at reactive programming as one of the methodologies or pieces of the puzzle for reactive as a broader term.
MH: And then can you move that up the stack to reactive architecture? What changes in that model?
DD: Sure. I think one of the best ways to take a look at it is by analogy. If you take a look at perhaps a team, an athletic team -- whatever the team is, baseball, football, basketball -- you can have a team comprised of fantastic athletes. They could be in fantastic physical shape, their cardio is great, and their actual skill at perhaps shooting the ball or kicking the ball is really good.
And so I would kind of equate those to being a reactive program or a reactive function, if you will. There's something that occurs, however, when they come together and they coalesce as a team. There's many examples where you have a team of fantastic players but the team in and of itself doesn't operate very well.
MH: And loses to the inferior team...
DD: Exactly, loses to the inferior team. And generally that's because of coaching or maybe the attitudes of the individual players or so forth. The difference I think is that when you move into the area of a reactive architecture or a team that is successful, a transformation occurs. In other words, the aggregate is not the sum of the parts.
The team in and of itself, or the architecture in and of itself, evolves, and it exhibits a feature or the ability of, to some degree, being proactive. In other words, the individual team players cannot really read each other's minds, but they've trained together for so long, they've worked together so well, that they know how to react in the moment of now in association with the other team members.
And I think that's the difference or the primary difference between just a simple reactive programming solution and a reactive architecture. The architecture in and of itself has to provide the tools so that you can program reactively. But when it coalescences together as a single unit, it has to present or it has to give the ability to proactively manage its surroundings by being able to react to whatever is occurring now.
And in our world, that's being resilient and being elastic, you know, automatically scaleable, and resilience -- not just fault tolerance, but being able to self-heal.
MH: Right. So self-healing and reactive, and does AI have any help in this new reactive model?
DD: That's an excellent question. So AI in the sense of machine learning or fast data analytics -- I come from the energy industry, and one of the things that we reactively/proactively managed was energy type grids. And all those types of systems are the same. They're distributed systems.
And you can think of a transformer or whatever as a [microservicer, as] a node, and what you need to do is you need to keep harmony in how that entire things operates as a system. It's when a spike in energy occurs, you get an oscillation, and that blows out a transformer. It's like a server going down.
So the idea is that each individual transformer, i.e., micro service, has to be able to react not only to its environment but also to the system as a whole. And then what you layer in is AI or fast data analytics...
MH: For predictive...?
DD: For predictive analysis, exactly. So what that allows you to do is that even though each microservice is very baked into its DNA, this notion of reactive, you are now able to optimize that ability because you're able to predict the future within some delta of error and give it more information, you -- based on historical analysis and some transformation of algorithms, you believe that the future is going to be X, and so you can begin to prepare for it.
So another example I like to use is you are going out in the woods or camping or something and you have the appropriate equipment for rain, snow, sleet, hail, all that kind of stuff. However, it would be nice if you could optimize it and you would know that, you know, it's going to rain, so I don't need a heavier coat. Even though the heavier coats will do both...
MH: But it's not freezing...
DD: It's not freezing, right. So you can perform more efficiently and less costly.
MH: So in today's world when someone thinks about the architecture that their company is trying to build for delivering software to their customers, how do they think reactively? How do they think about the architecture that they need to build rather than the one that they have? What are the first things that they do to build that reactive architecture?
DD: I think that in and of itself is one of the major challenges because the shift that you see a lot of companies making in today's world is they used to be some type of retailer or service provider or et cetera, and now they're rethinking what am I? And they're redefining themselves as, say, a technology company that happens to provide banking services or a technology company that happens to provide...
MH: Every company is a software company.
DD: Exactly. Every company is a software company. So the challenge becomes in order to break through that barrier, it's a philosophical way of thinking about what I am as a company. And I think that's really where the challenge lies. For the developer or the engineer or the architect, so forth, for them, it's a learning process. They've been trained to think one way.
You know, they're thinking more monolithic or whatever over the last 20 years, and now embracing a new approach. I think the first thing that an architect has to accept -- when I used to interview people, I said if you're going to come -- or somebody would ask me, can I get into comp sci? I said the one thing that you have to accept is you'll be in school forever, and if you don't accept that...
MH: Yeah, lifelong learning.
DD: Yeah -- you shouldn't, because this industry moves so fast, and it requires a lot of effort, it's a lot of sacrifice, to stay on top of things. So it's something you have to desire to do.
MH: So let me ask you this, then. So what is the second thing a company does? Let's say a company knows what they want to do. They want to, you know, from a banking company to a services-oriented company, and they're going to have APIs everywhere, and they're going to do everything that they can to really be a service first company. What's the second thing they do when they're thinking about their architecture of software development? I mean, is there some secret sauce for success versus people who just give it the good old try but it doesn't work?
DD: Sure, yeah. I would say that the first thing -- and I actually went through this caterpillar-to-moth genesis in my previous company. I realized that the way that we designed systems at the time was not going to be able to handle the volume of data that we were going to process. There was just no way. I had seen it crash. I had seen it had all kinds of problems.
The problem was I didn't really know how to solve the problem. And as I began to do the research and understand, I realized that distributed computing -- and from our perspective, at Lightbend, that's what we're talking about when we talk about reactive -- distributed computing was the solution. Now, which distributed computing model and all that had yet to be figured out at the time.
But I knew there were some things that -- about distributed computing that were markedly different from the traditional, more monolithic style. One of them is, for example, the domain model. The way that you design the domain model is influenced heavily by the fact that you're a distributed architecture. And things like immutability become incredibly important, whereas they're not so important in a monolithic style architecture.
So the first thing, I think, or the key thing is recognizing the fact that it's not just Google and Amazon and Netflix that need distributed systems. That your average company, now, of medium size or even smaller, if they want to be competitive, they're going to embrace that mentality. And once they do that, then they can begin to take a look at where they want to go.
If they're on the JVM, Lightbend is the choice, in my opinion -- obviously, but -- and as a customer, prior to joining the company, I did that same research and looked at the same things and realized that the solutions that we offer were the best choice. So that's what I would say would be kind of like the evolution --
MH: The journey?
DD: The journey.
MH: Yeah. So if you fast forward 12 months and let's say we sit down here in London next year, what would you like to say Lightbend is doing differently in this space, in specifically architecture and reactive architectures -- what would you like to say you're doing differently? And what should the industry be doing differently?
DD: So I think two primary areas is obviously continue honing and tuning the toolkits that we have and providing the feature set that we have. One of the things I like that we do is we listen to the customer and then we weigh what the customer is asking for to see if it makes sense with the direction that we're headed in.
And I think the toolkit that we have out there now is essentially focused in that direction and does a good job, and we'll continue to just optimize it. Things like Akka, Akka cluster, Akka cluster sharding, and all those types of things in play. I think that is one big part. It works well. You know, it just works. That's what we always used to say about Akka: we use it because it just works.
Then I think the other area is the data analytics. Having been in the energy industry and doing a fair amount of that to predict -- because it was tied to the energy market, where you could make money. You know, you make money by either saving it, not spending it, or earning it, right? Individual companies would allow their generator to be used to supply energy to the grid or whatever.
MH: Unlike Uber, energy doesn't surge price, do they?
DD: That's a good question. Now, the energy market is an interesting beast. So in order to be successful there, there's also incentives and so forth. And, for example, in the energy market, if you're -- on peak load days, if you were to not exceed a particular value, the general company that ran the five-state area would give -- you could make hundreds of thousands of dollars.
But you had to predict what your load would be or when those days would be. And I think that's where we're going to begin to offer some real value is the notion of being able to do predictive analytics in a timely fashion. The historical problem with that is it takes too long. Like, I can tell you what the temperature was yesterday with 100 percent accuracy.
And I think, as time goes on, hardware gets better, the software algorithms get better, things like Spark and the things that we're working on are more efficient, and streaming of data, so forth. You're going to be able to get answers and answer questions about what the future could be. And then kind of prepare yourself for it programmatically.
MH: Excellent. Duncan, we look forward to that conversation. Thank you.
DD: Thank you. It was nice meeting you.
Learn more about building Reactive Systems as a whole in Duncan's book, Reactive Application Development. Free sample chapters and a 40% Lightbend discount await you here:
