Next-Gen Corda is Here!

By: Dries Samyn, Principal Software Engineer at R3

Next-Gen Corda in Numbers

• 3 years in the making
• 2 open-source repositories
• Nearly 100 contributors
• At least 3 R3 engineering babies conceived and successfully delivered (we take nurturing in-house talent very seriously!)
• Over 1,500,000 lines of code added, deleted, or changed
• Over 4000 pull requests
• Over 5000 merged commits
• Over 70,000 Jenkins builds
• Over 500,000,000 test executions
• 2 alpha releases
• 4 beta releases
• Exactly 3.56 liters of blood, sweat, and tears

These are just some of the statistics to give an indication of the scale of the work that has gone into building the next generation of the best Distributed Ledger Technology (DLT) platform in the world. All while continuing the development of Corda 4, with four releases during this period, to maintain its status as “the go-to DLT platform”.
In this blog, we will reflect on the journey from the perspective of an engineer.

Why Did We Do This?

I would like to say that it was for the love of solving really complex engineering problems, but it wasn’t. Of course we loved solving these complex problems, often getting things wrong a couple of times before we got it right, but there were more solid reasons that we undertook what has been the largest technical transformation of the Corda platform to date.

R3 has been working with customers using Corda from the early days of DLT adoption. We have seen innovative use cases and organizations solving real world problems, sometimes using Corda in ways we hadn’t originally envisaged. Some of those cases hit limitations that could not be facilitated by modifying the original architecture of Corda 4.

Along came “Project Starfish” and with it, the product team presented a vision for what the next generation of Corda needed to be. This vision would ensure that Corda was fit for the next generation of financial market infrastructure, digital assets, and digital currency applications, as well as the many other innovative use-cases of private, permissioned blockchain technology.

What Does Next-Gen Corda Look Like?

It’s still the Corda we know and love

Before describing what is new and how this has changed Corda, we want to establish that Next-Gen doesn’t mean it’s a different thing. We made some improvements to the CorDapp API that will hopefully help developers to write cleaner, more testable code, but this should not feel like a big change to existing CorDapp developers. We’re keeping the Corda Flow programming model that we know and love, the UTXO ledger is what we’re familiar with, and smart contracts still look pretty much like they did before.

The big technical transformation that has happened has been all about the internals. We’ve taken Corda and given it a big engine transplant.

Highly available (HA) and scalable

“Highly Available” was the leading requirement for Project Starfish. The types of applications that we now know are being built on the Corda platform have challenging requirements around uptime and scalability. This was probably the biggest driver behind the need to re-imagining some of Corda’s foundations.

Previously, a Corda node was supported by a monolithic JVM process. This means that it’s quite easy to manage and it performs well. However, there are limits to how much it can scale. As one node was one process, scaling a Corda node meant finding a bigger JVM/Host.
This also means that achieving a highly available configuration was difficult, as only one process can support a node at any time. There is limited support for a hot or warm topology where you can have a second instance on standby to take over in case of failure.

Next-Gen Corda has taken on a distributed architecture designed around stateless worker processes that can scale up or down and provide redundancy.

Multi-Tenancy

Of course, Corda is still a distributed system, however, what we see is that users often operate multiple Corda nodes. Either because have multiple segregated identities on a network, or because it is part of a progressive decentralization model where multiple nodes are operated on behalf of others until they are ready to manage their own.

This is the reason that we separated the concept of the Corda node from the compute resource. In Next-Gen Corda, we call them Virtual Nodes and they are simply execution contexts in which worker processes within a Corda cluster can operate.

To see this in action, you can easily set up a Corda network with dozens or even hundreds of nodes on your laptop, or local development environment.

Network of networks

We believe in the value of private, permissioned, networks, but that doesn’t mean that we don’t believe in interoperability between these networks. Being able to complete an atomic Delivery vs Payment (DvP) transaction across Corda networks, or even between Corda and other blockchains such as Ethereum, is something that our customers and partners in the ecosystem want to do. This is why Interoperability was such an important requirement for Next-Gen Corda. We have already announced and launched Hyperledger Lab Harmonia – watch this space for more announcements.

REST API

You interact with the Next-Gen Corda using a standard REST interface over https. This makes it straight forward to integrate into any kind of application or workflow. Corda no longer takes any opinion of what language or framework that you use to write your client app.

Manage network membership through the new Membership Group Manager (MGM)

Next-Gen Corda introduces the Membership Group Manager (MGM). Its role is to register and manage members of the network. This registration process can be automatic, or custom registration logic can be integrated with your own workflow through integration with the REST API.

New peer-to-peer communications layer

P2P (peer-to-peer) communications in Next-Gen Corda uses a standard HTTP transport, making it easier for people to connect their Corda installations to others in the network. The layer was designed and built as an independent component, which was released as an open source preview just over a year ago.

Pluggable ledger / pluggable notary

The UTXO ledger and notary in Next-Gen Corda will be familiar to those who are already familiar with Corda. However, similar to the P2P layer, the ledger and notary are designed and built as layers on the top of the stack. They are independent of the Corda flow model and this enables adding alternative ledger models in the future, or in the case of the notary, custom implementations can be supported by Network Operators.

Operator Experience

We recognized that most of our customers wanted to host Corda in a containerized environment, typically in the cloud. We also recognized that Kubernetes is a good match for the new Corda architecture.

You don’t have to operate Next-Gen Corda in Kubernetes or the Cloud, but when you do, we’ll make your life as a platform operator easy, since Corda is designed to be operated in such environments.

Developers Developers Developers

Engineers of my vintage will probably remember the day in the early 2000s, when Steve Ballmer turned into a meme, but wasn’t he right about developers? He thought that the success of Microsoft was dependent on a large ecosystem of applications, supported by developers developing for the platform.

We want CorDapp developers to love the platform as much as we do and the “CorDapp Developer” persona has featured high in the Next-Gen Corda user stories. This means that we’ve introduced a REST API for easy integration and we’ve improved the CorDapp API so that it is now cleaner and more testable. We’re also giving CorDapp developers a way of running an entire Corda Network in a single process, easily runnable from a local development environment.

And this is just the start, because we have some very exciting developments in the pipeline – so watch this space!

What’s Next?

After reflecting over the journey of the last couple of years, it feels like we’ve reached a destination, but as my colleague and great mentor Katelyn Baker reminded me numerous times over the last 2 years, releasing Corda 5.0 is just the start of the road. We have built the foundations for a Corda that is ready to support the next generation of distributed applications at scale.

Yes, we will be improving and fine-tuning those foundations in the next few months, but mostly, we’ll be building on top of those foundations to help our customers and users to innovate.

For now though, we’re due a moment to reflect and recharge, but we hope that you will be busy testing the new platform and giving us the feedback that we need to make Corda even better.

Next-Gen Corda is available today. Download the Corda artifacts, learn more in the Corda docs, and start building for free.