Technical lessons from Ivno’s CorDapp Trial

The Trial was successful, with sub-second transaction times and $18bn in Trial tokens created and transacted within a live, production-parity environment. I’d like to share some of the important decisions and lessons prevailing from the Trial in the hope it provides a net positive knowledge transfer to the Corda community, and indeed the wider financial world.

Deployment

To consider the Trial a success, we determined that participants should have the opportunity to deploy and manage their own nodes — as is likely to be the case in production.

Corda utilises technology and stacks that most senior technologists will likely be comfortable with: Java (or Kotlin); AMQP; SQL, and others.

It also leans on infrastructure that should be equally familiar, with node provisioning needing: A Virtual Machine (VM) — in our case running Ubuntu 18.04 LTS; some inbound port rule configurations for the application; and an optional public DNS record for the GUI.

Ivno’s technical team made the decision to ensure the second stages of deployment (post-provisioning) was handled on behalf of participants.

This included installing OS pre-requisites, downloading and installing Corda’s platform to the VM, installing and configuring Ivno’s CorDapp on top of Corda, applying the correct settings to the node.config file, and finally, requesting access to R3’s pre-production network via a Certificate Signing Request (CSR).

This was achieved using a combination of Bash scripts and Docker Containers stored in a container registry, but ultimately allowed the participant to use a single command line entry to automatically serve all of the above.

This provided participants with the capability to spin up a single node within less than 5 minutes, pending entry to the pre-production environment to join the Trial’s business network.

It allowed users to ultimately test all three of our use-cases (Instant Settlement, Real-Time Collateral Mobility and Instant Effective Cash Transfer) internally within their own organisation, as well as externally with other participants. It simulated a real-world node deployment with minimal effort and maximal efficacy.

A lesson learned from the deployment stage is that the manual provisioning of the infrastructure on Microsoft Azure provided little to the participants in terms of learning opportunities, as many had already had technical experience with infrastructure provisioning in the past.

I would recommend investing time in infrastructure-as-code — in our case, Terraform — to undertake both the provisioning and the post-provisioning process on behalf of your participants. Ivno’s technical team has now incorporated an end-to-end node spin-up process to do exactly this in production.

Updating Your CorDapp

One of the most difficult computer science problems is how to manage, and update, a distributed network. Within a live environment, developers and architects need to consider that friction should be minimised when updating your CorDapp and it’s dependencies.

One cannot assume that updates will be deployed throughout the network at an agreed time between all participants (a lock-step upgrade), with the likely outcome being that new updates are instead factored in to an organisation’s existing update cycle. For financial institutions, this is often cumbersome and absolutely critical to deploy without bugs, which means some update cycles can be as irregular as bi-yearly, or even longer to allow for testing and fixes.

To minimise friction and to test an update script Ivno is intending to deploy in production, we designed an update script similar to the deployment script(s) described above. The update script would pull the newest version of the CorDapp with the newest features, enable the node’s drainingMode, and then power down and restart the node, as is best practice.

Ivno’s update script was used in a production-parity environment over a hundred times across deployed nodes without error, allowing participants to benefit from added features that were requested during and throughout the Trial.

You must also ensure backwards-compatibility. Given there is no guarantee as to when financial institutions will apply the new update in a live environment, you can assume by default that all CorDapps should be designed to be backwards-compatible for as long as is possible.

For some more technical information on how to do this effectively and a chance to learn by doing, I would recommend Matt Layton’s Cordacademy guide on how to upgrade CorDapps:

Explicit Upgrades and Implicit Upgrades.

Some might also find Ivan Schasny & Mike Hearn’s design on a CorDapp Distribution Service (CDS) useful, with the technical implementation here.

Environments

Consider your environment choice carefully. R3’s test networks, and entrance thereto, is subtly different than R3’s pre-production network.

The pre-production network is intended to mostly retain parity with CordaNet.

Our recommendation will be to use them as intended — begin testing on the test network, and then once the CorDapp is considered ready, port over to the pre-production network for the Trial, or simply for final checks prior to production deployment.

Timezones

The Global Token Trial was fortunate to have participating entities from all over the world from 18 countries, and (almost) every continent.

A byproduct of this incredible level of interest is that support to participants must be considered. Ivno is based out of London, so some late nights supporting our Asian-Pacific participants were necessary. We are lucky to have been assisted by R3, who have a global presence and were able to help support a multi-timezone deployment. We are incredibly grateful for this.

Governance

Governance is the plat du jour of all serious DLT communities. Deployment, networks, support and the application itself are all intensely difficult areas to successfully employ, but many participants were interested in the nuances of a production environment. We were able to extend R3’s Business Network Membership Service (BNMS) for our use cases. Some notable questions included:

• How do you guarantee effective KYC/AML implementations throughout the network?
  Entrance into CordaNet deals with this to a certain point, with all parties being identified prior to entry into the network. KYC can be further attested to by an existing KYC provider whose node has been given that specific role within the business network.
• How do you ensure Tokens are collateralised, and can only be issued by approved entities?
  Largely this is dealt with by CorDapp signing, and the utilisation of Corda’s contract design, whitelisting specific entities for Issuance. There are similar proprietary off-chain practices for military-grade protection during Issuance and Redemption workflows.
• How do you onboard/offboard members of the Business Network?
  Clearly, elements of off-chain integration with the incumbent financial system is needed to facilitate entry to Ivno’s token network, but the onboarding & offboarding process is made simple using the BNMS. I’d like to take this opportunity to give kudos to some community efforts such as Koncerto, which takes the concept of Business Networks many steps forward with a decentralised network design, allowing members of the network to create and upkeep the network without a centralised party.
• What else can be achieved with the BNMS?
  The opportunities are mostly endless. With the requisite design experience, you can design anything — from creating a role to include regulators and minimise reporting overheads as a result (a concept Ivno is working towards), to allowing all your members to create a decentralised society and voting system.

Tell Your Story

After months or years spent creating a product, it’s easy to fall into the trap of assuming participants will understand your product in as much detail as your internal team.

Don’t make such assumptions, DLT’s benefits can be hard to understand as is. Hire a UX expert or host User Story sessions with your team. Work out your customer profile(s), understand your product’s benefit with regards to these customer profiles and design specific storyboards with this knowledge. Your participants will be better able to understand your product in detail if you guide them through it with your greater insight.

Ensure your UI is intuitive and easy to use. It should remove any complexities that the underlying DLT layer creates. It should offer tips, guides, and an easy way for users to contact your support team. Look at incumbent systems you’re attempting to disrupt, and understand the market norms and best practices. You are in this industry to disrupt and innovate, but utilising existing practices within the industry allows you to focus on the differences and improvements your product offers, whilst removing excess complexity. Change is always best when incremental.

The Drive Towards Production

Ivno are driving towards production early next year, and working closely with partners directly from the Trial but equally with members prior to and subsequent to the Trial.

We endeavour to continue building a community that normalises token standards, assists with design decisions, and whom may take part in our own private ‘pre-production’ networks to help test and understand Ivno’s product in greater detail.

Should you like to become a part of Ivno’s community, either as a business or technical interest — please reach out to [email protected].

R3 And The Corda Community

It’s fair to say a Trial of this magnitude would have been substantially more difficult without the help of R3.

R3 encourages an open-source community, driven not only by organisations such as Ivno, but with equal credence given to standalone developers. I would encourage all to get involved with any of the concepts above, and we would be delighted to support in any new pursuits or contributions. After all, any contribution to Corda is a net positive to the entire community.

And after the recent festivities and successes on display at CordaCon, it is very clear to me how substantial, supportive and exceptionally talented this community is.

I’d like to finally thank all at R3, with specific emphasis given to Alex Oldfield, Nick Rogers, Dhruv Kapoor, Austin Moothart, Roger Willis, Neal Shah, and the Trial team as a whole who worked tirelessly to make the Trial a success.

I would also like to thank the whole team at Ivno. On top of having industry-leading talent to offer within their niche, they are also exceptionally hard-working. Thank you.

Let’s not forget all representatives from our participant organisations, who made the Token Trial truly global, and continue to help us sharpen our product.

Many thanks to all involved.