SDK Development

One of the key objectives we had when building the first group of SDKs was to enable developers to change quickly between programming languages without having to adapt to a completely different design. This document intends to guide developers to ship Catapult-based SDKs that share the same design to achieve interoperability.

Rationale

The NEM2-SDK shares the same design/architecture between programming languages to accomplish the next properties:

  • Fast language adaptation: For example, there is a NEM2-SDK for Java, but you need to work on C++. As both SDKs share the same design, you could re-write the library faster, adapting the syntax to your language. The same principle also applies to code examples, projects, applications…
  • Cohesion/shared knowledge across NEM developers: Every developer in the NEM ecosystem should be able to change between projects that use NEM. By sharing the same design, we also share the same best practices among projects.
  • Fast SDK updates: Migrating any improvement from a NEM2-SDK implementation to the rest is faster. Also, if any bug appears in one language, it is quicker to check and fix it.

Architecture

Characteristics

  • Standardized Contracts: Guaranteeing interoperability and harmonization of data models.
  • Loose Coupling: Reducing the degree of component coupling fosters.
  • Abstraction: Increasing the long-term consistency of interoperability and allowing underlying components to evolve independently.
  • Reusability: Enabling high-level interoperability between modules and potential component consumers.
  • Stateless: Increasing availability and scalability of components allowing them to interoperate more frequently and reliably.
  • Composability: For components to be effectively composable, they must be interoperable.

Package Organization

../_images/nem2-sdk-architecture.png

Package organization diagram

Infrastructure

The HTTP requests are made following the Repository Pattern, and they return NEM Domain immutable models via the Observable Pattern.

Models

The NEM Domain models are, usually, immutable by definition. The developer cannot change its attributes. Instead, the developers have to create new Transactions and dispatch them to NEM Blockchain via TransactionHTTP, to change the NEM Blockchain state.

Services

Common operations that require multiple REST API requests are handled by services provided.

Reactive

NEM2-SDK uses ReactiveX Library intensely.

  • Functional: Developers can avoid complex stateful programs using clean input/output functions over observable streams.
  • Less is more: ReactiveX’s operators often reduce what was once an elaborate challenge into a few lines of code.
  • Async error handling: Traditional try/catch is weak for error handling in asynchronous computations, but ReactiveX will offer developers the proper tools to handle errors.
  • Concurrency: Observables and Schedulers in ReactiveX allow the programmer to abstract away low-level threading, synchronization, and concurrency issues.
  • Frontend: Manipulation of UI events and API responses on the Web using RxJS.
  • Backend: Embrace ReactiveX’s asynchronicity, enabling concurrency and implementation independence.

Note

In case you are not familiar with ReactiveX and you still have to deliver something fast, you can convert an observable to Promise/Future by reviewing this example. However, we encourage you to learn ReactiveX.

Before starting

  1. Review the technical documentation to become familiar with the Catapult built-in features.
  2. Setup the catapult in local environment via docker.
  3. Check the API reference and play with the API endpoints.
  4. Become familiar with the current nem2-sdk via code examples & nem2-cli .
  5. Join our Slack to ask Catapult related questions.
  6. Be sure no one is already working on the SDK you want to create. Check the repository list and comment your intentions in nem2 slack #sig-api channel. If someone is already working on it, we suggest you collaborate with him/her.
  7. Claim the SDK forking this repository and add a new entry to the repository list.

Development

You can base your work in TypeScript. The TypeScript version is the first SDK getting the latest updates.

Regularly check the Changelog to be sure you didn’t miss any code change update.

Creating the project

  1. Add a README with the instructions to install the SDK.
  2. Add a Code of Conduct.
  3. Add a Contributors guidelines to help others know how they can help you.
  4. Setup the Continuous Integration system. We use travis-ci, but feel free to use the one that suits you best.

A project with good test coverage it’s more likely to be used and trusted by the developers!

We strongly suggest to do Test-Driven Development or Unit-Testing (test last). If you need inspiration, you can adapt the same tests we did.

API Wrapper

The OpenAPI Generator handles the API generation. It supports multiple languages, and hopefully, yours is on the list.

These are the steps we are following to generate the Typescript DTOs (data transfer objects):

  1. Download the latest Open API definition.
git clone https://github.com/nemtech/nem2-openapi.git
cd nem2-openapi && mkdir sdks && cd sdks
cp ../spec/openapi3.yaml .
  1. Copy the templates folder from {nem2-sdk-typescript-javascript}/infrastructure/ into a new folder.
  2. Download the OpenAPI generator and generate the DTOs.
brew install openapi-generator
openapi-generator generate -i ./openapi3.yml -g typescript-node -t templates/ -o ./nem2-ts-sdk/ && rm -R nem2-ts-sdk/test
  1. As the TypeScript generator does not recognize enum type alias, we need to manually move enum classes into the enumsMap list. You can jump this step if the code generator for your language supports them.
  • Open the generated file ./nem2-ts-sdk/model/models.ts in your favorite editor.
  • Search for line contains let enumsMap: {[index: string]: any}.
  • Move all xxxTypeEnum entries from below typeMap into enumsMap.

Example:

let enumsMap: {[index: string]: any} = {
    "AccountPropertyTypeEnum": AccountPropertyTypeEnum,
    "AliasTypeEnum": AliasTypeEnum,
    "MosaicPropertyIdEnum": MosaicPropertyIdEnum,
    "MultisigModificationTypeEnum": MultisigModificationTypeEnum,
    "NamespaceTypeEnum": NamespaceTypeEnum,
    "ReceiptTypeEnum": ReceiptTypeEnum,
    "RolesTypeEnum": RolesTypeEnum,
}

let typeMap: {[index: string]: any} = {
    "AccountDTO": AccountDTO,
    "AccountIds": AccountIds,
    "AccountInfoDTO": AccountInfoDTO,
    "AccountMetaDTO": AccountMetaDTO,
    "AccountNamesDTO": AccountNamesDTO,
    "AccountPropertiesDTO": AccountPropertiesDTO,
    "AccountPropertiesInfoDTO": AccountPropertiesInfoDTO,
    "AccountPropertyDTO": AccountPropertyDTO,
    "AliasDTO": AliasDTO,
    "AnnounceTransactionInfoDTO": AnnounceTransactionInfoDTO,
    "BlockDTO": BlockDTO,
    "BlockInfoDTO": BlockInfoDTO,
    "BlockMetaDTO": BlockMetaDTO,
    "BlockchainScoreDTO": BlockchainScoreDTO,
    "CommunicationTimestamps": CommunicationTimestamps,
    "Cosignature": Cosignature,
    "HeightInfoDTO": HeightInfoDTO,
    "MerklePathItem": MerklePathItem,
    "MerkleProofInfo": MerkleProofInfo,
    "MerkleProofInfoDTO": MerkleProofInfoDTO,
    "MosaicDTO": MosaicDTO,
    "MosaicDefinitionDTO": MosaicDefinitionDTO,
    "MosaicIds": MosaicIds,
    "MosaicInfoDTO": MosaicInfoDTO,
    "MosaicMetaDTO": MosaicMetaDTO,
    "MosaicNamesDTO": MosaicNamesDTO,
    "MosaicPropertyDTO": MosaicPropertyDTO,
    "MultisigAccountGraphInfoDTO": MultisigAccountGraphInfoDTO,
    "MultisigAccountInfoDTO": MultisigAccountInfoDTO,
    "MultisigDTO": MultisigDTO,
    "NamespaceDTO": NamespaceDTO,
    "NamespaceIds": NamespaceIds,
    "NamespaceInfoDTO": NamespaceInfoDTO,
    "NamespaceMetaDTO": NamespaceMetaDTO,
    "NamespaceNameDTO": NamespaceNameDTO,
    "NetworkTypeDTO": NetworkTypeDTO,
    "NodeInfoDTO": NodeInfoDTO,
    "NodeTimeDTO": NodeTimeDTO,
    "ResolutionEntryDTO": ResolutionEntryDTO,
    "ResolutionStatementDTO": ResolutionStatementDTO,
    "ServerDTO": ServerDTO,
    "ServerInfoDTO": ServerInfoDTO,
    "SourceDTO": SourceDTO,
    "StatementsDTO": StatementsDTO,
    "StorageInfoDTO": StorageInfoDTO,
    "TransactionHashes": TransactionHashes,
    "TransactionIds": TransactionIds,
    "TransactionInfoDTO": TransactionInfoDTO,
    "TransactionMetaDTO": TransactionMetaDTO,
    "TransactionPayload": TransactionPayload,
    "TransactionStatementDTO": TransactionStatementDTO,
    "TransactionStatusDTO": TransactionStatusDTO,
}
  1. Copy the generated files into the nem2-sdk infrastructure folder.
  2. Drop the generated client classes and implement them using the Repository pattern returning Observables of ReactiveX.

Example of a Repository and HTTP implementation:

  1. The repositories return models instead of DTOs. You will need to code the models before finishing the API wrapper.

Models

The models are by default immutable and aim to hide the complexity, like type conversion or relationship between objects.

You will find in the different implementations different invariants to ensure the object is well constructed and a nicer API is published.

Particular decisions to consider:

  • uint64 support: While Java supports big numbers, for example, JavaScript doesn’t. The JavaScript SDK has a custom class to handle the uint64 types. If your language supports uint64, use that implementation instead.
  • API conversions: Sometimes, the data returned by API is compressed. You might need to convert those types for the user.
  • Namespace ID: At creation time you add the string name, but when you receive the Namespace from the network, it comes in formatted as uint64 ID. A specific endpoint returns the Namespace string name.

Transaction Serialization

The catbuffer library defines the protocol to serialize and deserialize Catapult entities.

In combination with the catbuffer-generators project, developers can generate builder classes for a given set of programming languages. For example, the NEM2-SDK uses the generated code to operate with the entities in binary form before announcing them to the network.

KeyPair and Cryptographic functions

Note

This section is incomplete.

Implementing cryptographic functions is required to sign transactions.

Example: core/crypto

Documenting your SDK

The SDKs need to be adopted by other developers. As a contributor, no one knows better than you how a determined SDK works. Consider helping others and spread the usage of the SDK by providing the following documentation.

Publishing the SDK as official

To make an SDK officially supported, submit it as a NIP. The reason behind the NEM2 Improvement Proposal is to ensure that the new libraries are reviewed, tested, and shared among Catapult developers.