SDK Development

A key objective is that interoperability becomes a natural design of the NEM2-SDK. Follow this guideline to collaborate creating a NEM SDK, achieving the best quality with less effort.

NEM2-SDK shares the same design/architecture between programming languages to accomplish the next properties:

  • Fast language adaptation: There is a library for Java, but you need it for C# for example. As both SDKs share the same design, you can re-write the library faster, adapting the syntax to your language. It also applies to examples, projects, applications…
  • Cohesion/shared knowledge across NEM developers: Be able to change between projects that use NEM, sharing the same design accompanied by the best practices.
  • Fast SDK updates: Migrating any improvement from a NEM2-SDK implementation to the rest is faster.
  • Fewer bugs: If any bug appears in one language, it is faster to check and fix it.

Architecture

Characteristics

  • Standardized Contracts: Guaranteeing interoperability and harmonization of data models.
  • Loose Coupling: Reducing the degree of component coupling fosters.
  • Abstraction: Increasing long-term consistency of interoperability and allowing underlying components to evolve independently.
  • Reusability: Enabling high-level interoperability between modules and potential component consumers.
  • Stateless: Increasing availability and scalability of components allowing them to interoperate more frequently and reliably.
  • Composability: For components to be effectively composable they must be interoperable.

Package Organization

../_images/nem2-sdk-architecture.png

Package organization diagram

Infrastructure

The HTTP requests are made following the Repository Pattern, and they return NEM Domain immutable models via the Observable Pattern.

Models

The NEM Domain models are, usually, immutable by definition. The developer cannot change its attributes. Instead, the developers have to create new Transactions and dispatch them to NEM Blockchain via TransactionHTTP, to change the NEM Blockchain state.

Services

Common operations that require multiple REST API requests are handled by services provided.

Reactive

NEM2-SDK uses ReactiveX Library intensely.

  • Functional: Developers can avoid complex stateful programs using clean input/output functions over observable streams.
  • Less is more: ReactiveX’s operators often reduce what was once an elaborate challenge into a few lines of code.
  • Async error handling: Traditional try/catch is powerless for errors handling in asynchronous computations, but ReactiveX will offer developers the proper tools to handle these sort of errors.
  • Concurrency: Observables and Schedulers in ReactiveX allow the programmer to abstract away low-level threading, synchronization, and concurrency issues.
  • Frontend: Manipulation of UI events and API responses on the Web using RxJS.
  • Backend: Embrace ReactiveX’s asynchronicity, enabling concurrency and implementation independence.

Note

In case you are not familiar with ReactiveX and you still have to deliver something fast, you can convert an observable to Promise/Future by reviewing this example. However, we encourage you to learn ReactiveX.

Before starting

  1. Review the technical documentation to become familiar with the NEM built-in features.
  2. Setup the catapult in local environment via docker.
  3. Check the API reference and play with the API endpoints.
  4. Become familiar with the current nem2-sdk via code examples & nem2-cli .
  5. Join our Slack to ask Catapult related questions.
  6. Be sure no one is already working on the SDK you want to create. Check the repository list and comment your intentions in nem2 slack #sig-api channel. If someone is already working on it, we suggest you collaborate with him/her.
  7. Claim the SDK forking this repository and add a new entry to the repository list.

Development

You can base your work in TypeScript. The TypeScript version is the first SDK getting the latest updates. Meanwhile, Java takes longer to be updated.

Regularly check the Changelog to be sure you didn’t miss any code change update.

Creating the project

  1. Add a README with the instructions to install the SDK. You can find a template here.
  2. Add a Code of Conduct. Download a sample code of conduct here.
  3. Add a Contributors guidelines to help others know how they can help you. Find here a CONTRIBUTING.md template.
  4. Setup the Continuous Integration system. We use travis-ci, but feel free to use the one suits you best.

A project with a good test coverage it’s more likely to be used and trusted by the developers!

We strongly suggest to do Test-Driven Development or Unit-Testing (test last). If you need inspiration, you can adapt the same tests we did.

API Wrapper

The OpenAPI Generator handles the API generation. It supports multiple languages, and hopefully, yours is on the list.

These are the steps we are following to generate the Typescript DTOs (data transfer objects):

  1. Download the latest Open API definition.
git clone git@github.com:nemtech/nem2-docs
cd nem2-docs && mkdir sdks && cd sdks
cp ../source/resources/collections/swagger.yaml .
  1. Copy the templates folder from {nem2-sdk-typescript-javascript}/infrastructure/ into a new folder.
  2. Download the OpenAPI generator and generate the DTOs.
brew install openapi-generator
openapi-generator generate -i ./swagger2.yaml -g typescript-node -t templates/ -o ./nem2-ts-sdk/ && rm -R nem2-ts-sdk/test
  1. As the typescript generator does not recognize enum type alias, we need to manually move enum classes into the enumsMap list. You can jump this step if the code generator for your language supports them.
  • Open the generated file ./nem2-ts-sdk/model/models.ts in your favorite editor.
  • Search for line contains let enumsMap: {[index: string]: any}.
  • Move all xxxTypeEnum entries from below typeMap into enumsMap.

Example:

let enumsMap: {[index: string]: any} = {
    "AccountPropertyTypeEnum": AccountPropertyTypeEnum,
    "AliasTypeEnum": AliasTypeEnum,
    "MosaicPropertyIdEnum": MosaicPropertyIdEnum,
    "MultisigModificationTypeEnum": MultisigModificationTypeEnum,
    "NamespaceTypeEnum": NamespaceTypeEnum,
    "ReceiptTypeEnum": ReceiptTypeEnum,
    "RolesTypeEnum": RolesTypeEnum,
}

let typeMap: {[index: string]: any} = {
    "AccountDTO": AccountDTO,
    "AccountIds": AccountIds,
    "AccountInfoDTO": AccountInfoDTO,
    "AccountMetaDTO": AccountMetaDTO,
    "AccountNamesDTO": AccountNamesDTO,
    "AccountPropertiesDTO": AccountPropertiesDTO,
    "AccountPropertiesInfoDTO": AccountPropertiesInfoDTO,
    "AccountPropertyDTO": AccountPropertyDTO,
    "AliasDTO": AliasDTO,
    "AnnounceTransactionInfoDTO": AnnounceTransactionInfoDTO,
    "BlockDTO": BlockDTO,
    "BlockInfoDTO": BlockInfoDTO,
    "BlockMetaDTO": BlockMetaDTO,
    "BlockchainScoreDTO": BlockchainScoreDTO,
    "CommunicationTimestamps": CommunicationTimestamps,
    "Cosignature": Cosignature,
    "HeightInfoDTO": HeightInfoDTO,
    "MerklePathItem": MerklePathItem,
    "MerkleProofInfo": MerkleProofInfo,
    "MerkleProofInfoDTO": MerkleProofInfoDTO,
    "MosaicDTO": MosaicDTO,
    "MosaicDefinitionDTO": MosaicDefinitionDTO,
    "MosaicIds": MosaicIds,
    "MosaicInfoDTO": MosaicInfoDTO,
    "MosaicMetaDTO": MosaicMetaDTO,
    "MosaicNamesDTO": MosaicNamesDTO,
    "MosaicPropertyDTO": MosaicPropertyDTO,
    "MultisigAccountGraphInfoDTO": MultisigAccountGraphInfoDTO,
    "MultisigAccountInfoDTO": MultisigAccountInfoDTO,
    "MultisigDTO": MultisigDTO,
    "NamespaceDTO": NamespaceDTO,
    "NamespaceIds": NamespaceIds,
    "NamespaceInfoDTO": NamespaceInfoDTO,
    "NamespaceMetaDTO": NamespaceMetaDTO,
    "NamespaceNameDTO": NamespaceNameDTO,
    "NetworkTypeDTO": NetworkTypeDTO,
    "NodeInfoDTO": NodeInfoDTO,
    "NodeTimeDTO": NodeTimeDTO,
    "ResolutionEntryDTO": ResolutionEntryDTO,
    "ResolutionStatementDTO": ResolutionStatementDTO,
    "ServerDTO": ServerDTO,
    "ServerInfoDTO": ServerInfoDTO,
    "SourceDTO": SourceDTO,
    "StatementsDTO": StatementsDTO,
    "StorageInfoDTO": StorageInfoDTO,
    "TransactionHashes": TransactionHashes,
    "TransactionIds": TransactionIds,
    "TransactionInfoDTO": TransactionInfoDTO,
    "TransactionMetaDTO": TransactionMetaDTO,
    "TransactionPayload": TransactionPayload,
    "TransactionStatementDTO": TransactionStatementDTO,
    "TransactionStatusDTO": TransactionStatusDTO,
}
  1. Copy the generated files into the nem2-sdk infrastructure folder.

6. Drop the generated client classes and implement them using the Repository pattern returning Observables of ReactiveX.

Example of a Repository and HTTP implementation:

  1. The repositories return models instead of DTOs. You will need to code the models before finishing the API wrapper.

Models

The models are by default immutable and aim to hide the complexity, like type conversion or relationship between objects.

You will find in the different implementations different invariants to ensure the object is well constructed and a nicer API is published.

Particular decisions to consider:

  • uint64 support: meanwhile Java supports big numbers, for example JavaScript doesn’t. The JavaScript SDK has a custom class to handle the uint64 types. If your language supports uint64 use that implementation instead.
  • API conversions: Sometimes, the data returned by API is compressed. You might need to convert those types for the user.
  • Namespace ID: At creation time you add the string name, but when you receive the Namespace from the network, it comes in formatted as uint64 ID. A specific endpoint returns the Namespace string name.

Transaction Serialization

A Transaction needs a particular serialization schema in binary optimized in size.

Generate the buffer classes

Note

This section is incomplete. It will be updated with complete information once the first SDK integrates the builders generated with catbuffer library.

Create the schema classes

  1. Schema class.
  2. SchemaAttribute class.
  3. ScalarAttribute class.
  4. ArrayAttribute class.
  5. TableAttribute class.
  6. TableArrayAttribute class.
  7. Constants class.

Create the transaction schemas

Each transaction has a schemas. It has the same type as catbuffer schemas but using the Schema class. It’s used to know where each component is located in the catbuffer schema and remove the unnecessary bytes to create the optimized serialization.

Example: TransferTransactionSchema.

Using the schemas in the transaction models

The Transaction class has the abstract method generateBytes().

Each Transaction has to implement and use the previous classes, the Buffers and the Schemas, to serialize the transaction.

Example: TransferTransaction.generateBytes().

Note

Do not forget to implement the Cosignatory classes.

KeyPair and Cryptographic functions

Note

This section is incomplete.

Implementing the cryptographic functions is required to sign transactions.

Example: core/crypto

Documenting your SDK

The SDKs need to be adopted by other developers. As a contributor, no one knows better than you how a determined SDK works. Consider helping others and spread the usage of the SDK by providing the following documentation.

Publishing the SDK as official

To make an SDK officially supported, submit it as a NIP. The reason behind the NEM2 Improvement Proposal is to ensure that the new libraries are reviewed, tested and shared among NEM developers.

Future work

The current guideline shows what is done up to today since the SDK isn’t complete. It will get updates according to the latest architecture/features.