[NKP-0009] NKN Transport layer

The NKN message protocol isn’t reliable wiki - networking reliability which is fine in some cases (offers better performance, lower latency), but some use cases might need a reliable protocol implemented by client SDKs.

Current message protobuf encoding (simplified) is as follows:
Message { Recipient; EncryptedPayload { UseEncryption YES/NO; enc( Payload(Type; Data ) } } where Type can denote difference between BIN/TEXT/ACK packet.

I suggest a change that Payload would contain not just Data itself but split to Header(s) and Body where Header(s) can contain service information about the protocol state and if TCP like transport layer is used or not. (As well as other useful information)

User should be able to choose if they want the SDK to handle (initiate) connection that handles resending dropped packets, managing packet order, or both. Possibly with user-selectable hyperparameters, if they want to optimize for higher transfer speed or latency/session initiation. On incoming connection, the SDK would just continue using whatever type was used.

We will have to find a way to name things correctly to avoid confusion with the session mode (when released), that will create a direct tunnel and keep this session with one node only, not with one client using messages.

Possible Header types:

  • MessageType (as currently implemented)
  • ControlFlow { Control = NOTHING | DROPS | ORDER_AND_DROPS; some control bits required for this transfer (TCP protocol for inspiration) }
    When no ControlFlow header is prvided, one with Control = NOTHING is assumed. Control = NOTHING will not require any additional parameters

What are your thought/requirements for this protocol?

1 Like

I suggest going one level lower and create this new protocol directly on top of OutboundMessage/payload to avoid additional overhead. Also, that way we’ll be able to stack current nkn-client protocol either on top of raw OutboundMessage or this new protocol to seamlessly and effortlessly gain reliability (switch in nkn-client SDK)

hello,

thanks for your proposal, it looks good!

for me as one of the people trying to develop with NKN i think the most important thing would be that NKN tries to work using ACKnowledgement + resend - so send a package -

wait for confirmation or receiver, resend if in timeout x it did not get acknowledged. and provide reordering if packets arrive out of order.

for me its not so important to have lots of configurable options or parameters.

the new proposal i made today covers similar requirements: [NKN-0010] NVPN - Virtual private network functionality in the NKN core

hint: maybe instead of reiventing the wheel you guys can copy this retransmission, reorder etc. logic from existing networking stack code or maybe OPENVPN, they spent lots of time designing and testing it well.

What about using this control flow to replace the no_ack field in Payload message? It includes what no_ack is intended for but more general.

hi,

i am afraid i dont have an suggestion about this, i am not familiar with no_ack so someone else should answer.

A relevant (but not the same) stuff we just built: nkn-multiclient-js

The multiclient uses multiple concurrent (and importantly, independent) nkn client to greatly enhance reliability and reduce latency.

hi,

@yilun thanks but i am using the java sdk so i need an option to have retransmission & packet reordering either in the java sdk or better in the NKN core directly.

so e.g. i can be sure the packets coming in in the websocket are complete and in right order, when working with binary data specially lost or wrong ordered packets are useless.