Code quality, test cases, and speed of development

Research & Dev
#1

[Editor’s notes] Recently there was a community member raised concerns that there might be lack of test cases and automated testing in NKN code repository. Below is the full response from our CTO Yilun Zhang.

Yes we all know having automated testing is good, it’s just a matter of cost efficiency. We are a very small team building the world’ largest and most decentralized blockchain from scratch in just a year. We’d love to move fast, really fast, not like some other projects spending years building a testnet, or a network with just high profile, stable supernodes. That’s why for every move we do, every dev effort we spent, we have to consider cost efficiency. In Google, people spent around half of their time writing test cases, and we cannot tolerate that at the current stage. We will be there one day, but not today.

Actually we have our own internal process and large cluster that test every release throughly before release, and it’s more suitable to our current stage. Unlike other blockchain projects, we are an extremely network intense system, and most of our function do not work on single node or with a small networks. If you know about such kind of permisionless decentralized system, you will probably know that 99% (if not more) problems occurs only in a real network environment. I can ensure you that almost every problem occured since the first day of our testnet is not a problem in our internal large cluster. Yet, to accurately similate the real network environment, we need really complicated testing environment like network simulator, emulab, etc, and it’s definitely beyond our current stage.

Test cases is useful, but it’s never the only factor you should look at when judging code quality. Actually we are pretty confident at our code quality so I’d welcome you to have some real discussion, but not like oh I don’t want to look at xxx because they don’t have test cases! Trust me, you can’t find any other projects at the moment that has such an amount of consensus nodes and works really well on even on raspberry pi or router (there are a lot of openwrt routers in the network).

So back to your question, we will have test cases and detailed documentations, but just not today :slightly_smiling_face:

https://media.giphy.com/media/OyuoAUQNEXcu4/giphy.gif(image larger than 4096KB)

#2

@yilun:

if automated testing is not an option, consider making an manual testing guide which describes test steps e.g.:

  1. nknc — ip info -s
  2. Should: Display your external ip

when a new release is final atleast or even when developer makes changes in the code, they can simply run through all the steps to verify the product works still. every code change can break other parts by surprise.

if you do not have such a guide, it is likely people in a hurry forget about testing some use cases or think it will be alright still and produce issues.

a manal testing guide is easy to update and gives all team member a way to see what the product should be doing and can be used as an introduction for new developers to onboard them quickly!

this advice comes from my experience of developing for multiple years.