In the world of blockchain, upgrades are common occurrences, often executed with meticulous planning and minimal disruption. However, the recent upgrade event on the Lunc blockchain, which transitioned from SDK 45 to SDK 46, proved to be a challenging ordeal for validators and the community alike.
The upgrade process began as scheduled, with block 14,514,000 serving as the designated starting point. Validators geared up for what they thought would be a seamless transition. Little did they know, it would turn into a marathon.
So it's time for a breakdown of upgrade-events on #LUNC from my side.
Preliminary note: The chain is running (except some endpoints) currently and producing blocks with 84 validators being active and 26 still not back online. All big VP nodes are back online (kucoin came online… pic.twitter.com/xht6QWDduv
— StrathCole (@ColeStrathclyde) September 13, 2023
The first shock came when it became apparent that even servers equipped with 32GB or 64GB of RAM were insufficient to complete the migration of nodes. This unexpected hurdle caused many nodes to crash and forced validators to restart the process repeatedly. The upgrade itself was relatively quick, taking approximately 5 minutes, but the subsequent migration consumed a staggering 2 hours, with some validators experiencing never-ending stalls.
Around 70% of validators managed to come online, and the chain briefly resumed block production, only to face another setback. A group of validators, including some high-profile nodes, lost peer connections, leading to a frantic scramble to restore operations. Fortunately, Allnodes stepped in, offering a vital snapshot at the problematic block height, allowing affected validators to resume operations and those struggling with resource constraints to sidestep the arduous migration process.
The entire ordeal unfolded over a gruelling 12-hour period, with dedicated validators working tirelessly in collaboration. This episode underscored the strength of the LUNC community’s cooperation and resilience.
At approximately 1:55 a.m., after countless ups and downs, the Lunc chain finally achieved a stable consensus and resumed block production. Yet, some validators continue to face challenges, particularly concerning the oracle feeder.
In retrospect, questions arose about whether the upgrade could have been divided into smaller, more manageable chunks. However, given the nature of the transition from SDK 45 to SDK 46, it was an all-or-nothing situation. Nevertheless, some of the chaos could potentially have been mitigated had disconnected nodes been updated on the mainnet beforehand, revealing the resource-intensive nature of the process and the obstacles that lay ahead.
As the LUNC community rallies together to support validators still grappling with post-upgrade issues, it’s a testament to the resilience and camaraderie within the Lunc ecosystem. The journey may have been tumultuous, but it ultimately showcases the community’s determination to weather any storm and emerge stronger on the other side.