What Caused the Bitcoin Unlimited Node Crash?

Bitcoin Unlimited nodes went down like a rock yesterday as a vulnerability was exploited. They now appear to have recovered, but the speed with which nodes fell is unprecedented.

The cause appears to be the use of “asserts” in production. This will get a bit technical, but we asked a pseudo-anonymous Bitcoin Unlimited developer who would rather not be named to provide an explanation for a non-technical audience. He told CCN:

“Assertions are used to capture programming errors, i.e. to check whether a condition that the programmer *believes* should hold true at a particular point in the program, actually holds true. If an assertion is triggered (by the expected condition not holding true) then they usually result in a short message and aborting the program.

They are not intended for handling user or run-time errors (e.g. invalid input data), and are therefore generally disabled after a program exits its debugging phase. An assert(0) in C/C++ program code, if hit, will abort the program with certainty.

In bitcoind code (BU as well as Core), assertions are ENABLED in the production builds for reasons that are perhaps a bit too complex to go into here.

What happened: BU developers left an assert(0) in a code path that could be reached due to input data (Xthin protocol request) not being adequately checked. This left a door open to someone who wanted to trigger that assert(0) to craft a special message which would make the BU software enter that code path.”

In more simplified terms, it is a method coders use in testing, but generally this method is automatically disabled once the client is launched for users to download. In bitcoin, however, that’s not the case. This method remains enabled. Thus, if asserts are not removed, that allows anyone with some skill to remote crash the nodes.

The BU developer further states that “it is not good practice to leave asserts active in production code because they are NOT designed for handling run-time errors, should use exceptions for that.

Leaving them ENABLED in production code is dangerous. There is an NDEBUG definition (stands for “no debug”). Normally this will disable asserts for the production version, but for “reasons”, the bitcoind codebases don’t do this.”

Bitcoin Unlimited developers are currently quite busy, so we didn’t press for an explanation of the “reasons,” but he says that the practice of leaving asserts enabled has to be carefully re-evaluated. “It is clearly not good programming practice. It makes such incidents possible.” – he continued.

We reached out to Matt Corallo, a Bitcoin Core developer, and others, to ask if an explanation could be provided on why the practice of leaving asserts enabled is used, but have not received a response in time for publishing.

The BU developer tells CCN :

“[There] needs to be a detailed incident report… touching e.g. code review, release of security-critical fixes, and working with security researchers and other Bitcoin client projects on the topic of responsible disclosure of such vulnerabilities.

As a developer, I am certain that there are more bugs in Bitcoin code, not just BU code. So, this issue is bigger than BU.”

It is not yet clear who exactly discovered the bug, but a fix was merged in BU yesterday around midday London time. Shortly after, Peter Todd, a Bitcoin Core developer, tweeted a link to the bug fix prior to new clients being released. The attack followed soon after. Peter Tschipper, a Bitcoin Unlimited developer, stated yesterday:

“The bug fix was merged…and literally within 30 minutes the attack happened. They just have been monitoring our Git…I was actually talking with [Andrea Suisani] at the time about maybe we should be more careful about using Git for these kind of things… and then minutes later the attack happened.”

Just hours before the attack, CCN published an article on Bitcoin Core supporters threatening zero-day exploits.

Around 200 Bitcoin Unlimited nodes withstood the attack. Haipo Yang, founder of viaBTC, a pool that mines Bitcoin Unlimited, told CCN that they were affected by yesterday’s events, but the pool’s node/s uses auto restart, “so the effect was not big.”

BU’s hashrate appears unchanged, now standing at 33% of network’s share.

Image from Shutterstock.

Share
Andrew Quentson @Aquentson

Feel free to contact me at andrewquentson@gmail.com with any information you may have.

Leave a Comment

Dear reader. Please use this comment field to express your own thoughts and opinions. We do not allow harassment, profanity words or off-topic comments. All comments are manually reviewed before published.