The Case For Proactive Solvency and Security Audits

by | Dec 1, 2020 | proof of reserves, security, solvency

Anyone who has been involved in the cryptocurrency industry for more than a few months is no doubt well aware that data breaches, thefts, hacks, fake hacks are commonplace. This applies to custodial exchanges and services but also on an individual level with a continually increasing amount of cryptocurrency thefts and exchange account hacks. 

Furthermore, custodial services and exchanges have suddenly and sometimes indefinitely been unable to pay out customer funds. When this occurs, it’s often because a company has borrowed against user assets (and were unable to pay them back), which is a nicer way of saying that at best, they’re operating on a fractional reserve, but it also it may suggest they’re insolvent. A considerable number of custodial services operating right now are insolvent with debts they will be unable to recoup. Chances are you’re a user of some of them.

Individuals, companies, and services all have a tendency to overestimate how secure they are. All too often, this leads to the loss of personal cryptocurrency assets, or worse, other people’s assets and/or personal information. People also have a tendency to be lazy or apathetic about their security and only seriously care once they’ve been breached in an attack. Of course, it would have been a lot cheaper and easier if they had the foresight to deal with such vulnerabilities more proactively. The problem is one can’t pinpoint the things they don’t know about or things they’ve overlooked.

Anyone who works in any cybersecurity knows that it’s like pulling teeth to get others to spend time and resources on security proactively; they too often see it as a waste of money, despite cybersecurity professionals’ insistence. People only care once they’ve been breached or there’s a vulnerability. But by then, it’s often too late, and the damage has been done, with costs astronomically higher than what it would have been if a more proactive approach to security was taken.

Historical Context and Importance

Cryptocurrency platforms worldwide, both exchanges and non-exchanges, have been plagued by attacks resulting in cryptocurrency loss. Many of these have been poorly run and have had lackluster security, to put it mildly. While hacks, security breaches, and thefts are fairly common occurrences in the industry overall, including Kucoin, Cryptopia, Bitgrail, Upbit, DragonEX, Bitrue, Bitstamp, Bancor, IDAX, Coinrail, Coinsecure, Cryptsy, Poloniex, Coincheck, FCoin, Bithumb, Zaif, Eterbase and of course Mt Gox to name but a few of many. And this doesn’t just apply to cryptocurrency exchanges; it applies to any custodial service, such as Cred for instance, which was recently breached.

If we were to look at Canada specifically, four Canadian cryptocurrency exchanges have shut down and have not returned cryptocurrency assets belonging to their customers to a large extent in the past two years, specifically QuadrigaCX, Einstein Exchange, Ezbtc, and Maplechange. QuadrigaCX, Einstein, Ezbtc, and Maplechange all shut down for different reasons, but the bottom line is they did not have assets to back up the balances they claimed to have to their customers. Furthermore, Coinsquare was also hacked, resulting in a major client data breach that they took roughly a year to disclose publicly. They only ended up doing so once a VICE reporter uncovered it. Overall, the “industry standard” bar has been set extremely low; the vast majority of centralized platforms and exchanges have not taken proactive action to show that their business in no way resembles these shady exchanges that have come and gone, taking customer funds along with them.

The case of QuadrigaCX is the most well known of these four exchanges. QuadrigaCX was not a licensed Money Services Business (MSB) and allegedly grossly mismanaged customer funds in a number of ways. This includes utilizing poor security practices and allegedly not utilizing multi-signature wallets for cold storage despite publicly declaring that they did. It also includes the alleged misappropriation of funds by the founder by allegedly day-trading customer’s cryptocurrency assets. Events culminated in the disappearance of the founder, Gerald Cotten, who allegedly died suddenly in India due to complications from Crohn’s disease. QuadrigaCX evidently failed to have an appropriate contingency plan in place as QuadrigaCX now claims Gerald was supposedly the only one able to access cryptocurrency held cold storage wallets despite there not being any actual evidence suggesting much of the cryptocurrency in cold storage actually existed. Furthermore, QuadrigaCX employees had a history of utilizing fictitious names and had questionable pasts, to say the least. In the case of Einstein Exchange, they became insolvent for reasons not publicly known, but allegedly improperly managed customer assets and did not properly segregate funds. In the case of Ezbtc, one might question whether they were ever really legitimate or met the definition of a cryptocurrency exchange to begin with. Nonetheless, the founder has widely been accused of misappropriation of funds, having allegedly gambled with customer’s cryptocurrency assets on various gambling websites. In the case of Maplechange, they allegedly lost all cryptocurrency in a hack, which, if true, is suggestive of lackluster security and cryptocurrency storage practices. And that’s just Canada! 

QuadrigaCX disclaimer

Straight from the former QuadrigaCX website

A few months ago, European cryptocurrency exchange Eterbase was hacked and $5 million was stolen from their ‘hot wallets.’ In reality, they had no legitimate reason for having $5 million in hot wallets; either they got lazy, didn’t bother to properly use their cold wallets, and didn’t have an appropriate cold storage reserve ratio, or their ‘cold wallets’ weren’t actually cold, or their ‘cold wallets’ didn’t actually exist at all. Shortly thereafter, Kucoin was hacked allegedly to the tune of $280M USD all supposedly from their ‘hot wallets’.

While centralized exchanges have historically done a particularly poor job preventing breaches and closing down, unable to pay back customer’s assets that were supposed to be held securely in custody, throughout the cryptocurrency industry platform and exchange hacks occur relatively often, and platforms have been a frequent target for attempted thefts and frauds. These hacks have been perpetrated by a variety of individuals and entities. In some cases, they have been perpetrated by serial hackers. In other cases, nation-state actors, particularly North Korea’s Lazarus Group, have been found or are presumed to be responsible, which has undoubtedly proved to be a sizable revenue source for the North Korean regime. In other cases, an internal breach or theft by an employee or founder has been found to be the cause, which of course isn’t really a ‘hack’ at all. Nonetheless, custodial services need to be secure enough to prevent a breach from sophisticated external actors, but also internal actors, while at the same time having sufficient redundancy in place, such as in the event of the sudden death of the founder(s).

Solvency Audits and ‘Proof of Reserves’

Not your keys, not your Bitcoin is a phrase often repeated in cryptocurrency circles, and it’s worth repeating here for good reason. When you store or even briefly send your Bitcoin to a custodial service (even if you haven’t sold it and it appears in your ‘account’), the Bitcoin is no longer yours. Most cryptocurrency exchanges and services are custodial in nature; that is, the business controls the private keys to the Bitcoin, not the individual.

When a user sends funds to a custodial service, the service creates an IOU in their internal ledger and database, assigning that IOU to the account owner accordingly, and creating a liability on their books. You no longer own that Bitcoin. You own an IOU or credit for that Bitcoin. The service could take that IOU away for any reason (in which case you may have cause for a lawsuit), they could simply elect to not allow you to redeem that IOU (a liability of theirs), or they may be unable to pay the user perhaps due to them being (or becoming) insolvent.

Frankly, this isn’t so different from a bank. While a bank can become insolvent, one of the main differences is that they are regulated, audited regularly, and subject to capital control requirements.

Audits that assess a company’s assets and liabilities are commonplace in the traditional financial world and are largely required for any company that takes custody of people’s investments. Banks, credit unions, investment funds, brokerage firms, and insurance companies all undergo regular independent audits by third parties to assess their financial health. Why shouldn’t custodial cryptocurrency services be held to the same standard? If history is any indication, it’s no doubt needed given the number of custodial services that have ‘exit scammed’ or ended up becoming insolvent and unable to pay out user balances. And particularly given that custodial services generally don’t have the equivalent of FDIC insurance to protect users’ balances, solvency audits should be deemed essential.

Why Solvency Audits are (Currently) Uncommon

The reality is that custodial services generally won’t want to go through the trouble and costs of such a solvency audit just for the sake of it. There are really only two reasons why a service or exchange would voluntarily elect to go through such an audit:

  1. Regulatory requirements or pressure. Ultimately, most cryptocurrency businesses and users want to minimize the number of regulations in the cryptocurrency space, not increase them further. Nonetheless, there may come a day when regulators start requiring custodial services to undergo such audits if they don’t do it voluntarily, particularly if services continue to abscond with customer assets as the industry matures.
  2. Customers demand it. Without regulation, a custodial service will generally only go through an audit if users speak with their wallets and demand it. While ‘Proof of Reserves’ is increasingly being discussed more and more, the reality is the majority of customers don’t see it as important for exchanges to have before doing business with them. Such a stance is ridiculous; would an investor ever trust a brokerage platform with hundreds of thousands of dollars in fiat, stocks, options, and ETFs of theirs that wasn’t subject to a periodic audit? Of course not! The risk of the brokerage shutting down and/or not returning their assets would be far too high.

The Dirty Truth

If a customer were to query a custodial service about whether or not they’ve conducted a solvency audit, or if they would consider conducting one, they’ll generally provide a few reasons why they haven’t or why they believe it’s unnecessary, but there are also reasons they won’t tell you as well. The answers generally provided include:

  1. They already (allegedly) operate on a full-reserve basis (as opposed to running fractionally) and have already been ‘transparent’ with the public about this. The keyword, of course, is ‘allegedly.’ It has not been proven, and it isn’t verifiable either. QuadrigaCX pretended to be ‘transparent’ with their customers for years while they were insolvent to the tune of well over $100M USD. And they continued to be ‘transparent’ with their customers for a month after Gerald Cotten’s (alleged) death.
  2. Their team is composed of ‘extremely experienced professionals who would never allow the company to borrow against or steal customer assets.’ If you’re gullible enough to believe that spiel, then I have a bridge to sell you. Are all these employees of the custodial service going to have both intimate knowledge, details and control over the company’s finances? Of course not; such decisions are made strictly at the C-level.

Then there are the answers custodial services won’t tell you (keep in mind that generally only some of these are true; not all):

  1. They may not be able to justify the time and cost associated with it, particularly given that it’s not something that’s requested, much less demanded by their customer base. From a business perspective, this is arguably somewhat understandable. Why would they elect to bear any cost if it’s not going to result in more business for them (or at least not losing business due to a lack of confidence)? Ultimately, consumer demand for such an audit will change the cost/benefit equation here so that it’s more profitable for custodial services to undergo such an audit.
  2. They may be insolvent, so they don’t want an audit conducted for obvious reasons. Simply put, they borrow against customer assets to fund their operations. In many cases, they have the intention of paying the debt back. A surprising amount of custodial services fit into this categorization.
  3. They may be operating on a fractional reserve and/or engaging in misappropriation. Specifically, customers who elect (or are required to) store funds with the custodial service. However, the custodial service may not custody the assets themselves. Instead, they may lend out customer assets to receive interest in return. For example, they may lend customer assets out to margin traders, perhaps directly, or through DeFi platforms, or perhaps by using a service like Cred. This enables the custodial service to receive interest, which can considerably increase their profitability. However, it’s not done without risk. What if the debt goes bad? Who’s on the hook? The customers are. That’s precisely what happened with Cred right now which recently filed for Chapter 11 bankruptcy.
  4. In rare cases, there are a variety of concerns that might be considered even more sinister or even criminal in nature. This could involve anything from the founders embezzling cryptocurrency funds to the exchange having a Ponzi Scheme type structure to gambling or day-trading customer funds, to an exchange having suffered a hack or breach resulting in the loss of customer cryptocurrency assets which they aren’t disclosing.

Custodial services that operate on a fractional reserve model aren’t necessarily a deal-breaker; banks obviously operate on this model. But custodial services that operate fractionally ought to properly disclose that they don’t fully custody customer assets, so long as the risks are made aware to the customer, and so long their assets and liabilities are audited periodically; otherwise, they could hang on to bad debt indefinitely.

In the case of custodial exchanges in particular, they may be falsifying trading data or engaging in wash trading. Many exchanges falsify and inflate trading data through a variety of mechanisms. One only needs to go to Coinmarketcap and look at the top volume exchanges who no one has ever heard of to confirm this assumption. But falsifying trading data isn’t limited to a few exchanges; it’s widespread. Some just do it more than others.

There’s also wash trading that occurs. It arguably doesn’t constitute falsifying trading data since the trades do technically happen but would be better described as manipulating trading data, and it too is widespread.

There’s no reason users should have to ‘trust’ an exchange will actually still be around and able to pay out when requested. For any service that takes custody of people’s funds, the onus should be on them to have the ability to follow through. But more importantly, users should demand that such custodial services they utilize are audited on a semi-regular basis.

Security Audits

Almost everyone has a tendency to think their cryptocurrency funds are stored far more securely than they actually are. This applies to individuals and businesses alike. At CipherBlade, we get a lot of people who reach out to us after they’ve experienced a breach or theft. When this happens, people almost always express dismay about how it could have happened, thinking they had a highly secure setup and that they simply could not be hacked, or at least not in a way that would cause them to lose their cryptocurrency in such a theft. Below are some essential elements that a security audit needs to be able to tackle, and notably some of the aspects a security audit ought to encompass.

Client Data Breaches

It’s essential in any security audit to review how client data, such as email address, passwords, names, addresses, phone number, and KYC data is stored, how it’s accessed, and who has access. Users ought to care about just how secure their personal information is, particularly given the increasing prevalence of data breaches and the consequences for users if, or I should say, when their data is breached. From a user perspective, a breach of client data can lead to:

  • Hackers using the same credentials to access or attempt to access other exchange accounts or email addresses belonging to a victim, including via SIM-Swapping attacks as we’ve seen with Coinsquare. The effect of this cannot be overstated, even if you use different passwords for different exchanges or even if no passwords were stolen at all. The thief (or whoever he/she sells the data to) now knows you own cryptocurrency, making you a ripe target to be hacked or attacked in the future. This could easily result in an email account takeover if it’s not properly secured (e.g. via a hacker fraudulently ‘recovering’ your email), which can, in turn, lead to breaches of multiple other critical accounts.
  • Identity theft — you might notice a fraudster taking out a loan or credit card in your name, negatively affecting your credit score. In many cases, it won’t be the actual hacker that will have done this. Typically, the hackers sell this type of stolen data on darknet markets and private forums, often for Bitcoin.
  • Extortion. While a hacker could attempt to extort people individually, a better bet would be to extort the exchange themselves, requiring a ransom to not publicly release client data (ransomware attackers have frequently found this to often be just as effective as encrypting hard drives, and not providing the decryption key until the ransom is paid). The exchange has a heavy incentive to pay the ransom; otherwise, it publicly revealed that they were hacked (decimating trust from customers, and thus critical business revenue and trading volume), but it draws the ire of regulators as well. And hackers no doubt know the cryptocurrency exchanges have access to ample cryptocurrency holdings. So who ends up paying the ransom in the end? Well, they’re a custodial service, so the answer is YOU if you have funds on there.

Credential-related Security and Procedures

A sizable portion of exchange hacks can be attributed to employee credentials being breached by hackers, giving them access to perform key actions like draining funds from exchange ‘hot wallets.’ Aspects which need to be examined include:

  • What type of password policy are personnel required to utilize? Are they forced to use secure, unique passwords? Are passwords ever reused?
  • Is there a password manager used? If so, who holds the backup key to each password manager account? Do employees manage that themselves? If so, where do they store these backups? Is that location susceptible to a breach?
  • Is 2FA required for all logins? What type of 2FA? SMS 2FA, TOTP, or U2F? Hopefully, app-based or hardware-based 2FA is always required for anything the slightest bit important to minimize any damage caused by a SIM-Swap attack. But the question then becomes who stores the 2FA backup codes, how they store it and where it’s stored.
  • What recovery methods are available for accounts. Do employees have a backup recovery option set as a personal email, for example? Or a phone number? These are both breach vectors to consider. Any account recovery options need to be carefully considered because chances are if secure, unique passwords are used, there’s a good chance a hacker may have used the ‘account recovery ‘ feature.

Tiered Access Policies and Accessibility

Does the exchange or service employ an adequate tiered access policy? Is sensitive information restricted based on a need to know basis, or people given ‘all the keys to the castle’ when requested? What exactly are the employee onboarding and offboarding procedures, and are there any inherent risks that could be cause for concern?

Cold Storage Policies and Cold Storage Reserve Ratio

All custodial services should actually have cold wallets and make an effort to store as much customer funds in cold storage as would be practical. Funds in ‘cold storage’ are by no means immune to being breached as well, however.

First, it needs to be assessed whether a company actually holds funds in what could be considered ‘cold storage’ or not. How precisely are funds in cold storage accessed?  Can the founder do it by himself or herself? The answer should be a resounding no. There should be some sort of multi-party system in order to access funds in cold storage, such as a multi-signature setup or, alternatively, Multi-party Computation (MPC).

Who are the keyholders? For example, suppose the exchange employs a 2-3 multi-sig, where one signatory is the CEO, one signatory is the CFO, and 3rd key is held as a backup in a safety deposit box. One crucial question is how the safety deposit can be accessed, since if it can be accessed solely by the CEO or CFO, either could easily abscond with funds themselves.

Another vital thing to look at is what’s referred to as the ‘Cold Storage Reserve Ratio.’ We define this as the percentage of custodial cryptocurrency assets held in cold storage relative to the custodial cryptocurrency assets held overall (inclusive of ‘hot’ wallets, of course). This is important because funds in hot wallets will always necessarily be more susceptible to a breach than funds in cold storage; it’s designed that way. Since custodial services need to access and process transactions from hot wallets more frequently, the keys naturally need to be more accessible and often need to be stored online or on digital media in some form. This convenience naturally should make them somewhat more susceptible to a breach. 

Hence why custodial services should make an effort to store as much customer funds in ‘cold storage’ as reasonably possible. Although services and exchanges should encourage people to withdraw their funds to personal wallets, some people will inevitably be lazy, won’t want to pay the withdrawal fee, or don’t even have a personal cryptocurrency wallet and don’t care to go through the hassle of setting one up themselves. Thus, custodial services will end up holding a considerable percentage of customer funds “long term.” And it’s critical that a sufficient percentage of customer funds are in cold storage, so they’re less susceptible to a breach.

Exchanges and services do need to still keep a percentage of customer funds in hot wallets. This is because, in the case of cryptocurrency deposits by users, those deposit addresses will be part of a hot wallet controlled by the platform. Typically, there’s functionality that automatically forwards funds from user deposit wallets into a consolidation wallet sometime after the deposit, which is often ‘hot’ as well. As you can imagine, it would not be practical for multiple parties to manually sign all sending transactions from the deposit wallets.

A related case can be made for the hot wallet that processes customer withdrawals. Funds in cold storage aren’t meant to be accessed all the time; they’re meant to be accessed periodically. When a customer requests a withdrawal, the exchange or service needs to be able to process that withdrawal promptly and to do that, they need funds in an accessible wallet that can quickly process that ‘withdrawal’ transaction. Users often want the withdrawal to occur automatically as well and don’t want to wait for manual approval, so the withdrawal wallet already needs to be funded before the customer even requests the withdrawal. Also, given that exchanges often don’t have people on staff 24/7, they also need to be able to process withdrawal requests when no one else is in the office. To achieve this, the wallet needs to be ‘topped off’ with a sufficient amount of funds to continue processing withdrawals until someone is back in the office.

It’s thus necessary for exchanges and services to keep a portion of customer funds in hot wallets. But what is the appropriate amount? In our opinion, the answer really depends on the nature of the exchange or service in question. For most major exchanges, we tend to take the view that 90-95% of customer funds should be held in cold storage. The exact percentage does vary depending on a variety of factors, however. For example, exchanges that tend to cater to ‘beginners’ will naturally have more customers who store funds on the exchange. Thus, they will tend to want to have a reserve ratio on the higher end than an exchange targeted towards people who are already quite knowledgeable regarding cryptocurrency. The amount the exchange charges for cryptocurrency withdrawals is also a facto. If an exchange over-charges for withdrawals, it’ll incentive people to keep more funds on the exchange, meaning a higher reserve ratio is warranted, whereas if withdrawals are free, an exchange needs to be able to keep funds on hand more readily to process withdrawals since people are somewhat less likely to store funds on the exchange.

When exchanges fail to adhere to an adequate cold storage reserve ratio, it can easily result in a significant breach of customer funds, as was the case with the KuCoin hack, for example. There’s no reason that KuCoin should have had so much cryptocurrency in ‘hot wallets’ of theirs. And it appears that for many of the shitcoins they listed, they didn’t even bother to keep any of those tokens in cold storage at all, which led to considerable declines in the prices of many tokens when the hacker liquidated them.

Redundancy and Contingency Plans

Custodial services also must take care to implement redundancy and contingency plans. For example, even if an exchange elects to store wallet backups in a large secure safe in the office, there’s a fatal flaw; what if the building burns down or the building collapses? What if a founder dies, as was allegedly the case with QuadrigaCX? What if a keyholder isn’t available, as is allegedly the case with OKex founder Star Xu who has been detained indefinitely by the Chinese government, likely due to charges related to money laundering? 

Presumably, Okex didn’t have the foresight to even consider one of their founders might have been “unavailable,” and as a result, had to stop allowing customer withdrawals from the exchange! Don’t worry; they are still accepting deposits though, and OKex is now offering people up to $1000 if they deposit funds with them and continue to trade with them! To us, that sounds eerily like one of the many fraudulent investment scams that litter the cryptocurrency space. We tend to assume that OKex employed a 2-of-2 multi-sig for these cold wallets where Star Xu was one of the required signatories. Incompetence at its finest!

Cryptocurrency Wallet Management Practices

This is the meat and potatoes of the security audit. Questions that need to be asked, answered, and most importantly, independently verified when possible include:

  • What are the wallets they control, and what use case does each wallet have?
  • When and how, specifically, is each wallet accessed, and how exactly are the private keys and/or seed phrases stored?
  • What are the signature requirements for each wallet? Who are the keyholder(s)?
  • How and where does the keyholder(s) store their keys to these wallets?
  • Do keyholders back up their keys, and if so, how?
  • Are there keys that are not allocated to specific keyholders but perhaps accessible through a common shared but secure location such as a bank safety deposit box?
  • How does a company authorize a transaction internally before funds are sent? What protocols do they have, what verification methods are used, and are the protocols and procedures sound?
  • How do personnel and keyholders communicate with one another, particularly when sending or authorizing transactions? Is there any chance communication could be intercepted in any way, such as via a SIM-Swap, stolen electronic device, phishing link or email, or some other method that could result in a loss of funds?
  • How were the wallets generated? This is important so it can be determined whether there’s a chance they may not have been generated securely to begin with.
  • For their multi-sig wallets, do they have any procedures to detect if one of the keyholders may be under duress? If so, what are those procedures?
  • How are the key(s) for the hot wallet(s) stored, and how are transactions executed and broadcasted? Are there any security measures that limit the flow of funds out of hot wallets?
  • Is there a threat that funds could be lost due to a social engineering attack?

These are just some of the questions that need to be asked, answered, and verified as part of this portion of the security audit. It’s essential for services and exchanges to ensure they have a sound security setup since these are the exact types of questions hackers ponder and try to exploit every single day. Remember, these exchanges are under attack almost every single day. The best hackers in the world are trying to breach these exchanges every single day because there’s a massive bounty on the line.

Staff Risks & Risks of an Internal Breach

The risk of theft by an insider such as an employee or even a company founder is critical to assess in a security audit. Custodial services should employ a setup whereby no single person, not even a founder, would able to abscond with customer funds if they wanted to. However, there are more businesses should do as well.

Background checks should be conducted on all exchange personnel to assess any criminal history and employment history; something QuadrigaCX clearly failed to do with Michael Patryn / Omar Dhanani. While it’s perfectly acceptable for not all exchange personnel to be ‘public-facing,’ at least some exchange personnel should be, all exchange personnel’s real identity should be known to the executive team.

Physical Security and the $5 Wrench Attack

People who work at cryptocurrency exchange, and especially when they’re in an executive role, need to be incredibly conscious of physical security. For this reason, it’s not uncommon for exchanges to refuse to disclose their office addresses; to mitigate any security threat by a thief. But if an exchange’s real address is found, what could happen if a thief broke in? Could they gain access to computer systems? Or to wallets themselves that hold customer funds?

Five dollar wrench attack

Personal security is another factor exchange personnel need to be conscious of since an attacker may threaten harm to them or hold them at gunpoint or at ransom unless they hand over a considerable amount of funds. What mitigation efforts, if any, has an exchange taken to reduce the likelihood of such an event transpiring? What’s been done to mitigate the threat of the $5 wrench attack? The best defence against the $5 wrench attack is to ultimately take proactive action to mitigate it from happening to begin with. Are there actions one could easily perform and/or keywords one could indicate to show they are in distress?

Red Teaming and Pentesting

Red teaming and pentesting, while related, should not be confused with one another, and it’s extremely important for custodial to go through both. Penetration tests are focused on discovering and then exploiting network-related and infrastructure or computer-system related vulnerabilities. An example of what pentesting would naturally encompass would include scanning and probing of open ports, which an attacker could enter through and then install malicious software on the host or server.

A red teaming exercise, in contrast, often relies on the human element and social engineering susceptibility. Elements that would often be tested in any red teaming test include attempts to get employees to breach company protocols, impersonation attempts, and attempts to get employees to reveal confidential information, which could then be used in a future attack. Red teaming is designed to simulate a real-life adversary and not something the exchange would already be prepared or waiting for, thus allowing to see how they’d really fair against an attacker.

Demonstrating Solvency & Security

There is no single ‘correct’ way a custodial service can demonstrate to the public that they are solvent nor secure. There are multiple routes that can be taken.

Regarding exchange security, CipherBlade ultimately believes there needs to be a trusted third party involved when conducting such security audits. The reasons for this should be fairly obvious. A company should not want to publicly disclose the methods they are using to manage and secure cryptocurrency that they are responsible for custodying. Otherwise, threat actors also have this information, and it makes them more susceptible to an attack. If there are publically available details that outline the credential-related security measures the exchange takes, a prospective hacker then knows that as well and can use that to their advantage in an attack. Thus, a trusted and respected party must be involved that can publicly attest to security measures taken while at the same time not disclosing any information that could in any way lead to a security breach.

For solvency, there are more options available, but it does depend on the type of setup employed by the exchange or service. And all have various advantages and disadvantages. Verifying solvency necessarily involves verifying both assets, including cryptocurrency assets in this case (the easy part) but also liabilities. There are two main ways that are practical to utilize to attempt to assess solvency, and both do have some shortcomings. Let’s start with a comparison between the two main approaches.

Merkle Tree-based Proof of Reserves

A Proof of Reserves scheme can help to determine whether or not a custodial service or exchange is solvent through an approach that involves users being able to verify cryptocurrency liabilities (balances) on-exchange, as well as total customer cryptocurrency liabilities. That can then be contrasted with cryptocurrency assets held, which can be verified in a number of ways, some of which are auditor-assisted, and others which can involve an exchange simply publishing their addresses and providing applicable digital signatures as verification. It’s important to note that are a variety of Proof of Reserves schemes that use this ‘Merkle approach’ so one should not consider them all the same since even within this categorization, some are superior to others.

merkle tree example

This type of audit has several advantages and disadvantages compared to the trusted third-party approach.

Advantages

  • No need to place trust in an exchange regarding whether their claim that they hold enough cryptocurrency assets to cover cryptocurrency liabilities (with some exceptions)
  • Users can independently verify that cryptocurrency account balances or liabilities to customers at the applicable moment in time were properly accounted for in the audit (this generally applies to both their individual account balance, but also total balances of all user accounts).
  • Much more transparent to the general public with regards to cryptocurrency holdings
  • Doesn’t rely on trusted third-party; it relies on cryptographic proofs.
  • Can be performed entirely or mostly using in-house personnel; may not need to hire an external party to assist
  • Able to independently verify cryptocurrency assets they control (usually, but not always; depends on the scheme implemented by the exchange)
  • Individuals can independently verify that the exchange holds the requisite private key(s) which control applicable cryptocurrency funds (usually, but not always; depends on the scheme implemented by the exchange). However, do note that at a technical level, this doesn’t inherently prove the funds belong to the service/exchange; it just proves they have the keys to access those funds which presumably belong to the service/exchange since they have the private key(s).

Disadvantages and Shortcomings

  • Very difficult for non-technical users to understand. Let’s be honest; even if we were to narrow the field down to long-time Bitcoin enthusiasts, what percentage of people would even be able to describe Merkle tree is, much less be capable of independently verifying the data themselves? A very, very low percentage.
  • It doesn’t inherently verify that cryptocurrency controlled are customer cryptocurrency assets specifically (as opposed to corporate assets).
  • Unable to verify customer fiat assets nor liabilities (without involving an auditor). For example, the service may have secretly taken out a loan that isn’t disclosed to increase reserve assets, which is not something that would be picked up.
  • Focuses on solvency at a specific moment in time.
  • Audit cannot be performed instantly or on-demand.
  • Audit is not automatically renewed. The vast majority of exchanges that have published a Proof of Reserves audit have only done so once. Coinfloor is a notable exception, as they have been publishing one every month.
  • Not possible to do when funds are held in cold storage by a custodian because the exchange or service won’t have access to the private keys.
  • There are some instances where this will not be possible to do securely when the exchange or services controls the private keys depending on the SOP’s surrounding exactly how those private keys can be accessed.
  • Possibility of manipulation. In the absence of a competent and independent auditor, it’s possible to manipulate the results to make themselves seem solvent even if they are not, hence why it’s somewhat common to involve an auditor even when utilizing the Merkle-based approach.

Solvency Audit or Proof of Reserves Audit by a Trusted Third Party

This type of audit has a closer resemblance to a traditional financial audit. Cryptocurrency wallets can often be easily gathered via extended public keys (xPubs, yPubs, and zPubs), and control over cryptocurrency assets can be verified in a number of ways, including through the use of digital signatures, Satoshi tests and blockchain forensic analysis. Customer cryptocurrency liabilities can be gathered from financial records and back-end systems and can be assessed in multiple ways for any irregularities. Fiat assets and liabilities can be assessed through traditional means.

Despite having a closer resemblance to a traditional financial audit, very few auditors are currently both capable and qualified to perform this type of audit due to a lack of knowledge, experience, and tools. However, this is something we expect to change in the coming years.

This type of audit has several advantages and disadvantages compared to the Merkle-based approach.

Advantages

  • Much easier for non-technical users to understand. Newcomers to Bitcoin will likely still have difficulty understanding the verbiage in such an audit, but it should not sound foreign to those more well-acquainted with Bitcoin.
  • No need to place trust in an exchange that claims they hold enough cryptocurrency assets to cover cryptocurrency liabilities.
  • Able to verify customer fiat assets.
  • Possible to verify customer fiat liabilities, depending on the scope of the audit.
  • Able to verify customer cryptocurrency assets.
  • Able to verify customer cryptocurrency liabilities.
  • Although the primary focus is on solvency at a moment in time, an auditor is able better observe the ebb and flow of funds through applicable accounts over time, which can spot ‘irregularities’ or possible ‘concerns,’ which can then be assessed further.
  • Builds more trust with mainstream finance (e.g. banking partners) and regulators.

Disadvantages and Shortcomings

  • Relies on a trusted third party of which their credibility is of the utmost importance. I’ve listed this as a disadvantage since public transparency is something that’s typically sought after by cryptocurrency enthusiasts for intrinsic reasons. However, traditional financial audits have always relied on a trusted third party, namely the auditor. It’s incredibly rare to see a trusted auditor lie about verifying information since that’s the primary reason they exist.
  • Not always able to verify all financial liabilities since some liabilities, particularly non-customer fiat liabilities, could easily be missed due to a lack of documentation. However, the auditor is in a better position to be able to detect it than the Merkle-based PoL system, which cannot detect it.
  • Less publicly transparent
  • Individuals cannot independently verify most claims made by the auditor.
  • Primary focus is on solvency at a moment in time.
  • Audit cannot be performed instantly or on-demand. Preplanning is required.
  • Audit is not automatically renewed.

Which is Better?

That really depends on who you ask and who your audience is. If the audience is unwilling to place any trust in a qualified third party, there’s really only one option. Both options are good, in our opinion, but exchanges and services will often lean towards one or the other depending on a variety of factors. Regardless, it’s important for both businesses and cryptocurrency users to be cognizant of the limitations of both. A solvency audit or report by an auditor or trusted third party, such as those CipherBlade conducted for Shakepay and Bitbuy can cover slightly more in terms of scope, but users will not be able to independently verify the vast majority of the claims made by the auditor themselves, and that may prove to be a deal-breaker for some people. In contrast, other people may see the trusted third party option as superior as it’s broader in scope and may be willing to trust it as long as the auditor is trustworthy. Furthermore, those without the technical prowess to be able to verify claims from the Merkle-based approach independently may gravitate to the trusted third party approach.

At CipherBlade, we see a couple of important limitations with the Merkle-based approach that are worth discussing further. First, the Merkle-based approach doesn’t verify fiat assets or liabilities at all, which could have some profound consequences. Namely, a nefarious exchange could easily take customer fiat assets they have on-hand and use that to acquire more cryptocurrency elsewhere, which would then be added to their reserves, and could then suggest they have enough cryptocurrency assets to cover cryptocurrency liabilities, which would technically be true for the time being but would ignore the discrepancy between fiat assets and liabilities.

In our opinion, Kraken’s Proof of Reserves report is an optimal implementation of the Merkle-based approach. Kraken had an external auditor present to address precisely to address this concern about manipulation. However, another key role the auditor played in their audit was with regards to verification of cryptocurrency assets. Kraken was not publicly transparent about cryptocurrency assets they held, and users were not able to verify those assets either; rather, the auditor attested to them. The reason why Kraken chose this approach was related to security and privacy concerns. As but one example, if an attacker knew all of Kraken’s addresses they could leverage that in a $5 wrench attack, creating a security risk, and it also reduces user privacy if the general public has access to all Kraken addresses since they can see what wallet addresses transacted with Kraken’s addresses. In our opinion, both these concerns are well-founded, and it’s why exchanges often don’t simply want to release a list of addresses to the general public. Thus, in Kraken’s Proof of Reserves Audit, users were only able to verify cryptocurrency balances held on exchange at the time of the audit (and not Kraken’s assets), and because an auditor was present, it’s quite unlikely that the results were manipulated.

In contrast with the HBTC Proof of Reserves report, they just published their alleged addresses. They didn’t even bother to provide digital signatures, thus whether or not HBTC actually controls the wallets they’ve indicated hasn’t been independently proven. CipherBlade has no reason to doubt them, in fact, we can already ascertain through blockchain forensics that most of the wallets they’ve indicated are actually theirs with certainty, and we don’t have any reason to doubt their claims about the ones we aren’t 100% sure of, but just because we know something, doesn’t mean the public knows it as well — it’s not publicly transparent. They also haven’t worked with an external auditor, and thus there’s a variety of ways results could have been manipulated to make it seem like they were solvent (although to be clear, we are not suggesting that they did manipulate any results, we’re suggesting it would have been possible to do so), hence why involving an auditor would have stymied such concerns.

So we’ve determined that it’s quite important to have an external auditor present to ensure results aren’t manipulated in the Merkle-based approach. Given that some people prefer this approach because they can independently verify themselves, it should be noted that it still relies on a trusted third party to ensure there’s no manipulation. This trusted third party is sometimes responsible for verifying assets as well, again without public transparency. Thus, one logical question that follows is what’s the point of taking the Merkle-based approach if only some things can be independently verified, and other things rely on the auditor’s attestation? The user would still need to trust that the auditor in the Merkle-based approach is both competent and didn’t conspire with the exchange to manipulate results, which are also the main downfalls of the Trusted Third Party approach. These are things that are undoubtedly worth discussing further. We think both approaches have merit and we believe more and more exchanges will be undergoing such audits in the coming years. However, we believe that the Merkle-based approach still needs an auditor to ensure results related to solvency have not been manipulated.

Other Considerations

Whether or not a solvency and/or security audit is warranted depends on a number of factors. Let’s say the entity in question is not a custodial exchange but rather a corporation that trades and owns cryptocurrency. Obviously, there’s no point in conducting a solvency audit (unless they’re a publicly-traded company). There’s certainly no public obligation for them to conduct any type of security audit either, since if they lose those funds to a hack, no one is out of pocket but themselves. Nonetheless, they still may want to go through a security audit since the cost of doing so would be minuscule compared to the amount they could lose. Granted, proportionality is another consideration as well; the more cryptocurrency held, the more important it is to go through a security audit, and vice versa. However, if/when that trading firm starts utilizing other people’s assets for trading or investment purposes, that changes things.

All too often, people just don’t give a damn and don’t want to bother taking proactive measures since they already think they’re secure and won’t be hacked. People would rather be stingy and not spend $10k on an audit to protect tens of millions of dollars of customer funds, and only once they’ve been hacked do they suddenly care and want to try to find a way to recoup funds. However, proactive measures are the most effective and are certainly far, far cheaper than succumbing to a hack and experiencing a loss of customer funds.

At CipherBlade, we believe that these type of solvency and security audits become more of an industry-standard than it is today. Without public solvency and security audits, no person would want to do business with them for the same reason an investor wouldn’t invest in a stock on a public stock exchange if the company hasn’t gone through any independent scrutiny and hasn’t provided any financial records (much less audited financials). But that’s only going to happen if customers start seriously caring about these things; if customers don’t care, neither will exchanges, unless regulators start forcing exchanges to care.

Summary

It’s fairly standard practice across multiple sectors, particularly the retail sector, to allocate resources to loss prevention. When an exchange is responsible for custodying hundreds of millions of dollars of cryptocurrency, allocating resources to loss prevention is really a no-brainer. It’s far, far cheaper than dealing with a breach and then trying to recover those funds. But just importantly, it’s critical for cryptocurrency businesses to be as transparent with their customers about measures they’ve taken as possible, and self-proclaiming “we assure you, we have the most amazing security imagine” and to “rest assured, we’re solvent” simply isn’t acceptable. Cryptocurrency exchanges need to be more transparent with their users via Proof of Reserves audits, proving whether or not they are solvent, and must independently prove if they’re actually secure or if it’s just a facade.

After all the hacks, thefts, breaches and frauds that have happened and which are continuing to happen, cryptocurrency users are clearly interested in aspects like security and solvency as they should, but what remains to be seen is just how much consumers demand it. Will they downright refuse to use exchanges and services that haven’t taken measures which independently show or prove it? Or is it just a ‘nice to have’ for them? Our general opinion is that the former should be the case and that things are moving in that direction, slowly but surely.