At around 9 pm on June 17, 2021, a Philippine Election Monitoring’s — lets call it as “NGO”- webserver was attacked. This caused the webserver to become unresponsive and it later on had to be shutdown to prevent possible data loss.
The attack started when monitoring systems showed a sudden spike in OUTBOUND traffic activity. This is shown in the MRTG graph below:
The blue line shows a surge in outbound traffic growing past 50mbps of bandwidth. This triggered NGO’s top hierarchy to approve the incident responses. With the authorization, the security team went into action.
While we now know that something ‘strange’ was afoot, we still needed to find out the “how” and the “why”.
The first priority was to mitigate the risk of data leakage. Vital data could be ‘sucked’ out of NGO’s server. The pattern of traffic was consistent with data leakage. In addition, a few days before, Manila Bulletin Editor Art Samaniego shared his discovery of a potential exploit in NGO’s app that “will lead to leaks of private data of over 3000 volunteers.” The NGO had to consider this a real threat. So the server had to be “taken offline” to avoid potential data loss.
There is also the RISK that the data could be WIPED as well AFTER the data has been copied. It is typical for an attacker to either encrypt or delete all the data from the server AFTER the data has been uploaded somewhere to the cloud.
Why did Hackers target NGO?
As to the “Why”, this attack came on the heels of a recent article by highly influential author, RJ Nieto in Thinking Pinoy about alleged links between 1Sambayan and the NGO. Could this article have incited the attack? I reached out to the hacker group Pinoy Vendetta that had claimed responsibility for the attack. They did acknowlege it in part by saying “Tungkol po kase yan doon sa na issue sila connected sa 1 sambayan”.
The second priority was to respond to this DDoS. This stands for “Distributed Denial of Service”. This type of cyber attack will flood a web server with hundreds of thousands of requests. The requests will come from multiple sources - hence the “Distributed” nature of the Denial of service attack. You might mistake it for the “Reddit effect”.
The goal of the DDos hack is to deny service to the public. Imagine if the NGO’s server was attacked on Election night. No one can browse the website and its elections results. This could set the stage for neutralizing an election monitoring organization.
Additionally, Pinoy Vendetta group offered this insight:
“During a DDoS attack, systems no longer operate normally. The heavy load or overload causes some systems to suddenly become vulnerable and opens up new vectors of attack.” So a DDoS from a hacker’s perspective is a jumping off point to subsequent exploits.
The DDoS attack generated high demand that was designed to bleed the capacity of the webserver. And ultimately, the webserver will become unresponsive and ‘die’. My favorite analogy would be take a pickup truck and load it with higher loads carried by18-wheeler trailer. — ‘Hinde kakayanin”
How was the Attack orchestrated?
This DDoS used a novel type of attack and admittedly, the attacker gave NGO something new to learn. Does it fit the pattern of a “UAM Bypass” attack as claimed by some? Let us see what the collected evidence shows.
By looking at the web and firewall logs, NGO noted the unusually high number of http request using HTTP HEAD. While normally seen in the wild, a massive amount of HTTP HEAD will cause the server to ASYMETRICALLY return a lot more data (over 50mbps) in response to a small byte size request (around 5mbps). In terms of economics, the hacker paid for 5mbps while NGO pays for 50mbps of bandwidth.
Trivia: Pinoy Vendetta shared the “market rate” for a raw botnet would be around 500 to 1000 USD a week. These networks can be engaged via Telegram or discord channels. There is no more need to go to the “Dark web” to engage these DDoS platforms.
The attack was also particularly insidious and sneaky. Each of the web HTTP Head request came with a series of cryptic characters after a . IT security friends had the same consensus: it was a cache bypass code designed to waste even MORE server resources. Much like adding salt to an injury. It further maximized the DDoS effect.
Normally, webservers will serve web request from cache. This speeds up the service and uses considerably LESS server resources. But when the HTTP HEAD requests contain in the URL, the server is FORCED to fetch the content DIRECTLY. It won’t come from the cache. Hence, the term “Cache Busting Code”.
Lessons Learned
The quick and easy way to fix this first round of attack was to only allow HTTP GET request. NGO re-configured their server to drop all other types of HTTP requests. This neutralized most of the HTTP HEAD based DDoS attack.
So take note fellow webmasters, if your clients’ applications don’t need the other HTTP verbs, allow only HTTP GET. This is an important lesson for this first wave of attack on NGO’s server.
Second, remember to preserve all the logs as evidence for any legal action that NGO should decide to take later on.
Third, from time to time, conduct a ‘Table-Top Exercise” to ensure that Incident Response (IR) teams can respond properly and execute recovery procedures to maximize uptimes, minimize disaster recovery time, and conduct forensics to identify and neutralize cyber attacks.
In the next part, we will share what the hacker group did to escalate the attacks once they figured out that their HTTP HEAD wasn’t working:
“about naman po dyan may GET method po kami na can even bypass firewalls more than the head method we had.”
Note: Some security procedures could not be divulged at the time of this writing. This enables NGO to pursue legal charges against the hackers, should they choose to do so.