Telstra Cybersecurity Job Simulation

About

https://www.theforage.com/simulations/telstra/cybersecurity-cyyo

GitHub repository: https://github.com/h4m5t/Telstra_Cybersecurity

Telstra is Australia’s largest telecommunications company, offering services like mobile phones, internet, and data solutions to millions of customers nationwide. Known for its reliability and innovation, Telstra connects people and businesses, ensuring smooth and effective communication.

The Telstra Cybersecurity Job Simulation Project is a training program designed to replicate real-world cybersecurity challenges. Participants work through tasks such as detecting threats, responding to incidents, collaborating with different teams, and implementing technical solutions to protect digital systems. This simulation helps individuals build the skills needed to defend against cyber attacks and keep Telstra’s services secure.

In the dynamic realm of cybersecurity, organizations must remain vigilant and responsive to emerging threats to safeguard their infrastructure and services. This blog post presents a detailed case study of how Telstra’s Security Operations Centre (SOC) effectively responded to a Spring4Shell (CVE-2022-22965) malware attack targeting the NBN Connection service. We will walk through the entire incident response process, encompassing initial threat triage, inter-team communication, technical mitigation using Python-based firewall rules, troubleshooting, and a post-incident analysis.

Task 1: Initial Threat Triage and Notification

Incident Identification and Severity Assessment

On March 20th, 2024, at 14:20 UTC, the SOC detected unusual activity targeting the NBN Connection service (nbn.external.network), which operates on Spring Framework 5.3.0. The attack manifested through multiple malicious POST requests to the /tomcatwar.jsp endpoint, indicating an exploitation attempt of the Spring4Shell vulnerability.

Affected Infrastructure and Prioritization

An analysis of firewall logs revealed that the NBN Connection service was under direct attack. Given its critical role in providing high-speed internet connectivity, the incident was classified as P1 - Critical. Other services, including Mobile Tower Connection, Home & Business Lines, and ADSL Connect, were evaluated and found to be unaffected based on the current logs. Nonetheless, continuous monitoring was recommended to ensure comprehensive security.

Notification of the Respective Team

Prompt communication was essential to coordinate an effective response. An urgent email was drafted and sent to the NBN Team, alerting them of the ongoing attack and the necessity to initiate immediate incident response measures.

Email to NBN Team:

From: Telstra Security Operations  
To: NBN Team (nbn@email)  
Subject: Urgent: Malware Attack Impacting NBN Connection Service

---
Body:

Hello NBN Team,

At **14:20 on March 20, 2024**, a malware attack targeting the **NBN Connection service** running on the **Spring Framework** was detected, resulting in service disruption and impaired functionality. This incident has been assessed as **P1 - Critical** and requires the immediate initiation of incident response measures to restore services and prevent further impact.

Please review the relevant logs promptly and take the necessary mitigation actions. If you need assistance or have any questions, feel free to contact us.

Kind regards,  
Telstra Security Operations

This communication ensured that the NBN Team was promptly informed, enabling them to take swift action to mitigate the threat.

Task 2: Collaborating with the Networks Team to Mitigate the Attack

Analyzing Firewall Logs and Identifying Attack Patterns

Upon identifying the attack, the SOC conducted a thorough analysis of the firewall logs. The logs indicated that the attack originated from multiple IP addresses within the AU region, utilizing specific malicious payloads designed to exploit the Spring4Shell vulnerability. The attack pattern involved POST requests to the /tomcatwar.jsp endpoint with parameters like class.module.classLoader.resources.context.parent.pipeline.first.pattern and others.

Drafting an Email to the Networks Team

To address the distributed nature of the attack without blocking individual IP addresses, the SOC collaborated with the Networks Team to implement a firewall rule that filters incoming traffic based on the identified malicious request characteristics.

Email to Networks Team:

From: Telstra Security Operations  
To: Networks Team (networks@email)  
Subject: Create Firewall Rule to Mitigate Spring4Shell Attack

---
Body:

Hello Networks Team,

We would like to request the creation of a firewall rule and provide you with more information about the ongoing attack.

**Type of Attack:**
Our analysis of the firewall logs indicates a Spring4Shell (CVE-2022-22965) malware attack targeting the **NBN Connection service (nbn.external.network)**. The attack involves multiple POST requests to `/tomcatwar.jsp` with malicious payloads designed to exploit the Spring Framework vulnerability.

**Characteristics to Block:**
- **Request Path:** `/tomcatwar.jsp`
- **HTTP Method:** `POST`
- **Specific Payload Patterns:** Requests containing parameters such as `class.module.classLoader.resources.context.parent.pipeline.first.pattern` and related malicious payloads.

**Request:**
Please implement a firewall rule to block incoming POST requests to the `/tomcatwar.jsp` endpoint and inspect for the presence of the aforementioned malicious payload patterns in the request data. This should help mitigate the distributed nature of the attack by targeting the specific exploit characteristics rather than individual IP addresses.

**Additional Information:**
The attack has been distributed across multiple IP addresses within the AU region. Blocking the specific request patterns will provide a more effective mitigation strategy. Attached is a proof of concept payload that demonstrates how the attacker scripts this attack, which may aid in refining the firewall rules.

For any questions or issues, don’t hesitate to reach out to us.

Kind regards,  
Telstra Security Operations

This email provided the Networks Team with the necessary details to develop targeted firewall rules, enhancing the organization’s defensive measures against the attack.

Task 3: Implementing Firewall Rules with Python

Developing a Python-Based Firewall Rule

To mitigate the attack effectively, a Python script was developed to implement a firewall rule that filters incoming traffic based on the identified malicious request characteristics. The goal was to block malicious POST requests to the /tomcatwar.jsp endpoint without relying on IP-based blocking, which is less effective against distributed attacks.

Firewall Server Script (firewall_server.py):

# firewall_server.py
# www.theforage.com - Telstra Cyber Task 3
# Firewall Server Handler

from http.server import BaseHTTPRequestHandler, HTTPServer
import urllib.parse

host = "localhost"
port = 8000

#########
# Handle the response here 
def block_request(self):
    print("Blocking malicious request from:", self.client_address)
    self.send_response(403)
    self.send_header("Content-Type", "application/json")
    self.end_headers()
    response = '{"message": "403 Forbidden: Malicious request blocked."}'
    self.wfile.write(response.encode('utf-8'))

def allow_request(self):
    self.send_response(200)
    self.send_header("Content-Type", "application/json")
    self.end_headers()
    response = '{"message": "200 OK"}'
    self.wfile.write(response.encode('utf-8'))

#########

# Define malicious parameters associated with Spring4Shell vulnerability
MALICIOUS_PARAMS = [
    'class.module.classLoader.resources.context.parent.pipeline.first.pattern',
    'class.module.classLoader.resources.context.parent.pipeline.first.suffix',
    'class.module.classLoader.resources.context.parent.pipeline.first.directory',
    'class.module.classLoader.resources.context.parent.pipeline.first.prefix',
    'class.module.classLoader.resources.context.parent.pipeline.first.fileDateFormat'
]

class ServerHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        allow_request(self)

    def do_POST(self):
        # Check if the request path is the targeted endpoint
        if self.path == '/tomcatwar.jsp':
            # Retrieve and decode the request body
            content_length = int(self.headers.get('Content-Length', 0))
            body = self.rfile.read(content_length).decode('utf-8')
            params = urllib.parse.parse_qs(body)
            
            # Check for the presence of any malicious parameters
            if any(param in params for param in MALICIOUS_PARAMS):
                block_request(self)
                return
            else:
                allow_request(self)
                return
        else:
            # For all other POST requests, allow them
            allow_request(self)

    def log_message(self, format, *args):
        # Override to suppress default logging
        return

if __name__ == "__main__":        
    server = HTTPServer((host, port), ServerHandler)
    print("[+] Firewall Server")
    print("[+] HTTP Web Server running on: %s:%s" % (host, port))

    try:
        server.serve_forever()
    except KeyboardInterrupt:
        pass

    server.server_close()
    print("[+] Server terminated. Exiting...")
    exit(0)

Testing the Firewall Rule

A complementary script, test_requests.py, was utilized to simulate both malicious and benign requests to ensure the firewall rule functioned as intended.

Test Requester Script (test_requests.py):

# Test Requester.py
# www.theforage.com - Telstra Cyber Task 3
# Test Requester

import http.client

host = "localhost"
port = 8000

def main():
    target = f"{host}:{port}"
    print(f"[+] Beginning test requests to: {target}")
    successful_responses = 0

    for x in range(5):
        payload = (
            "class.module.classLoader.resources.context.parent.pipeline.first.pattern=%25%7Bc2%7Di%20if("
            "%22j%22.equals(request.getParameter(%22pwd%22)))%7B%20java.io.InputStream%20in%20%3D%20%25%7Bc1%7Di."
            "getRuntime().exec(request.getParameter(%22cmd%22)).getInputStream()%3B%20int%20a%20%3D%20-1%3B%20byte%5B%5D%20b%20"
            "%3D%20new%20byte%5B2048%5D%3B%20while((a%3Din.read(b))!%3D-1)%7B%20out.println(new%20String(b))%3B%20%7D%20%7D%20%"
            "7Bsuffix%7Di&class.module.classLoader.resources.context.parent.pipeline.first.suffix=.jsp&class.module."
            "classLoader.resources.context.parent.pipeline.first.directory=webapps/ROOT&class.module.classLoader."
            "resources.context.parent.pipeline.first.prefix=tomcatwar&class.module.classLoader.resources.context.parent."
            "pipeline.first.fileDateFormat="
        )
        print(f"[{x + 1}/5]: Making test request to {target} with payload: {payload}")
        conn = http.client.HTTPConnection(target)

        conn.request('POST', '/tomcatwar.jsp', payload,  {
            "suffix": "%>//",
            "c1": "Runtime",
            "c2": "<%",
            "DNT": "1",
            "Content-Type": "application/x-www-form-urlencoded",
        })
        response = conn.getresponse()
        status_code = response.status
        if status_code == 200:
            successful_responses += 1
        print(f"Response status code: {status_code}")
        print("=============")

    print("[+] Test completed.")
    print(f"[+] Successful responses: {successful_responses}/5")

if __name__ == "__main__":
    main()

Troubleshooting: Addressing Port Conflicts

During the deployment of the firewall_server.py script, an error was encountered:

1	OSError: [Errno 48] Address already in use

This indicated that port 8000 was occupied by another process, preventing the firewall server from binding to it. The following steps were undertaken to resolve the issue:

Identifying the Occupying Process:

Using the lsof command:
1
lsof -i :8000
This command revealed the Process ID (PID) of the application using port 8000.
Terminating the Conflicting Process:

The identified process was terminated using the kill command:
1
kill -9 <PID>
Replace <PID> with the actual Process ID obtained from the previous step.
Verifying Port Availability:

Ensuring that port 8000 was free by rerunning the lsof command:
1
lsof -i :8000
No output indicates that the port is now free.
Restarting the Firewall Server:

After freeing up the port, the firewall_server.py script was successfully executed:
1
python3 firewall_server.py
The server started without issues, indicating that it was listening on the designated port.

Alternative Solution: Changing the Server Port

If port 8000 remains consistently in use, an alternative approach involves changing the server to listen on a different port (e.g., 8080). This requires updating both the firewall_server.py and test_requests.py scripts to reflect the new port number.

Edit firewall_server.py:

Modify the port variable:
1
2
host = "localhost"
port = 8080
Edit test_requests.py:

Update the port number accordingly:
1
2
host = "localhost"
port = 8080
Run the Modified Server:
1
python3 firewall_server.py
Run the Test Requester:
1
python3 test_requests.py
This ensures that the firewall rule is correctly applied on the new port.

Task 4: Incident Postmortem and Lessons Learned

Incident Postmortem: Spring4Shell Malware Attack on NBN Connection Service

Summary

On March 20th, 2024, at 14:20 UTC, Telstra’s Security Operations Centre (SOC) detected a P1 - Critical malware attack targeting the NBN Connection service (nbn.external.network), operating on Spring Framework 5.3.0. The attack involved multiple malicious POST requests to the /tomcatwar.jsp endpoint, exploiting the Spring4Shell (CVE-2022-22965) vulnerability. The incident was identified through firewall log analysis and was successfully mitigated two hours after detection by implementing a targeted firewall rule. Key teams involved in the response included the Security Operations Centre and the NBN Team.

Impact

Service Disruption: The NBN Connection service experienced significant downtime, impairing high-speed internet connectivity for customers relying on this infrastructure.
Operational Impairment: Critical services dependent on the NBN Connection, such as remote communications and business operations, were temporarily affected.
Potential Data Exposure: Although no data breaches were confirmed, the nature of the attack posed a risk of unauthorized command execution and potential data exfiltration.

Detection

The incident was discovered through routine monitoring of firewall logs by the SOC. Analysis revealed a pattern of multiple POST requests to the /tomcatwar.jsp endpoint originating from several IP addresses within the AU region. These requests contained specific malicious payloads characteristic of the Spring4Shell vulnerability, including parameters like class.module.classLoader.resources.context.parent.pipeline.first.pattern and others designed to execute remote commands.

Root Cause

The root cause of the incident was the exploitation of the Spring4Shell (CVE-2022-22965) vulnerability within the Spring Framework 5.3.0 used by the NBN Connection service. Attackers crafted malicious POST requests to the /tomcatwar.jsp endpoint, embedding payloads that leveraged this vulnerability to execute arbitrary commands on the server, leading to service disruption and impaired functionality.

Resolution

To mitigate the attack, the SOC collaborated with the Networks Team to implement a targeted firewall rule using a Python-based HTTP server (firewall_server.py). The rule specifically blocked incoming POST requests to the /tomcatwar.jsp endpoint that contained the identified malicious parameters. This measure effectively halted the ongoing attack within two hours of its initiation, restoring the NBN Connection service to operational status and preventing further unauthorized access.

Action Items

Immediate Actions:
- Firewall Rule Implementation: Successfully deployed a Python-scripted firewall rule to block malicious POST requests targeting the /tomcatwar.jsp endpoint.
- Service Restoration: Coordinated with the Networks Team to ensure the NBN Connection service was promptly restored to normal operations.
Short-Term Actions:
- Vulnerability Patching: Upgrade the Spring Framework to the latest version to eliminate the exploited Spring4Shell vulnerability.
- Enhanced Monitoring: Increase the frequency and depth of firewall log reviews to detect similar or new attack patterns more swiftly.
- Incident Documentation: Complete detailed documentation of the incident for future reference and compliance purposes.
Long-Term Actions:
- Security Training: Conduct training sessions for the SOC and relevant teams on identifying and responding to similar vulnerabilities and attack vectors.
- Comprehensive Security Audit: Perform a thorough security audit of all critical services to identify and remediate potential vulnerabilities.
- Automation of Response Mechanisms: Develop automated scripts and tools to detect and mitigate such attacks in real-time, reducing response times.
- Collaboration with Development Teams: Work closely with development teams to ensure secure coding practices are followed, particularly when using frameworks like Spring.
Future Prevention:
- Regular Updates and Patching: Establish a routine schedule for updating and patching all software frameworks and dependencies to minimize vulnerability exposure.
- Advanced Threat Detection Systems: Invest in more sophisticated threat detection and prevention systems that can identify and block complex attack patterns.
- Incident Response Drills: Conduct regular incident response drills to ensure all teams are prepared to handle similar attacks efficiently.

Lessons Learned

Proactive Monitoring: Continuous and proactive monitoring of firewall logs is essential in the early detection of potential threats.
Collaborative Response: Effective communication and collaboration between the SOC and infrastructure teams are critical in swiftly mitigating attacks.
Automation and Scripting: Utilizing scripting languages like Python for developing automated firewall rules can significantly enhance response times and accuracy.
Regular Patching: Keeping all software frameworks and dependencies up-to-date is vital in preventing exploitation of known vulnerabilities.
Comprehensive Documentation: Maintaining detailed incident postmortems aids in future governance, risk management, and compliance efforts while educating the team on handling similar incidents.

Conclusion

This incident underscored the importance of robust monitoring, swift response mechanisms, and collaborative efforts in combating sophisticated malware attacks. By implementing targeted firewall rules and adhering to best practices in incident response, Telstra effectively mitigated the Spring4Shell attack, ensuring the continuity of its critical services and reinforcing its commitment to cybersecurity excellence.

Prepared by:
Telstra Security Operations
Date: April 27, 2024