Network Forensics Deep Packet Inspection using NetWitness

Network ForensicsDeep Packet Inspection

Topic: User Agent Strings

Created by Jonathan TomekSenior Threat AnalystiSIGHT PartnersThe User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations. User agents SHOULD include this field with requests. The field can contain multiple product tokens and comments identifying the agent and any subproducts which form a significant part of the user agent. By convention, the product tokens are listed in order of their significance for identifying the application. RFC2616What is a User-Agent?In many cases, a user agent acts as aclientin anetwork protocolused in communications within aclientserverdistributed computing system. In particular, theHypertext Transfer Protocolidentifies the client software originating the request, using a "User-Agent" header, even when the client is not operated by a user. The SIP protocol (based on HTTP) followed this usage. http://en.wikipedia.org/wiki/User_agent

Examples:

What is a User-Agent?

These strings are collected in the machine logs and reviewed for statistics.Many applications trust a User-Agent to be exactly what it specifies.Servers have the ability to represent data differently based upon User-Agent.I.E. If their User-Agent says they are on linux and they download a program, the server will deliver a .deb or .rpm instead of .exe or .dmgThis string can be modified to just about anything as long as it ends in newline hex(0D0A)Why is this important?The best way to know what is anomalous is by reviewing what is normalReview RFC 2616 by IETF to truly understand how the User-Agent string works:http://www.ietf.org/rfc/rfc2616.txt(search for section 14.43)If a User-Agent looks suspicious because it deviates from the normal standard, find out why it is behaving that way or find other suspicious triggers to correlate against.

Understanding what is Suspicious

Normal Browser User-AgentThe User-Agent starts with Mozilla/5.0. Has no real meaning anymore; kept for historical purposesChrome/19.0.1084.56 specifies the browser and version of Chrome that is runningWindows NT 6.1 specifies they are on Windows 7WOW64 means it is a 32-bit application running on a 64-bit processorNormal Browser User-Agent

The User-Agent starts with Mozilla/5.0. Basically means compatible with Mozilla.Macintosh Platform running OS X 10.6 Snow LeopardBuild date 2010 Jan 01Browser is Firefox version 13Other Legitimate User-Agents

There are other products out there with custom stringsUnderstand how they are being usedLegitimate products usually include an easy to recognize identifier

Command Line User-Agents

This user agent is curl running on a Solaris machineCurl is a command line tool for transferring data with a URL SyntaxThis tool can be used in an automated method to download a redirected fileCommand Line User-Agents

This user agent is Wget running on a Linux machineWget is command line tool used for downloading files without processing themNote: This session raises the level of suspicion because it is checking what the external IP address is of the host machine from a known good site but it does not make it malicious.Scripted User-Agents

These user agents were created by the coding libraries for these scripting languagesIn these cases it is Python and PerlDepending on the environment, this could raise the level of suspicionJava User-Agents

This is the JVM being used to crawl a siteTypically not too suspicious in natureIf there are lots of 404 Error responses, this could likely be an scanning attackCustom User-Agents

This will raises the level of suspicion when it is vagueIf possible, check with the OwnerSearch Engine Bot User-Agents

These are used to index web pages for Search EnginesDiscobot doesnt advertise

Another bot used to index web pages for Search EnginesUsed for Updates

Some vendors will dismiss RFC2616 completely and use the User-Agent string for their own needsThis is a good example to show that the string can be modified to anythingBe mindful that this User-Agent would be suspicious if it was not from a well known service

Vulnerability Scanning

Nessus, Qualys, and other vulnerability scanning tools often keep their product name in the user agentThis could be malicious if the scanning host is not approved

SIP vulnerability Scanning

These user agents are used by SIP vulnerability scanning typicallyThe User-Agent string is now more vague in recent versionsAs with any scanning, make sure you verify the Source IP addressKnowing how normal User-Agents are supposed to behave shows what they should not be doing unless specifiedWhat happens if there is a deviation from the normal, does it mean it is bad? Not reallyLook at these examples to see what stands out in comparison to the other User-Agent stringsWhat will raise the red flag?

User-Agent Missing

??This could mean numerous things but does raise the level of suspicionIt is missing many other HTTP header valuesMalicious

This User-Agent includes Javascript, highly suspiciousUser-Agent strings are written to a log file on the remote machineIf a successful exploit occurs, that string can be called from the log file and used to aid the attackerOther known malicious tags in User-Agent strings:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; AntivirXP08; .NET CLR 1.1.4322)Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; PeoplePal 7.0; .NET CLR 2.0.50727)Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; .NET CLR 2.0.50727)(Red is a form of fake antivirus and malicious. Orange is adware/spyware which sends data back to a remote host) Malicious

After knowing what is good, we can see what looks anomalous and suspicious.These steps will help you understand what to look for and find the low hanging fruit.Good malware authors will deceive by abiding by the rulesThey wanted to not be detected by blending in with other normal trafficThe malware could be used to download a special version of malware

Understanding the differenceAnalyze User Agent Stringshttp://www.useragentstring.com/index.phpCopy paste a user agent string to break it downList of User-Agent Stringshttp://www.useragentstring.com/pages/useragentstring.phpSummaries on what each User-Agent String meansMicrosoft.comhttp://msdn.microsoft.com/en-us/library/ms537503(v=vs.85).aspxInformation on how Microsofts user agent works

Sites that will helpBe familiar with the current browsers that are being used in the wildCommon: Opera, Firefox, ChromeUncommon: Lynx, Gecko, AOL

Note the difference between a mobile device, desktop, and command line User-AgentsMobile: Fennec, Blackberry, AndroidCLI: Curl, Wget, BinGet

ConsiderationsUser-Agent strings can contain useful information to determine a browser or systemUser-Agents can be visibly suspiciousIt alone is *not* enough to determine if something is malware but may be important during a pivot in an investigationKnowing how User-Agents function can determine how certain malware is delivered

If you dont understand a value, google it ;-)Conclusion