This is a new series of articles that will cover the complete penetration testing methodology based largely on case studies of previous hacks. These articles will cover initial reconnaissance, picking an attack vector, gaining a foothold, maintaining presence, lateral movement, and finally going after the prize.

A guide to Open Source Intelligence

One of the main case studies these articles will borrow from is the APT1 report published by Mandiant in 2013. Though the report focuses on a nation-state conducting a long-term hack against an opposing government and industry, the methodology is very similar to standard penetration tests. The main differences in methodology between penetration testing and nation-state hacking is resources and intent. These articles are not a how-to on hacking a network but more of a broad overview of the concepts with some tool demonstrations to solidify thoughts and ideas. With that, let’s get on to the first step: Open Source Intelligence.

In all penetration tests, one must always assess the battlefield. A penetration tester must know the internet footprint of the network they are dealing with. In order to do that, you need to start with the basics. Before sending even one packet at the target network, the first thing to accomplish is gathering Open Source Intelligence or OSINT for short. OSINT is comprised of two different but distinct parts; Technical OSINT and Human OSINT. Technical OSINT focuses on the technical details of the target network/organization, what IP spaces they own, their registrar, domain records, devices attached, errant servers on random ports, etc. Whereas Human OSINT is the details and relationships that are connected with the network or organization and how to leverage that information. This article will discuss the aspects of Technical OSINT, where to find various information and why it is critical information.

Technical OSINT

For the purposes of this article, we are going to use the domain of nmap.org. Nmap.org is the main page for the scanning tool, nmap. The nmap tool itself will be covered more extensively later on in this series of articles.

Before sending a packet at the target network, we need to find what addresses it owns and where our packets might go if we send them at a domain. This also relates to receiving packets and traffic. Before APT1 sent their spear fishing emails, they canvased the internet to find all the Technical OSINT they could about the target they were going after. They knew IP information in advance when someone opened a malicious email attachment. If a penetration tester sent a spear fishing email out with a malicious attachment and then received a callback from an IP space they were not expecting, that would be a serious cause for concern and potentially a legal liability as it might be outside the scope of the penetration test.

As penetration testers, we can find basic information about the domain itself using a site called Robtex.com. Following this link, we can see that it provides everything from the IP addresses it owns, previous IP addresses, reverse DNS lookup, registrar information, web hosting information, whois, etc. It even shows if that IP address is home to other websites. One thing to think about is that if an IP address attached to your target is also shared with an organization that isn’t your target, it could cause a problem with trying to enter the network through technical means. All of this information is good at building a profile on the target. 

Tools of the trade for cyber security professionals

Domain Tools is a great storehouse of information that a penetration tester can use to gather information on a potential target. It doesn’t have as fancy graphics as Robtex, but it has the majority of information needed to start the job. The information presented is very clear and concise. It does give you a screenshot gallery of the web page and links to whois information for other related domains like nmap.net and nmap.com. You can see more information at this link. http://whois.domaintools.com/nmap.org

Moving away from websites to more Linux command line-based tools, there is a tool called “dnsenum” at our disposal. Dnsenum is a simple program that can take different command line switches to give us different information. In this example we just use nmap.org as the command line argument and nothing more. You’ll notice dnsenum shows us the mail server information. This valuable information tells us that the mail server is run by Google and that they don’t have their own in-house mail server. For future reference, a mail server can serve as a potential entry point into a network. It also specifically tells us that *.nmap.org will redirect to www.nmap.org due to wildcard domains.

Staying with the Linux command line options for technical OSINT, another tool available to us is “theHarvester.” Instead of using a web browser to search and deliver the requested information, theHarvester works from the Linux command line and derives its information from various search engines. More information on how this tool works can be found at: https://github.com/laramies/theHarvester.

Below is an example of how to use theHarvester.

You’ll notice that the four tools previously mentioned all show data in different ways and have different interfaces. Using all these tools in conjunction can provide a mostly complete picture of the target network. This happens to be the case in our example of using nmap.org. In the search results from theHarvester, we find that there are more subdomains. Some of these subdomains are to help in the event of a typo, others are completely dormant or haven’t been in use for some time. Others are listed that aren’t publicly listed on a site like Robtex. If we take a look at svn.nmap.org, you’ll notice that it isn’t exactly a website but more of a file repository. This website is a front-end for Subversion. Subversion is a software versioning and revision system used by developers to maintain source code. This could be a potentially useful tool later on in a penetration test to find exploitable bugs in a company’s source code. For right now, we will leave svn.nmap.org alone and continue to focus on fleshing out the Technical OSINT on nmap.org.

Another way to find more Technical OSINT is using Google search, or more specifically using Google Dorks. Google Dorks are search terms that are used in conjunction with the target network. For instance, if we wanted to see if nmap.org hosted Microsoft Word documents we could use a search term something like this: “site:nmap.orgfiletype:doc”. This would show any search results with nmap.org being the hosting server and any files of the “doc” type. Another example of a Google Dork would be “site:nmap.org inurl:login”. This could potentially show if there were a hidden login page for the website that isn’t linked anywhere on the website. It is also important to note that Google Dorks isn’t limited to just the Google search engine but also the DuckDuckGo search engine.

As shown above, documents can be discovered using Google Dorks. Documents can lead to some great Technical OSINT on the company but can also lead to Human OSINT through metadata. We will focus more on that in the next article.