Have you Tried Googling It?
I have a whole section (here) on different search engines online, and some common advanced search terms that can be used to assist in finding data. However, I figured a good first walk through would be how to best use Google to find whatever it is we are looking for.
Before getting into how to best use Google, let's briefly talk about how it and most other search engines work.
They use web crawlers to go out to scrape sites on the internet and save all of the text, urls for images, etc on a website and all of its pages (If you really want to nerd out and see exactly what parts of a webpage are saved you could look at the robots.txt section of the website.) After they go out and collect of that data they save it, and organize it on their own servers. So essentially when a user conducts a search, all they are doing is asking Google if they have seen and saved any data that matches your search.
With that all being said, hopefully you at least now have a real basic understanding of how search engines work and lets get into Google. Starting with the settings... (Highlighted in the picture below)
Once we click on settings we are going to get into our "Search Settings" first. So click that if you haven't already.
The first thing I like to change is the Results per page. I set it to 50, but the world is your oyster. Do I recommend this? It is because most people have no issues scrolling down a website for forever, but cannot stand to click through more than a couple pages of search results. What if you stopped looking at page 5 because you thought it wasn't there, but the result you needed was on page 6? Now you have more results per page and if you only look at three pages of results at 50 results per page, it is like looking at 15 pages with the default pages.
All the way at the bottom of the page is the "Region Settings" This little gem is often overlooked and is super useful. This because it allows me to conduct my searches and get regionally specific results. Am I looking for an individual in Ireland? I am going to change my region settings to Ireland. Don't forget to save your region after selecting it!
*Quick Disclaimer: This offers no type of IP address protection. This is not like using a proxy or a VPN. All you are doing is asking Google to conduct your search on a different set of servers.
Now let's conduct a basic search. Let's search "cafes" with your normal Google Settings. You will see below that when I conduct this search my results are based off what Google believes my current location to be, and I see a lot of cafes that I frequent.
Now I will change my regional settings to Ireland and conduct the same search, and just look at the difference below! Note how all of the recommended cafes are located in Ireland and the top level domain of the websites returned.
***If you tried the same thing and noticed that you did not get many changes in your results, it is probably because of several factors. One is you have your locational services on and you have a lot of cookies in your browser. Try clearing your cookies and cache, put your region settings back on ireland, and try again. Did you notice a difference this time?
We just did our first two basic searches together (it is a moment I will never forget). So let's get on with the walk through. Next we are going to briefly talk about the map tab and the tool tab. With your cafe search already conducted (regional settings still set to Ireland) click on the map below the search bar.
What did you notice? Did your map center over Ireland and show you cafes? Most likely not, it probably showed you the city you were in and highlighted cafes there. Why the change?
If you were thinking because maps are based off of your IP address and the regional settings does not change that, that is correct! I am not going to really cover Google Maps right now. Google Maps is an awesome tool and it deserves a walk through of its own. Stay tuned for that at some point soonish (hopefully).
RIght now here are a couple of things to know about Google Maps. You can search in the search bar in the upper left for a location, change the map layers near the bottom left, and access street view by dragging the little yellow man on the bottom right onto the map. If you have never done that before, play with that for a bit. It can provide you street level views of the area you are interested in (unless it is the middle of nowhere).
Hopefully you had fun on your virtual tour, but it is time to get back to work. Go back to your main search, and this time I want you select Tools. It is the right most option directly below the search bar.
You should now see two additional options populate. "Any time" and "All results"
Any time will all you put a date filter on your search. This can be useful for finding recently published news or articles related to a topic of interest, or go back in time to look at the results of your search before breaking news.
All results if useful if what you are searching is spelt unusually and Google wants to correct your spelling. If you want Google to search exactly what you typed select "Verbatim". This can also be accomplished by putting your term in quotation marks. And this is a great transition to Google Dorking. Which, in all honesty is probably the reason you clicked on this to begin with. So let's get into it...
Welcome to Google Dorking 101
Google Dorking is the bread and butter of OSINT. Wikipedia has a decent definition of the term, "Google hacking, also named Google dorking, is a hacker technique that uses Google Search and other Google applications to find security holes in the configuration and computer code that websites are using." Essentially we will be using how google saves and structures its data when it caches websites to find nuggets of information.
In this section we will be going over some of the advanced Google search operators that will aid us in finding whatever it is we are looking for (hopefully). With that all being said let us get started with the basics. AND OR and NOT...
By default when you type multiple words into Google, it will automatically place an AND between the words you are searching. The "AND" operator is telling Google to show you results that match all of the words you entered, regardless of the order. So if I typed, cute corgi puppies into google it will showing me results that have cute AND corgis AND puppies on the page . I could conduct the same search by typing cute AND corgi AND puppies into the google search bar and get the same results. Another way to use the AND operator is by using the + sign instead of AND. So our search from earlier could also be written as cute+corgi+puppies.
Using the "OR" operator between words is telling Google to show me results with either this term, that term, or both. If we go ahead and put Chocolate OR Vanilla into the Google search bar you will see that you get results related to only chocolate, only vanilla, or pages that have both of your search terms on them. You could use the | character as well. This can usually be fun two keys above the right shift key on most keyboards. Our search would look like Chocolate|Vanilla if we used that operator's shorthand.
Using the "NOT" operator in a Google search will exclude the word that follows it from any of your search results. So if we typed Chocolate NOT vanilla what did you notice? If you said, "Results for vanilla are still showing up in my search." You would be correct. When we use NOT in a search we have to use the - sign. Retry your search, but this time construct it like this Chocolate -vanilla
Your results should now only have the term chocolate in them. Please note that if a page have both chocolate and vanilla it will not include it in your search results, because any result that has vanilla in it will be eliminated even if it has words that you want.
Those are our three basic operators, AND OR and NOT. I would take some time constructing several different search with those operators before moving on to the next section. It only gets more fun from here.
Next, let's get into a couple more of the easier operators we can use. In these next couple of paragraphs we will cover quotation marks "", parenthesis (), and the asterisk *
This will force Google to search exactly whatever you in input in between them. This can be useful when searching for something that is spelt a certain way and Google is trying to search for something else. Names can be a great example of this. Mohammed is one of the most popular names in the world, but I have several friends named Mohammed and I have seen it spelt several different ways including Muhammed, Muhammad, Mohamed. I was going to search for my friend Mohamed using google I would use quotation marks and search it like this, "Mohamed LASTNAME". Quotation marks can also be used to search for words you care about in an exact order. Remember our cute corgi puppies from earlier? If I wanted search results that had cute corgi puppies exactly in that order without any word in between I would search "cute corgi puppies"
These bad boys will essentially allow us to group multiple search terms or operators together. What do I mean? What if you wanted to search multiple things at once, or you did not want to write a certain word over again in your search? Parenthesis help you in simplifying a potentially long and complex search. What if you wanted to search for information about apple ipads and apple iphones? You could search knowing what we learned from above apple iphone OR apple ipad And this would return good results but why type more than what you need to? I could construct the same using parenthesis like this:
apple (ipad OR iphone)
Does that make sense? If not think all the back to math class when you learned about the distributive property. AB+AC could also be rewritten as A(B+C). Google works the same way.
Our searches do not have to be as simple as looking for iphones and ipads. What if you were writing a paper on gun violence in several of the major US cities? Sure you could several search terms into Google, for example I have seen people when researching this topic enter Gun deaths in New York and then do the same thing for Washington, and Philly, and Chicago, but who wants to do all of that when one search could do it for me? I could have google search for gun deaths in all of those cities by searching gun AND deaths AND in AND (new york OR Chicago OR Philadelphia OR Washington DC).
This one is pretty simple. Think as this a wildcard search. If we put An * a day keeps the * away into Google, Google will treat those * as potentially being any word and we will probably see An apple a day keeps the doctor away as our top result.
And there are our next three operators for Google. Take a couple of minutes and play with all of the operators I have talked about already. Try to get familiar with what they do and get comfortable constructing searches with them.
Feeling pretty confident with all of the operators talked about so far?
This operator will limit results to a certain website. Sticking to my chocolate theme from before, If I was looking for a chocolate cake recipe from allrecipes.com I could put the following into Google,
This will search for your search terms in the url of Google's cached results. If we only wanted results that had chocolate cake recipe in the url we could put the following into Google,
This will only return results that match your requested file type. This can be combined with a keyword search as well. I will also note that not every single file type is supported by this but most of the major ones will be including .png, .docx, .pptx, and .xlsx. So let's say we wanted to find a pdf from WHO related to hunger. We could accomplish this by putting the following into Google,
site:who.int filetype:pdf hunger
Did you see what we also just did there? We combined several advanced operators together. In this case we combined the site and filetype operator together to find even more specific results.
The intitle operator searches the title of web pages and articles for terms that match your search. Not sure what is the title? If you run a search in Google this we will be section that is usually returned in blue below the URL preview. Let's combine this with the filetype operator to find a PDF with any documents with Gun violence in the title and couple of different cities in the United States for our keywords. This search would look something like this,
filetype:pdf intitle:"Gun Violence" AND ("New York City" OR Camden OR Newark)
And the results look like they are pulling back exactly what I am looking for.
The intext operator restricts results that have my search terms in the text of a web result. Pretty simple concept, but extremely powerful when combined with other operators.
The around() operator is useful for finding words that fall within a certain number of each other. The number goes between the parenthesis. So if was looking for results that had the words corgis and cute with no more than two words in between I would type the following into Google,
cute around(2) corgis
And there we go! Our first walk through is done! I think at some point I will come up I will come up with some exercises that you can do to practice Google, but if you play with some of these operators daily you will be amazed at how quickly your Google Dorking skills increase!