When pentesting a website, it is important to find hidden information which could possibly be attack vectors but are not visible on the public. Finding hidden files can be considered the first step in searching for potential attacks in a website. There are different tools created for this purpose but they each have their way of usage. Websploit, for example, is one of those tools.
In this guide, however, we are going to look at how to install and use dirsearch in Ubuntu. Dirsearch is written in Python and is used in brute-forcing hidden web directories and files. It is a simple to use, yet powerful command line tool, and is available for Linux, Windows and MacOS. Some of the features that make dirsearch stand out include: multithreading, proxy support, request delaying, user agent randomization and support for multiple extensions. Being Python, it is easy to integrate into scripts and existing projects.
Dirsearch works best when it comes to recursive scanning. When it finds any file, it goes back through it and crawl for any additional directories. Due to this recursive scanning, its simplicity and speed that comes with the use of command line, dirsearch remains a powerful tool that every pentester should know.
Install Dirsearch on Linux
For my installation, I am going to be using Ubuntu. We are going to be installing from github. Update your system repository and install git.
sudo apt-get update && apt-get install git
Next, use git command to clone the directory where the dirsearch will be
$ git clone https://github.com/maurosoria/dirsearch Cloning into 'dirsearch'... remote: Enumerating objects: 37, done. remote: Counting objects: 100% (37/37), done. remote: Compressing objects: 100% (37/37), done. remote: Total 4059 (delta 16), reused 0 (delta 0), pack-reused 4022 Receiving objects: 100% (4059/4059), 18.59 MiB | 4.08 MiB/s, done. Resolving deltas: 100% (2523/2523), done.
Change to the dirsearch created directory
List the content to ensure that is is properly installed.
$ ls CHANGELOG.md CONTRIBUTING.md db default.conf dirsearch.py Dockerfile lib logs README.md reports thirdparty
Once installed, dirsearch can be run in different ways which we are going to discuss below:
Run Dirsearch Using Python
To run dirsearch with Python, ensure that you have python3 installed in your system. The syntax to use is:
python3 dirsearch.py -u <target-url>
Run dirsearch using bash
To dirsearch with bash, we simply run .py executable file as below:
Run dirsearch using symbolic link
Here, we will create a symbolic link in the /bin directory which then allows us to run dirsearch from anywhere as opposed to just running it from its directory. Run the below commands to create the symbolic link:
cd /bin/ sudo ln -s ~/dirsearch/dirsearch.py dirsearch
After that simply run the command ‘dirsearch’ from anywhere
$ dirsearch URL target is missing, try using -u <url>
How to use Dirsearch to Scan files and directories
Note that using -h flag gives more information on how to use dirsearch.
$ dirsearch -h Mandatory: -u URL, --url=URL URL target -l URLLIST, --url-list=URLLIST URL list target -e EXTENSIONS, --extensions=EXTENSIONS Extensions list separated by comma (Example: php,asp) -E, --extensions-list Use predefined list of common extensions -X EXCLUDEEXTENSIONS, --exclude-extensions=EXCLUDEEXTENSIONS Exclude extensions list, separated by comma (Example: asp,jsp) Dictionary Settings: -w WORDLIST, --wordlist=WORDLIST Customize wordlist (separated by comma) --prefixes=PREFIXES Add custom prefixes to all entries (separated by comma) --suffixes=SUFFIXES Add custom suffixes to all entries, ignores directories (separated by comma) -f, --force-extensions Force extensions for every wordlist entry. Add %NOFORCE% at the end of the entry in the wordlist that you do not want to force --no-extension Remove extensions in all wordlist entries (Example: admin.php -> admin) --no-dot-extensions Remove the "." character before extensions -C, --capitalization Capital wordlist -U, --uppercase Uppercase wordlist -L, --lowercase Lowercase wordlist General Settings: -d DATA, --data=DATA HTTP request data (POST, PUT, ... body) -r, --recursive Bruteforce recursively -R RECURSIVE_LEVEL_MAX, --recursive-level-max=RECURSIVE_LEVEL_MAX Max recursion level (subdirs) (Default: 1 [only rootdir + 1 dir]) --suppress-empty Suppress empty responses --minimal=MINIMUMRESPONSESIZE Minimal response length --maximal=MAXIMUMRESPONSESIZE Maximal response length --scan-subdir=SCANSUBDIRS, --scan-subdirs=SCANSUBDIRS Scan subdirectories of the given URL (separated by comma) --exclude-subdir=EXCLUDESUBDIRS, --exclude-subdirs=EXCLUDESUBDIRS Exclude the following subdirectories during recursive scan (separated by comma) -t THREADSCOUNT, --threads=THREADSCOUNT
At the very least, dirsearch requires a URL and at least one file extension to run. For example, we can give a valid URL with the -u flag, and a file extension to search for using the -e flag:
dirsearch -u http://10.10.2.15/site1 -e php
The above command is suppose to provide information about the extensions, http methods used, threads number and the size of current word list. It then crawls the directories and returns the findings including status code, size and name of directory.
We also pass -x flag to exclude certain HTTP status codes.
dirsearch -u http://10.10.2.15/site1 -e php -x 403
We can tell dirsearch to use a wordlist of our choice by setting the -w flag:
dirsearch -u http://10.10.2.15/site1 -e php -x 403,301,302 -w /usr/share/wordlists/wfuzz/general/common.txt
To run the recursive search, simply tack on the -r flag.The command completes the initial scan then go back through and scan each directory it found recursively.
dirsearch -u http://10.10.2.15/dvwa -e php -x 403,301,302 -r
It is possible to pause the scan at any time with a keyboard interrupt. Pressing e completely exits the scan c will continue where it left off, and n will make it to move on to the next directory. These give us some control over the results since recursive scanning can often take quite some time.
Note that to set the recursion level to a deeper value, use the -R flag and indicate how many levels deep to go.
dirsearch -u http://10.10.2.15/site1 -e php -x 403,301,302 -r -R 3
That’s it about how to install and use Dirsearch on Ubuntu. Practice more and enjoy! Check below more interesting guides on Linux: