Forem: Zack Allen

Absolute measurement corrupts alert severity, absolutely

Zack Allen — Fri, 18 Nov 2022 12:50:58 +0000

I sometimes get so excited about an alert or a finding, then immediately feel a sense of dread because of this comic. Is this a bun? Who else cares besides me?

Tell me if you’ve been in this situation before: you are working as a (SOC Analyst, Detection Engineer, Security Researcher), and just created a new rule for your SIEM. It took blood, sweat, and tears to get it exactly how you wanted it. It has lots of unit tests, links to articles, and blogs that document the TTP you are trying to capture, and it is a high visibility rule because management asked for it. Now comes the dreaded question: what should the severity of the rule be, and how can you logically explain why it is set to a certain level?

You sit back and think. You’re a professional. A logical, creative and driven individual. “Well, management asked for it”, you say to yourself, “..especially after they read about that breach that this rule helps prevent, so I can’t just mark it an insignificant info or low. They’d question my judgment”! Alright, so it’s gotta be higher than low. That leaves medium, high, and critical.

You pull up your organization’s Risk Severity Matrix document. In it, you sift through all kinds of mathematical equations to give you direction. The goal here is to apply rigor to your decision because without math, you’d just be making this up anyways.

The document is several matrices lined up with different colors and selection parameters: likelihood of occurrence, impact if successful, countermeasures in place to mitigate damage, should someone be woken up for this. It starts to feel like you are reading a different language. How the hell are YOU responsible for computing this?

After 30 mins, your brain starts to melt, and you let out a big sigh. “Screw it,” you say, “critical. Management wants it, my team wants it, it just seems important”. You deploy to production, pat yourself on the back and move on.

What’s the problem here?

I’ve been in this exact scenario a myriad of times in my career. You need to label an alert within a rule with a severity that you hope is accurate enough. You want an adequate severity because it would really suck to mess somebody’s night up because they get paged, and the alert is a nothing burger. Or worse - the alert triggered, but the severity wasn’t set high enough, so you did not wake that person up, and now the company is burned. So what gives? How did we get here? Why do we have so much power?

There are a lot of ways to measure the severity of a potential security alert. And I think we’ve taken it a bit far with the best ways to do it. The security industry latched onto this concept of being on-call because the bad guys never sleep, so why should we? So after we made THAT decision, we now need some logic behind what constitutes an alert that’s interesting from an alert that makes you go, “Oh, Shit.”

Here’s the problem. Brilliant people are in security. And because we have so many brilliant people, we came up with clever ways to find bad things. And once we found those bad things, we let our lizard brains take us the next step further and say, “well, the only way to explain how bad this evil thing is, is to make something equally brilliant to quantify its severity.”

Look - I am not going to name and shame severity models. The only question I am asking - does this explain enough to my boss or my organization why someone should be interrupted, paged, or woken up? How does this help my alert explainability?

Alert severity explainability

I joined Mastodon after the crazy Twitter exodus and started thinking about ways I could write about this subject. I also had COVID. And so what do you do when you are bored, isolated, and have COVID? You start ~~tweeting~~ tooting your boss, the CISO!

I had a performance management review scheduled with Emilio the week after this toot, so I thought tagging him in this investigation would get me into his good graces 😇😇

After more and more Googling, I realized I should be careful, there has been a lot of smart people working on this problem. Instead, I realized the issue: everything I investigated focused on Absolute Measurement. A series of inputs are sent into an alert, and the output should be some severity on a scale from informational -> critical.

Severity = Probability of occurrence * Threat Actor Jerk Coefficient * (Compensating Control - Cups of Coffee) - abs(Last time I took a vacation - Burnout)
Not all threat actors are jerks, but many jerks are threat actors

Aha! Mathematics saves the day yet again! But here is the problem: humans are generally bad at absolute measurement.

Exhibit A is a study about how different outsourcing companies could give an absolute estimate of hours of work provided a set of specifications. It turns out they were all over the place. Culture, bias, expertise, and specification details (or lack thereof) all played a role in how firms decided the cost of doing a project. So what can we do instead?

Security has to learn yet again from Dev and Ops: relative measurements to the rescue!

Estimating work for software is difficult. But developers and operations folks figured this out long ago with a little thing called Agile Development. Rather than being all waterfall-y, you can get a couple of sprints under your belt and compare feature requests and bug reports with work you’ve already done!

"So I believe that a button on our website is a 2-point story because I worked on a slider for our website 2 sprints ago, and that was a 2-pointer, and the code is relatively the same.” - An unknown enlightened developer, Initech, 1998

Bingo! Relative measurement. Humans rock.

So how does this apply to alert severity? Well, we should embrace what we’ve done before. What does this specific rule look like compared to our library of rules? Similar TTPs, similar environment? Does it alert on a very specific case, or a broad case? These may seem like absolute measurements, but you'd be surprised how good humans are at taking very vague details like above and clustering them to other similar rules.

Once you can cluster that, it is easy to assign the severity because it’s just the same severity as the rules it was clustered in. And yeah, finger to the wind is totally fine here! No more math, make a justification of why your PHP Webshell rule is a Medium that seems awfully similar to an ASP.NET Webshell rule, and bam! Deploy away.

Alert severity can be alert urgency. Different name, similar outcomes

What does it mean for day-to-day operations to have an informational, low, medium, high, and critical alert now that you can cluster? Well, this is where documentation and an SLA come into play. I’m pretty bullish on treating security as a ship in the ocean rather than an immovable fortress. You have to keep the ship afloat. So how would you classify problems within the ship, with increasing levels of severity, that go from something to note, to something interesting, to something catastrophic?

Sailing analogies aside, I am impressed with how the DevOps profession solved this issue years ago. My employer has a great blog on Alert Urgency (not severity!) and classifies alerts as Records, Notifications and Pages. Atlassian has a 5 tier model that talks about service degradation and customer annoyance. Notice these all can fit into one page and can be quickly adapted.

Conclusion

TL;DR focus on relative measurement. There will be some upfront costs and pain, but it gets easier as time goes on. That pain is the same thing Agile teams go through with assigning points, but it quickly fades. Next, identify your severity model from a notification perspective rather than a threat perspective. You might realize they achieve the same thing, but without as much absolute measurement in trying to quantify threat scores.

Table stakes for Detection Engineering

Zack Allen — Fri, 18 Nov 2022 01:38:32 +0000

What is a rule, really?

Alucard will not be sold any snake oil

For as long as I have been in the security industry, there has been a concerted effort to sort through massive troves of data with powerful and mysterious tools called “rules”. It allows us mere mortals to take a million line logfile and separate each line into two buckets: interesting or not interesting, malicious or not malicious, vulnerable or not vulnerable. If you know what “bad” or “vulnerable” is, then you can codify it and let the computer do the sorting for you.

I cut my teeth in security research writing WAF rules for modsecurity and looking for interesting HTTP-based attacks on behalf of a customer base. I also launched the security detection and research team at startups that are now public. At my current gig, I help my organization write detection content against 100s of data sources with terabytes of cloud-based control-plane and data-plane events flowing through our systems. Seeing how detection and research have evolved in my 10+ year career has been rewarding and tiring.

The security elders at my previous companies would scoff at my WAF rules and talk to me about a time when vulnerability scanner rules were the only thing that mattered. A team of researchers would feverishly comb through binaries and RE tools like the Matrix, and when they would find a vulnerability, they would rush out a rule so their company would be the first to disclose it and have a detection before their competitors.

A security researcher from McAfee deploys a new rule to their vulnerability scanner (2003, colorized)

At the end of the day, this fell in the realm of "security research". Companies would scoop up new grads and oldheads alike, put them on a security research team, and put them to work. They would then measure how many rules and detections they could push into production in a month, and hopefully it was enough to claim that their products protected customers from more attacks and vulnerabilities than their competitors.

This go-to-market strategy can be effective but suffers diminishing returns. It begs the question: why is "more" better, and why is "lots more", lots better? In the age of vulnerability scanners, more rules meant more vulnerabilities being detected. This translates into having better coverage, which is a great sales statistic. The same pervasiveness of coverage crept into threat detection products, but threats are not equal to vulnerabilities. Sure, you want to have coverage against an overwhelming number of threats, but is that going to help protect you and your firm? Can you do “all” threats, forever? More than a competitor, more than a threat actor? Probably not.

This culture of more is better has caused burnout and pain for researchers at these companies. It doesn't matter if you wrote an exceptional rule that was relevant, contextual, and precise: it carried the same weight as another bad rule with bad results within the game of quotas. When detection counts go up, the sales engine gets revved up, and they rush to their pipelines to close more deals.

Detorction rules are like stonks, they can only go up

Threat detection is dead. Long live threat detection!

The security research team in these times (maybe not as much now, but I have recency bias) was treated like wizards. They were the identity of the company. They had cringe-inducing named research teams, such as the IBM Hacker Ninjas or the McAfee Alpha Bro Exploiter Extraordinaires. The wizards would come down from their spire and preach to the world their latest findings, present at Blackhat and DEFCON, then afterward head back up the spire and close the door behind them. Their rules, research, and detections would then be left for other people to deal with. They had bigger things to worry about, like writing more rules to hit that damn quota.

In my opinion, this concept of "more is better" for detection rules is a sign that a company's go-to market is either a) stuck in the past of vulnerability research coverage or b) don't know what they are doing so they just do as much as possible to hide that fact. Believe me, I was part of this a few times in my career.

Now, I am not saying that you shouldn’t crank rules out for the sake of coverage. There are legitimate reasons to write, deploy and maintain a vast ruleset. What I am saying is that I think we got into this mess because we still continue to focus too much on deploying as much as possible, rather than thinking about the problems we are trying to solve. And the more I get into my career, the more I realize that I can’t blame sales or marketing people for this problem, this is the security research team’s fault.

When a company relies heavily on a research team to pump out content, they need to make sure that the team has the right people supporting them so they can focus on the nuances of security detection. Company’s should provide access to project management resources, software engineering capabilities to scale rule writing efforts and infrastructure, and consider the impact of rules using tried and tested methods in everyone’s favorite high school class: statistics. And no, I am not saying machine learning.

I think the industry is starting to see that security detection and research, for the sole purpose of writing threat detection rules, is evolving into a new and exciting type of security engineer: the Detection Engineer!

Detection Engineering is the new hotness but requires solid foundations in more than just security subject matter expertise

Detection Engineering, in my opinion, is the next level of security research. It's an evolution because companies have realized that it's more scalable to require security researchers to have skills in software engineering, project management, and statistics before they are released to go work through a detection backlog. If you want to scale your detection program, you need to hire a Detection Engineering team that can complement each other in the following areas:

Subject matter expertise in security
Software engineering
Statistics

That's it. That's all you need. Of course, this list can be picked apart, stretched, and folded under other areas like DevOps or Infrastructure, but at the end of the day, these 3 pillars can get you far without having to hire a ton of bodies.

You can't write detections for your network security product if you don't have network security experts. This is the same for endpoint, cloud, application and host-based detections. It’s like having a bunch of data scientists build a machine learning model to detect asthma in patients but they forgot to bring in a doctor to show them how pneumonia patients would give the model false positives. You need the subject matter experts. This has not changed in the industry, nor should it.

What has changed is that these experts need a solid basis in software engineering principles. You can't scale all of those detections and deploy them in a modern environment, manage sprints (yes this is software engineering :)), or write unit, integration, and regression tests without lots of bodies or lots of automation. I can reliably say my boss would rather hear that I can scale the problem away with software than with hiring more people.

Lastly, and I think this is the next step in the evolution of security research to detection engineering: we all must improve the explainability, and thus impact, of our rules, and statistics is how you do it. You can't reliably create, improve, deprecate or justify your detections to your sales teams, internal leadership, or customers without a background in statistics. This does not mean you need a graduate degree, but I think if security engineers and researchers spent some time looking at concepts like sampling bias and error, confusion matrices, precision and recall, they could better understand how rules perform under certain conditions and spot errors much earlier on before a rule hits production.

The more you learn, the more you realize you don't know anything

Conclusion

I am excited to see these 3 pillars being talked about more in the detection engineering and security realm. It shows how much we've matured as an industry. I wrote this post as a rant but also as a warning: do not do what I did. Do not fall victim to the "more is better" farce. I have a few more post ideas going into detail on what separates a good detection from a great detection (my team asks this question all the time), or what a go-to-market strategy for security detection rules should be (it's covering the right things, not more things). But for now, my parting advice for aspiring researchers and engineers is this Einstein quote:

"If I had only one hour to save the world, I would spend fifty-five minutes defining the problem, and only five minutes finding the solution."

Also, turns out, Einstein may not have said this, but the premise is still great. We write solutions (detections) trying to find problems (threats) without focusing on the problem (threat) beforehand. Don't do what I did. Don't commit to a quota!

Table stakes for Detection Engineering

Zack Allen — Tue, 18 Oct 2022 00:56:41 +0000

What is a rule, really?

Dracula refuses a call with a security vendor

The security elders at my previous companies would scoff at my WAF rules. They would talk to me about a time when vulnerability scanner rules were the only thing that mattered. A team of researchers would feverishly comb through binaries and RE tools like the Matrix. When they would find a vulnerability, they would rush out a rule so their company would be the first to disclose it and have a detection before their competitors.

A security researcher from McAfee deploys a new rule to their vulnerability scanner (2003, colorized)

At the end of the day, this fell into the realm of "security research". Companies would scoop up new grads and old heads alike, put them on a security research team, and put them to work. They would then measure how many rules and detections they could push into production in a month. Hopefully, it was enough to claim that their products protected customers from more attacks and vulnerabilities than their competitors.

This go-to-market strategy can be effective but suffers diminishing returns. It begs the question: why is "more" better, and why is "lots more", lots better? In the age of vulnerability scanners, more rules meant more vulnerabilities being detected. This translates to better coverage, which is a great sales statistic. The same pervasiveness of coverage crept into threat detection products, but threats are not equal to vulnerabilities. Sure, you want to have coverage against an overwhelming number of threats, but is that going to help protect you and your firm? Can you do “all” threats, forever? More than a competitor, more than a threat actor? Probably not.

This culture of more is better has caused burnout and pain for researchers at these companies. It doesn't matter if you wrote an exceptional rule that was relevant, contextual, and precise: it carried the same weight as another bad rule with bad results within the game of quotas. When detection counts are up, the sales engine gets revved up, and they rush to their pipelines to close more deals.

Detorction rules are like stonks, they can only go up

Threat detection is dead. Long live threat detection!

The security research team in these times (maybe not as much now, but I have recency bias) was treated like wizards. They were the identity of the company. They had cringe-inducing named research teams, such as the IBM Hacker Ninjas or the McAfee Alpha Bro Exploiter Extraordinaires. The wizards would come down from their spire and preach to the world their latest findings, present at Blackhat and DEFCON. Afterwards, they would head back up the spire and close the door behind them. Their rules, research, and detections would then be left for other people to deal with. They had bigger things to worry about, like writing more rules to hit that damn quota.

In my opinion, this concept of "more is better" for detection rules is a sign that a company's go-to market is either a) stuck in the past of vulnerability research coverage or b) doesn't know what they are doing so they just do as much as possible to hide that fact. Believe me, I was part of this a few times in my career.

Now, I am not saying that you shouldn’t crank them out for the sake of coverage. There are legitimate reasons to write, deploy and maintain a vast ruleset. What I am saying is that I think we got into this mess because we think more coverage is more secure. This fallacy can lead internal teams, or in my case a product detection team, down rabbit holes that aren't fruitful in the longrun. And the more I get into my career, the more I realize that I can’t solely blame sales or marketing people for this strategy. It's up to us, the researchers, to let them know which path is the more fruitful and why.

When a company relies heavily on a research team to pump out content, they need to make sure that the team has the right people supporting them. This will enable the team to focus on the nuances of security detection. Companies should provide access to project management resources, software engineering capabilities to scale rule writing efforts and infrastructure, and consider the impact of rules using tried and tested methods in everyone’s favorite high school class: statistics.

I think the industry is starting to see that security detection and research, for the sole purpose of writing threat detection rules, is evolving into a more advanced and exciting type of security engineer: the Detection Engineer!

Detection Engineering is the new hotness but requires solid foundations in more than just security subject matter expertise

Detection Engineering, in my opinion, is the next level of security research. It's an evolution because companies have realized that it's more scalable to require security researchers to have skills in software engineering, project management, and statistics. If you want to scale your detection program, you need to hire a Detection Engineering team that can complement each other in the following areas:

Subject matter expertise in security
Software engineering
Statistics

That's it. That's all you need. Of course, this list can be picked apart, stretched, and folded under other areas like DevOps or Infrastructure. However, at the end of the day, these 3 pillars can get you far without having to hire a ton of bodies.

You can't write detections for your network security product if you don't have network security experts. This is the same for endpoint, cloud, application and host-based detections. It’s like having a bunch of data scientists build a machine learning model to detect asthma in patients. However, they forgot to bring in a doctor to show them how pneumonia patients would give the model false positives. You need the subject matter experts. This has not changed in the industry, nor should it.

The more you learn, the more you realize you don't know anything

Conclusion

"If I had only one hour to save the world, I would spend fifty-five minutes defining the problem, and only five minutes finding the solution."

Deploying Django Rest Framework with Postgres on fly.io

Zack Allen — Thu, 30 Dec 2021 22:45:18 +0000

Background

In this post, I will be demonstrating how to deploy a Django Rest Framework (DRF) application on fly.io. DRF is built on top of Django and is my choice when it comes to building small to massive APIs. I have used it extensively professionally and personally, and as a cybersecurity researcher, it's been my favorite to build as a backend for some complex applications that track malicious actors and their infrastructure. I noticed the fly website did not have a Django example, so I wanted to provide the community with a template to get started to avoid the headaches that I endured building this :)

You should have an understanding of Docker, docker compose, DRF and postgres for this post. Although it's possible to use this template without too much knowledge of these concepts, you'd get a ton of knowledge going through the following tutorials:

Why DRF?

From the DRF website:

Some reasons you might want to use REST framework:

-  The Web browsable API is a huge usability win for your developers.
-  Authentication policies including packages for OAuth1a and OAuth2.
-  Serialization that supports both ORM and non-ORM data sources.
-  Customizable all the way down - just use regular function-based views if you don't need the more powerful features.
-  Extensive documentation, and great community support.
-  Used and trusted by internationally recognised companies including Mozilla, Red Hat, Heroku, and Eventbrite.

The biggest issue with DRF is that it can be somewhat challenging to get up and running (can be lots of up front work). My experience has been that once you get a good template for an API up, it's the fastest to build, most scalable and most intuitive ORM.

Why fly.io?

I've used Heroku for one-off apps for a few years. Fly came across my Twitter feed and I've been following it closely. The company has a great section on why you should use fly, and this section caught my eye and has kept me interested ever since I read it:

Despite the benefits of location-smart, time-agile and cloud-clever applications, there’s been no good platform for building applications that work like this. This is what Fly has set out to fix. In the process we want to make application distribution platforms as ubiquitous as CDNs.

You can think of fly as a Heroku competitor, although some folks might disagree with me. I like it because it does what it says it does well, is focused, and isn't as bloated as the Heroku stack.

Getting started

Bookkeeping

We'll be making a DRF app, the Silly Simple API, or ss-api for short, on fly. This will have the following features:

Postgres backend, courtesy of fly
No session authentication, only using Tokens (you can use some tricks from fly to manage this). This is especially nice for APIs and simplifies authentication to a token, which I prefer for microservices
Swagger and OpenAPI capabilities using drf-yasg, where you can only see endpoints and Swagger docs if you have a valid Token
TCP & HTTP health checks using fly. The HTTP health check will be somewhat useful by issuing a query to our DRF app under /ping/, which connects to the DB and issues an innocuous select 1 statement to make sure things are working
Using docker with a Dockerfile and gunicorn to launch the app. The cool thing about fly is that you can give it a Dockerfile and a fly.toml and you have a full-fledged app running on their infrastructure

This app will only have 1 endpoint, users, that you can use to manage your users. You must be authenticated to see it.

I will leave the following for later blog posts:

Metrics exposure and app tracing via Datadog (p.s., we're hiring https://www.datadoghq.com/careers/)
Tables for an app (this isn't a tutorial on building DRF apps, rather, to get you a template to get started)
Scaling primitives in DRF (avoiding n+1, indexing, replicas for postgres)

If this all sounds interesting for you still, let's get started :D

Clone and run locally

Make sure to have the following installed locally:

Latest Docker (with docker compose)
git
fly via here

then clone from the repo here:

git clone git@github.com:zmallen/ss-api.git

Run docker compose up and connect locally by navigating to:

http://localhost:8000

The docker-compose.yml file overrides the RUN command in the Dockerfile by issuing the following command on every docker compose up:

command: bash -c "python manage.py migrate && python manage.py runserver 0.0.0.0:8000"

This will differ from when we deploy on fly.io, where we use gunicorn to serve the app and we run migrations manually.

Token Auth & Authentication in DRF

Unauthenticated users can only see the /ping/ endpoint on Swagger. This is by design - the app will render endpoints based on permission, and under /ping/views.py on Line 16, the permission for this endpoint is:

permission_classes = (AllowAny,)

Compare this to ssapi/views.py, under the UserViewSet on Line 17:

authentication_classes = (authentication.TokenAuthentication,)

This is achieved via some magic in settings.py Lines 88-111:

# DRF settings
REST_FRAMEWORK = {
    "DEFAULT_PERMISSION_CLASSES": ("rest_framework.permissions.IsAuthenticated",),
    "DEFAULT_AUTHENTICATION_CLASSES": (
        "rest_framework.authentication.TokenAuthentication",
    ),
    "DEFAULT_RENDERER_CLASSES": ("rest_framework.renderers.JSONRenderer",),
}

# SWAGGER_SETTINGS
SWAGGER_SETTINGS = {
    "USE_SESSION_AUTH": False,
    "LOGIN_URL": "rest_framework:login",
    "LOGOUT_URL": "rest_framework:logout",
    "VALIDATOR_URL": None,
    "SECURITY_DEFINITIONS": {
        "api_key": {
            "type": "apiKey",
            "name": "Authorization",
            "in": "header",
        },
    },
    "REFETCH_SCHEMA_WITH_AUTH": True,
}

Under DRF_SETTINGS, I forced TokenAuthentication and isAuthenticated for viewing endpoints, so no more sessions! 🎉🎉

So how do you get an API token if you can't authenticate? This is where some magic with manage.py comes into play.

Authenticating - `getdevtoken` and `/users/`

To get a local API key, run the following command in a separate tab, in the same directory as docker-compose.yml:

└> docker compose run api python manage.py getdevtoken
[+] Running 1/0
 ⠿ Container ss-api-db-1  Running                                          0.0s
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(｡◕‿‿◕｡)☞☞  Token 9884a551be31b80a61b49becf7c3640224a9ec42  ☜☜(｡◕‿‿◕｡)

Note, this Token is for my local deployment :)

Copy the Token abc and navigate over to your browser, then click 'Authorize', paste and press submit. You should get the /users/ endpoint to return in the Swagger frontend, and issuing a GET request will list the superuser!

You can do a lot more with Swagger documentation than just the defaults, I suggest checking out these resources to learn about Swagger docs in Django:

Deploying to fly

Everything is running smoothly in your local environment, now let's get it to a prod environment!

First, we need to create a toml file for fly. This is a configuration file used by fly to deploy your app. I generated one within the Github repo, but you can explore how to generate and configure other toml files on fly here.

A few things need to happen to finish our "deploy to prod" for ss-api:

Create a fly app
Create a postgres db with fly, and retrieve the DATABASE_URL string
Set a fly secret with the DATABASE_URL from Step 1 so our app can dynamically render the secret in a fly environment and use the db created in step 1
Deploy the app on fly, make sure TCP & HTTP health checks pass (they wont on first pass :D)
Create a database and run a migration using fly ssh
Get a devtoken for prod

Step 1: Create your fly app

Simply run fly create and name your app!

└> fly create
? App Name: ssapiblog
automatically selected personal organization: Zack Allen
New app created: ssapiblog

Step 2: postgres

Launch a new postgres instance via fly with fly postgres create. Accept all the defaults (minimal DB settings, aka the cheapest!) and wait for fly to give you the DATABASE_URL.

└> fly postgres create
? App Name: ssapidb
Automatically selected personal organization: Zack Allen
? Select region: iad (Ashburn, Virginia (US))
? Select VM size: shared-cpu-1x - 256
? Volume size (GB): 10
Creating postgres cluster ssapidb in organization personal
Postgres cluster ssapidb created
  Username:    postgres
  Password:   SECRETPASSWORD
  Hostname:    ssapidb.internal
  Proxy Port:  5432
  PG Port: 5433
Save your credentials in a secure place, you won't be able to see them again!

Monitoring Deployment
...
...
2 desired, 2 placed, 2 healthy, 0 unhealthy [health checks: 6 total, 6 passing]
--> v0 deployed successfully

Connect to postgres
Any app within the personal organization can connect to postgres using the above credentials and the hostname "ssapidb.internal."
For example: postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432

See the postgres docs for more information on next steps, managing postgres, connecting from outside fly:  https://fly.io/docs/reference/postgres/

You want the 12factor string after For example:, which in this example is:
postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432

Step 3: Set DATABASE_URL as a `fly secret`

You want to set the DATABASE_URL with a 12factor string from before, as well as a database name (which we will create).

Note the /ssapidb at the end of the DATABASE_URL

fly secrets set DATABASE_URL="postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432/ssapidb"

Step 4: Deploy your app

Change the following line in fly.toml to whatever you want:

app = "ssapiblog"

Run fly deploy:

└> fly deploy 
Deploying ssapiblog
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8000
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.8 linux x86_64
Sending build context to Docker daemon  153.1kB
...
==> A bunch of Docker output

You can detach the terminal anytime without stopping the deployment
Monitoring Deployment

v0 is being deployed
2021-12-30T22:26:21.000 [info] 145.40.89.203 - - [30/Dec/2021:22:26:21 +0000] "GET /ping/ HTTP/1.1" 500 114326 "http://172.19.10.66:8000/ping" "Consul Health Check"

1 desired, 1 placed, 0 healthy, 0 unhealthy [health checks: 2 total, 1 passing, 1 critical]

Step 5: Build a DB then migrate

Notice how 1 health check is passing (tcp), and 1 is critical. This is because we did not do a database migration and our /ping healthcheck is failing. This will most likely fail after a certain amount of time, so in a separate tab navigate to the project directory to run a few fly ssh commands.

First, make the ssapidb database by running a handy dandy bash script I added into the repo. We can use this via fly ssh console -C command:

└> fly ssh console -C 'bash /app/provision_db.sh'
Connecting to ssapiblog.internal... complete
Database does not exist. Creating now..
CREATE DATABASE

If you run fly logs in a separate tab, you should see Consul health checks returning 200, which is healthy \o/:

2021-12-30T22:31:27.116 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping HTTP/1.1" 301 0 "-" "Consul Health Check"
2021-12-30T22:31:27.119 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping/ HTTP/1.1" 200 2 "http://172.19.10.66:8000/ping" "Consul Health Check"

Let's migrate and get a devtoken:

└> fly ssh console -C 'python /app/manage.py migrate'
Connecting to ssapiblog.internal... complete
Operations to perform:
  Apply all migrations: admin, auth, authtoken, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying authtoken.0001_initial... OK
  Applying authtoken.0002_auto_20160226_1747... OK
  Applying authtoken.0003_tokenproxy... OK
  Applying sessions.0001_initial... OK

Step 6: Get your devtoken and open the app

└> fly ssh console -C 'python /app/manage.py getdevtoken'
Connecting to ssapiblog.internal... complete
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(｡◕‿‿◕｡)☞☞  Token TOKEN  ☜☜(｡◕‿‿◕｡)

Woot! Run fly open and go through the same workflow as your local deployment: put Token TOKEN into the Authorize panel, and you can now see the authenticated users endpoint, and issue a GET request to get your token!

If you want to add more users, just use the POST request endpoint here to create a new user.

Conclusion

I enjoyed writing this app and this blog post! Fly is a cool concept and I will definitely play with it more. There are some sharp edges with DRF, so I tried to simplify it, but please study the DRF tutorials and the api/settings.py file for other configuration options I used.

For my next posts, Im looking to develop an app to do some basic cybersecurity threat intelligence tracking and correlation. If you have ideas for other apps, or have a question on this app, please leave a comment or open an issue on the ss-api repo here:

https://github.com/zmallen/ss-api

Cross-post: Threat Detection Opportunities Learned from the Ubiquiti indictment

Zack Allen — Wed, 22 Dec 2021 15:09:44 +0000

My coworker and I just published a blog post for Datadog on the techniques used by the Ubiquiti hacker. We go into detail on different threat detection opportunities you can leverage in Amazon to catch a highly technical and malicious insider.

Check it out!

https://dev.to/anskedatadog/threat-detection-opportunities-learned-from-the-ubiquiti-indictment-56i8

Forem: Zack Allen

Absolute measurement corrupts alert severity, absolutely

What’s the problem here?

Alert severity explainability

Security has to learn yet again from Dev and Ops: relative measurements to the rescue!

Alert severity can be alert urgency. Different name, similar outcomes

Conclusion

Table stakes for Detection Engineering

What is a rule, really?

Threat detection is dead. Long live threat detection!

Detection Engineering is the new hotness but requires solid foundations in more than just security subject matter expertise

Conclusion

Table stakes for Detection Engineering

What is a rule, really?

Threat detection is dead. Long live threat detection!

Detection Engineering is the new hotness but requires solid foundations in more than just security subject matter expertise

Conclusion

Deploying Django Rest Framework with Postgres on fly.io

Background

Why DRF?

Why fly.io?

Getting started

Bookkeeping

Clone and run locally

Token Auth & Authentication in DRF

Authenticating - getdevtoken and /users/

Deploying to fly

Step 1: Create your fly app

Step 2: postgres

Step 3: Set DATABASE_URL as a fly secret

Step 4: Deploy your app

Step 5: Build a DB then migrate

Step 6: Get your devtoken and open the app

Conclusion

Cross-post: Threat Detection Opportunities Learned from the Ubiquiti indictment

Authenticating - `getdevtoken` and `/users/`

Step 3: Set DATABASE_URL as a `fly secret`