Diagnose webtraffic in Azure using KQL and Log Analytics

In an older post we discussed the benefits of using and protecting your web applications with Azure Web Application Firewall, you can find the post in full here
We touched on some best practices which were summarized in the post:
- Best Practices: I recommend using Detection mode rather than Prevention mode (actively blocks requests) initially to catch false positives and use diagnostics logging to review traffic in a Log Analytics Workspace.
Today we will expand on the topic of reviewing traffic logs using Log Analytics and dive into some KQL-queries.
Pre-requisites
First off you need a log analytics workspace and a web application firewall instance with diagnostic settings forwarding logs to said workspace. If you want to quickly deploy this I have created the resources needed in this repository
The following objects will be deployed into one Resource Group so we can easily delete everything afterwards:

In order to deploy the code you need Terraform and the Azure CLI installed on your machine.
az login
az account set -s <sub-id>
git clone git@github.com:carlzxc71/waf-diagnostic-deployment.git
cd /waf-diagnostic-deployment/terraform
terraform init
terraform apply -auto-approve
Additional configuration
Once we have deployed our resources there is some additional configuration that we need to complete.
- In the Azure Portal search for Application Gateway and select our new load balancer
- Select Diagnostic Settings and select Add diagnostic setting

- Select Save
It will take a few minutes for logs to start to ingest into the workspace. I should also note that this post is not aimed at configuring a proper loadbalancer infront of a website type of guide. This is merely to deploy resources and provide you some configuration out of the box to get to play with actual KQL-queries and see why they are useful with WAF.
Browse the logs
So after a few minutes you should see some logs starting to populate in your workspace if you search for log-prod-we-webdemo and expand LogManagement

As you can see I have run a simple AGWFirewallLogs
command and I can see that I have a match or hit for a rule.
Match means that the web application firewall has matched an incoming request with a rule in the managed ruleset, in this case OWASP. If we search for the RuleId which in this case is 920350 in the WAF we can see what rule it is:

In this case it is complaining about the host header being an IP address. This makes sense in our case as I browsed to the public IP of the load balancer to get to the application service but in a production application you would not want this.
Action Anomaly score means that this client IP is adding upp points to their anomaly score for when it reaches a certain threshhold where it has added up enough points the requests will be blocked. This means that occassional anomalies will not be blocked but repeated offenses gets taken care of.
Once the score is 5 or greather and your WAF is in prevention mode as we have deployed the request is blocked
So if you are investigating the logs then, what are some useful queries you can use? Well here are some I use.
To find requests coming from a specific IP where the action is Blocked (You could change to Matched as well)
AGWFirewallLogs
| where ClientIp startswith "<IP>"
| where Action startswith "Blocked"
To search for hits for specific rules
AGWFirewallLogs
| where RuleId startswith "<rule-ID>"
You can then combine the two above
AGWFirewallLogs
| where ClientIp startswith "<IP>"
| where Action startswith "Blocked"
| where RuleId startswith "<rule-ID>"
Find hits generated from certain RequestUri (Useful if certain functions in your app services is not working)
AGWFirewallLogs
| where RequestUri startswith "/images"
There you go with some useful tips and trix. As you can see these are some pretty basic but very powerful examples of queries you can leverage, just imagine what more complex things you can do.
Of course this is not limited to Web Application Firewall you have resource logs for a whole host of Azure Resources such as Firewall, Virtual Machines, Storage Accounts and so forth..
References

Read more about Managed Rule sets here
About me
