Friday, July 25, 2025

Writing KQL Queries to Analyse Azure Costs in Log Analytics

Azure Cost Management provides excellent visual tools for cost analysis, but there are scenarios where custom querying gives more flexibility, particularly when you need to correlate cost data with operational logs or produce custom aggregations that the portal does not surface directly.

Log Analytics supports several tables relevant to cost analysis. This post covers the key tables and practical KQL queries for understanding Azure spend from within a Log Analytics workspace.

1. The Usage Table

The Usage table in Log Analytics records data ingestion volumes per data type. This is the primary table for understanding Log Analytics workspace costs, as ingestion pricing is based on the volume of data written.

Following is a query to identify the top data types by ingestion volume over the past 30 days:

Usage
| where TimeGenerated > ago(30d)
| summarize TotalGB = sum(Quantity) / 1024 by DataType
| order by TotalGB desc
| take 20

This query is particularly useful when a workspace bill has increased unexpectedly. It quickly identifies which log source is responsible.

2. Estimating Ingestion Cost by Data Type

The Usage table volume can be converted to an approximate cost estimate using the current Log Analytics pricing for your region. As of 2025, Pay-As-You-Go ingestion in Australia East is approximately $3.63 per GB.

Usage
| where TimeGenerated > ago(30d)
| summarize TotalGB = sum(Quantity) / 1024 by DataType
| extend EstimatedCostAUD = round(TotalGB * 3.63, 2)
| order by EstimatedCostAUD desc

Note that this is an approximation. Commitment tiers and reserved capacity will produce different effective rates. Use this query for relative comparisons between data types rather than absolute billing reconciliation.

Understanding whether ingestion is growing, stable, or declining helps with capacity planning and budget forecasting.

Usage
| where TimeGenerated > ago(90d)
| summarize DailyGB = sum(Quantity) / 1024 by bin(TimeGenerated, 1d)
| render timechart

Rendering as a timechart in the Log Analytics query editor provides an immediate visual of ingestion trends. A sustained upward trend warrants a review of diagnostic settings and data collection rules.

4. Identifying Resources Generating the Most Logs

When a specific data type has high ingestion volume, the next step is identifying which resources are generating it. The following query works for AzureDiagnostics, which is commonly the largest data type in environments with many Azure services:

AzureDiagnostics
| where TimeGenerated > ago(7d)
| summarize EventCount = count(), EstimatedGB = count() * 0.0005 by ResourceType, Resource
| order by EstimatedGB desc
| take 20

The 0.0005 multiplier is a rough approximation of average event size in GB. The primary value of this query is identifying the top contributing resource types, not a precise cost figure.

5. Running Queries from the Portal

All queries above can be run directly from Log Analytics workspace > Logs. Results can be:

  • Exported to CSV using the Export button
  • Pinned to an Azure Dashboard using Pin to dashboard
  • Saved as a named query using Save > Save as query for reuse by the team
  • Converted to an alert rule using + New alert rule directly from the query toolbar

Summary

KQL queries against the Usage table provide a level of cost analysis granularity that the Cost Management portal does not offer natively, particularly for diagnosing workspace ingestion costs and correlating data volumes with specific resources.

Friday, June 20, 2025

AgentCon Perth - 20-06-2025

I had the privilege of helping organize the AgentCon Perth event, which drew an impressive turnout of over 300 AI enthusiasts.

We had an outstanding lineup of speakers who shared insights on Agentic AI and how the Microsoft ecosystem can be leveraged to harness AI capabilities



Thursday, June 19, 2025

Azure Savings Plans vs Reservations: When to Use Which

Commitment-based discounts are among the most impactful cost optimisation levers available in Azure. Both Azure Savings Plans and Azure Reservations offer significant discounts over pay-as-you-go pricing, but they serve different scenarios and the distinction is not always obvious.

This post outlines what each commitment type offers, their key differences, and a practical framework for deciding which to use.

1. What Is an Azure Reservation

An Azure Reservation is a commitment to a specific resource type, size, and region for one or three years. In exchange, Azure offers discounts of up to 72% compared to pay-as-you-go.

Reservations are available for a wide range of services including:

  • Virtual Machines (e.g., Standard_D4s_v5 in Australia East)
  • Azure SQL Database and Managed Instance
  • Azure Cosmos DB
  • Azure Blob Storage (reserved capacity)
  • Azure Machine Learning compute clusters

The key characteristic of a reservation is its specificity. A reservation for a Standard_D4s_v5 in Australia East applies only to that VM size in that region. It does not apply to a Standard_D8s_v5 or a VM in Southeast Asia.

2. What Is an Azure Savings Plan

Azure Savings Plans, introduced in 2022, offer a more flexible commitment model. Rather than committing to a specific resource, you commit to a fixed hourly spend (for example, $5/hour) across eligible compute services for one or three years.

The savings plan discount applies automatically to any eligible compute usage across VM sizes, regions, and even across Azure services such as AKS, Azure Functions, and App Service.

Savings Plan discounts are slightly lower than equivalent reservation discounts (typically 15–65% depending on the term and service), in exchange for the flexibility.

3. Key Differences

DimensionReservationSavings Plan
CommitmentSpecific resource, size, regionHourly spend amount
FlexibilityFixed to a specific resource and regionApplies across VM sizes, regions, and services
DiscountUp to 72%Up to 65%
Best forStable, predictable workloadsDynamic or diverse compute workloads
Exchange/RefundAllowed with limitsNot exchangeable

Use a Reservation when:

  • The workload has run at consistent size and scale for at least 60 days
  • The resource is tied to a specific region due to latency or data residency requirements
  • The service is reservation-eligible and you can accurately forecast utilisation above 80%

Use a Savings Plan when:

  • The workload spans multiple VM sizes or regions (common with autoscaling environments)
  • The team is early in cloud maturity and forecasting specific resource consumption is not yet reliable
  • You need a simpler commitment model that covers future architectural changes

5. Reviewing Recommendations Before Purchasing

Azure Advisor provides reservation and savings plan recommendations based on your actual usage history. I recommend reviewing these before making any commitment.

Navigate to Azure Advisor > Cost to see recommendations including estimated annual savings, recommended term, and utilisation confidence based on the past 30 days of usage.

Summary

Reservations deliver the highest discount for stable, well-understood workloads. Savings Plans offer flexibility for environments that change frequently. In practice, a combination of both often yields the best outcome: reservations for core infrastructure and savings plans for dynamic compute. Azure Advisor removes much of the guesswork by surfacing recommendations backed by actual usage data.

Wednesday, May 14, 2025

Automating Monthly Cost Reports Using Azure Cost Management Exports

Manually reviewing Azure costs through the portal each month is time-consuming and inconsistent. Cost Management Exports solve this by delivering structured cost data to a storage account on a scheduled basis, enabling automation, Power BI reporting, and finance integration without manual intervention.

This post walks through setting up scheduled exports and accessing the exported data for downstream processing.

1. What Are Azure Cost Management Exports

Azure Cost Management Exports are scheduled jobs that write cost and usage data as CSV files to an Azure Blob Storage container. Exports can be configured at the subscription, resource group, or management group scope.

There are three export types available:

  • Daily export of month-to-date costs: appends data daily, useful for real-time tracking
  • Weekly export of the last 7 days: a rolling weekly snapshot
  • Monthly export of the last billing month: the most common choice for finance reporting

2. Creating a Scheduled Export

  1. Navigate to Cost Management + Billing > Cost Management > Exports
  2. Select + Add
  3. Provide an Export name and select the Export type (Monthly is recommended for finance use cases)
  4. Under Storage, select an existing storage account or create a new one
  5. Specify a Container name and Directory path. A path such as cost-exports/monthly keeps exports organised
  6. Select Create

Following is the recommended storage account configuration for cost exports:

  • Redundancy: LRS is sufficient, as this data can be regenerated
  • Access tier: Cool, since cost data is written once and read infrequently
  • Soft delete: Enabled with a 7-day retention window

Once created, the export runs automatically on schedule. You can also select Run now to trigger an immediate export for testing.y on schedule. You can also select Run now to trigger an immediate export for testing.

3. Accessing the Exported Data

Exported files are delivered to the storage container as CSV files named with a date suffix, for example:

cost-exports/monthly/20250501-20250531/export_20250601.csv

Each row represents an individual resource's cost for the period, including fields such as ResourceId, ResourceType, ResourceGroupName, MeterCategory, CostInBillingCurrency, and any resource tags.

Following is the recommended approach for accessing the data:

  • Azure Storage Explorer: for ad hoc review and validation
  • Power BI: connect directly to the blob container using the Azure Blob Storage connector for monthly dashboards
  • Logic App or Azure Function: for automated processing such as sending cost summaries by email or posting to a Teams channel

4. Filtering and Enriching Export Data

The raw CSV export includes all resources. For team-level reporting, filter rows by the CostCenter or Team tag columns. Consistent tagging is therefore a prerequisite for meaningful cost reports.

If tags are missing from a significant portion of resources, review the tagging coverage in Azure Policy > Compliance before relying on exports for chargeback purposes.

Summary

Azure Cost Management Exports provide a reliable, low-maintenance mechanism for delivering cost data to storage for downstream reporting and automation. Configuring a monthly export early in a project's lifecycle ensures that historical data is available when finance teams need it, removing the need for manual extraction from the portal each month.