Streamlining Web Application Diagnostics with Azure Functions and Headless Chrome

Discover how to enhance web application performance monitoring and testing using Azure Functions and headless Chrome. Our solution captures comprehensive diagnostic data, including HAR files, screenshots, performance profiles, and memory dumps, all stored securely in Azure Blob Storage. Azure Service Bus ensures reliable and ordered message processing for efficient distribution of scan results.

In I approach, I chose Azure Service Bus over Event Grid due to its advanced messaging features, reliability, and seamless integration with Azure services. However, we remain open to insights from the Azure community. If Event Grid is more suitable for this use case, we welcome your feedback and suggestions.

Consumption Plan Execution
The Azure Function operates on a consumption plan, avoiding the need for premium plans or container-based deployments. This ensures cost-effective scalability, as the function only incurs charges based on actual compute usage during scan execution.

Platform-Independent Headless Chrome Installation
The challenge of installing headless Chrome in an Azure Function is resolved, allowing it to be installed on any plan and platform without requiring premium or container-based deployments. Installation will done during cold start of the function. This solution ensures compatibility and flexibility across different environments and allows us to use azure function consumption plan which is pure serverless.

Flow

Service Bus Topic (Scan Trigger)
Initiates Scan: The process starts when a scan trigger is initiated by an event in the Service Bus Topic. This can be based on specified events or schedules.

Azure Function Execution
The Azure Function, equipped with headless Chrome, is triggered by the scan initiation event. Upon the first trigger, the function app installs headless Chrome if it is not already installed. From the second execution onward, Chrome does not need to be reinstalled. This deferred installation process addresses compatibility issues that arise due to the OS/platform-dependent nature of Chrome installation when included as part of the function deployment. This approach ensures smooth execution across different environments.

Data Collection
The HAR files, Screenshots, Performance Profiles, and Memory Dumps are stored securely and reliably in Azure Blob Storage. Where as this storage provides scalable space for large amounts of data like screenshots and json blobs, perforce metrics and health specific data

Scan Result
After collecting the scan data, the results are sent to another Azure Service Bus Topic. This approach allows multiple subscribers to process the scan results simultaneously. For example, one subscriber can store the data in an Azure SQL Database or Cosmos DB for long-term storage and analytics. Another subscriber can process the screenshots using Machine Learning models and Computer Vision tools to detect visual anomalies or changes. Additionally, the results can be integrated with alerting systems to notify administrators of any detected issues, ensuring timely responses to potential problems.

Use cases

Visual Regression Testing
Capture and compare screenshots across different versions of the web application to detect visual regressions. To ensures that visual changes are intentional and that the user interface remains consistent across updates, enhancing user experience.

Network Performance Analysis
Analyse HAR files to understand network performance and identify bottlenecks or slow-loading resources. To provides detailed insights into network performance, helping to optimize load times and improve the overall responsiveness of the application.

Integration with Monitoring and Alerting Systems
Send scan results to monitoring and alerting systems for real-time analysis and notifications. To enhances proactive monitoring by integrating with existing systems, enabling real-time alerts and faster response to issues.

Synthetic Monitoring
Conduct synthetic scans to simulate user interactions and measure the performance and availability of the web application. To provides consistent and controlled testing scenarios to monitor application performance from different geographic locations and identify potential issues before they affect real users.

Azure Service Bus over Event Grid
Azure Service Bus is chosen over Event Grid for its advanced messaging capabilities, including message ordering, duplication detection, and robust reliability features such as guaranteed delivery, dead-letter queues, and message deferral. Service Bus supports complex workflows with patterns like request-response and publish-subscribe, and handles larger message sizes and complex payloads, making it ideal for detailed scan results. Its seamless integration with various Azure services and legacy systems provides a comprehensive messaging solution. I welcome insights from Azure experts if Event Grid is more suitable for this use case, recognizing the value of leveraging the most effective technology.

Terraform code
https://github.com/resilience-tech/insight-track/tree/main/packages/synthetic-scan-ic-tf

Linkedin Post
https://www.linkedin.com/pulse/streamlining-web-application-diagnostics-azure-functions-rayudu-weiie/