The Enterprise Snowflake Usage Monitor by Infocepts is a comprehensive app built using Streamlit. It enables organizations to monitor and analyze Snowflake account usage, costs, and performance metrics in real time. Designed for enterprise environments, this solution offers interactive visualizations and detailed analytics to help teams optimize their Snowflake operations. The Key features of the app include:
Account Overview
- Real-Time Metrics: View current credit usage, total jobs executed, and storage consumption.
- Credit Usage Trends: Analyze credit consumption patterns across different warehouses.
- Storage Analytics: Track storage usage over time and identify key contributors.
- Query Performance: Examine the longest-running successful and failed queries.
- Execution Monitoring: Visual breakdown of query success and failure rates.
- Query Type Analysis: Performance metrics segmented by query type.
- Storage Insights: Identify top tables by active and failsafe storage usage.
Warehouse Analytics
- Cost Breakdown: Detailed credit and cost analysis per warehouse.
- Role-Based Attribution: Understand cost distribution by user roles within each warehouse.
- Job Distribution: Analyze query execution patterns across warehouses.
- Variance Tracking: Monitor warehouse usage against 7-day rolling averages.
- Cloud Services Monitoring: Track cloud services credit usage per warehouse.
User Analytics
- Login Monitoring: Review user login activity and failure rates.
- Query Activity: Track query execution per user.
- Performance Metrics: View average query execution times by user.
Technology Stack
- Frontend: Streamlit
- Database: Snowflake (Account Usage Schema)
- Authentication: Snowflake Permissions Framework
System Requirements
- Python 3.7 or higher
- Active Snowflake account with necessary permissions
- Access to the Snowflake Account Usage Schema
Required Snowflake Privileges
To operate correctly, the application requires the following privileges:
- IMPORTED PRIVILEGES ON SNOWFLAKE DB
- Access to the following views in SNOWFLAKE.ACCOUNT_USAGE:
- METERING_HISTORY
- QUERY_HISTORY
- STORAGE_USAGE
- WAREHOUSE_METERING_HISTORY
- QUERY_ATTRIBUTION_HISTORY
- TABLE_STORAGE_METRICS
- LOGIN_HISTORY
- Additionally, the application uses Snowflake’s get_active_session() function, which requires:
- An active Snowflake session
- Valid authentication credentials
- Network connectivity to Snowflake
Dashboard Navigation
The dashboard is organized into three primary tabs:
Account Tab
- Displays overall usage metrics and trends
- Provides query performance insights
- Offers detailed storage utilization analytics
Warehouse Tab
- Presents warehouse-specific cost and usage metrics
- Tracks performance variance
- Highlights resource utilization
Users Tab
- Monitors user activity and login patterns
- Displays individual query performance metrics
Date Range Selection
Users can filter data using:
- Quick-select buttons for 7, 30, 60, 90, 180, and 365-day ranges
- A custom date picker for specific date ranges
- Default view loads data for the last 30 days
Key Metrics Explained
Credits
- Total Credits Used: Sum of compute credits consumed across the account
Storage
- Billable Storage (TB): Includes active, staged, and failsafe storage
- Active Storage: Data currently accessible
- Failsafe Storage: Data retained during the failsafe period
Query Performance
- Execution Time: Duration of query execution
- Success Rate: Percentage of successful queries
- Warehouse Distribution: Load distribution across warehouses
Performance Optimization Tips
- Use shorter date ranges to improve dashboard responsiveness
- Monitor warehouse usage during dashboard operation
- Consider implementing data caching for frequently accessed metrics
Contact
This application is developed by Infocepts, a global leader in enterprise data analytics and AI solutions. For implementation, support, customization, or integration services, please contact the Infocepts team.
Infocepts
