Overview
The ZigmaAI Metadata Manager is an enterprise-grade platform designed to streamline metadata discovery, management, and governance for modern data platforms. Built with security and scalability in mind, it provides intelligent automation for metadata generation while maintaining full control and auditability.
Key Features
- AI-Powered Generation: Automated metadata creation using advanced language models
- Multi-Source Integration: Support for Trino, DataHub, and various AI providers
- Role-Based Access: Secure admin and user access controls
- Bulk Operations: CSV upload and batch processing capabilities
- Audit Trail: Comprehensive logging and change tracking
- Flexible Deployment: Docker support with environment-based configuration
Getting Started
Step 1: Connect to Data Sources
Load catalogs from your Data Sources, select a catalog and schema, then load the tables you want to manage.
Step 2: Add Metadata
You have three options for adding metadata to your tables:
Manual Entry
- Select a table and column
- Add descriptions, tags, and data types
- Best for small datasets or specific customizations
CSV Upload
- Prepare a CSV file with the required columns
- Upload to add metadata for multiple tables at once
- Best for bulk operations and data governance workflows
AI Generation
- Generate metadata automatically using AI
- Review and accept/discard AI suggestions
- Best for quick starts and large datasets
Step 3: Review and Emit
Review your metadata, then emit it to DataHub. Only tables with metadata will be available for emission.
CSV Format Requirements
Your CSV file must contain these columns:
- SchemaName - Name of the schema (required)
- TableName - Name of the table (required)
- ColumnName - Name of the column (required)
- ColumnDescription - Description of the column (required)
- Domain - Business domain (optional)
- OwnerName - Data owner (optional)
- TableDescription - Description of the table (optional)
- TableTag - Tag for the table (optional)
- ColumnTag - Tag for the column (optional)
- ColumnDataType - Data type of the column (optional)
AI Metadata Generation
The AI metadata generator analyzes table and column names to suggest:
- Table descriptions based on naming patterns
- Column descriptions based on data types and names
- Appropriate tags (PII, Financial, Primary Key, etc.)
- Business domains based on table context
Important: Always review AI-generated metadata before accepting it. The AI makes educated guesses based on naming conventions.
Metadata Sources
All metadata is tagged with its source:
- Manual - Added through the manual entry form
- CSV - Uploaded via CSV file
- AI - Generated by AI and accepted
Access Control
Admin Access
Full access to all features:
- Metadata management
- System logs
- User management
- System configuration
Default: admin/admin
User Access
Limited access to core features:
- Home dashboard
- About page
- Health monitoring
- Profile management
Note: Any username/password
Tips
- Use the "Test Connections" feature to verify your Trino and DataHub connections
- Start with AI generation for quick metadata, then refine manually
- Use CSV upload for standardized metadata across multiple tables
- Review the combined metadata view before emitting to DataHub
- Only tables with metadata will appear in the emission section
Quick Links
Need Help?
If you encounter issues:
- Check your Trino connection settings
- Verify DataHub is running and accessible
- Ensure your CSV follows the required format
- Use the debug buttons for troubleshooting
Security
This application implements:
- Environment-based configuration
- Secure credential handling
- Input validation and sanitization
- Comprehensive audit logging
- Role-based access control