DataPig provides near real-time data ingestion from ADLS to SQL Server for D365/Dataverse applications, empowering organizations with fast, reliable, and actionable insights while simplifying data architecture.
Features
Real-Time Ingestion Start
Experience near real-time data ingestion as DataPig instantly triggers ingestion upon CSV package drops to ADLS, significantly reducing wait times for data availability.
SynapseLink Multiprofile support
Manage up to 10,000 tables with the ability to handle large datasets effortlessly, overcoming single-profile system limitations.
Flexible CDM to SQL Transformation
Offers flexibility in transforming CDM data to SQL datatypes, allowing for customized SQL tables and eliminating rigid schema constraints.
Delta Ingestion Speed
Ingest delta changes rapidly, moving updates for up to 200 tables in less than 10 seconds with high-speed parallel processing.
Pipeline Health Visibility
Provides end-to-end visibility of ingestion pipelines with a detailed interface, improving tracking and troubleshooting.
Robust Pipeline Reliability
Ensures data pipeline reliability with automated error recovery and robust retry logic to minimize disruptions.
Change Data Tracking
Captures every data change in CDC tables, managing storage costs with smart data rehydration and archiving.
Automated Archiving
Reduces storage costs significantly by automatically archiving processed data to cost-effective tiers.
Scheduled Purging
Automates data lifecycle management by scheduling purges at the database profile level, optimizing storage and compliance.