Overview
The Data Quality (DQ) tab on the pipeline page is the command‑center for authoring, organizing, and evaluating rules that measure the health of each entity in your Syncari hub.
Key capabilities
| Capability | What it lets you do |
|---|---|
| Rule authoring | Create granular checks—ranging from simple field validations to complex multi‑field logic—that score records as Pass or Fail. |
| Category management | Group rules into logical buckets (e.g., Completeness, Conformity). Syncari ships several starter categories; you can add your own to match internal standards. |
| Pipeline integration | Leverage temporary variables computed in pipelines to express rich, reusable logic. |
| Data Quality Dashboard | The Data Quality Dashboard in Insights Studio visualizes scores by entity, category, and individual rule over time, with built‑in DQ trends. |
Authoring Data Quality Rules
Navigate to the Data Quality Tab
From Sync Studio → Your Entity → Data Quality, you’ll see the existing rule grid (above). Use the Create Rule button to open the rule designer.
Create a Rule
- Click Create Rule.
- Fill out:
- Name – clear, action‑oriented (Is Unique Record).
- Policy – choose Report, Warn, or Fail.
- Scope – select the fields or records the rule evaluates.
- Category – pick an existing category or Add Category.
- Condition – combine fields, TVs, and operators.
- Validate to preview hit counts → Save.
Need advanced operators? Open the operator dropdown for options such as Is PhoneNumber, In Reference Data, Not Equals, and more.
Leverage Pipeline‑Derived Variables
Complex logic is often easier to compute upstream in a pipeline and stored in a temporary variable (TV).
- In Pipelines → Your Pipeline, add relevant functions and actions that evaluate your DQ rule.
- Output the result to a temporary variable via the Set Value function.
- Save the pipeline.
- Now you can use this temporary variable in DQ Rules
Managing Categories
Categories help stakeholders filter dashboards and bulk‑toggle rule groups.
Syncari ships the following starters:
| Starter Category | Purpose |
| Completeness | Mandatory field presence |
| Conformity | Format and pattern checks |
| Uniqueness | Duplicate detection |
| Validity | Field‑to‑field cross checks |
Add or edit categories via Manage Categories. Re‑assign rules at any time without losing history.
Monitoring Quality in Insights Studio
Open the Dashboard
- Insights Studio → Dashboards → Data Quality Dashboard presents entity‑level and category‑level scorecards.
- Hover over any bar or line to view exact scores.
Built‑in Reports
| Report | What it shows | Drill‑down |
| Current Score by Entity | Weighted score per entity. | Entity → Record list |
| Overall Score by Category | Pass/Fail counts per category. | Category → Rule list |
| Score Over Time | Trendline per entity or category. | Point → Daily variance |
Filtering & Time Windows
Use the filter bar to adjust Date Range, Entity, Category, or Severity. Trendlines adjust in real time for pre/post remediation comparison.
Best Practices
- Start small—enable a narrow set of critical rules to avoid alert fatigue.
- Compute once, reuse everywhere—centralize complex formulas in pipelines, exposing TVs to rules.
- Version intentionally—clone rules before major logic changes to keep historical comparability.
- Align categories to owners—map each category to a steward (e.g., Finance owns Billing Integrity).
- Schedule reviews—monitor dashboard trendlines weekly; investigate spikes quickly.
Troubleshooting & FAQs
| Symptom | Resolution |
| Rule not evaluating | Confirm pipeline temporary variable is populated before DQ runs; check node execution order. |
| Dashboard missing entity | Ensure at least one active rule exists for that entity. |
| Performance lag on large entities | Reduce the number of rules |
Appendix: Regular Expression Cookbook
Regular expressions are very powerful tools to match patterns. Here are some common regular expressions you can use in DQ rules
Common Regular Expressions for Data Quality
| # | Purpose | Regular Expression |
| 1 | Email address (RFC 5322 trimmed) | ^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}$ (use case‑insensitive flag) |
| 2 | US phone number (E.164) | ^\+1\d{10}$ |
| 3 | Generic international phone (E.164, 8–15 digits) | ^\+\d{8,15}$ |
| 4 | US ZIP or ZIP+4 | ^\d{5}(-\d{4})?$ |
| 5 | ISO‑3166‑1 alpha‑2 country code | ^[A-Z]{2}$ |
| 6 | State/Province 2‑letter code (US/CA) | ^[A-Z]{2}$ |
| 7 | UUID v4 | ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$ |
| 8 | Date YYYY‑MM‑DD | ^\d{4}-\d{2}-\d{2}$ |
| 9 | URL (HTTP/HTTPS) | ^(https?):\/\/[\w.-]+(?:\/[\w\-._~:/?#\[\]@!$&'()*+,;=]*)?$ |
| 10 | Alphabetic name with spaces & punctuation | ^[A-Za-z][A-Za-z \-']*$ |
Related to