metabase_ingest

What it does

Transforms Metabase collection cards into ktx Semantic Layer source definitions. Uses cards/<id>.json data to build sources, mapping resultMetadata base_type to KSL types, identifying PK/FK candidates, time columns, and applying deduplication and pre-aggregation rules. Reacts to priorProvenance guidance when provided. Ingests only when WorkUnit contains cards/<id>.json files under a Metabase bundle. Uses parameters and resultMetadata fields to drive mapping and join discovery, and enforces verification steps before writing any schema.table.column entries.

How it works

• Each WorkUnit represents one Metabase collection's cards for one database; every cards/<id>.json includes resolvedSql, resultMetadata, and related metadata.
• Map each resultMetadata entry to a KSL type: Integer/Float/Decimal/BigInteger → number; Text/TextLike → string; DateTime/Date/DateTimeWithTZ → time; Boolean → boolean. Identify grain candidates via semantic_type: type/PK and join candidates via semantic_type: type/FK with fk_target_field_id. Identify time columns via semantic_type: type/CreationTimestamp or type/UpdatedTimestamp and set role: time. Use display_name for measure descriptions when available.
• Use parameters to guide SQL translation steps if SQL resolution fell back and to drop optional clauses for Step A of translation.
• Use lastRunAt and dashboardCount to assess staleness and skip signals.
• Before writing, verify identifiers with entity_details and, if needed, perform sql_execution probes to confirm column resolution and values.
• For each card, perform decision tree steps: determine base tables/joins, call sl_discover for candidate targets, decide whether to edit existing sources or write new ones, and include rawPaths: ["cards/<id>.json"] on write/edit calls. When dealing with manifest-only names, bootstrap an overlay first with sl_write_source and add measures, then use sl_edit_source for tweaks. Ensure proper join usage and avoid incorrect joins (FK to non-key display values).
• Apply priorProvenance guidance to prefer edits over new writes when appropriate, and to avoid duplicating existing work.

When to use it

Use when ingesting a Metabase bundle where the WorkUnit includes cards/<id>.json files; triggers include the presence of such files in the Metabase bundle and the need to convert Metabase card metadata into ktx Semantic Layer sources with proper mappings and joins.

What it can touch

• Commands and tools named: sla_discover, sl_read_source, sl_write_source, sl_edit_source, sql_execution, read_raw_file. It references the file path pattern cards/<id>.json. It uses the mapping rules described in resultMetadata to derive KSL column types and roles.

Caveats

• License: Apache-2.0
• Declared tools: claude-code, codex
• Behavior depends on presence of priorProvenance to guide edits vs writes; may skip cards if non-trivial business logic cannot be translated reliably.
• Requires verification steps before emitting schema.table.column into wiki or SL sources; may require overlay creation for manifest-backed names to avoid shadowing manifest schema.
• If a card is determined to be a near-duplicate, a generalized source is created and others are marked skipped.