Looker ingest

--- name: lookeringest description: Extract durable KTX knowledge and semantic-layer contribution proposals from staged Looker runtime dashboard, Look, and explore JSON. Load for WorkUnits whose raw files are located in explores/, dashboards/, or looks/. callers: [memoryagent] --- # Looker Runtime Ingest Looker runtime ingest converts API-staged dashboards, Looks, and explores into persistent ktx memory. Runtime entities are evidence. They are not themselves the final form of knowledge. ## Required Workflow 1. Read every rawFiles entry for the WorkUnit. 2. Read relevant dependencyPaths before making a decision. For dashboard and Look WUs, this usually includes the referenced explore JSON, signal files, folders/tree.json, and users/<id>.json. 3. Treat signals/*.json, owners, folders, schedules, and favorites as prioritization or provenance context only. 4. Extract generalizable metric formulas, segment definitions, field semantics, and domain conventions. 5. Use wiki_search, sl_discover, and sl_read_source before writing so that new content merges with existing data rather than duplicating it. 6. Use context_evidence_search or context_evidence_read to obtain evidence chunk IDs for any wiki-bound knowledge candidate. 7. Use context_candidate_write for durable wiki-bound knowledge. Do not call wiki_write from a Looker WorkUnit; Stage 4 reconciliation promotes candidates and writes wiki pages. 8. Use looker_query_to_sl for each Look query or dashboard tile query that has a query object. 9. Write SL from Looker runtime evidence only through the staged warehouse target contract. For explores and inherited dashboard/Look queries, check targetTable.ok; if true, write to targetWarehouseConnectionId and use targetTable.canonicalTable as source.table. If false or missing, write wiki knowledge candidates and record emit_unmapped_fallback with the staged reason. 10. Run sl_validate after every SL write. If validation fails, fix the source or roll it back before the WorkUnit ends. For every Looker field reference, call entitydetails on the underlying schema.table.column before promoting it to slrefs or quoting it in the wiki body. ## Identifier Verification Protocol Before writing a wiki page or SL source on any topic: 1. discover_data({query: "<topic>"}) - see which wikis, SL sources, and raw tables already exist. Prefer updating existing pages over creating new ones. Before including any schema.table or schema.table.column in a wiki body, SL source, tables: frontmatter, sl_refs, or emit_unmapped_fallback: 2. entity_details({connectionId, targets: [{display: "<identifier>"}]}) - confirm that the identifier resolves; inspect native types, FK/PK, and sampleValues. 3. For literal values from the source, such as status codes or plan tiers, check whether they appear in entity_details sampleValues for the relevant column. If sampleValues is short or the sample may have missed real values, run a sql_execution probe with the same warehouse connection ID: sql_execution({connectionId, sql: "SELECT DISTINCT <col> FROM <ref> LIMIT 50"}). 4. If the candidate identifier still does not resolve, do one of the following: - Use sql_execution({connectionId, sql: "SELECT 1 FROM <ref> LIMIT 0"}). If it returns an error, the identifier is fictional. - Wrap the identifier in [unverified - from <rawPath>] in the wiki body, citing the exact raw path where it was mentioned. - When recording emit_unmapped_fallback with no_physical_table, include the failing probe error in clarification. 5. Never copy <schema>.<table> placeholder strings from these instructions into the output. ## Explore WorkUnits Explore WUs have raw files such as explores/<model>/<explore>.json and usually depend on lookml_models.json. Use the deterministic API -derived source key: ``text looker__<model>__<explore> For example, modelName: "b2b" and exploreName: "sales_pipeline" map to lookerb2bsales_pipeline. Mapped explore write shape: ``json { "connectionId": "22222222-2222-4222-8222-222222222222", "sourceName": "lookerb2bsales_pipeline", "source": { "name": "loo