Skip to content

3 - Avoid recreating snapshot_by_name in evaluator #6019

Description

@cmgoffena13

File: sqlmesh/core/snapshot/evaluator.py
Function: promote , _promote_snapshot
Lines: 267 - 330 , 1253 - 1285

Issue with _promote_snapshot

In _promote_snapshot we are passed the snapshots, which is keyed by snapshot_id. The function needs to key by snapshot_name though, so we keep recreating:

snapshot_by_name = {s.name: s for s in (snapshots or {}).values()}
render_kwargs["snapshots"] = snapshot_by_name

We should just create this once in promote before we pass it in.

        # Fetch the view data objects for the promoted snapshots to get them cached
        self._get_virtual_data_objects(target_snapshots, environment_naming_info)

       snapshot_by_name = {s.name: s for s in (snapshots or {}).values()}    <------NEW

        deployability_index = deployability_index or DeployabilityIndex.all_deployable()
        with self.concurrent_context():
            concurrent_apply_to_snapshots(
                target_snapshots,
                lambda s: self._promote_snapshot(
                    s,
                    start=start,
                    end=end,
                    execution_time=execution_time,
                    snapshots=snapshots,
                    snapshots_by_name=snapshots_by_name,    <-------- NEW
                    table_mapping=table_mapping,
                    environment_naming_info=environment_naming_info,
                    deployability_index=deployability_index,  # type: ignore
                    on_complete=on_complete,
                ),
                self.ddl_concurrent_tasks,
            )

Roughly. I would also evaluate if we can remove the snapshots input to self._promote_snapshot. Maybe snapshots_by_name fulfills all the functions downstream.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions