Skip to main content

Remote Entities

Remote entities in this SDK provide the interface for managing and interacting with the Flyte/Union platform. They act as a bridge between local Python code and the remote execution environment, allowing you to manage projects, monitor task executions, and retrieve results.

The core entities are Projects, Runs, and Actions, each serving a specific role in the execution lifecycle.

Project Management​

The Project class in flyte.remote._project is used to manage top-level containers for your Flyte entities. Projects organize domains and executions.

Creating and Updating Projects​

You can create new projects or update existing ones directly through the SDK. When updating a project, the SDK specifically clears the domains field before sending the request, as the backend rejects update requests that include them.

from flyte.remote import Project

# Create a new project
project = Project.create(
    id="ml-research",
    name="ML Research Project",
    description="Project for tracking model experiments",
    labels={"department": "data-science"}
)

# Update project metadata
Project.update(id="ml-research", description="Updated description for research")

Listing and Archiving​

Projects can be listed with filters and sorting. By default, listall returns active projects. You can also archive or unarchive projects to manage their visibility.

# List all active projects
active_projects = Project.listall(archived=False)

# Archive a project
project = Project.get("ml-research")
project.archive()

Task Executions and Runs​

A Run (defined in flyte.remote._run) represents a single execution of a task on the remote platform. It is the primary entity used to monitor progress and retrieve data.

Fetching and Monitoring Runs​

You can retrieve a run by its unique name. Internally, Run.get fetches RunDetails to populate the run's state.

from flyte.remote import Run

# Fetch an existing run
run = Run.get(name="af6b2c8d4e1f")

# Wait for the run to reach a terminal state (Succeeded, Failed, etc.)
# This displays a rich progress panel by default
run.wait()

if run.done():
    print(f"Run finished in phase: {run.phase}")

Accessing Data and Logs​

Once a run is executing or finished, you can access its inputs, outputs, and logs.

# Retrieve inputs and outputs
inputs = run.inputs()
outputs = run.outputs()

# Stream logs to the console
run.show_logs()

# Or get logs as an iterator
async for line in run.get_logs.aio(filter_system=True):
    print(line)

Lifecycle Control​

Runs can be aborted if they are no longer needed, which terminates the remote execution.

run.abort(reason="Parameters were incorrect")

Execution Units and Actions​

An Action (defined in flyte.remote._action) is the fundamental unit of execution. In this codebase, a Run is composed of actions, which can be linearly nested. For example, a complex task might consist of multiple sub-actions.

Action Hierarchy​

Actions have unique identifiers within their parent action. You can list all actions associated with a specific run to inspect granular execution steps.

from flyte.remote import Action

# List all actions for a specific run
actions = Action.listall(for_run_name="af6b2c8d4e1f")
for action in actions:
    print(f"Action: {action.name}, Task: {action.task_name}, Phase: {action.phase}")

Watching State Transitions​

The watch method provides an asynchronous stream of ActionDetails, allowing you to react to state changes in real-time.

async for details in action.watch():
    print(f"Action {action.name} is now in phase {details.phase}")
    if details.done():
        break

Filtering and Discovery​

The SDK provides robust filtering capabilities for discovering entities, particularly through the listall methods of Run and Action.

Time-Based Filtering​

The TimeFilter class in flyte.remote._common allows you to filter entities based on created_at or updated_at timestamps. Note that the after bound is inclusive (Greater Than or Equal), while the before bound is exclusive (Less Than).

from flyte.remote import Run, TimeFilter
from datetime import datetime, timezone

# Find runs created after a specific date
start_time = datetime(2024, 1, 1, tzinfo=timezone.utc)
runs = Run.listall(
    project="my-project",
    created_at=TimeFilter(after=start_time)
)

Phase Filtering​

You can filter runs and actions by their execution phase using the ActionPhase enum from flyte.models.

from flyte.models import ActionPhase

# List only successful or failed runs
runs = Run.listall(
    in_phase=(ActionPhase.SUCCEEDED, ActionPhase.FAILED)
)

The ActionPhase enum also provides a is_terminal property to easily check if an execution has reached a final state (Succeeded, Failed, Aborted, or Timed Out).