AsyncConnector

This is the base class for all async connectors, and it defines the interface that all connectors must implement. The connector service is responsible for invoking connectors. The executor will communicate with the connector service to create tasks, get the status of tasks, and delete tasks.

Attributes

Attribute	Type	Description
name	string = "Async Connector"	Human-readable identifier for the connector used for logging and identification purposes.
task_type_name	string	The unique string identifier for the specific task type this connector is responsible for handling.
task_type_version	integer = 0	The version number of the task type, used by the Connector Service to look up the correct connector implementation.
metadata_type	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The specific ResourceMeta class type used to store and retrieve stateful information about the external task.

---

Methods

---

create()

@classmethod
def create(
    task_template: tasks_pb2.TaskTemplate,
    output_prefix: str,
    inputs: Optional[Dict[str, typing.Any]] = None,
    task_execution_metadata: Optional[TaskExecutionMetadata] = None
) - > [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)

Return a resource meta that can be used to get the status of the task.

Parameters

Name	Type	Description
task_template	tasks_pb2.TaskTemplate	The protobuf definition containing the configuration and requirements for the task to be executed
output_prefix	str	The storage location prefix where task execution results and artifacts should be persisted
inputs	Optional[Dict[str, typing.Any]] = None	A dictionary of input parameters and values required for the task execution
task_execution_metadata	Optional[TaskExecutionMetadata] = None	Contextual metadata about the current execution environment and workflow identity

Returns

Type	Description
[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The metadata object containing unique identifiers and state information for the newly created external resource

---

get()

@classmethod
def get(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)
) - > [Resource](resource.md?sid=flyte_connectors__connector_resource)

Return the status of the task, and return the outputs in some cases. For example, bigquery job can't write the structured dataset to the output location, so it returns the output literals to the propeller, and the propeller will write the structured dataset to the blob store.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The metadata handle used to identify and locate the specific task in the external system

Returns

Type	Description
[Resource](resource.md?sid=flyte_connectors__connector_resource)	The current state of the external resource, including its status and any available output data

---

delete()

@classmethod
def delete(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)
)

Delete the task. This call should be idempotent. It should raise an error if fails to delete the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The metadata handle identifying the external resource to be terminated or removed

---

get_metrics()

@classmethod
def get_metrics(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)
) - > GetTaskMetricsResponse

Return the metrics for the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The metadata handle identifying the task for which metrics are being requested

Returns

Type	Description
GetTaskMetricsResponse	A response object containing performance and execution metrics collected from the external task

---

get_logs()

@classmethod
def get_logs(
    resource_meta: [ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)
) - > GetTaskLogsResponse

Return the metrics for the task.

Parameters

Name	Type	Description
resource_meta	[ResourceMeta](resourcemeta.md?sid=flyte_connectors__connector_resourcemeta)	The metadata handle identifying the task for which logs are being retrieved

Returns

Type	Description
GetTaskLogsResponse	A response object containing log links or log data generated during the task execution

---