Secret and Credential Management

To securely handle sensitive information in this SDK, you can inject secrets directly into tasks or manage them programmatically on a remote Flyte cluster. Additionally, the SDK provides a secure mechanism for caching authentication tokens using the system keyring.

Injecting Secrets into Tasks

You can request secrets for a task by passing them to the @task decorator. Secrets are typically injected as environment variables.

import os
from flyte import task, Secret

# Simple usage: secret name is converted to uppercase env var (MY_SECRET)
@task(secrets="my-secret")
def secret_task():
    print(os.environ["MY_SECRET"])

# Custom environment variable name
@task(secrets=Secret(key="api-key", as_env_var="OPENAI_API_KEY"))
def custom_env_task():
    print(os.environ["OPENAI_API_KEY"])

# Mounting as a file (restricted to /etc/flyte/secrets)
@task(secrets=Secret(key="config", mount="/etc/flyte/secrets"))
def file_secret_task():
    with open("/etc/flyte/secrets/config", "r") as f:
        print(f.read())

Secret Injection Rules

When using flyte.Secret in src/flyte/_secret.py:

• Default Naming: If as_env_var is not provided, the SDK automatically converts the key to uppercase and replaces hyphens with underscores (e.g., my-key becomes MY_KEY).
• Environment Variable Validation: Custom names must match the regex ^[A-Z_][A-Z0-9_]*$.
• Mount Path Restriction: Currently, the only supported mount path is /etc/flyte/secrets. Any other path will raise a ValueError.

Managing Remote Cluster Secrets

The flyte.remote.Secret class allows you to perform CRUD operations on secrets stored in a remote Flyte cluster. These methods are synchronized and can be called directly in a standard Python script.

Creating and Retrieving Secrets

Use Secret.create to upload a new secret and Secret.get to fetch its metadata.

from flyte.remote import Secret

# Create a regular secret
Secret.create(name="database-password", value="super-secret-string")

# Create an image pull secret for private registries
# Note: project and domain must NOT be set for image pull secrets
Secret.create(name="registry-creds", value='{"auths":...}', type="image_pull")

# Retrieve secret metadata
secret = Secret.get(name="database-password")
print(f"Secret {secret.name} was created at {secret.pb2.secret_metadata.created_time}")

Listing and Deleting Secrets

You can iterate through all secrets in your current project and domain or delete them by name.

from flyte.remote import Secret

# List all secrets (returns an iterator)
for secret in Secret.listall(limit=20):
    print(f"Found secret: {secret.name} [Type: {secret.type}]")

# Delete a secret
Secret.delete(name="database-password")

Managing Authentication Credentials

The SDK uses KeyringStore to securely cache OAuth2 tokens (access and refresh tokens) in your system's keyring. This prevents you from having to re-authenticate frequently.

Storing and Retrieving Tokens

The KeyringStore in src/flyte/remote/_client/auth/_keyring.py handles the interaction with the system keyring.

from flyte.remote._client.auth._keyring import Credentials, KeyringStore

# Create a credentials object
creds = Credentials(
    access_token="your-access-token",
    refresh_token="your-refresh-token",
    for_endpoint="flyte.example.com"
)

# Store in system keyring
KeyringStore.store(creds)

# Retrieve from system keyring
stored_creds = KeyringStore.retrieve(for_endpoint="flyte.example.com")
if stored_creds:
    print(f"Retrieved token: {stored_creds.access_token}")

Disabling the Keyring

In ephemeral environments like CI/CD pipelines or when running inside a Flyte task (in-cluster), you should disable the keyring to avoid errors or unnecessary overhead.

# Retrieve without using keyring
creds = KeyringStore.retrieve(for_endpoint="flyte.example.com", disable=True)

# Delete stored credentials
KeyringStore.delete(for_endpoint="flyte.example.com")

Troubleshooting

Invalid Environment Variable Names

If you manually specify as_env_var, ensure it follows standard naming conventions. The SDK will raise a ValueError if the name contains lowercase letters or invalid characters:

# This will raise ValueError: Invalid environment variable name
Secret(key="my-key", as_env_var="my_env_var") 

Mount Path Errors

The SDK strictly enforces the mount path for file-based secrets. If you attempt to use a custom directory, it will fail:

# This will raise ValueError: Only /etc/flyte/secrets is supported
Secret(key="my-key", mount="/tmp/secrets")

Keyring Availability

If the keyring library is missing or the system keyring is unavailable (e.g., in a headless Linux environment without a configured backend), KeyringStore will log a debug message and fail gracefully without raising an exception. Tokens will simply not be cached.