Skip to main content

Secrets and Security

Securely inject sensitive information like API keys, database credentials, or private certificates into your tasks using the Secret class. This project supports mounting secrets as environment variables or as files within a specific directory.

Injecting Secrets into Tasks​

To use a secret in a task, pass a Secret object or a string (representing the secret key) to the secrets parameter of the @task decorator. By default, secrets are mounted as environment variables.

import os
from flyte import task, Secret

# Simple injection using the secret key name
@task(secrets="my-api-key")
async def task_with_simple_secret():
    # The environment variable is automatically uppercased: MY_API_KEY
    api_key = os.environ["MY_API_KEY"]
    print(f"Using key: {api_key}")

# Explicitly naming the environment variable
@task(secrets=Secret("my-openai-api-key", as_env_var="OPENAI_API_KEY"))
async def task_with_custom_env_var():
    api_key = os.environ["OPENAI_API_KEY"]
    print(f"Using OpenAI key: {api_key}")

Environment Variable Naming Logic​

When you don't provide as_env_var, the flyte._secret.Secret class automatically generates the environment variable name:

  1. It combines the group and key (if a group is provided): "{group}_{key}".
  2. It replaces all hyphens (-) with underscores (_).
  3. It converts the entire string to uppercase.

Note: If you provide a custom as_env_var, it must match the regex ^[A-Z_][A-Z0-9_]*$.

Mounting Secrets as Files​

You can mount secrets as files by providing a mount path. Currently, the only supported mount path is /etc/flyte/secrets.

import pathlib
from flyte import task, Secret

SECRET_PATH = "/etc/flyte/secrets"
GROUP = "user-credentials"
KEY = "password"

@task(secrets=Secret(group=GROUP, key=KEY, mount=pathlib.Path(SECRET_PATH)))
async def task_with_file_secret():
    # Secrets are mounted at: {mount}/{group}/{key}
    # Note: The key in the file path is converted to lowercase
    path = pathlib.Path(f"{SECRET_PATH}/{GROUP}/{KEY.lower()}")
    password = path.read_text()
    return password

Using Secrets in Task Environments​

If multiple tasks share the same security requirements, you can define secrets at the TaskEnvironment level.

import flyte
from flyte import Secret

env = flyte.TaskEnvironment(
    "secure-env",
    secrets=[
        Secret(key="database-password", as_env_var="DB_PASS"),
        Secret(key="api-token", as_env_var="API_TOKEN")
    ]
)

@env.task
async def database_operation():
    import os
    db_pass = os.environ["DB_PASS"]
    # ... perform operation

Overriding Secrets at Call-time​

You can override the secrets required by a task when calling it using the .override() method. This is useful for testing or when different environments require different secret keys.

from flyte import task, Secret

@task(secrets="default-secret")
async def my_task():
    pass

# Override the secret for a specific execution
my_task.override(secrets=Secret("production-secret", as_env_var="DEFAULT_SECRET"))

Remote Secret Management​

The flyte.remote.Secret class allows you to manage secrets stored on the Flyte backend programmatically.

Creating and Deleting Secrets​

You can create "regular" secrets for tasks or "image_pull" secrets for private registries.

from flyte.remote import Secret

# Create a regular secret
Secret.create(name="my-secret", value="super-secret-value", type="regular")

# Create an image pull secret (must be organization-level, no project/domain)
Secret.create(name="my-registry-auth", value='{"auths": {...}}', type="image_pull")

# Delete a secret
Secret.delete(name="my-secret")

Retrieving and Listing Secrets​

You can inspect existing secrets to check their status and metadata.

from flyte.remote import Secret

# Get a specific secret
secret = Secret.get(name="my-secret")
print(f"Secret Name: {secret.name}")
print(f"Secret Type: {secret.type}")

# List all secrets in the current project/domain
async for s in Secret.listall(limit=20):
    print(s.name)

Troubleshooting and Limitations​

  • Mount Path Restriction: The Secret class in flyte._secret strictly enforces /etc/flyte/secrets as the only valid mount path. Attempting to use any other path will raise a ValueError.
  • Environment Variable Names: Custom environment variable names must be uppercase and start with a letter or underscore. They cannot contain hyphens.
  • Reusable Containers: If you use TaskEnvironment(reusable=True), you cannot override secrets at the task level unless you set reusable='off' in the task.override() call.
  • Image Pull Secrets: When creating an image pull secret via Secret.create, ensure that your configuration does not have a specific project or domain set, as these secrets are managed at the organization level.