ShardConfig

Configuration for model sharding.

Attributes

Attribute	Type	Description
engine	Literal["vllm"] = vllm	The sharding engine to use (currently only "vllm" is supported).
args	[VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()	Arguments for the sharding engine.

---

Constructor

Signature

def ShardConfig(
    engine: Literal["vllm"] = "vllm",
    args: [VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()
) - > null

Parameters

Name	Type	Description
engine	Literal["vllm"] = "vllm"	The sharding engine to use (currently only "vllm" is supported).
args	[VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()	Arguments for the sharding engine.

---

Signature

def ShardConfig(
    engine: Literal["vllm"] = vllm,
    args: [VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()
) - > null

Parameters

Name	Type	Description
engine	Literal["vllm"] = vllm	The sharding engine to use for model distribution, currently restricted to "vllm".
args	[VLLMShardArgs](vllmshardargs.md?sid=flyte_prefetch__hf_model_vllmshardargs) = VLLMShardArgs()	A collection of engine-specific arguments and settings used to configure the sharding process.

---