API Reference

Schema

Core data structures for capacity planning.

Core data structures for capacity planning.

This module defines the schema classes used throughout slosizer for representing request traces, capacity profiles, SLO targets, and planning results.

class slosizer.schema.OutputTokenSource(*values)[source]

Source for output token counts in capacity planning.

OBSERVED

Use actual observed output token counts from trace data.

MAX_OUTPUT_TOKENS

Use max_output_tokens limit for worst-case planning.

class slosizer.schema.LatencyMetric(*values)[source]

Latency metric for SLO evaluation.

E2E

End-to-end latency including baseline model latency and queue delay.

QUEUE_DELAY

Queue delay only, excluding baseline model latency.

class slosizer.schema.RequestSchema(time_col='ts', class_col='class_name', input_tokens_col='input_tokens', cached_input_tokens_col='cached_input_tokens', output_tokens_col='output_tokens', thinking_tokens_col='thinking_tokens', max_output_tokens_col='max_output_tokens', latency_col='latency_s')[source]

Column mapping for request trace DataFrames.

Parameters:

  • time_col (str)

  • class_col (str | None)

  • input_tokens_col (str)

  • cached_input_tokens_col (str | None)

  • output_tokens_col (str)

  • thinking_tokens_col (str | None)

  • max_output_tokens_col (str | None)

  • latency_col (str | None)

time_col

Column containing request arrival timestamps.

Type:

str

class_col

Column containing request class labels.

Type:

str | None

input_tokens_col

Column containing input token counts.

Type:

str

cached_input_tokens_col

Column containing cached input token counts.

Type:

str | None

output_tokens_col

Column containing output token counts.

Type:

str

thinking_tokens_col

Column containing thinking/reasoning token counts.

Type:

str | None

max_output_tokens_col

Column containing max output token limits.

Type:

str | None

latency_col

Column containing observed latency in seconds.

Type:

str | None

class slosizer.schema.RequestTrace(frame, schema, provider=None, model=None, region=None, metadata=<factory>)[source]

Normalized request trace with canonical columns.

Parameters:
frame

DataFrame with canonical columns (arrival_s, input_tokens, etc.).

Type:

pandas.core.frame.DataFrame

schema

Original schema used to parse the trace.

Type:

slosizer.schema.RequestSchema

provider

Cloud provider name (e.g., “vertex”, “azure”).

Type:

str | None

model

Model identifier.

Type:

str | None

region

Deployment region.

Type:

str | None

metadata

Additional trace metadata.

Type:

collections.abc.Mapping[str, Any]

class slosizer.schema.CapacityProfile(provider, model, unit_name, throughput_per_unit, purchase_increment=1, min_units=1, input_weight=1.0, cached_input_weight=0.0, output_weight=4.0, thinking_weight=4.0, long_input_threshold=None, long_input_input_weight=None, long_input_cached_input_weight=None, long_input_output_weight=None, long_input_thinking_weight=None, source='', notes=())[source]

Provider-specific capacity configuration.

Defines how tokens translate to reserved capacity units and the constraints on purchasing those units.

Parameters:

  • provider (str)

  • model (str)

  • unit_name (Literal['GSU', 'PTU', 'capacity_unit'])

  • throughput_per_unit (float | None)

  • purchase_increment (int)

  • min_units (int)

  • input_weight (float)

  • cached_input_weight (float)

  • output_weight (float)

  • thinking_weight (float)

  • long_input_threshold (int | None)

  • long_input_input_weight (float | None)

  • long_input_cached_input_weight (float | None)

  • long_input_output_weight (float | None)

  • long_input_thinking_weight (float | None)

  • source (str)

  • notes (tuple[str, ...])

provider

Cloud provider name.

Type:

str

model

Model identifier.

Type:

str

unit_name

Name of capacity unit (e.g., “GSU”, “PTU”).

Type:

Literal[‘GSU’, ‘PTU’, ‘capacity_unit’]

throughput_per_unit

Tokens per second per capacity unit.

Type:

float | None

purchase_increment

Minimum increment for purchasing units.

Type:

int

min_units

Minimum number of units that can be provisioned.

Type:

int

input_weight

Token weight multiplier for input tokens.

Type:

float

cached_input_weight

Token weight multiplier for cached input tokens.

Type:

float

output_weight

Token weight multiplier for output tokens.

Type:

float

thinking_weight

Token weight multiplier for thinking tokens.

Type:

float

long_input_threshold

Input token count above which long-context weights apply.

Type:

int | None

long_input_input_weight

Input weight for long-context requests.

Type:

float | None

long_input_cached_input_weight

Cached input weight for long-context requests.

Type:

float | None

long_input_output_weight

Output weight for long-context requests.

Type:

float | None

long_input_thinking_weight

Thinking weight for long-context requests.

Type:

float | None

source

Documentation or calibration source for the profile.

Type:

str

notes

Additional notes about the profile.

Type:

tuple[str, …]

class slosizer.schema.LatencySLO(threshold_s, percentile=0.99, metric=LatencyMetric.E2E)[source]

Latency service level objective.

threshold_s

Maximum acceptable latency in seconds.

Type:

float

percentile

Target percentile (e.g., 0.99 for p99).

Type:

float

metric

Latency metric to measure (E2E or QUEUE_DELAY).

Type:

slosizer.schema.LatencyMetric

Raises:

ValueError – If threshold_s <= 0 or percentile not in (0, 1).

Parameters:
class slosizer.schema.ThroughputTarget(percentile=0.99, max_overload_probability=None, windows_s=(1.0, 5.0, 30.0))[source]

Throughput-based capacity planning target.

percentile

Target percentile for required capacity.

Type:

float | None

max_overload_probability

Maximum acceptable probability of overload.

Type:

float | None

windows_s

Time window sizes for bucket analysis.

Type:

tuple[float, …]

Raises:

ValueError – If percentile not in (0, 1) or max_overload_probability not in [0, 1].

Parameters:
label()[source]

Generate a human-readable label for this target.

Returns:

Descriptive label string.

Return type:

str

class slosizer.schema.LatencyTarget(slo)[source]

Latency-based capacity planning target.

Parameters:

slo (LatencySLO)

slo

The latency SLO to meet.

Type:

slosizer.schema.LatencySLO

label()[source]

Generate a human-readable label for this target.

Returns:

Descriptive label string.

Return type:

str

class slosizer.schema.BaselineLatencyModel(intercept_s=0.15, input_token_s=3e-05, cached_input_token_s=8e-06, output_token_s=0.0009, thinking_token_s=0.0007)[source]

Linear model for baseline request latency.

Predicts latency as a linear combination of token counts, useful for estimating processing time independent of queueing.

Parameters:
intercept_s

Base latency in seconds.

Type:

float

input_token_s

Seconds per input token.

Type:

float

cached_input_token_s

Seconds per cached input token.

Type:

float

output_token_s

Seconds per output token.

Type:

float

thinking_token_s

Seconds per thinking token.

Type:

float

predict(frame)[source]

Predict baseline latency for each request.

Parameters:

frame (DataFrame) – DataFrame with token count columns.

Returns:

Array of predicted latencies in seconds.

Return type:

ndarray

class slosizer.schema.PlanOptions(output_token_source=OutputTokenSource.OBSERVED, max_units_to_search=200, headroom_factor=0.0, baseline_latency_model=None)[source]

Options for capacity planning.

output_token_source

Use OBSERVED or MAX_OUTPUT_TOKENS for planning.

Type:

slosizer.schema.OutputTokenSource

Maximum capacity units to consider during search.

Type:

int

headroom_factor

Additional capacity buffer as a fraction (e.g., 0.1 for 10%).

Type:

float

baseline_latency_model

Custom latency model; if None, one is fitted.

Type:

slosizer.schema.BaselineLatencyModel | None

Raises:

ValueError – If max_units_to_search < 1 or headroom_factor < 0.

Parameters:
class slosizer.schema.SimulationResult(units, unit_name, request_level, latency_summary, slack_summary, assumptions)[source]

Results from a capacity simulation.

Parameters:

  • units (int)

  • unit_name (str)

  • request_level (DataFrame)

  • latency_summary (DataFrame)

  • slack_summary (DataFrame)

  • assumptions (dict[str, Any])

units

Number of capacity units simulated.

Type:

int

unit_name

Name of capacity unit.

Type:

str

request_level

Per-request simulation results.

Type:

pandas.core.frame.DataFrame

latency_summary

Aggregate latency statistics.

Type:

pandas.core.frame.DataFrame

slack_summary

Spare capacity statistics by time window.

Type:

pandas.core.frame.DataFrame

assumptions

Simulation parameters and settings.

Type:

dict[str, Any]

class slosizer.schema.PlanResult(objective, target, recommended_units, unit_name, metrics, slack_summary, latency_summary=None, request_level=None, assumptions=<factory>)[source]

Results from capacity planning.

Parameters:

  • objective (str)

  • target (str)

  • recommended_units (int)

  • unit_name (str)

  • metrics (dict[str, Any])

  • slack_summary (DataFrame)

  • latency_summary (DataFrame | None)

  • request_level (DataFrame | None)

  • assumptions (dict[str, Any])

objective

Planning objective (“throughput” or “latency”).

Type:

str

target

Human-readable target description.

Type:

str

recommended_units

Recommended number of capacity units.

Type:

int

unit_name

Name of capacity unit.

Type:

str

metrics

Planning metrics and statistics.

Type:

dict[str, Any]

slack_summary

Spare capacity statistics.

Type:

pandas.core.frame.DataFrame

latency_summary

Latency statistics (for latency planning).

Type:

pandas.core.frame.DataFrame | None

request_level

Per-request results (for latency planning).

Type:

pandas.core.frame.DataFrame | None

assumptions

Planning parameters and settings.

Type:

dict[str, Any]

as_dict()[source]

Convert result to a flat dictionary.

Returns:

Dictionary with all metrics and metadata.

Return type:

dict[str, Any]

class slosizer.schema.PaygoPricing(input_cost_per_million, output_cost_per_million)[source]

Per-token pricing for overflow traffic.

input_cost_per_million

Cost per million input tokens.

Type:

float

output_cost_per_million

Cost per million output tokens.

Type:

float

Raises:

ValueError – If costs are negative.

Parameters:

  • input_cost_per_million (float)

  • output_cost_per_million (float)

class slosizer.schema.ProvisionedPricing(cost_per_unit_hour)[source]

Hourly cost for provisioned capacity.

cost_per_unit_hour

Cost per capacity unit per hour.

Type:

float

Raises:

ValueError – If cost is negative.

Parameters:

cost_per_unit_hour (float)

class slosizer.schema.HybridPricingModel(provisioned, paygo)[source]

Combined pricing for hybrid capacity planning.

Parameters:
provisioned

Hourly cost for provisioned capacity.

Type:

slosizer.schema.ProvisionedPricing

paygo

Per-token pricing for overflow traffic.

Type:

slosizer.schema.PaygoPricing

class slosizer.schema.HybridTarget(strategy, provision_percentile=None, latency_slo=None)[source]

Target for hybrid capacity planning.

strategy

Planning strategy - “cost_optimal” or “percentile_split”.

Type:

Literal[‘cost_optimal’, ‘percentile_split’]

provision_percentile

Percentile to provision for (required if strategy=”percentile_split”).

Type:

float | None

latency_slo

Optional latency SLO constraint.

Type:

slosizer.schema.LatencySLO | None

Raises:

ValueError – If strategy is “percentile_split” but provision_percentile is not set, or if provision_percentile is not in (0, 1).

Parameters:

  • strategy (Literal['cost_optimal', 'percentile_split'])

  • provision_percentile (float | None)

  • latency_slo (LatencySLO | None)

label()[source]

Generate a human-readable label for this target.

Returns:

Descriptive label string.

Return type:

str

class slosizer.schema.HybridPlanResult(provisioned_units, unit_name, provisioned_cost_hourly, paygo_cost_hourly, total_cost_hourly, full_provision_units, full_provision_cost_hourly, savings_vs_full_provision, savings_percent, overflow_fraction, overflow_input_tokens_hourly, overflow_output_tokens_hourly, slack_summary, assumptions=<factory>)[source]

Results from hybrid capacity planning.

Parameters:

  • provisioned_units (int)

  • unit_name (str)

  • provisioned_cost_hourly (float)

  • paygo_cost_hourly (float)

  • total_cost_hourly (float)

  • full_provision_units (int)

  • full_provision_cost_hourly (float)

  • savings_vs_full_provision (float)

  • savings_percent (float)

  • overflow_fraction (float)

  • overflow_input_tokens_hourly (float)

  • overflow_output_tokens_hourly (float)

  • slack_summary (DataFrame)

  • assumptions (dict[str, Any])

provisioned_units

Number of provisioned capacity units.

Type:

int

unit_name

Name of capacity unit (e.g., “GSU”, “PTU”).

Type:

str

provisioned_cost_hourly

Hourly cost of provisioned capacity.

Type:

float

paygo_cost_hourly

Hourly cost of overflow to paygo.

Type:

float

total_cost_hourly

Total hourly cost (provisioned + paygo).

Type:

float

full_provision_units

Units needed if provisioning for 100% of traffic.

Type:

int

full_provision_cost_hourly

Hourly cost if fully provisioned.

Type:

float

savings_vs_full_provision

Dollar savings per hour vs full provision.

Type:

float

savings_percent

Percentage savings vs full provision.

Type:

float

overflow_fraction

Fraction of time buckets with overflow.

Type:

float

overflow_input_tokens_hourly

Average overflow input tokens per hour.

Type:

float

overflow_output_tokens_hourly

Average overflow output tokens per hour.

Type:

float

slack_summary

Spare capacity statistics by time window.

Type:

pandas.core.frame.DataFrame

assumptions

Planning parameters and settings.

Type:

dict[str, Any]

as_dict()[source]

Convert result to a flat dictionary.

Returns:

Dictionary with all metrics and metadata.

Return type:

dict[str, Any]

Ingestion

Request trace parsing and normalization.

Request trace ingestion and normalization.

This module provides functions to convert raw DataFrames into normalized RequestTrace objects with canonical column names.

slosizer.ingest.from_dataframe(df, *, schema, provider=None, model=None, region=None, validate=True, metadata=None)[source]

Create a RequestTrace from a DataFrame.

Normalizes column names and validates data according to the schema.

Parameters:

  • df (DataFrame) – Source DataFrame with request data.

  • schema (RequestSchema) – Column mapping for the DataFrame.

  • provider (str | None) – Cloud provider name.

  • model (str | None) – Model identifier.

  • region (str | None) – Deployment region.

  • validate (bool) – Whether to validate data constraints.

  • metadata (dict[str, Any] | None) – Additional trace metadata.

Returns:

Normalized RequestTrace.

Raises:

ValueError – If required columns are missing or validation fails.

Return type:

RequestTrace

Simulation

Capacity simulation for queue-based latency modeling.

Capacity simulation for queue-based latency modeling.

This module simulates request processing with finite capacity to estimate latency distributions and capacity utilization.

slosizer.simulation.fit_baseline_latency_model(trace)[source]

Fit a linear latency model from observed latencies.

Uses ordinary least squares to fit latency as a function of token counts. Coefficients are constrained to be non-negative. Only rows with valid (non-NaN) latency values are used for fitting.

Parameters:

trace (RequestTrace) – Request trace with observed latencies.

Returns:

Fitted baseline latency model, or default model if insufficient data.

Return type:

BaselineLatencyModel

slosizer.simulation.bucket_required_units(frame, profile, *, units, windows_s, output_token_source)[source]

Compute required capacity units per time bucket.

Divides the trace into fixed-width time windows and calculates the capacity units needed to serve all requests in each window.

Parameters:

  • frame (DataFrame) – DataFrame with canonical columns.

  • profile (CapacityProfile) – Capacity profile with throughput settings.

  • units (int) – Reserved capacity units to compare against.

  • windows_s (Iterable[float]) – Time window sizes in seconds.

  • output_token_source (str) – Source for output tokens.

Returns:

DataFrame with required_units, spare_units, and overflow_units per bucket.

Raises:

ValueError – If profile.throughput_per_unit is not set.

Return type:

DataFrame

slosizer.simulation.bucket_with_tokens(frame, profile, *, units, window_s, output_token_source)[source]

Compute required units and token counts per time bucket.

Similar to bucket_required_units but also tracks raw token counts per bucket for overflow cost calculation. Uses a single window size for efficiency.

Parameters:

  • frame (DataFrame) – DataFrame with canonical columns.

  • profile (CapacityProfile) – Capacity profile with throughput settings.

  • units (int) – Reserved capacity units to compare against.

  • window_s (float) – Time window size in seconds.

  • output_token_source (str) – Source for output tokens.

Returns:

DataFrame with required_units, overflow metrics, and token counts per bucket.

Raises:

ValueError – If profile.throughput_per_unit is not set.

Return type:

DataFrame

slosizer.simulation.summarize_slack(slack_table)[source]

Summarize spare capacity statistics by time window.

Parameters:

slack_table (DataFrame) – Output from bucket_required_units.

Returns:

DataFrame with aggregate statistics per window size.

Return type:

DataFrame

slosizer.simulation.simulate_capacity(trace, profile, *, units, options=None, windows_s=(1.0, 5.0, 30.0))[source]

Simulate request processing with fixed capacity.

Models a simple FIFO queue where requests arrive and are processed at a rate determined by the reserved capacity.

Parameters:

  • trace (RequestTrace) – Request trace to simulate.

  • profile (CapacityProfile) – Capacity profile with throughput settings.

  • units (int) – Number of reserved capacity units.

  • options (PlanOptions | None) – Planning options including output token source.

  • windows_s (tuple[float, ...]) – Time window sizes for slack analysis.

Returns:

SimulationResult with latency and slack statistics.

Raises:

ValueError – If profile.throughput_per_unit is not set or if trace contains fewer than 2 requests.

Return type:

SimulationResult

Planning

Capacity planning algorithms.

Capacity planning algorithms.

This module provides functions to determine optimal reserved capacity based on throughput or latency targets.

slosizer.planning.plan_capacity(trace, profile, target, *, options=None)[source]

Determine optimal reserved capacity for a target.

Searches over candidate capacity levels to find the minimum that satisfies the given throughput or latency target.

Parameters:
Returns:

PlanResult with recommended capacity and metrics.

Raises:

  • ValueError – If profile.throughput_per_unit is not set.

  • TypeError – If target is not a ThroughputTarget or LatencyTarget.

Return type:

PlanResult

slosizer.planning.compare_scenarios(scenarios, profile, targets, *, options=None)[source]

Compare capacity requirements across scenarios and targets.

Parameters:
Returns:

DataFrame with planning results for each scenario/target combination.

Return type:

DataFrame

Plotting

Visualization functions.

Visualization functions for capacity planning results.

slosizer.plotting.plot_latency_vs_units(trace, profile, *, units, options=None, target=None, path=None)[source]

Plot latency percentiles as a function of reserved capacity.

Parameters:

  • trace (RequestTrace) – Request trace to simulate.

  • profile (CapacityProfile) – Capacity profile.

  • units (Iterable[int]) – Capacity unit values to plot.

  • options (PlanOptions | None) – Planning options.

  • target (LatencyTarget | None) – Optional latency target to show as horizontal line.

  • path (str | Path | None) – Optional path to save the figure.

Return type:

None

slosizer.plotting.plot_required_units_distribution(trace, profile, *, windows_s=(1.0, 5.0, 30.0), options=None, path=None)[source]

Plot histogram of required capacity units per time window.

Parameters:

  • trace (RequestTrace) – Request trace to analyze.

  • profile (CapacityProfile) – Capacity profile.

  • windows_s (tuple[float, ...]) – Time window sizes to plot.

  • options (PlanOptions | None) – Planning options.

  • path (str | Path | None) – Optional path to save the figure.

Return type:

None

slosizer.plotting.plot_capacity_tradeoff(comparison, *, path=None)[source]

Plot recommended capacity across scenarios and targets.

Parameters:

  • comparison (DataFrame) – Output from compare_scenarios.

  • path (str | Path | None) – Optional path to save the figure.

Return type:

None

slosizer.plotting.plot_slack_tradeoff(comparison, *, path=None)[source]

Plot spare capacity fraction across scenarios and targets.

Parameters:

  • comparison (DataFrame) – Output from compare_scenarios.

  • path (str | Path | None) – Optional path to save the figure.

Raises:

ValueError – If comparison is missing the avg_spare_fraction_1s column.

Return type:

None

Synthetic Workloads

Synthetic workload generation for testing.

Synthetic workload generation for testing and demonstration.

slosizer.synthetic.optimize_trace(trace)[source]

Apply prompt optimization to reduce token usage.

Simulates the effect of prompt engineering and caching improvements by reducing input, output, and thinking tokens.

Parameters:

trace (RequestTrace) – Original request trace.

Returns:

Optimized trace with reduced token counts.

Return type:

RequestTrace

slosizer.synthetic.make_synthetic_trace(*, horizon_s=14400, seed=42, scenario='baseline')[source]

Generate a synthetic request trace for testing.

Parameters:

  • horizon_s (int) – Trace duration in seconds.

  • seed (int) – Random seed for reproducibility.

  • scenario (str) – Either “baseline” or “optimized”.

Returns:

Synthetic RequestTrace.

Return type:

RequestTrace

Provider Adapters

Vertex AI

Google Cloud Vertex AI capacity profiles.

This module provides built-in capacity profiles for Vertex AI Generative AI models using the GSU (Generative Service Unit) provisioned throughput model.

See: https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput/supported-models

slosizer.providers.vertex.available_vertex_profiles()[source]

List available built-in Vertex AI model profiles.

Returns:

Sorted list of model identifiers.

Return type:

list[str]

slosizer.providers.vertex.vertex_profile(model)[source]

Get a built-in Vertex AI capacity profile.

Parameters:

model (str) – Model identifier (e.g., “gemini-2.5-flash”).

Returns:

CapacityProfile configured for the specified Vertex model.

Raises:

KeyError – If the model is not in the built-in registry.

Return type:

CapacityProfile

Azure OpenAI

Azure OpenAI PTU capacity profiles.

This module provides a factory function for creating Azure OpenAI capacity profiles using the PTU (Provisioned Throughput Unit) model.

Azure PTU throughput is workload-sensitive and must be calibrated per deployment. See: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/provisioned-throughput

slosizer.providers.azure.azure_profile(model, *, throughput_per_unit, purchase_increment=1, min_units=1, input_weight=1.0, cached_input_weight=0.0, output_weight=4.0, thinking_weight=4.0, notes=())[source]

Create an Azure OpenAI PTU capacity profile.

Azure PTU capacity varies by workload, so profiles must be calibrated using the Azure capacity calculator and benchmark data.

Parameters:

  • model (str) – Model identifier (e.g., “gpt-4.1”).

  • throughput_per_unit (float) – Tokens per second per PTU.

  • purchase_increment (int) – Minimum PTU increment for purchasing.

  • min_units (int) – Minimum number of PTUs.

  • input_weight (float) – Token weight for input tokens.

  • cached_input_weight (float) – Token weight for cached input tokens.

  • output_weight (float) – Token weight for output tokens.

  • thinking_weight (float) – Token weight for thinking tokens.

  • notes (tuple[str, ...]) – Additional notes about the profile.

Returns:

CapacityProfile configured for Azure OpenAI.

Return type:

CapacityProfile