RoutersClient

The RoutersClient provides methods for creating, managing, and using routers. Routers are used to intelligently direct traffic between different LLM models based on your requirements.

Router Lifecycle

  1. Initial Creation: When a router is first created with create_router, it only routes to a single model (specified by base_model).

  2. Training: To enable routing between multiple models, use run_training_job to train the router with a set of models.

  3. Routing: After training, the router can intelligently route between any of the models it was trained on.

Creating a Router

router = client.routers.create_router(
    router_id="my-router",
    base_model="anthropic/anthropic/claude-3-opus-latest",  # Initial model before training
    description="Routes between different models based on quality and cost"
)

Training a Router

To enable routing between multiple models, train the router using run_training_job. You can specify which models to include:

from martian_apart_hack_sdk.models.llm_models import ANTHROPIC_MODELS, OPENAI_MODELS

# Train on all Anthropic and OpenAI models
training_job = client.routers.run_training_job(
    router=router,
    judge=quality_judge,
    llms=list(ANTHROPIC_MODELS | OPENAI_MODELS),  # Combine model sets using set operations
    requests=example_requests
)

# Or train on specific models
training_job = client.routers.run_training_job(
    router=router,
    judge=quality_judge,
    llms=[
        "anthropic/anthropic/claude-3-opus-latest",
        "openai/openai/gpt-4o",
        "together/mistralai/Mistral-Small-24B-Instruct-2501"
    ],
    requests=example_requests
)

Available Model Sets

The SDK provides predefined sets of models in martian_apart_hack_sdk.models.llm_models:

  • OPENAI_MODELS: All OpenAI models (GPT-4 variants)

  • ANTHROPIC_MODELS: All Anthropic models (Claude variants)

  • TOGETHER_MODELS: All models available through Together

  • GEMINI_MODELS: All Google Gemini models

  • ALL_MODELS: Union of all available models

You can use these sets to easily specify which models to include in router training:

from martian_apart_hack_sdk.models.llm_models import ALL_MODELS

# Train on every available model
training_job = client.routers.run_training_job(
    router=router,
    judge=quality_judge,
    llms=list(ALL_MODELS),  # Convert set to list
    requests=example_requests
)
class martian_apart_hack_sdk.backend_clients.routers.RoutersClient(httpx, config)[source]

The client for the Martian Routers API. Use the RoutersClient to create, update, and list routers.

Normally, you don’t need to create a RoutersClient directly. Instead, use the MartianClient.routers property to access the RoutersClient.

Parameters:

  • httpx (httpx.Client) – The HTTP client to use for the API.

  • config (utils.ClientConfig) – The configuration for the API.

create_router(router_id, base_model, description=None)[source]

Create a router.

Parameters:

  • router_id (str) – An arbitrary identifier (chosen by you) for the router. You’ll need to use this identifier to reference the router in other API calls.

  • base_model (str) – The initial model the router will use. The router will only route to this model until it is trained with additional models using run_training_job.

  • description (Optional[str], optional) – The description of the router, for your own reference.

Returns:

The newly created router resource.

Return type:

router_resource.Router

Notes

When first created, a router only routes to its base_model. To enable routing between multiple models, you must train the router using run_training_job with a list of models to include.

Raises:

  • ResourceAlreadyExistsError – If a router with the given ID already exists.

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.

Parameters:

  • router_id (str)

  • base_model (str)

  • description (str | None)

Return type:

Router

update_router(router_id, router_spec, description=None)[source]

Update an existing router’s specification and/or description.

Parameters:

  • router_id (str) – The ID of the router to update.

  • router_spec (Dict[str, Any]) – The new router specification.

  • description (Optional[str], optional) – Optional new description for the router.

Returns:

The updated Router resource.

Return type:

router_resource.Router

Notes

Router updates are non-destructive. The updated router will have an incremented version number. You can use this version number to reference the router in other API calls. You can also access previous versions of the router by passing the previous version number to the get method.

Raises:

  • ResourceNotFoundError – If the router doesn’t exist.

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.

Parameters:

  • router_id (str)

  • router_spec (Dict[str, Any])

  • description (str | None)

Return type:

Router

list()[source]

List all routers.

Returns:

A list of all available routers.

Return type:

list[router_resource.Router]

Raises:

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.

get(router_id, version=None)[source]

Get a specific router by ID and optionally version.

Parameters:

  • router_id (str) – The ID of the router to retrieve.

  • version (Optional[int], optional) – The specific version of the router to retrieve. If not provided, the latest version will be returned.

Returns:

The router resource. OR None if the router does not exist.

Return type:

router_resource.Router

Raises:

  • httpx.HTTPError – If the request fails for reasons other than a missing router.

  • httpx.TimeoutException – If the request times out.

run(router, routing_constraint, completion_request, version=None)[source]

Run a router with the given constraints and completion request.

Parameters:

  • router_id (str) – The ID of the router to run.

  • routing_constraint (RoutingConstraint) – The routing constraints to apply.

  • completion_request (Dict[str, Any]) – The completion request parameters.

  • version (Optional[int], optional) – Optional router version to use.

  • router (Router)

Returns:

The router’s response string.

Return type:

str

Raises:

  • ResourceNotFoundError – If the router doesn’t exist.

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.

run_training_job(router, judge, llms, requests)[source]

Train a router to intelligently route between multiple models.

After training, the router will be able to route between any of the models specified in the llms parameter, based on the quality vs latency preferences specified in routing constraints.

Parameters:

  • router (Router) – The router to train.

  • judge (Judge) – The judge to use for evaluation.

  • llms (List[str]) – List of model names to include in routing decisions. After training, the router will only be able to route between these models. You can use predefined model sets from martian_apart_hack_sdk.models.llm_models (e.g., ANTHROPIC_MODELS, OPENAI_MODELS, ALL_MODELS).

  • requests (List[Dict[str, Any]]) – List of request objects containing messages for training.

Returns:

The training job response metadata.

Note: The response metadata contains information about the training job itself, but does not contain any information about the results of training or router configuration.

Return type:

RouterTrainingJob

Raises:

  • ResourceNotFoundError – If the router or judge doesn’t exist.

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.

Examples

Create a judge and router, then train the router with multiple models:

>>> from martian_apart_hack_sdk.models.llm_models import ANTHROPIC_MODELS, OPENAI_MODELS
>>>
>>> # Create a basic judge that evaluates response quality
>>> judge_spec = RubricJudgeSpec(
...     model_type="rubric_judge",
...     model="gpt-4",
...     rubric="Rate the response quality from 1-10 based on accuracy and completeness.",
...     min_score=1,
...     max_score=10
... )
>>> judge = client.judges.create("quality_judge", judge_spec)
>>>
>>> # Create a router (initially only routes to base_model)
>>> router = client.routers.create(
...     router_id="test_router",
...     base_model="anthropic/anthropic/claude-3-opus-latest"
... )
>>>
>>> # Example training requests
>>> requests = [
...     {
...         "messages": [
...             {"role": "user", "content": "What is Python?"}
...         ]
...     },
...     {
...         "messages": [
...             {"role": "system", "content": "You are a machine learning expert who explains concepts clearly and concisely."},
...             {"role": "user", "content": "Explain machine learning."}
...         ]
...     }
... ]
>>>
>>> # Train the router with multiple models
>>> training_job = client.routers.run_training_job(
...     router=router,
...     judge=judge,
...     llms=list(ANTHROPIC_MODELS | OPENAI_MODELS),  # Train on all Anthropic and OpenAI models
...     requests=requests
... )
wait_training_job(job_name, poll_interval=10, poll_timeout=1200)[source]

Poll a training job until it completes or fails.

Parameters:

  • job_name (str) – The job name or ID. If it contains ‘/’ it’s treated as a full name (e.g. ‘organizations/org-name/router_training_jobs/job-id’) and the last part is used as ID.

  • poll_interval (int, optional) – Number of seconds to wait between polls. Defaults to 10.

  • poll_timeout (int, optional) – Maximum time to poll in seconds. Defaults to 1200 (20 minutes).

Returns:

The final training job state. Check the status field to determine

if the job completed successfully (“SUCCESS”) or failed (“FAILURE”, “FAILURE_WITHOUT_RETRY”).

Return type:

RouterTrainingJob

Raises:

  • httpx.HTTPError – If any API request fails.

  • TimeoutError – If the job doesn’t complete within the poll_timeout period.

Examples

>>> # Start a training job
>>> training_job = client.routers.run_training_job(...)
>>>
>>> # Poll until completion
>>> final_job = client.routers.poll_training_job(
...     job_name=training_job.name,
...     poll_interval=15,    # Check every 15 seconds
...     poll_timeout=600     # Wait up to 10 minutes
... )
>>>
>>> if final_job.status == "SUCCESS":
...     print("Training completed successfully!")
... else:
...     print(f"Training failed with status: {final_job.status}")
poll_training_job(job_name)[source]

Get the current status of a training job.

Parameters:

job_name (str) – The job name or ID. If it contains ‘/’ it’s treated as a full name (e.g. ‘organizations/org-name/router_training_jobs/job-id’) and the last part is used as ID.

Returns:

The current state of the training job. Check the status field to determine

if the job is still running (“RUNNING”), completed successfully (“SUCCESS”), or failed (“FAILURE”, “FAILURE_WITHOUT_RETRY”).

Return type:

RouterTrainingJob

Raises:

  • ResourceNotFoundError – If the training job doesn’t exist.

  • httpx.HTTPError – If the request fails.

  • httpx.TimeoutException – If the request times out.