Running AWS Strands Decider 2B Locally: A Complete Setup Guide for AI Routing & Multi-RAG Systems

Running AWS Strands Decider 2B Locally for Multi-RAG and Agentic AI Applications

Making reliable decisions before invoking a large language model (LLM) remains a challenge as GenAI applications grow more sophisticated. AWS has addressed this with Strands Decider 2B, a lightweight decision model focused on routing, classification, scoring, and orchestrating agent workflows. This setup guide outlines the process of installing and testing Strands Decider 2B locally on Windows using WSL2.

Why Use a Decision Model?

Traditionally, LLMs handle both decision-making and response generation, leading to inefficiencies. By separating concerns, the LLM focuses on reasoning and generating responses, while the decision model takes care of routing and orchestration.

Prerequisites

The guide assumes the following environment:

- Windows 11

- WSL2 Ubuntu

- Python virtual environment

- Strands Decider 2B

Installation Steps

1. Verify WSL2 Installation

- Open PowerShell and run: wsl -l -v

- This confirms that WSL2 is properly installed and running.

2. Create a Workspace Directory

- Open a terminal and execute: mkdir -p /mnt/c/GENAI/strands

- This creates a dedicated directory for the project.

3. Install Required Packages

- Update Ubuntu: sudo apt update

- Install dependencies: sudo apt install -y python3 python3-pip python3-venv python3-dev build-essential gcc g++

4. Create a Virtual Environment

- Create the environment: python3 -m venv .venv

- Activate it: source .venv/bin/activate

- Upgrade pip: pip install --upgrade pip setuptools wheel

5. Install Strands Decider

- Run: pip install strands-decider

- Verify installation by executing: strands-decider --help

6. Discover Available Models

- Install Hugging Face Hub: pip install huggingface_hub

- List available Strands models using: python -c "from huggingface_hub import list_models; [print(m.id) for m in list_models(search='strands')]"

- The specific model used in this guide: StrandsAgents/strands-decider-2B-hobson-v19

7. Start the Model Server

- Launch the model with: strands-decider serve StrandsAgents/strands-decider-2B-hobson-v19 --device cpu

- Expected output: Application startup complete. Uvicorn running on http://127.0.0.1:8000

- Note that the first startup downloads and caches the model automatically.

8. Verify the API

- Open a web browser and navigate to http://127.0.0.1:8000/docs

- Alternatively, use curl to check the OpenAPI specification: curl http://127.0.0.1:8000/openapi.json

Understanding Question Types

Strands Decider supports three decision formats:

1. Choice Question: Choose one option from a list.

- Type: choice

- Instructions: Select the best datasource.

- Criteria:

- PLM: Engineering changes and parts

- JIRA: Issue tracking system

- CONFLUENCE: Documentation repository

- UNKNOWN: No suitable source

2. Noul Question: Yes/No decision.

- Type: noul

- Instructions: Determine whether this statement is true.

3. Score Question: Rate against an ordered scale.

- Type: score

- Instructions: Rate the sentiment.

- Criteria:

- Very Negative

- Negative

- Neutral

- Positive

- Very Positive

Testing with Python

Create a file called decider_demo.py with the following content:

```python

import requests

import json

payload = {

"state": "User wants ECO information",

"questions": {

"datasource": {

"type": "choice",

"instructions": "Select the best datasource."

}

}

}

```

This payload represents a choice question where the user wants ECO information and needs to select the best datasource.