DEV AI Assistant

Introduction

The Hydden AI Assistant is a comprehensive artificial intelligence framework integrated into the Hydden identity platform. It provides a unified abstraction layer for working with multiple AI providers, orchestrating intelligent agents, executing tools, managing vector embeddings, and enabling semantic search capabilities. The module is designed to support both automated identity governance workflows and interactive user assistance.

Architecture

The AI module follows a layered architecture inspired by modern agent frameworks (LangGraph, Eino, OpenAI Swarm):

Diagram description: A top-down layered architecture diagram showing five layers of the AI module. The Application Layer (Identity Posture Analysis, Mapping Assistants, Chat UI) connects down to the Agent Layer (Orchestration, Multi-turn Conversations, Sessions), then to the Tool Execution Layer (Tool Registry, Parallel Execution, UI Tools, RPC), then to the Provider Layer (OpenAI, Anthropic, Google AI, Ollama, Grok), and finally to Supporting Services (Embeddings, Vector Store, Document Extraction, Storage).

Core Components

1. Provider Abstraction Layer

The Provider interface provides a unified API for interacting with multiple LLM providers, abstracting away provider-specific details.

Supported Providers

1. OpenAI - GPT-4, GPT-4 Turbo, GPT-3.5
2. Anthropic - Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Haiku
3. Google AI - Gemini 2.0 Flash, Gemini Pro
4. Ollama - Local open-source models
5. Grok - xAI's Grok models

Provider Interface

type Provider interface {
    Complete(ctx context.Context, request Request) (*Message, error)
    Embed(ctx context.Context, request EmbeddingRequest) (*EmbeddingResponse, error)
    EmbeddingModel() string
    Model() string
}

Key Features:

• Unified message format: Single format works across all providers
• Automatic translation: Provider-specific formats handled internally
• Tool calling: Native tool support with provider-specific optimizations
• Streaming: Support for streaming responses (where available)
• Retry logic: Built-in retry with exponential backoff for rate limits
• Caching: Prompt caching for Anthropic (reduces costs)
• Response formatting: Support for JSON output modes

2. Agent Orchestration

The Agent component orchestrates complete conversation loops with automatic tool execution.

Agent Configuration

type AgentConfig struct {
    // Core
    Name         string           // Agent name
    SystemPrompt string           // Agent instructions
    Provider     Provider         // LLM provider
    ToolExecutor *ToolExecutor    // Available tools

    // Execution Control
    MaxIterations          int
    Timeout                time.Duration
    AllowParallelToolCalls bool

    // Final Iteration Handling
    FinalIterationStrategy FinalIterationStrategy
    FinalIterationPrompt   string
    OnMaxIterationsReached func(...)

    // Observability Callbacks
    OnBeforeComplete       func(...)
    OnAfterComplete        func(...)
    OnBeforeToolExecution  func(...)
    OnAfterToolExecution   func(...)
    OnError                func(...)

    // Human-in-the-Loop
    RequireToolApproval bool
    ApprovalFunc        func(...) (bool, error)
}

Agent Workflow

The agent implements an autonomous loop:

1. LLM Call - Send messages to the LLM provider
2. Tool Detection - Check if LLM wants to call tools
3. Tool Execution - Execute requested tools (parallel or sequential)
4. Result Integration - Add tool results to conversation
5. Loop Continue - Repeat until completion or max iterations
6. Final Response - Return final answer to user

Final Iteration Strategies

When an agent reaches max iterations without completing:

• FinalIterationNoTools (Default): Omits tools on final call to force completion
• FinalIterationExplicitPrompt: Adds explicit "final iteration" message
• FinalIterationError: Returns error if task incomplete
• FinalIterationPartial: Returns partial results with metadata

3. Tool Execution Framework

The ToolExecutor manages tool registration, validation, and execution.

Tool Definition

type Tool struct {
    Name        string          // Unique tool name
    Description string          // What the tool does
    Parameters  any             // JSON Schema for parameters
    Handler     ToolHandler     // Execution function
    IsUITool    bool            // Requires user interaction
}

type ToolHandler func(ctx context.Context, args map[string]any) (any, error)

Tool Types

Standard Tools - Execute automatically:

• Database queries
• API calls
• File operations
• Calculations
• Data transformations

UI Tools - Require user interaction:

• Confirmations
• Form inputs
• File selections
• Multi-choice selections
• Notifications

Tool Execution Modes

Sequential Execution:

results, err := executor.Execute(ctx, toolCalls)

Parallel Execution:

results, err := executor.ExecuteParallel(ctx, toolCalls)

Built-in Tool Categories

The Hydden platform provides numerous built-in tools:

1. Identity Tools

   • Query users, groups, roles
   • Search accounts
   • Analyze access patterns
   • Check entitlements

2. Collector Tools

   • Run collectors
   • Test connections
   • Validate configurations
   • Get collection status

3. Configuration Tools

   • Read/write configurations
   • Manage credentials
   • Update settings

4. Query Tools

   • Execute SQL queries
   • Run graph queries
   • Search entities

5. Document Tools

   • Search documentation
   • Retrieve help content
   • Find examples

4. Session Management

Sessions enable multi-turn conversations with full context preservation.

Session Features

• State Persistence: Full conversation history maintained
• Serialization: Sessions can be saved and resumed
• Context Tracking: Automatic context management
• UI Tool Support: Pause/resume for user interactions
• Error Recovery: Orphaned tool call detection and repair

Session API

// Create session
session := agent.NewSession()

// Multi-turn conversation
response1, _ := session.Send(ctx, "What users have admin access?")
response2, _ := session.Send(ctx, "Which of those logged in today?")
response3, _ := session.Send(ctx, "Send them a notification")

// Get full history
messages := session.Messages()

// Export/import
data := session.Export()
agent.LoadSession(data)

UI Tool Workflow

When a UI tool is needed:

1. Session.Send() returns with RequiresUIInteraction=true
2. Application displays UI based on ToolCalls
3. User provides input
4. Application calls Session.Resume(ctx, results)
5. Session continues automatically

5. Embedding & Vector Store

The AI module includes a complete vector database for semantic search.

Embedding Generation

Supported Providers:

• OpenAI (text-embedding-3-small, text-embedding-3-large)
• Anthropic (via Voyage AI)
• Google AI (text-embedding-004)
• Ollama (local models)

Embedding Types:

• Document: For content to be indexed/stored
• Query: For search queries (some models use different prefixes)

Vector Store Architecture

Diagram description: A top-down diagram showing the Vector Store Module structure. The module contains Collections (named vector spaces), which connect to three components: Documents (with embeddings), Metadata (JSON attributes), and an HNSW Index (for fast similarity search).

Features:

• HNSW Indexing: Hierarchical Navigable Small World graphs for fast ANN search
• Metadata Filtering: Filter by attributes before similarity search
• Multiple Collections: Separate vector spaces for different use cases
• Built-in Collections: Pre-populated with documentation and examples
• Real-time Updates: Add/update/delete documents dynamically

Collection Operations

// Create collection
collection := &api.Collection{
    Name:        "documentation",
    Description: "Product documentation",
    Model:       "text-embedding-3-small",
}

// Add documents
document := &api.Document{
    Name:       "user-guide",
    Collection: collection.ID,
    Content:    "How to use the product...",
}

// Semantic search
query := "How do I reset my password?"
results := vectorStore.Query(collection.ID, query, 5)

6. Document Extraction

The extraction system converts various document formats into text chunks suitable for embedding.

Supported Formats

1. Text - Plain text with intelligent chunking
2. JSON - Flattened key-value pairs with context
3. JSON Schema - Complete schema definitions
4. HTML - Clean text extraction with structure preservation
5. Markdown - Chunked by headers
6. CSV - Row-based document extraction
7. PDF - Text extraction with cleanup (dehyphenation, sentence reconstruction)

Extraction Features

• Smart Chunking: Preserves semantic boundaries (paragraphs, sentences)
• Overlap: Configurable overlap between chunks for context
• Structure Preservation: Maintains document hierarchy
• Cleanup: Automatic whitespace normalization, header/footer removal
• Configurable: Chunk size, overlap, minimum chunk size

Extraction Configuration

config := ExtractionConfig{
    MaxChunkSize:      2000,  // ~500 tokens
    OverlapSize:       200,   // 10% overlap
    PreserveStructure: true,
    MinChunkSize:      100,
}

chunks := ExtractWithConfig(data, DocumentTypePDF, config)

7. Built-in Agents & Prompts

The platform includes specialized agents for identity governance tasks:

Identity Posture Agent

Purpose: Analyze identity security posture and provide recommendations

Capabilities:

• Analyze access patterns
• Identify over-privileged accounts
• Detect inactive accounts
• Find orphaned accounts
• Recommend access reviews
• Generate compliance reports

Mapping Assistant

Purpose: Help create data mappings between source systems and Hydden entities

Capabilities:

• Analyze source schemas
• Suggest field mappings
• Generate mapping rules
• Validate transformations
• Create test cases
• Generate sandboxed Python code

Code Completion Assistant

Purpose: Provide intelligent code completion for sandboxed Python scripts

Capabilities:

• Context-aware suggestions
• API documentation lookup
• Example code generation
• Syntax help
• Best practices recommendations

Prompt Templates

Prompts are stored as reusable templates with:

• Variables: Dynamic content substitution
• Context: Relevant documentation injection
• Examples: Few-shot learning samples
• Constraints: Output format requirements

8. Integration with Hydden Platform

The AI module deeply integrates with Hydden's identity platform:

RPC Integration

All AI capabilities exposed via RPC:

• ai.agent.stream - Streaming agent responses
• ai.agent.session - Session management
• ai.vector.query - Semantic search
• ai.embed - Embedding generation
• ai.extract - Document extraction

Tool Integration

Platform tools available to agents:

• Identity Queries: Query the identity graph
• Collector Operations: Run and manage collectors
• Configuration Management: Read/write system config
• Audit Logging: Log AI operations for compliance

Authentication & Authorization

• Per-tenant AI config: Separate API keys per tenant
• Credential Management: Secure storage via vault
• Audit Logging: All AI operations logged
• Permission Checks: Tools respect user permissions

Multi-tenancy

• Tenant Isolation: Each tenant has separate:
  • AI configurations
  • Vector collections
  • Agent sessions
  • Tool access
• Grid Owner Support: Primary tenant fallback for shared resources

Use Cases

1. Interactive Identity Analysis

Scenario: Security analyst investigating access patterns

agent := NewAgent(AgentConfig{
    Name:         "IdentityAnalyst",
    SystemPrompt: "You are an identity security expert...",
    Provider:     anthropic.New(...),
    ToolExecutor: executor, // Has identity query tools
})

session := agent.NewSession()

// Analyst asks questions in natural language
session.Send(ctx, "Show me users with admin access")
session.Send(ctx, "Which haven't logged in for 90 days?")
session.Send(ctx, "Generate a review for their manager")

2. Automated Compliance Reporting

Scenario: Generate compliance reports automatically

agent := NewAgent(AgentConfig{
    Name:         "ComplianceReporter",
    SystemPrompt: "Generate compliance reports...",
    Provider:     openai.New(...),
})

response := agent.Run(ctx, "Generate SOX compliance report for Q4")

3. Semantic Documentation Search

Scenario: Find relevant documentation from natural language queries

// User asks: "How do I configure LDAP authentication?"

// 1. Generate query embedding
queryEmbed := provider.Embed(ctx, EmbeddingRequest{
    Input:         []string{query},
    EmbeddingType: EmbeddingTypeQuery,
})

// 2. Search vector store
results := vectorStore.Query("documentation", queryEmbed, 5)

// 3. Use results to answer question

4. Intelligent Mapping Creation

Scenario: Map source data to identity entities

agent := NewAgent(AgentConfig{
    Name:         "MappingAssistant",
    SystemPrompt: "You help create data mappings...",
})

response := agent.Run(ctx, `Create a mapping from this JSON:
{
  "user_name": "jdoe",
  "email_addr": "jdoe@example.com",
  "dept": "Engineering"
}
to a Hydden UserAccount`)

5. Collector Configuration Assistant

Scenario: Help configure a new collector

// Agent has access to:
// - Collector documentation
// - Example configurations
// - Validation tools
// - Test connection tools

session.Send(ctx, "Help me configure a SQL collector for Active Directory")
session.Send(ctx, "Test the connection")
session.Send(ctx, "Generate the sandboxed Python script to collect users")

Advanced Features

1. Prompt Caching (Anthropic)

The Anthropic provider automatically caches:

• System prompts: Static instructions cached
• Tool definitions: Tool list cached (marked on last tool)
• Large tool results: First substantial result cached (>1024 chars)

Benefits:

• Reduced token costs (50-90% savings)
• Faster response times
• Efficient multi-turn conversations

2. Parallel Tool Execution

Execute multiple independent tools concurrently:

config := AgentConfig{
    AllowParallelToolCalls: true,
}

// Agent can call multiple tools at once
// Example: "Check users in US, UK, and Germany"
// -> 3 queries execute in parallel

3. Human-in-the-Loop Approval

Require approval before executing sensitive operations:

config := AgentConfig{
    RequireToolApproval: true,
    ApprovalFunc: func(ctx context.Context, calls []ToolCall) (bool, error) {
        for _, call := range calls {
            if call.Name == "delete_user" {
                fmt.Printf("Approve deletion of %v? (y/n): ", call.Arguments)
                var response string
                fmt.Scanln(&response)
                return response == "y", nil
            }
        }
        return true, nil
    },
}

4. Observability Callbacks

Monitor agent behavior in production:

config := AgentConfig{
    OnBeforeComplete: func(ctx context.Context, messages []Message) error {
        metrics.IncrementLLMCalls()
        return nil
    },

    OnAfterToolExecution: func(ctx context.Context, result *ToolResult) error {
        if result.IsError {
            metrics.IncrementToolErrors(result.ToolCallID)
        }
        log.Info("Tool executed", "name", result.ToolCallID)
        return nil
    },
}

5. Response Format Control

Request structured output:

// JSON object mode
response := session.Send(ctx, "List top 5 users",
    WithResponseFormat(ResponseFormatJSONObject))

// JSON schema mode (OpenAI)
response := session.Send(ctx, "Get user details",
    WithResponseFormat(ResponseFormatJSONSchema))

6. Session Serialization

Save and restore conversations:

// Save session
sessionData := session.Export()
json, _ := json.Marshal(sessionData)
saveToDatabase(json)

// Later: restore session
json, _ := loadFromDatabase()
var sessionData SessionData
json.Unmarshal(json, &sessionData)
session := agent.LoadSession(sessionData)

// Continue conversation
session.Send(ctx, "Continue from where we left off")

Configuration & Deployment

Provider Configuration

Each tenant configures their AI providers:

message Provider {
  string id = 1;
  string name = 2;
  string provider = 3;      // "openai", "anthropic", etc.
  RawMessage options = 4;   // Provider-specific options
  Credential credential = 6; // API key reference
}

Module Configuration

AI module settings per tenant:

message Config {
  string id = 1;
  string provider = 2;      // Default provider ID
  string model = 3;         // Default model
  bool enabled = 4;
  string vault_secret = 5;  // Encrypted API keys
}

Built-in Documentation

The system includes embedded documentation for:

• sandboxed Python API: Complete language reference
• Collector API: How to write collectors
• Mapping Guide: Creating data mappings
• Examples: Real-world scripts
• Best Practices: Recommendations

Performance Considerations

Token Management

• Chunk size: Default 2000 chars ≈ 500 tokens
• Context windows:
  • GPT-4: 128k tokens
  • Claude 3.5 Sonnet: 200k tokens
  • Gemini 2.0 Flash: 1M tokens

Cost Optimization

1. Prompt caching: Reduce token costs (Anthropic)
2. Smaller models: Use Haiku/3.5-turbo for simple tasks
3. Max iterations: Limit agent loops
4. Tool batching: Execute multiple tools in parallel
5. Local models: Use Ollama for development/testing

Latency Optimization

1. Streaming responses: Show partial results immediately
2. Parallel tools: Execute independent tools concurrently
3. Provider selection: Choose fastest provider for use case
4. Caching: Cache frequent queries in vector store
5. Connection pooling: Reuse HTTP connections

Scalability

• Stateless agents: Each request independent
• Session storage: KV store for conversations
• Vector store: HNSW indexes scale to millions of vectors
• Multi-tenant: Complete isolation per tenant
• Horizontal scaling: Module can run on multiple nodes

Security & Compliance

Credential Protection

• Vault storage: API keys encrypted at rest
• No logging: Credentials never logged
• Rotation: Support for key rotation
• Per-tenant: Separate keys per tenant

Audit Logging

All AI operations logged:

• LLM requests/responses
• Tool executions
• Session creation/updates
• Configuration changes
• Access to sensitive data

Permission Enforcement

• Tool-level: Each tool checks permissions
• Entity-level: Tools respect data access controls
• Tenant isolation: Complete separation
• Human-in-the-loop: Optional approval workflows

Data Privacy

• No training: Opt-out of training on all providers
• Retention: Control message retention
• Sanitization: Automatic PII removal (optional)
• Regional compliance: Provider selection per region

Forward Direction: Modern Agentic Orchestration

The AI module is evolving toward a modern agentic orchestration model. This section describes the architectural direction, informed by patterns from advanced agent systems such as Claude Code, OpenAI Agents SDK, and AutoGen.

Architectural Principles

Three concepts compose the full orchestration model:

• Agent (system prompt + provider) — The system prompt defines who an agent is: its role, reasoning style, and domain knowledge. This is agent-level and is not shared.
• Skill (reusable tool bundle) — A named, versioned capability that encapsulates a set of tools and an optional prompt hint. Skills can be attached to multiple agents without repeating tool declarations.
• Sub-agent (agent-as-tool) — Register one agent as a callable tool within another to enable multi-agent orchestration. The parent agent's system prompt governs when and why it delegates. The child agent runs its own conversation loop with its own provider, tools, and iteration budget.

Skills

A skill is a first-class configuration entity. It encapsulates:

• A coherent set of tools grouped around a specific domain of capability
• An optional system prompt fragment that hints at how to use the capability
• Optional provider or model constraints appropriate to the skill's scope

Skills are attached to agents at configuration time. An agent with a skill gains access to all of that skill's tools and prompt context. The same skill can be attached to many agents, eliminating the need to repeat tool registration.

Example IAM Skills:

Skill	Tools it provides
access-policy-review	Query roles, entitlements, policy assignments
credential-hygiene	Identify stale, over-scoped, or unrotated credentials
audit-log-reasoning	Query and interpret audit event streams
compliance-assessment	Evaluate evidence against control frameworks
collector-framework	Read collector docs, browse existing collector configs
starlark-tools	Write, validate, and lint Starlark scripts
api-explorer	Make HTTP requests, introspect API schemas, test endpoints

Sub-agents

A sub-agent is a fully configured Agent entity registered as a tool within a parent agent at runtime. From the parent's perspective, calling a sub-agent is identical to calling any other tool: it passes a goal and receives a result.

The child agent runs its own conversation loop with its own system prompt, skills, provider, and iteration budget. This keeps the child isolated from the parent's context window.

Structured Return via Output Tool: When a sub-agent has a defined output schema, a special return_result tool is registered in the child agent's executor. The child must call this tool to signal completion, producing typed arguments that conform to the schema. This gives pipeline stages type-safe, validated handoffs between agents.

Planning Mode

Complex tasks benefit from reasoning before tool execution. Two mechanisms support planning:

Mode	Behavior
auto	Use extended thinking if the provider supports it; fall back to the plan tool otherwise
thinking	Extended thinking only; error if provider does not support it
tool	Explicit plan tool always, regardless of provider capability
(unset)	Planning disabled; agent acts immediately (current behavior)

Extended thinking — Providers that support native extended thinking (Anthropic Claude 3.7+) are given a token budget for internal reasoning before producing a response.

Explicit plan tool — For providers without extended thinking, a create_plan tool is registered. The system prompt instructs the agent to call this tool before taking any other action. The plan is stored in session state and surfaced in the administration UI.

Context Window Management

Long sessions accumulate message history that can exceed model context limits. Two mechanisms address this:

• Message windowing — A MaxContextMessages limit trims the oldest non-system messages before each LLM call.
• Summarization — Configure a SummarizeAt token threshold to enable summarization. The loop replaces dropped messages with a single summary to preserve semantic continuity.

Orchestration Patterns

The sub-agent-as-tool mechanism enables three orchestration patterns:

Supervisor-Worker — A lightweight routing agent classifies requests and delegates to specialized workers via sub-agent tools.

Diagram description: A top-down diagram showing the Supervisor-Worker orchestration pattern. A Request flows to a Router Agent (using a fast model), which delegates to three specialized worker agents via tool calls: IAM Policy Agent, Credential Audit Agent, and Compliance Report Agent.

Parallel Fork-Join — A coordinator calls multiple sub-agent tools concurrently and synthesizes their results.

Diagram description: A top-down diagram showing the Parallel Fork-Join orchestration pattern. A Coordinator Agent dispatches parallel calls to three sub-agents: AWS Access Agent, Azure Access Agent, and Okta Access Agent. All three agents feed their results into a single Results Synthesized node.

Sequential Pipeline — An orchestrating agent calls pipeline stages in order, passing each stage's structured output to the next.

Diagram description: A left-to-right diagram showing the Sequential Pipeline orchestration pattern. An Orchestrator calls pipeline stages in order: first the Research Agent, which passes structured output to the Planning Agent, which in turn passes structured output to the Implementation Agent.

Phased Implementation

The features described above are sequenced in dependency order. Each phase is independently releasable:

Phase	Feature	Dependency
1	Planning Mode	None
2	Context Window Management	None
3	Skills	None
4	Sub-agents	Phase 3
5	Streaming Responses	None
6	Proactive & Context-Aware Assistance	Phase 5

Summary

The Hydden AI Module is a production-ready, enterprise-grade AI framework that brings advanced language model capabilities to identity governance. Its key strengths include:

Flexibility:

• Multi-provider support (5 providers)
• Pluggable tool system
• Configurable agents
• Extensible architecture

Power:

• Autonomous agent orchestration
• Automatic tool execution
• Semantic search with vectors
• Multi-turn conversations

Reliability:

• Retry logic and error handling
• Session persistence
• Orphaned call recovery
• Comprehensive testing

Security:

• Credential protection
• Audit logging
• Permission enforcement
• Multi-tenant isolation

Integration:

• Deep platform integration
• RPC-based architecture
• Built-in identity tools
• Reusable components

This makes it ideal for organizations seeking to leverage AI for identity governance, compliance automation, security analysis, and user assistance while maintaining enterprise-grade security and reliability.