@sashbot/uibridge

🌉 UIBridge - AI-Powered Browser Automation for AI Agents

Primary Method: Use PowerShell helper functions for reliable, tested browser automation with built-in error handling and JSON validation.

🎯 What UIBridge Does

UIBridge provides reliable browser automation specifically designed for AI agents. It runs automation against a real browser with visual feedback and comprehensive error handling.

Perfect for: AI agents, automation testing, web scraping, form filling, UI testing.

🚀 Quick Start for AI Agents (Recommended Method)

Step 1: Install UIBridge

npm install @sashbot/uibridge

Step 2: Start the Server

# Start the UIBridge server (keep this running)
node node_modules/@sashbot/uibridge/api-server.cjs

✅ Server Status: You should see UIBridge API Server running on http://localhost:3002

Step 3: Load PowerShell Helper Functions

# Download the helper functions (includes JSON corruption fixes)
curl -o uibridge-powershell-helpers.ps1 https://unpkg.com/@sashbot/uibridge@latest/uibridge-powershell-helpers.ps1

# Load the functions (do this first in every session)
. .\uibridge-powershell-helpers.ps1

Step 4: Verify Everything Works

# Test server connection
Test-UIBridgeServer

# Start a complete automation session (opens browser and takes screenshot)
Start-UIBridgeSession -Url "https://example.com"

🎉 That's it! You now have reliable browser automation with proper error handling.

🤖 Core AI Agent Commands (Primary Method)

🔗 Connection & Navigation

# Always load helpers first in every session
. .\uibridge-powershell-helpers.ps1

# Test server connection
Test-UIBridgeServer

# Navigate to any URL
Open-UIBridgePage -Url "https://example.com"
Open-UIBridgePage -Url "http://localhost:3000"

# Get current page information
Get-UIBridgePageInfo

🖱️ Clicking Elements (Robust with Multiple Strategies)

# Click by text content (RECOMMENDED - most reliable)
Click-UIBridgeText -Text "Submit"
Click-UIBridgeText -Text "Login"
Click-UIBridgeText -Text "Save Changes"

# Click by test ID (best for testing)
Click-UIBridgeTestId -TestId "submit-btn"
Click-UIBridgeTestId -TestId "login-button"

# Click by CSS selector (when other methods don't work)
Click-UIBridgeElement -Selector "#save-button"
Click-UIBridgeElement -Selector ".submit-btn"
Click-UIBridgeElement -Selector "button[type='submit']"

📸 Screenshots (Auto-Saved with Validation)

# Basic screenshot (recommended)
Take-UIBridgeScreenshot

# Full page screenshot with options
Take-UIBridgeScreenshot -FullPage $true -Format "png" -Quality 90

# Screenshot with specific background color
Take-UIBridgeScreenshot -BackgroundColor "white"
Take-UIBridgeScreenshot -BackgroundColor "transparent"
Take-UIBridgeScreenshot -BackgroundColor "auto"  # Auto-detect page background

# List all saved screenshots
Get-UIBridgeScreenshots

🔄 Complete Automation Workflows

# Complete automation session with error handling
Start-UIBridgeSession -Url "https://example.com"

# Example: Form automation workflow
. .\uibridge-powershell-helpers.ps1
Test-UIBridgeServer
Open-UIBridgePage -Url "https://example.com/form"
Take-UIBridgeScreenshot  # Before
Click-UIBridgeText -Text "Fill Form"
Take-UIBridgeScreenshot  # After
Click-UIBridgeTestId -TestId "submit-btn"
Take-UIBridgeScreenshot  # Final result
Get-UIBridgeScreenshots  # Review all screenshots

✅ Why PowerShell Helpers Are the Primary Method

🛡️ Built-in Reliability (v1.2.5+ includes JSON corruption fixes)

• JSON Validation: Every command validates JSON before sending to prevent corruption errors
• Consistent Formatting: All JSON uses proper depth and escaping
• Error Handling: Clear error messages with troubleshooting guidance
• Special Character Support: Handles quotes, apostrophes, unicode, emoji correctly

🎯 AI Agent Optimized

• Simple Function Calls: Click-UIBridgeText -Text "Submit" instead of complex JSON
• Multiple Click Strategies: Automatic fallback between text, testId, and CSS selectors
• Auto-Save Screenshots: Timestamps and organized file management
• Connection Testing: Built-in server health checks

🧪 Comprehensive Testing

• 17 Unit Tests: JSON generation, special characters, error handling
• Integration Tests: Real server and browser testing
• Regression Prevention: Prevents the return of JSON corruption bugs

# Run tests to verify everything works
pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1

🔧 When to Use Alternative Methods

The PowerShell helpers are the recommended primary method, but use alternatives for these specific scenarios:

🌐 Live Session Method (Visual Debugging)

Use when: You need to see automation happen in real-time in your own browser

<!-- Add to your web app for live visual feedback -->
<script src="http://localhost:3002/uibridge-client.js"></script>

# Download live session helpers
curl -o uibridge-live-session-helpers.ps1 https://unpkg.com/@sashbot/uibridge@latest/uibridge-live-session-helpers.ps1
. .\uibridge-live-session-helpers.ps1

# Control YOUR browser live
Click-UIBridgeLiveText -Text "Submit"  # You see it happen in your browser!

🔧 Direct HTTP API

Use when:

• Working from non-PowerShell environments (Python, Node.js, etc.)
• Need custom JSON structures not supported by helpers
• Building your own wrapper functions

# Direct HTTP for other languages
curl -X POST http://localhost:3002/execute \
  -H "Content-Type: application/json" \
  -d '{"command":"click","selector":{"text":"Submit"}}'

📦 NPM Package Integration

Use when: Building JavaScript/TypeScript applications with UIBridge

import { UIBridge } from '@sashbot/uibridge';
const bridge = new UIBridge();
await bridge.click({ text: 'Submit' });

🚨 Troubleshooting & Error Resolution

Problem: JSON corruption errors

✅ FIXED in v1.2.5+ The PowerShell helpers now generate valid JSON consistently:

# Verify the fix works
pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1
# Should show: All tests passed! ✅

Problem: Server not accessible

Test-UIBridgeServer
# If this fails:
# 1. Restart server: node node_modules/@sashbot/uibridge/api-server.cjs
# 2. Check firewall/antivirus blocking port 3002
# 3. Try different port: . .\uibridge-powershell-helpers.ps1 -ServerUrl "http://localhost:3003"

Problem: Elements not found

# Try multiple strategies in order:
Click-UIBridgeText -Text "Button Text"        # Try this first (most reliable)
Click-UIBridgeTestId -TestId "button-id"      # If text doesn't work  
Click-UIBridgeElement -Selector "#button-id"  # If test ID doesn't work

# Take screenshot to see current page state
Take-UIBridgeScreenshot

Problem: Special characters in text

# These all work correctly with the fixed helpers:
Click-UIBridgeText -Text "Text with ""quotes"" and 'apostrophes'"
Click-UIBridgeText -Text "Button with emoji 🚀"
Click-UIBridgeText -Text "Unicode: 测试 العربية русский"

🎯 AI Agent Best Practices

✅ DO (Primary Method):

# 1. Always load helpers first
. .\uibridge-powershell-helpers.ps1

# 2. Test connection before automating
Test-UIBridgeServer

# 3. Use text-based clicking first (most reliable)
Click-UIBridgeText -Text "Submit"

# 4. Take screenshots to verify actions
Take-UIBridgeScreenshot

# 5. Use complete workflows
Start-UIBridgeSession -Url "https://example.com"

⚠️ AVOID:

• Don't construct JSON manually - use the helper functions
• Don't skip Test-UIBridgeServer - always verify connection
• Don't use complex CSS selectors if text clicking works
• Don't forget to handle errors with try/catch blocks

📊 Helper Functions Reference

Core Functions

Advanced Usage

# Custom timeout for slow pages
Click-UIBridgeText -Text "Submit" -Timeout 10000

# High quality screenshot
Take-UIBridgeScreenshot -Quality 95 -Format "png"

# Multiple screenshots for comparison
Take-UIBridgeScreenshot  # Before action
Click-UIBridgeText -Text "Change Background"
Take-UIBridgeScreenshot  # After action

🔬 Testing & Validation

Run Built-in Tests

# Test JSON generation and helper functions
pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1

# Expected result:
# 📊 Test Results Summary
# Total Tests: 17
# Passed: 17 ✅
# Failed: 0 ✅

Integration with CI/CD

# In your build scripts
$testResult = & pwsh -ExecutionPolicy Bypass -File test-powershell-helpers.ps1
if ($LASTEXITCODE -ne 0) {
    throw "UIBridge PowerShell helper tests failed"
}

🎉 Why UIBridge PowerShell Helpers Are Perfect for AI Agents

1. 🛡️ Reliability: Built-in JSON validation prevents corruption errors
2. 🎯 Simplicity: Click-UIBridgeText -Text "Submit" instead of complex APIs
3. 📸 Auto-Management: Screenshots auto-save with timestamps
4. 🔍 Error Handling: Clear error messages with troubleshooting guidance
5. 🤖 AI-Optimized: Functions designed specifically for AI agent workflows
6. ⚡ Fast Setup: Load helpers, test connection, start automating
7. 🧪 Tested: 17 unit tests ensure reliability
8. 🌐 Universal: Works with any web application

📚 Additional Resources

• 🔧 Advanced Configuration
• 🤖 AI Agent Guidelines
• 📦 NPM Package
• 🐛 GitHub Issues

🌉 Built for AI Agents: UIBridge provides the most reliable browser automation for AI agents with comprehensive error handling, JSON validation, and intuitive PowerShell functions.