Appearance

October 2025 ​

🎯 Major Features ​

Evaluation & Testing System ​

We're excited to introduce a comprehensive Evaluation System that transforms how you test and improve your AI peers.

Key Features:

  • Evaluation Suites: Organize your tests into reusable suites with automatic scoring and detailed analytics
  • Multiple Evaluator Types:
    • Exact Match: Precise keyword and phrase matching
    • LLM Judge: AI-powered semantic evaluation using GPT-4o
    • Semantic Similarity: Vector-based similarity scoring
  • Bulk Import: Import test questions via CSV or JSON files for rapid suite creation
  • Comprehensive Metrics: Track pass rates, average scores, response times, and identify improvement areas
  • Historical Tracking: Compare evaluations over time to measure peer improvement

Benefits:

  • Systematically test peer performance before deployment
  • Identify weak areas and optimize responses
  • Ensure consistent quality across updates
  • Reduce manual testing time by 90%

Documentation:

AI-Powered Analysis & Optimization ​

Take peer optimization to the next level with AI-Powered Analysis that automatically suggests improvements.

Key Features:

  • Automated Analysis: GPT-4o analyzes evaluation results and peer configuration
  • Actionable Suggestions: Receive specific, prioritized improvement recommendations
  • One-Click Application: Apply suggestions individually or in bulk
  • Impact Estimates: See expected performance improvements before applying changes
  • Suggestion Types:
    • System prompt enhancements
    • Datasource recommendations
    • Temperature and parameter adjustments
    • Tool integration suggestions

Real Results:

  • Average improvement: +15-25% in peer performance
  • Optimization time reduced from hours to minutes
  • Data-driven insights replace trial-and-error

Documentation:

Widget System ​

Embed your AI peers anywhere with the new Widget System - fully customizable React components for seamless integration.

7 Pre-Built Widget Types:

  1. Chat Widget: Classic chat interface for customer support and assistance
  2. Search Widget: Google-style search for knowledge bases
  3. Form Widget: Multi-step forms with AI-powered assistance
  4. Q&A Widget: Stack Overflow-style question and answer interface
  5. Sidebar Widget: Persistent sidebar for multi-step interactions
  6. Popup Widget: Modal-style widget for focused interactions
  7. Inline Widget: Embedded directly in page content

Key Features:

  • Full Customization: Match your brand with custom themes, colors, and styling
  • Dynamic Context: Pass real-time user data for personalized interactions
  • Event Tracking: Monitor widget interactions and conversions
  • Lightweight: Optimized bundle sizes (18-45KB gzipped)
  • Programmatic Control: JavaScript API for complete widget management

Use Cases:

  • Customer support chat on any page
  • Documentation search
  • E-commerce shopping assistants
  • Lead qualification forms
  • Interactive help systems

Documentation:

Loop Steps for Workflow Automation ​

Build more powerful workflows with Loop Steps - iterate over data and automate repetitive tasks.

Each Loop (Array Iteration):

  • Iterate over arrays and collections
  • Execute nested steps for each item
  • Collect results and aggregate data
  • Break conditions for early termination
  • Error handling per iteration

While Loop (Conditional Iteration):

  • Continue until a condition is met
  • Polling and retry mechanisms
  • Pagination handling
  • API rate limiting and delays
  • Maximum iteration safety limits

Common Use Cases:

  • Process multiple files or documents
  • Poll APIs until completion
  • Retry failed operations
  • Paginate through large datasets
  • Batch process records

Performance Features:

  • Parallel execution support
  • Progress tracking
  • Error isolation
  • Memory-efficient processing

Documentation:

Vector Storage Providers ​

Choose the right vector storage for your needs with support for Multiple Vector Providers.

Available Providers:

  1. System Default (MongoDB Atlas)

    • Zero configuration, included in platform
    • Perfect for getting started (up to 100K vectors)
    • No additional costs

  2. Pinecone

    • Production-grade, unlimited scale
    • Sub-100ms queries at billions of vectors
    • Managed service with 99.9% uptime SLA
    • Best for: High-scale production applications

  3. Qdrant

    • Self-hosted or managed cloud options
    • Excellent performance with full data control
    • Cost-effective at scale
    • Best for: Privacy-conscious teams, on-premise deployments

  4. Elasticsearch ⭐ Enhanced

    • Hybrid search (vector + full-text)
    • Three deployment options:
      • Elastic Cloud: Fully managed, auto-scaling
      • Self-Hosted: Full control, on-premise
      • Legacy: Backward compatibility
    • Advanced filtering and aggregations
    • Production-ready with high availability
    • Best for: Existing Elasticsearch infrastructure, hybrid search needs

  5. Orama

    • Runs in browser or edge workers
    • Ultra-fast client-side search
    • Offline capability
    • Best for: Documentation sites, edge computing

  6. S3 Vectors

    • Cheapest storage ($0.023/GB/month)
    • Unlimited scale for massive archives
    • Best for: Infrequent queries, archival search

Key Features:

  • Easy Migration: Zero-downtime migration between providers
  • Dual-Write: Write to multiple providers during migration
  • Decision Framework: Comprehensive guide to choose the right provider
  • Cost Optimization: Tips for reducing storage and query costs

Documentation:

🔧 API Enhancements ​

New API Endpoints ​

Evaluation API:

  • POST /api/v1/evaluation/suite - Create evaluation suite
  • GET /api/v1/evaluation/suite/:id - Get suite details
  • POST /api/v1/evaluation/:suiteId/run - Execute evaluation
  • GET /api/v1/evaluation/run/:runId - Get run results
  • POST /api/v1/evaluation/:runId/suggest-improvements - Get AI suggestions
  • Full CRUD operations for questions and evaluators

Widget API:

  • POST /api/v1/widget - Create widget
  • GET /api/v1/widget/:id - Get widget configuration
  • PUT /api/v1/widget/:id - Update widget
  • DELETE /api/v1/widget/:id - Delete widget
  • GET /api/v1/widget/:id/analytics - Get widget analytics
  • Real-time WebSocket support for widget interactions

Documentation:

💻 User Interface Improvements ​

Datasource Enhancements ​

  • Playground Feature: Test queries and preview results before deployment ⭐ New
    • Real-time query testing with relevance scores
    • Configurable result limits (1-20)
    • Full content vs. chunk comparison
    • Debug and optimize datasource configuration
    • See Datasource Playground Guide

WebChat Improvements ​

  • Feedback Buttons: Thumbs up/down for AI responses ⭐ New
    • Collect user satisfaction metrics
    • Identify problem areas automatically
    • Anonymous feedback collection
    • Integrated with analytics dashboard
    • Localization support for global users
    • See WebChat Integration Guide

Evaluation System UI ​

  • Suite Management: Create, edit, and organize evaluation suites
  • Question Editor: Bulk import via CSV/JSON or manual entry
  • Results Dashboard: Comprehensive metrics with visual charts
  • Analysis Modal: View AI suggestions with impact estimates
  • History Tracking: Compare results over time

Widget Management UI ​

  • Visual Editor: Create widgets with live preview
  • Code Generator: Auto-generate embed code
  • Theme Customizer: Visual theme editor with real-time preview
  • Analytics Dashboard: Track widget usage and conversions
  • Permission Management: Control widget access and sharing

Flow Editor Enhancements ​

  • Loop Step UI: Visual loop configuration with nested step support
  • Persistent Side Panel: Inspect nodes and view validation errors ⭐ New
  • Step Selector Popover: Quick step selection with search ⭐ New
  • Variable Explorer: See all available variables in context
  • Debug Mode: Step-by-step execution with variable inspection
  • Performance Metrics: Track execution time per step
  • Improved DAG Layout: Better visualization of complex flows ⭐ New

Design & UX Improvements ⭐ New ​

  • Enhanced form layouts across model and vector index pages
  • Improved active menu indicators and navigation
  • Better spacing and alignment in settings pages
  • Responsive design improvements for mobile devices
  • Code formatting and readability across all components

📊 Performance & Optimization ​

Query Performance ​

  • Vector Search: Up to 3x faster with optimized indexing
  • Evaluation Runs: Parallel execution for faster results
  • Widget Loading: Lazy loading and code splitting

Cost Optimization ​

  • Token Usage: Reduced token consumption in evaluations
  • Storage Efficiency: Compressed vector storage
  • Caching: Improved caching for repeated queries

Scalability ​

  • Horizontal Scaling: Better load distribution across services
  • Resource Management: Optimized memory usage in loop operations
  • Connection Pooling: Improved database connection handling

🔒 Security & Compliance ​

Widget Security ​

  • API Key Management: Separate public/private keys
  • CORS Configuration: Granular origin control
  • Rate Limiting: Prevent abuse with configurable limits
  • Data Privacy: GDPR-compliant data handling

Evaluation Security ​

  • Test Data Isolation: Secure storage of test questions
  • Result Access Control: Role-based access to evaluation results
  • Audit Logging: Track all evaluation activities

🐛 Bug Fixes & Improvements ​

General Fixes ​

  • Fixed race conditions in loop step execution
  • Resolved memory leaks in long-running evaluations
  • Corrected widget theme inheritance issues
  • Fixed CSV import encoding problems
  • Resolved vector migration data consistency issues

UI/UX Fixes ​

  • Improved error messages across evaluation system
  • Fixed widget preview rendering in Safari
  • Corrected pagination in evaluation history
  • Enhanced mobile responsiveness for widget configuration
  • Fixed dark mode styling inconsistencies

📚 Documentation Updates ​

New Documentation (62,500+ words) ​

  • 6 User Guides: Evaluation, AI Analysis, Widgets, Each Loop, While Loop, Vector Providers
  • 2 API References: Evaluation API, Widget API
  • 4 Blog Posts: Testing, Optimization, Widgets, Vector Storage
  • Updated Guides: Peer overview, Settings, Datasource, App steps

Documentation Improvements ​

  • Added comprehensive examples to all guides
  • Included troubleshooting sections
  • Created decision frameworks and comparison matrices
  • Added real-world use cases and success stories

🚀 Migration Guide ​

Upgrading to October 2025 ​

No Breaking Changes - All new features are additive and fully backward compatible.

Recommended Actions:

  1. Explore Evaluation System: Create a test suite for your critical peers
  2. Try AI Analysis: Run analysis on existing evaluations
  3. Test Widgets: Create a widget for your most-used peer
  4. Review Vector Providers: Consider migration if you have > 100K vectors
  5. Update Documentation: Check new guides for optimization tips

Optional Migrations:

  • Vector Storage: Migrate to dedicated provider if needed (see migration guide)
  • Widget Integration: Replace iframe embeds with new widget system
  • Workflow Loops: Refactor repeated steps to use loop steps

🔮 Coming Soon ​

Q4 2025 Preview ​

  • Advanced Evaluators: Custom evaluator plugins
  • A/B Testing: Built-in A/B test framework for peers
  • Multi-Language Evaluation: Support for non-English test suites
  • Widget Templates: Pre-configured widgets for common use cases
  • Evaluation Automation: Scheduled evaluation runs
  • Vector Search Analytics: Query performance insights

📖 Resources ​

Getting Started ​

API Documentation ​

Blog Posts ​

Community ​

💬 Feedback ​

We'd love to hear your thoughts on these new features:

Thank you for being part of the Cognipeer community!

These updates represent months of development based on your feedback. We're excited to see what you build with these new capabilities.

For detailed commit history, see our GitHub repository.

Built with VitePress

© 2025 cognipeer. All rights reserved.