A/B Experimenting with conversational interfaces

AI Assitants


Data Product

Beyond the Split: diving into the deep end of conversational experimentation. Build better conversatons with dynamic experimentation.

AB Experiment

The playground for experimentation is vast - experiment with different LLM, different embedding provider, different ways to query vector DB.
Prompt enginering have many experiment and user try different tone, persona, design variation. br/>In addition test different visual design and see how user interact with website and chatbot.
Response design is another area to explore and determine whether user like detail, summary, graphs format.

Traditional A/B Testing

Harness the power of qualitative and quantitative data to drive the future of your Data Product. Run experiments, segment users, analyze results, and make data-driven decisions - all through A/B experiment tools or simply integrae with existing tools and dataset to use part of functionality needed.

Experiment with LLM

Access the power of multiple LLM providers with ease through Dataknobs. Our pre-built code and intuitive interface allow you to seamlessly switch between OpenAI, Gemini, Hugging Face, and more—tailoring your experience to your exact needs

Open AI

Unleash OpenAI's versatility: Craft creative content, automate tasks, search knowledge bases with reasoning, and refine models (including fine-tuning, datasets, and multi-model evaluation).


Unleash the power of Google's Gemini LLM to create intelligent search apps, engaging conversational agents, and personalized recommendation systems that anticipate user needs.
Partner with Dataknobs to unlock the full potential of Gemini. Our team specializes in fine-tuning, dataset preparation, and model evaluation, ensuring optimal performance for your unique projects.

Try Vector DBs

Try different vector DB options, play with different chunk size, experiment diferent retrieval strategies.

Experiment with Vector DBs

Try available options like Pinecone, Mongo DB Vector, Chroma DB, Lance DB, Milvus, Weaviate and others. Consider factors like scalability, query speed, accuracy, ease of use, and supported data types. Select the database that best suits your specific use case, hardware constraints, and performance requirements.

Play with RAG approaches

Try different search indexing methods to find the most efficient and accurate way to retrieve relevant context for your queries. Play with how you break down your retrieved content into chunks for feeding to the LLM. Experiment with different chunk sizes and overlapping strategies

Experiment with Prompt

Prompt Engineering, Prompt Template, Temperature Control, Fusion Methods, Persona for Assistant, Persona for users, effective evaluation are capabilties ab experiment has.

Prompt Refinement and Iteration

Try different prompting styles and information inclusion strategies.Let domain expert try different prompt, change temperature to balance creativity with factuality in the generated text. Test with user profile and see whether chatbot and assitant work well with all user profile. Create and manage collections of pre-defined prompts to guide conversations and elicit specific information. Facilitate adaptive responses based on user input and conversation context.

Evaluate withh Different User Profiles

Create user profile and evaluate results on different user profiles. Track metrics like task completion rate, time saved, time to completion, user satisfaction surveys, and error rates, segmented by user profile. Evaluate and fine tune AI assistant for your user base.

A/B Experiment - Website and Assitant

A/B testing set up, generation of differnt layout, user segmentation. Use pre built tools and notebooks for - Sample Size determination, running various statistical test e.g. Chi Squared Test, Two Sample T Test, Paired T-Test, Mann Whitney U Test, Wilcoxon Signed Rank Test etc.

Pre Built Tools

Unleash data-driven decision making with our A/B testing tools: choose our cloud-based solution "abexperiment". Run insightful A/B tests with ease! Customize the underlying code with integrate notebooks.

Pre Built Packages

Deploy our package on your servers for deeper customization. Optimize your website and AI Assitant with confidence. Deploy in your enviornment without the need for moving real data to cloud.

Benefits and Impact

For Business Analyst

Raw Signal Intelligence

AI Twin goes beyond simply collecting data. It ingests raw signals from across your factory floor, from sensors on machines to environmental indicators.

For Data Scientists

Experimentation Playground

Want to test new production tweaks without risking real-world disruptions? AI Twin's virtual environment lets you experiment safely and see the potential impact before making changes.

Well governed data

Data Lineage and Extensibility

To build a commercial data product, create a base data product. Then add extension to these data product by adding various types of transformation. However it lead to complexity as you have to manage Data Lineage. Use knobs for lineage and extensibility


Higher-Level Data Products:

Forget data overload! AI Twin transforms raw data into actionable insights in the form of easily digestible data products,

Knowledge worker

For Machine and Human

AI Twin's data products are designed for both humans and machines. Business users gain clear, actionable insights, while automated processes can leverage the data for real-time optimization. With AI Twin, you can:

Business and Finance

Extend equipment life

Implement predictive maintenance to avoid costly breakdowns and keep your machines running smoothly


Pre Built Capabilities

  • Capability deployed in production
  • Pre Built Agents with configurability
  • Ability to integrate withcustomer knowledgebase
  • Capability to get user feedback
  • Conversation History and Logging
  • Login functionality
  • User Maangement
  • Prompt Templates
  • Fractional CTO for Bots

    Startup and enterprise who wish to build their own AI Asssitant can hire expertise to build

  • Conversation Agent Experise
  • LLM - OpenAI, Gemini
  • Vector DB and RAG
  • FienTuning LLM
  • Cloud - AWS, GCP,Azure
  • Customization Team

    Choose a partner with deep experience in delivering LLm based chatbot, Agents and Assistant

    Hire experts who have built kreatebots Stock, Finance AI assitant, real estate AI Agent, chatbot for travel