Elephant and Monkey story in research| Dataknobs


Elephant and Monkey story

There were 2 villages. First village had an elephant. Second village had a monkey. Everyday elephants and a monkey used to find food in the jungle for their villagers.

Elephant used to go thru known paths in the Jungle. If an elephant found food, he could easily get it by the power of his trunk. Elephant could carry plenty for the whole village. If an elephant could not find food in a given direction, it was very difficult to try a second direction. Villagers have to sleep without eating food if the elephant does not find food in two attempts.

Villagers were unhappy with the Elephant as they could not get food everyday.

On another hand, the 2nd village used to get some food every day. Their Monkey was quick. Monkey used to climb and jump through trees. If Monkey did not find food in one direction, Monkey went in 20 different directions. Monkey could invent new paths. Monkey always found food. However Monkey was small. Monkey has to struggle to pluck food from trees. Monkey have to make multiple trips to carry food. Even after multiple trips the monkey could not carry enough for the whole village.

Villagers were unhappy with Monkey too as Monkey could not bring required food.

A tourist went to both villages and noticed both methods. He brought two villages together and asked Elephant and Monkey to collaborate.

Everyday in Morning, Monkey used to go in different directions and check where food was. Then Monkey used to sit on the Elephant back and navigate him to go to a place where food is. Elephant could pluck and carry plenty of food for both villages.

Both villagers were happy with this approach.

In above analogy

  • Elephant = Waterfall method, Monkey = Agile, Villagers = customers, Tourist = Data science Leader.

  • Elephant = Execution, Monkey = Research. Villagers = Customers, Tourist = Data science leader

  • We go through the same situation in Machine Learning and in research projects. We do not know which experiment will give the result. We should use a light weight agile approach in areas of ambiguity & research. (monkey approach)

  • We should resolve ambiguous items in the beginning. As we get confidence in experiment outcomes, we should use a more structured & heavy weight approach to get results. In the beginning of research it is hard to find how much food (ROI) a team will get, after the research team can draw conclusions on what ROI, Success criteria can be established for the project.


    Link on my Linkedin post - https://www.linkedin.com/pulse/monkey-elephant-story-manage-machine-learning-prashant-k-dhingra/

  • From the blog

    Build Dataproducts

    How Dataknobs help in building data products

    Building data products using Generative AI (GenAI) and Agentic AI enhances automation, intelligence, and adaptability in data-driven applications. GenAI can generate structured and unstructured data, automate content creation, enrich datasets, and synthesize insights from large volumes of information. This helps in scenarios such as automated report generation, anomaly detection, and predictive modeling.

    AI Agent for Business Analysis

    Analyze reports, dashboard and determine To-do

    DataKnobs has built an AI Agent for structured data analysis that extracts meaningful insights from diverse datasets such as e-commerce metrics, sales/revenue reports, and sports scorecards. The agent ingests structured data from sources like CSV files, SQL databases, and APIs, automatically detecting schemas and relationships while standardizing formats. Using statistical analysis, anomaly detection, and AI-driven forecasting, it identifies trends, correlations, and outliers, providing insights such as sales fluctuations, revenue leaks, and performance metrics.

    KreateHub

    Create new knowledge with promot library

    At its core, KreateHub is designed to enable creation of new data and the generation of insights from existing datasets. It acts as a bridge between raw data and meaningful outcomes, providing the tools necessary for organizations to experiment, analyze, and optimize their data processes.

    Build Budget Plan for GenAI

    CIO Guide to create GenAI Budget for 2025

    CIOs and CTOs can apply GenAI in IT Systems. The guide here describe scenarios and solutions for IT system, tech stack, GenAI cost and how to allocate budget. Once CIO and CTO can apply this to IT system, it can be extended for business use cases across company.

    What is KREATE and KreatePro

    Kreate - Bring your Ideas to Life

    KREATE empowers you to create things - Dataset, Articles, Presentations, Proposals, Web design, Websites and AI Assistants Kreate is a platform inclide set of tools that ignite your creatviity and revolutionize the way you work. KReatePro is enterprise version.

    What is KONTROLS

    KONTROLS - apply creatvity with responsbility

    KONTROLS enable adding guardrails, lineage, audit trails and governance. KOntrols recogizes that different use cases for Gen AI and AI have varying levels of control requirements. Kontrols provide structure to select right controls.

    What is KNOBS

    KNOBS - Experimentation and Diagnostics

    Well defined tunable paramters for LLM API, LLM fine tuning , Vector DB. These parameters enable faster experimentation and diagosis for every state of GenAI development - chunking, embedding, upsert into vector DB, retrievel, generation and creating responses for AI Asistant.

    Kreate Articles

    Create Articles and Blogs

    Create articles for Blogs, Websites, Social Media posts. Write set of articles together such as chapters of book, or complete book by giving list of topics and Kreate will generate all articles.

    Kreate Slides

    Create Presentations, Proposals and Pages

    Design impactful presentation by giving prmpt. Convert your text and image content into presentations to win customers. Search in your knowledbe base of presentations and create presentations or different industry. Publish these presentation with one click. Generate SEO for public presentations to index and get traffic.

    Kreate Websites

    Agent to publish your website daily

    AI powered website generation engine. It empower user to refresh website daily. Kreate Website AI agent does work of reading conent, website builder, SEO, create light weight images, create meta data, publish website, submit to search engine, generate sitemap and test websites.

    Kreate AI Assistants

    Build AI Assistant in low code/no code

    Set up AI Assistant that give personized responss to your customers in minutes. Add RAG to AI assistant with minimal code- implement vector DB, create chunks to get contextual answer from your knowlebase. Build quality dataset with us for fine tuning and training a cusom LLM.

    Create AI Agent

    Build AI Agents - 5 types

    AI agent independently chooses the best actions it needs to perform to achieve their goals. AI agents make rational decisions based on their perceptions and data to produce optimal performance and results. Here are features of AI Agent, Types and Design patterns

    Develop data products with KREATE and AB Experiment

    Develop data products and check user response thru experiment

    As per HBR Data product require validation of both 1. whether algorithm work 2. whether user like it. Builders of data product need to balance between investing in data-building and experimenting. Our product KREATE focus on building dataset and apps , ABExperiment focus on ab testing. Both are designed to meet data product development lifecycle

    Innovate with experiments

    Experiment faster and cheaper with knobs

    In complex problems you have to run hundreds of experiments. Plurality of method require in machine learning is extremely high. With Dataknobs approach, you can experiment thru knobs.

    RAG For Unstructred and Structred Data

    RAG Use Cases and Implementation

    Here are several value propositions for Retrieval-Augmented Generation (RAG) across different contexts: Unstructred Data, Structred Data, Guardrails.

    Why knobs matter

    Knobs are levers using which you manage output

    See Drivetrain appproach for building data product, AI product. It has 4 steps and levers are key to success. Knobs are abstract mechanism on input that you can control.

    Our Products

    KreateBots

  • Pre built front end that you can configure
  • Pre built Admin App to manage chatbot
  • Prompt management UI
  • Personalization app
  • Built in chat history
  • Feedback Loop
  • Available on - GCP,Azure,AWS.
  • Add RAG with using few lines of Code.
  • Add FAQ generation to chatbot
  • KreateWebsites

  • AI powered websites to domainte search
  • Premium Hosting - Azure, GCP,AWS
  • AI web designer
  • Agent to generate website
  • SEO powered by LLM
  • Content management system for GenAI
  • Buy as Saas Application or managed services
  • Available on Azure Marketplace too.
  • Kreate CMS

  • CMS for GenAI
  • Lineage for GenAI and Human created content
  • Track GenAI and Human Edited content
  • Trace pages that use content
  • Ability to delete GenAI content
  • Generate Slides

  • Give prompt to generate slides
  • Convert slides into webpages
  • Add SEO to slides webpages
  • Content Compass

  • Generate articles
  • Generate images
  • Generate related articles and images
  • Get suggestion what to write next