Day 25 - Why Trying Different AI Models Matters: Lessons from a Real Task

By -

Context:

Initially, I thought all LLMs were pretty much the same. But after testing them on a real-world task, I realized the choice of model can significantly impact the quality of results.

What I Learned:

The Task:
Summarize the YouTube video:
“Software Testing A Web Application - Case Study - complete course video by EvilTester.”

I used the same prompt across different AI models:

  • OpenAI:
    • Searched the web, found the YouTube link, and provided a solid summary including:
      • Course overview
      • Why the case study matters
      • Actionable next steps
  • DeepSeek:
    • Used its web search feature and returned a detailed summary with:
      • Key objectives
      • Course structure
      • Testing techniques
      • Additional resources
  • Microsoft Copilot:
    • Pulled from various sources and provided:
      • Course overview
      • Key concepts
      • Learning outcomes
  • Gemini 2.5 Flash:
    • Accessed the YouTube video directly (Google owns YouTube) and gave:
      • Course structure
      • Key concepts
    • Surprisingly, it didn’t summarize as well as others.
  • Llama-3-70b-Groq:
    • Provided summary from the video with:
      • Overview
      • Module-wise takeaways

Why It Matters for QA / AI Testing:

  • Different models handle tasks differently — accuracy, depth, and context vary.
  • Choosing the right model isn’t just technical; it’s strategic for productivity and quality.
  • Tools like poe.com make it easy to compare responses across multiple models.

My Takeaway:

Model selection matters. Test before you trust — the right AI model can make or break your workflow.

Popular Posts

JMeter Producing Error: Windows RegCreateKeyEx(...) returned error code 5

By -
After running JMeter.bat on Command Line, Problem java.util.prefs.WindowsPreferences <init> WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5. Cause Environment Resolving the problem The error occurs because java.util.prefs.WindowsPreferences is trying to save information in HKEY_LOCAL_MACHINE\Software\JavaSoft\Prefs instead of under HKEY_CURRENT_USER\Software\JavaSoft\Prefs. Windows 7 64-bit The work around is to Open JMeter.bat file as the administrator and so it creates the key HKEY_LOCAL_MACHINE\Software\JavaSoft\Prefs.
Read more

Understanding about Contract Testing

By -
Image
Yesterday Morning (9th May 2020), I had good discussion with  Sundaresan  Krishnaswami on " Contract Testing ". I thank  Sundaresan   for his time and sharing his knowledge. It started with an example: The Owner provides the house or Place of Residence to the Tenants. "Rent Agreement" is provided for the house and It contains agreement to use the house. The house comes with like kitchen, living room, bed room, fans, geysers etc.. It also contains who resides and what is the contractual period, obligation (money) etc.. Here, Owner is the Server and Tenant is Client.                                         Client-Server Architecture. Illustrating the same in the E-Commerce Context: In Some Development Companies, There can be two teams developing the Payments and Orders APIs. In B2B or B2C Ecommerce applications, we place an order using web browser through payment methods...
Read more