My Journey of Testing, Learning & Growing!

Context: After learning about Tokenizers in my previous post, I wondered: Does an AI model remember the entire conversation in a chat? The answer is no — it has a memory limit called the Context Window . What I Learned: Context Window: The maximum number of tokens a model can process at once. Acts as the model’s short-term memory for a single interaction. What Fits in the Context Window: Input prompt Model’s response Chat history Fixed Size per Model: GPT-4o: 128,000 tokens GPT-4-Turbo: 128,000 tokens Claude 3 Opus/Sonnet: 200,000 tokens Older models (e.g., GPT-3.5): 4,096 tokens Pro Tip: Calculate cost from tokens: Example: Total tokens: 470 GPT-4-Turbo input price: $10 per 1M tokens Cost = (470 / 1,000,000) * $10 = $0.0047 The chat cost about half a cent ! Why It Matters for QA / AI Testing: Context window limits affect how much history the model can consider when generating responses. Testers need to design prompts and workflows that fit within token limits for consistent results...

Context: I always wondered how AI models like ChatGPT “read” text. Turns out, they don’t read words like humans — they see numbers . What I Learned: Tokenizer: A core component of every LLM. Converts words into tokens (pieces of words) that the model can process as numbers. Why It Matters: Every LLM API cost is based on tokens , not words. More tokens = higher cost. A prompt that costs $0.01 with one model might cost $0.015 with another due to tokenization differences. Efficiency Tip: Writing shorter, well-structured prompts reduces token count and cost. Resource: OpenAI provides a great tool to visualize tokenization: Tokenizer Tool. Why It Matters for QA / AI Testing: Tokenization impacts cost and performance in AI-driven testing workflows. Testers need to optimize prompts for efficiency without losing clarity. Understanding tokenization helps predict API usage and budget planning. My Takeaway: LLMs don’t see words — they see tokens. Efficient prompt design saves cost and improves ...

Context: We’ve discussed Fine-Tuning and RAG for improving AI responses. But there’s a third, simpler way: Prompt Engineering — mastering how we ask questions. What I Learned: Prompt Engineering is like giving the model a detailed map instead of just a destination . It guides the model’s internal attention mechanisms to focus on the most relevant patterns learned during training. Example: ❌ Weak Prompt: Write test cases for a login page. ✅ Engineered Prompt: Act as an expert QA engineer with 10 years of experience in security and usability testing. Your task is to generate a comprehensive test suite for a standard web login screen with the following fields: Username, Password, a 'Remember Me' checkbox, and a 'Forgot Password?' link. Why Use Prompt Engineering? ✅ No infrastructure changes. ✅ Iterate and get output in seconds. ✅ No additional training or data. Challenge: Requires creativity and multiple iterations to get the best answer. Why It Matters for QA / AI Test...

Context: When we have a powerful general-purpose LLM like GPT-4 but need deep expertise in a specific domain, the solution is Fine-Tuning . I explored how this technique works and why it matters. What I Learned: Fine-Tuning: Retrains a base model to make it an expert in a specific field. Adds additional training on a focused, specialized dataset. How It Works: Base model has broad knowledge. Fine-tuning adjusts internal weights using supervised learning with input-output pairs. Example: To build a legal assistant, train the model on thousands of legal documents. Advantages over RAG: Gains real domain-specific knowledge without maintaining a separate vector database. Challenges: Requires large labeled datasets. Needs more GPUs and compute resources. May require full retraining and ongoing maintenance. Risk of losing general capabilities while specializing. Why It Matters for QA / AI Testing: Fine-tuned models can deliver highly accurate results for domain-specific testing. Testers must...

Context: I was curious about what happens when you ask an LLM, “Who is Srinivas Kadiyala?” The responses vary across models because they depend on training data and knowledge cutoff. This made me wonder: How can we make AI models smarter, more accurate, and up-to-date? What I Learned: LLM responses are limited by their training data and knowledge cutoff dates . To overcome this, we can use RAG (Retrieval-Augmented Generation) . RAG Workflow: Retrieves real-time information from external sources (e.g., web search, APIs). Augments the model’s static knowledge with fresh data. Generates responses that are more reliable and relevant. RAG doesn’t just rely on what the model was trained on — it combines retrieval + generation for better accuracy. Why It Matters for QA / AI Testing: Ensures AI-driven testing tools provide up-to-date results. Reduces hallucinations caused by outdated or incomplete knowledge. Critical for scenarios where real-time data validation is required. My Takeaway: RA...

Context: Initially, I thought all LLMs were pretty much the same. But after testing them on a real-world task, I realized the choice of model can significantly impact the quality of results. What I Learned: The Task: Summarize the YouTube video: “Software Testing A Web Application - Case Study - complete course video by EvilTester.” I used the same prompt across different AI models: OpenAI: Searched the web, found the YouTube link, and provided a solid summary including: Course overview Why the case study matters Actionable next steps DeepSeek: Used its web search feature and returned a detailed summary with: Key objectives Course structure Testing techniques Additional resources Microsoft Copilot: Pulled from various sources and provided: Course overview Key concepts Learning outcomes Gemini 2.5 Flash: Accessed the YouTube video directly (Google owns YouTube) and gave: Course structure Key concepts Surprisingly, it didn’t summarize as well as others. Llama-3-70b-Groq: Provided summary...

Context: I was wondering — with so many AI models available, is there a way to access all of them through a single platform? And what if one model goes down — can it automatically switch to another? What I Learned: OpenRouter.ai is a unified API platform that provides access to hundreds of AI models through a single endpoint and one API key. It automatically handles fallbacks , seamless switching between providers like OpenAI, Google, DeepSeek, and more. It even selects the most cost-effective options for your requests. Essentially, it acts like a smart router between your app and the best available model. Why It Matters for QA / AI Testing: Simplifies integration by reducing dependency on individual model APIs. Ensures continuity in testing workflows even if one provider experiences downtime. Helps testers experiment with multiple models without complex setup. My Takeaway: OpenRouter.ai makes multi-model access simple, reliable, and cost-efficient — a game-changer for developers a...

Context: I wanted to understand the flow of data in Large Language Models (LLMs) — how they learn during training and respond during inference. What I Learned: Training Data: Used during the training phase of the LLM. Includes grammar, facts, reasoning, code, articles, books, etc. Stored as the model’s built-in knowledge. Inference Data (Input): The user’s prompt or question during inference time. No data is stored or learned from the user’s prompt. Output Data: The model’s response to the user’s input during inference time. No data is stored or reused. Key Insight: Training builds the brain. Inference is the conversation. Better inputs (prompt engineering) → Better outputs. Why It Matters for QA / AI Testing: Helps testers understand why models can’t “learn” from prompts in real-time. Emphasizes the importance of prompt engineering for accurate and useful responses. Critical for designing tests that separate training limitations from inference behavior. My Takeaway: LLMs learn durin...

Context: I used to wonder about the term “hallucinations” in AI: How do they happen? What causes them? Is it just how the model works, or is there a deeper reason? After reading and reflecting, I discovered two key concepts behind hallucinations: What I Learned: Token Bias: Models tend to choose certain words more often than others based on training patterns, not necessarily logic. When given a prompt, the model calculates the most likely next word. Sometimes, it picks a word because it’s “used to” it, not because it’s correct. Example: Prompt: “5 of the mangoes are smaller out of 10 mangoes” The word “smaller” might make the model think of “subtract” or “minus,” even if that’s not intended. Hallucination: The visible mistake in the output — when the model gives a wrong answer. Key Insight: Statistical Pattern Matching → Token Bias → Hallucinations Hallucinations are not random; they stem from how models learn and predict. Why It Matters for QA / AI Testing: Understanding token bias ...

Context: In my previous post, I learned that Probabilistic Pattern Matching is used by LLMs to predict the next word. Today, I explored how models learn patterns during training — and discovered the role of Statistical Pattern Matching . What I Learned: Statistical Pattern Matching: Builds an internal “map” of relationships between words, tokens, and contexts. Example: The model learns that “river” often appears near “water” or “bank.” Probabilistic Pattern Matching: Given a context, predicts the most likely next word by ranking probabilities. Example: "The cat sat on ___" mat (60%), floor (25%), spaceship (15%) → selects "mat" In short: Statistical = The How (internal learned structures). Probabilistic = The What (final output choice). Why It Matters for QA / AI Testing: Understanding both mechanisms helps testers anticipate why LLMs produce certain outputs. Statistical learning explains biases from training data; probabilistic prediction explains variability in...

Context: We often wonder: How do Large Language Models (LLMs) like GPT “think”? The answer might surprise you — they don’t reason like humans. Instead, they rely on Probabilistic Pattern Matching . What I Learned: LLMs predict the next token (word or part of a word) based on surrounding context. They compare billions of patterns from training data and pick the most probable continuation. Example: Input: "Peanut butter and ___" Prediction: "jelly" (because it’s statistically the most likely next token) It’s not logic or true understanding — it’s statistical prediction of patterns. Yet, this approach is powerful enough to simulate reasoning, writing, coding, and conversation. Why It Matters for QA / AI Testing: Helps testers understand why LLMs sometimes produce unexpected outputs — it’s probability-driven, not fact-driven. Knowing this mechanism aids in designing prompts and validating AI responses for accuracy. Critical for testing edge cases where statistical li...

Context: In my previous post, I explored the difference between Foundation Models and LLMs . Today, I’m adding the third piece of the puzzle: Deployed AI Models — the ones we actually interact with in real-world applications. What I Learned: Foundation Models: The “brain base” of AI, trained on massive, diverse datasets (text, images, audio, etc.). Examples: GPT-4, LLaMA, Gemini, Claude. Large Language Models (LLMs): A specialized type of foundation model focused on language understanding and generation. Examples: GPT-4, Claude 3, Gemini Pro, Mistral Medium. Deployed AI Models: Real-world, user-facing versions of models, integrated into apps, APIs, and tools with performance and safety layers. Examples: GPT-4 → Foundation Model (LLM) ChatGPT → Deployed AI Model powered by GPT-4 GitHub Copilot → Deployed AI Model powered by Codex Why It Matters for QA / AI Testing: Testers need to understand this hierarchy to design effective test strategies. Foundation Models define capabilities, L...

Context: I recently came across the term Foundation Models and wanted to understand how they relate to Large Language Models (LLMs) . Are they the same or different? What I Learned: Foundation Models: Large, general-purpose models trained on diverse data types (text, images, audio, etc.). Examples: GPT, CLIP, DALL·E, Whisper. LLMs (Large Language Models): A specialized type of foundation model focused only on language tasks (text). Examples: GPT, Claude, LLaMA, PaLM, Gemini. Key Relationship: Every LLM is a Foundation Model, but not every Foundation Model is an LLM. Example: Mistral Medium 3 is both: An LLM (specialized in text/language tasks). A Foundation Model (broad, adaptable, and part of the foundation model family). Why It Matters for QA / AI Testing: Understanding these categories helps testers choose the right AI tools for tasks like test automation, documentation analysis, or multimodal testing. Foundation models enable cross-domain testing (text + image), while LLMs focus ...

Context: Today, I explored how Large Language Models (LLMs) work behind the scenes and why they seem so intelligent when completing sentences or answering questions. What I Learned: LLMs are trained on massive datasets — books, articles, websites — far beyond any single library. Inside the model are parameters (weights) that adjust whenever the model makes mistakes, improving predictions over time. Training happens on GPUs , compressing centuries of reading into weeks. Two key phases: Pre-training: Learn to predict words and complete sentences. RLHF (Reinforcement Learning with Human Feedback): Humans guide the model to be helpful, kind, and respectful. The magic lies in the Transformer architecture: Attention Mechanism: Looks at all words in a sentence at once, understanding context (e.g., “river bank” vs “piggy bank”). Feed-Forward Neural Network: Stores patterns learned during training. Backpropagation Algorithm adjusts weights when predictions are wrong, making the model sm...

Context: While working with Pandas, I discovered a subtle but important change in the get_dummies() method that impacts how categorical encoding is handled. What I Learned: In older versions of Pandas, get_dummies() returned 1s and 0s for categorical data. In newer versions, the default return values are now True and False . This change reflects a shift toward boolean representation for better clarity and consistency. Interestingly, most LLMs (Large Language Models) still generate code using 1s and 0s because they were trained on older Pandas versions and haven’t adapted yet. Why It Matters for QA / AI Testing: Outdated code examples can lead to unexpected behavior in automation scripts. Testers need to validate encoding logic when working with categorical data in ML pipelines. Always check the latest documentation to avoid compatibility issues. My Takeaway: Tools evolve. Code examples may lag behind. Always verify with official documentation before implementing.

Context: While learning Pandas, I explored how outliers are detected and why they matter in data analysis. To make it fun, I imagined pandas in a jumping contest — most pandas jump within bounds, but one panda leaps way beyond… that’s an outlier! What I Learned: Outliers are data points that sit far outside the normal range. Tukey’s Rule of Thumb: Any value beyond: Q1 − 1.5 × IQR OR Q3 + 1.5 × IQR is flagged as an outlier. This method is widely used for detecting anomalies in datasets. Why It Matters for QA / AI Testing: Outliers can skew test results and model predictions. Detecting and handling outliers ensures data quality and reliable AI outcomes. Helps testers validate preprocessing steps in ML pipelines. My Takeaway: Outliers may look like Rocket Ronny in a jumping contest — rare but impactful. Knowing how to spot them is key for accurate analysis.

Context: Today, I explored RLHF , a technique used to align AI models with human preferences by applying reinforcement principles similar to how we guide children. What I Learned: Reinforcement = Encouraging good behavior Positive Reinforcement: Add something pleasant (e.g., praise, cookie 🍪). Negative Reinforcement: Remove something unpleasant (e.g., stop nagging once the task is done). Punishment = Discouraging bad behavior Positive Punishment: Add something unpleasant (e.g., timeout ⏱️). Negative Punishment: Remove something good (e.g., no screen time 🎮). These principles help AI models learn desired behaviors and avoid undesired ones. Why It Matters for QA / AI Testing: RLHF ensures AI outputs align with ethical and user-centric expectations. Testers need to validate reinforcement strategies to prevent bias or harmful responses. Understanding RLHF helps design better test cases for AI safety and compliance. My Takeaway: RLHF is like parenting for AI — guiding behavior through...

Context: Today, I explored the concept of Supervised Learning , one of the foundational approaches in machine learning. What I Learned: Supervised Learning is like teaching with an answer key — the model learns from labeled data. Formula: Training Data + Labels → Model → Predict / Classify Prediction (Numerical values): Example: Predicting house prices Algorithm: Linear Regression Classification (Categories/Labels): Example: Spam vs. Not Spam emails Algorithms: Logistic Regression, Decision Trees, Random Forests Why It Matters for QA / AI Testing: Understanding supervised learning helps testers validate AI-driven predictions and classifications. Knowing the algorithms behind predictions ensures better test coverage for edge cases. Helps design test scenarios for both numeric predictions and categorical classifications. My Takeaway: Supervised Learning = Learn from the past to predict the future.

Context: Today, I explored the concept of knowledge cut-off dates in popular AI models. These dates define what each model “knows” and what it doesn’t. Beyond these dates, the model may not be aware of the latest products, features, or events. What I Learned: GPT-4 – April 2023 GPT-4o – October 2023 GPT-5 – October 2024 DeepSeek-V3 – July 2024 Qwen2.5 – End of 2023 QwQ-32B – November 28, 2024 Grok 4 – July 2025 Claude 4.1 – March 2025 Gemini 2.5 Pro – January 2025 Why It Matters for QA / AI Testing: Knowing the cut-off helps testers ask the right questions. If fresh product info is needed, combine LLMs with live data sources like RAG , APIs , or search integration . Avoid relying solely on static knowledge for dynamic testing scenarios. My Takeaway: Knowledge cut-off dates are critical for planning AI-assisted testing strategies — always pair LLMs with real-time data for accuracy.

Context: I wanted to understand the major AI models currently influencing the industry and how they differ in capabilities and use cases. What I Learned: Qwen (Alibaba) – Strong multilingual capabilities, especially for Asian languages. OpenAI (ChatGPT, GPT-4, GPT-5) – Versatile and widely used for coding, testing, documentation, and productivity tasks. DeepSeek – Optimized for reasoning and efficiency. Grok (xAI) – Elon Musk’s conversational AI integrated into X (Twitter). Groq – Not a model, but a hardware company building ultra-fast chips for LLM inference. Why It Matters for QA / AI Testing: Knowing model strengths helps testers choose the right AI tool for tasks like test automation, documentation analysis, or brainstorming test ideas. Different models offer unique advantages in reasoning, language support, and integration, impacting testing workflows. Hardware innovations like Groq influence performance and scalability for AI-driven testing solutions. My Takeaway: Understand...

Context: Google’s Gemini recently introduced Storybook , a feature that transforms ideas into stunning visual narratives with just a prompt. I wanted to see how it could simplify explaining complex concepts in an engaging way. What I Learned: Storybook creates multi-page visual stories from a single prompt. It includes an audio option to listen to the story, making it more accessible. Great for explaining technical concepts in a fun, easy-to-understand format. I tested it by visualizing a data science concept for a 10-year-old , and the results were surprisingly engaging. Why It Matters for QA / AI Testing: Helps testers and trainers explain technical workflows to non-technical stakeholders. Can be used for knowledge sharing and training sessions in a creative way. Encourages better communication of AI concepts across teams. My Takeaway: AI tools like Gemini Storybook aren’t just for content creators — they can make technical education and collaboration more engaging and accessible....

Context: As a tester, I often deal with large volumes of product and technical documentation. I explored Google NotebookLM to see how AI can simplify this process. What I Learned: Quickly understand documentation by asking natural language questions. Generate overviews and study guides from dense material. Summarize workflows and identify requirements to derive core and end-to-end test scenarios. Brainstorm test ideas and edge cases using document context. Share structured notes and insights with the team for better collaboration. Why It Matters for QA / AI Testing: Saves time by reducing manual effort in reading and interpreting documentation. Improves test coverage by uncovering hidden requirements and edge cases. Enhances collaboration through AI-generated summaries and structured insights. My Takeaway: AI isn’t just about writing code — tools like NotebookLM can supercharge a tester’s productivity and creativity. https://notebooklm.google

Context: While learning Pandas, I realized that the library evolves constantly, and keeping up with changes is essential for writing clean, future-proof code. What I Learned: Pandas deprecated the old way of filling missing values in v2.1.0 (Aug 2023) : # Old way df .fillna ( method = "bfill" ) The recommended approach now is: # New way df .bfill ( )   # or df.ffill() Even small changes like this can break code if we don’t read release notes. Why It Matters for QA / AI Testing: Outdated code can lead to unexpected errors during automation or data validation. Staying updated ensures compatibility with the latest libraries and avoids technical debt. Release notes are a critical resource for testers working with data-driven AI models. My Takeaway: A tiny tweak in syntax highlights a big truth: reading release notes is not optional if you want reliable, maintainable code. https://pandas.pydata.org/docs/whatsnew/index.html

Context: I attended the “Prompt Engineering for Testers” webinar by Rahul Parwal on ShiftSync by Tricentis to learn how Generative AI can be applied in software testing. What I Learned: Practical tips on using Generative AI in testing workflows. How prompt engineering can accelerate test design and analysis. Real-world examples that make it easy for testers to start experimenting today. Why It Matters for QA / AI Testing: Prompt engineering helps testers leverage AI effectively for faster, smarter testing. Reduces manual effort in creating test cases and analyzing results. Encourages innovation in QA by integrating AI-driven approaches. My Takeaway: Generative AI and prompt engineering are not just buzzwords — they’re practical tools that can transform how testers work. https://shiftsync.tricentis.com/events/prompt-engineering-for-testers-with-rahul-parwal-45

Context: I’ve been using GitHub Copilot inside Visual Studio Code recently, and it has become my go-to companion for learning and coding. What I Learned: Copilot helps debug issues when I get stuck. It suggests new coding options to explore. It explains why an error happens, not just how to fix it. Why It Matters for QA / AI Testing: Speeds up troubleshooting during test automation development. Encourages deeper understanding of errors, improving test reliability. Makes learning new frameworks or tools less intimidating for QA engineers. My Takeaway: It feels like having an AI mentor right beside me while I code — highly recommended for Python learners and AI enthusiasts! https://visualstudio.microsoft.com/github-copilot/

  Context: I wanted to see how GitHub’s Ask Copilot feature can make code reviews more efficient by providing instant context on Pull Request changes. What I Learned: You can chat directly with Copilot and ask questions about files changed in a PR. It helps quickly understand changes made by other teams. Saves time by reducing the need to read every line of code. Why It Matters for QA / AI Testing: Speeds up PR reviews for testers and developers. Improves collaboration by providing clear context without manual digging. Reduces review fatigue and accelerates release cycles. My Takeaway: Ask Copilot makes collaboration smoother and PR reviews faster — a game-changer for productivity. https://docs.github.com/en/enterprise-cloud@latest/copilot/how-tos/chat-with-copilot/chat-in-github

  Context: I enrolled in the AI For All program by Intel, CBSE, and Digital India to understand how AI education can be made inclusive and impactful. What I Learned: The program is divided into two sections: AI Aware – Basics of AI, concepts, and applications. AI Appreciate – Real-world impact, ethical use, and AI for social good. AI education can be simple, structured, and accessible to non-technical audiences. Ethical considerations are central to responsible AI adoption. Why It Matters for QA / AI Testing: Understanding AI fundamentals helps QA professionals validate AI-driven features effectively. Awareness of ethical AI ensures testing aligns with fairness and transparency principles. Programs like this bridge the gap between technology and society, influencing how we test for real-world impact. My Takeaway: AI is not just for tech experts — it’s for everyone. Accessible education empowers people from all walks of life.

Context: While learning Pandas, I stumbled upon an interesting difference in how quartiles are calculated compared to what we learned in school. What I Learned: In school, quartiles were calculated by splitting sorted data into halves and finding medians manually. Pandas uses describe() which relies on NumPy’s percentile logic. NumPy applies linear interpolation (default = "linear" ), making results slightly different for small or odd-sized datasets. Why It Matters for QA / AI Testing: Statistical nuances can impact data validation and interpretation in AI models. Understanding how libraries compute metrics ensures accurate testing and reporting. Helps avoid confusion when comparing manual calculations with automated outputs. My Takeaway: Pandas is optimized for large-scale statistics, but knowing these subtle differences is key for accurate analysis.

Context: I wanted to see how integrating Jupyter Notebook within VS Code could improve my workflow for Python and AI learning. What I Learned: You can run code and see outputs instantly. Visualize results right next to your code. No need to switch between multiple tools — everything is in one place. Why It Matters for QA / AI Testing: Reduces friction when experimenting with AI models or scripts. Makes debugging and validating outputs faster and more intuitive. Encourages seamless learning and prototyping without tool fatigue. My Takeaway: VS Code + Jupyter Notebook = A powerful combo for Python and AI exploration.

Context: I wanted to explore how AI tools like GitHub Copilot can go beyond code generation and assist in improving collaboration during code reviews. What I Learned: Copilot can quickly review code changes. It provides detailed summaries of Pull Request updates. These summaries help testers and developers understand exactly what changed in the PR. Why It Matters for QA / AI Testing: Faster and clearer code reviews improve overall quality assurance. Helps QA teams validate changes without diving deep into every line of code. Enhances collaboration between developers and testers, reducing miscommunication. My Takeaway: AI is not just writing code — it’s helping us review, understand, and collaborate better. https://github.blog/changelog/2025-04-04-copilot-code-review-now-generally-available/