Lessons Learned Posts: #3 - Heuristics and Oracles

As part of the "Learn Something New Every Day" challenge, I had decided to read an article a day and write short notes on Lessons Learned.


Image result for LESSONS LEARNED



The topic of the week: Heuristics and Oracles

The words "Heuristics and Oracles" are new concepts, for testers who have not heard about it.
I have not used these terminologies in the last 6 years at my job.

I heard about the words "Heuristics, Oracles" in Weekend Testing session.
From a long time, I wanted to learn about Heuristics and Oracles more in detail and to Understand it in a better way.

Heuristics:

When we have a Testing Problem in Application, we try with different options or test ideas which we know how to test it and see if it works. The different options which we try are the test heuristics.

Heuristics are simply experience-based techniques for problem-solving, learning, and discovery. 

In Real Time, you might be experienced with testing an application which has scheduling functionality.
Now, the testing problem in an application is "To Test and Validate if scheduled Jobs are running or not"

The Test Heuristics can be:
  • Jobs can be scheduled in different ways: Day, Week, Month, Year and Time.
    • We would be testing if the job is scheduled for 5.00 am UTC is started to execute at a specified time or not, considering the Application server is running on UTC Timezone. --  The test heuristic is running at a specified time.
    • We would also test if the job at the specified day of the week and time is started to execute -- The test heuristic is waiting for the specified day of the week and time.

When we run out of test ideas, we can use heuristics to assist in exploratory testing.
Note: We cannot apply all heuristics, but we must try as much as possible.

Test Heuristics Cheat Sheet

Heuristic Test Strategy Model


Oracles:

While testing an application, we might discover a bug and immediately shouts "Got a Bug".
But there might be some cases, where the developer would not agree and asks for "why it is a bug? Is it part of the requirement document?"

we might think that "It should not work like this right?  Is it really a Bug?" 
There are number of ways, we can determine if its a bug or not. These are called as "Test oracles"

Oracles are simply the principle or mechanism by which we recognize a problem.

Oracles help to discover the real reason, why I think it is a bug.

Testers often say, We recognized the problem as "Product does not meet its expectations/requirements"



Few Hiccups


Posted in , | Leave a comment

Lessons Learned Posts: #2 - Security Testing Terminologies

As part of the "Learn Something New Every Day" challenge, I had decided to read an article a day and write short notes on Lessons Learned.





The topic of the day: Security Testing Terminologies


Last month, I have encountered with new terminology in security testing: False Positive.

I have understood in short, It is not an issue to fix. But wanted to learn more in detail.

Below are the references which I have used to learn about Security Terminologies:

https://www.contrastsecurity.com/security-influencers/the-true-cost-of-false-positive-vulnerabilities-in-application-security

https://www.owasp.org/index.php/Benchmark

https://community.softwaregrp.com/t5/ArcSight-User-Discussions/what-is-false-positive-false-negetive-true-positive-and-true/td-p/1582039


After going through these links, It was easy for me to correlate and understand easily.

What I have learned:

Below is the representation of Security Testing Terminologies.

1. False Positives
2. False Negatives
3. True Positives
4. True Negatives


Posted in , , | Leave a comment

Lessons Learned Posts: #1 - A Transpection Session: Inputs and Expected Results

As part of the "Learn Something New Every Day" challenge, I had decided to read an article a day and write short notes on Lessons Learned.





The topic of the day: A Transpection Session: Inputs and Expected Results

https://www.developsense.com/blog/2010/05/a-transpection-session-inputs-and-expected-results/


First, I was not aware of the word: Transpection.
The definition I understood from http://www.satisfice.com/blog/archives/62 :

Transpection is Learning about a product, by putting yourself in someone's place.
Asking someone a question, then thinking through the same question and comparing the answers with others while listening to them.


Definitions of Testing:
James - "Ask Questions in Order to Evaluate It"
Jerry Weinberg - "Gather Information with the intention of informing a decision"

Testing is not just about "Inputs and Expected Results"

What is Input?
Different types of Inputs: Symbolic Input, Non-Symbolic Input; Explicit Input and Implicit Input.

Symbolic input is data processed by the computer; data meaning “bits”, and “processed” meaning processed using the microprocessor. 
Non-symbolic input would be anything else, such as heat or shock.

Explicit input—the input you knowingly provide.
Implicit input: the input that influences the system without your knowledge or intent. Once you set an option, that option becomes implicit input for the function that refers to it.

Tests consist of "Coverage, Oracles, Procedures"


  • Coverage means observation of some aspect of the product in action. 
  • Oracle means a principle or mechanism by which we recognize a problem. 
  • Procedures mean “knowing how to do the test”


a. Observing and Evaluating the Product.
b. Recognizing the Problem if it occurs.
c. Knowing when to Stop the Tests by applying a Stopping Heuristic.
d. Reporting the Results.


I have learned about:
  • Different types of Inputs - Symbolic, Non-Symbolic, Explicit, Implicit Inputs.
  • Transpection Session
  • What is Testing all about?
    • Coverage
    • Oracles
    • Procedures

Posted in , | Leave a comment