Three hats of quality
A tale about how different people see quality in companies
from small to big, and what to do to always win
Ilya Fomin
fomin@fom.info
@fom_info
About me
✔
8 years in testing
✔
12 years in software engineering
✔
10 years in management
✔
2 years at Oscillations
✔
1 month at Google
(disclaimer)
Ilya Fomin
fomin@fom.info
@fom_info
Epic journey
startup
enterprise
This is how it all began
Maybe it’s time to
write autotests?
In a startup
We’re not an
enterprise
Too
early!
No, it’s
too late
We’re not a startup!
Your project is huge!
Automated tests?
-
Why is the company size an excuse?
Can’t the same quality approaches
work for all kinds of companies?
How do you even agree on
a quality process?
Is my experience relevant here?
What we know about company sizes
Company size 1: Building
Company size 1:
Building
Shaping
Funding: Bootstrap, Seed stage
Employees: 1 ~ 10
Goal:
find product-market fit,
find demand
Company size 2: Growing
Funding: Series A-B
Employees: 10 ~ 100
Goal:
get mature product,
more customers
Company size 3: Replicating
Funding: Series C+, IPO
Employees: 1,000 ~ 10,000
Goal:
become mainstream,
satisfy all the demand
Company size 4: Sustaining
Funding: Revenue
Employees: 10,000++
Goal:
stay in business,
incremental improvements,
new products and segments
Company sizes
1. Shaping
2. Growing
3. Replicating
4. Sustaining
What is quality?
Quality
From Wikipedia, the free encyclopedia
In business, engineering, and manufacturing, quality has a
pragmatic interpretation as the
non-inferiority or superiority
of
something; it's also defined as being suitable for its intended
purpose (
fitness for purpose
) while satisfying customer
expectations...
https://en.wikipedia.org/wiki/Quality_(business)
Quality
From Wikipedia, the free encyclopedia
… Quality is a perceptual, conditional, and somewhat subjective
attribute and
may be understood differently by different people.
https://en.wikipedia.org/wiki/Quality_(business)
Quality
TL;DR:
-
Measurement of how well
the product fits the purpose
-
Subjective and relative
Quality process
quality
TL;DR:
-
Measurement of how well
the product fits the purpose
of the company
-
Subjective and relative
Quality process
quality
TL;DR:
-
Measurement of how well
the product fits
the purpose
of the company
-
Subjective and relative
Shape
Grow
Replicate
Sustain
Quality process
quality
TL;DR:
-
Measurement of how well
the product fits the purpose
of the company
-
Subjective and relative
Shape
Grow
Replicate
Sustain
Let’s talk about subjective...
Subjective is great!
Because:
-
Companies consist of people
-
Every decision is made by some
one
-
Consensus can create big impact
About architecture:
https://youtu.be/GON1Q8zEEqA
4 simple things about people
-
Like to do
what they’re
good at
4 simple things about people
-
Are good
at what they
like to do
-
Don’t want to get fired
-
Agree with their own decisions
-
Typically, are lazy
Now that we know how
people work, let’s look at
Three hats of SW Quality
Product lead (PM)
-
Talks to everyone
-
Decides what gets done
-
Knows the business
-
Knows the customer
-
Wears
Development lead (Dev)
-
Solves problems
-
Likes to move fast
-
Not very good at talking
-
Wears
Quality lead (QA)
-
Ensures that everything works
as it should
-
Looks for defects
-
Approves good work
-
Wears
Meet Zmei Gorynich
One heart
(vulnerable)
Three heads
(breathe fire)
One ass
(not fireproof)
Me in startups
I choose quality
approaches that make
sense from all
perspectives to avoid
cognitive dissonance.
So can you!
Real-world examples
Automated testing
For PM:
Ensures customer needs are addressed
For Dev: Automated tests is a problem!
For QA:
Helps prevent defects
Shape: Add critical tests to make sure the main flow works
Grow: Add tests for bugs, especially the ones that reoccur
Replicate: Add behavior-driven test, add performance tests
Sustain: Add full regression, security, compliance, failure
testing
Customer research
For PM:
Helps understand customer segments and needs
For Dev: Helps understand where the problems are
For QA:
Helps structure testing, prioritize user journeys
Shape: Get your friends to try to use your service, is it useful?
Grow: Do an early access test. Does your solution scale?
Replicate: Run marketing studies, does your product work for all?
Sustain: Test on your users, is your solution improving?
Product metrics
For PM:
Helps ensure customers use the service
For Dev: Enables fast experimentation with less talking
For QA:
Helps identify where to look for problems
Shape: Conversion of ads and landing pages, new daily users
Grow: Total users of the service, number of “sales”
Replicate: Complete tracking of customer journey, by user type
Sustain: User engagement, time spent on service, financial$
Customer feedback
For PM:
Helps understand customer concerns
For Dev: Identifies more problems to work on
For QA:
Customers do their job for them
Shape: Record subjective feedback, follow-up in person
Grow: Structure feedback to find themes, satisfaction surveys
Replicate: Collect community ideas, aggregate reviews, beta testing
Sustain: Bug bounties, customer support metrics, enterprise support
Your QUALITY_METHOD
1.
Start with why: explain
how this method helps the 3 hats
1.
List all the practices and tools relevant to the method.
Ask
others for help!
1.
Select the practices that align with your company’s current stage/goal
P.S. Practices from earlier stages should be covered, too
P.P.S. Practices from later stages
might
work, but it’s a harder sell
But what about other areas?
How do I
always
win?
The Answer to the Ultimate Question of
Life, the Universe, and Everything SQA:
Q = N
customers
Problems
solved
Problems
all
Success
in %
Q = N
customers
Problems
solved
Problems
all
Success
in %
And they lived happily ever after
The end
Questions?
Ilya Fomin
fomin@fom.info
@fom_info
Three hats of quality: a tale about how different people see quality in companies from small to big, and what to do to always win
