Like most paths I'm passionate about in my life, quality is something attracted me since I was a kid. But in my young and immature mind I kind of mixed it up with luxury, which probably lead me to enroll in a business degree, and took me many years to realize that engineering has direct access to the beauty of quality, without all the loud and shiny marketing wrap around.
So, what is quality?
Quality is not luxury
First let me rule out luxury; Generally luxury product use to be high quality, but high quality products are not necessarily luxury. As luxury is an economic definition, based on the increase of demand of the good directly correlated with the increase of income. And I'll argue that quality is something that run in the opposite direction: from the creator to the product, and from the product to the customer, and finally from the customer usage to their friends, family, customers, employees, etc.
There are huge amount of quality definitions
So we have to be opinionated about it; For this exercise I'm ruling out all the «customer subjective value» based definitions, as they tend to lead to make investments in marketing instead engineering. And before going to my ideas, let me start by third party quotes:
Number of defects per million opportunities.
– Sig Sixma
I like the concept, but still too much Jack Donaghy for me.
Degree to which a set of inherent characteristics fulfills requirements.
– ISO 9000
We are getting somewhere here, but without getting too metaphysics-spooky on the interpretation, this definition still outside the realm of exceeding expectations.
Then comes Noriaki Kano, who invite us a to go further customer needs and the one-dimension quality of performance attributes, adding a new dimension of «Attractive Quality»;
...Excitement attributes are for the most part unforeseen by the client but may yield paramount satisfaction...The beauty behind an excitement attribute is to spur a potential consumers’ imagination...The key behind the Kano Model is for the engineer to discover this “unknown need” and enlighten the consumer, to sort of engage that “awe effect”...
Then Kano also offer us a model on how "must-be" and one-dimension performance attributes v/s «attractive qualities» shift over the time in customer-product relationship.
Applied arts as quality
Historically, before engineering dominated the quality goods delivery game, applied arts was the mayor player. And today, specially on the «attractive qualities», it’s still relevant, a classic case it's: historical concern about the fonts from Apple. And contemporary product engineering, still have a lot learn from applied arts, thus keeping them on your radar as discipline is a good idea. But also exchange with people of the applied arts professions should be in our quality pursue journey.
Quality on software
These are cool ideas for general products and engineering, but in software we have specifics to deal with. Here I’ll propose you that there are three main areas of care to server customer with high quality software.
Bullshit warning: Most formalization of software quality comes from big business, and training / consultancy / conference / certification selling companies. So expect to find outdated ideas from first ones, and implementations over values to sell you stuff on the second ones.
Delivering quality (AKA project management)
Some purist-coders may dispute this point, but delivering the requested features on time and budget is key to customer satisfaction. And many of us, have to perform not just as developers, but also as project managers, and even sales person sometimes.
This is not a PM post, and there is a lot to learn on the subject, but at least you need to internalise the PM triangle: specifications, budget and times. And the rule is pretty simple: you can’t have it all.
If you increase the specs (scope), you will need to increase times or/and budget. If you want to reduce the budget, you will need to increase the time and/or reduce specs, and so on. And this need to be internalised through all our team, and specially salespeople, so it can be part of the “givens” in to all —and from the first— conversation.
I will also argue, that in the sales / PM realm, another important component for the success, and final quality of a software project, is the professional culture and contract terms match between customer and provider. In my case, I try to avoid vanity customers —customer who don’t need the project, but they want to “have an entrepreneurship” in technology, without really understating the market, usually with money made in other industry or inherited, as they tend to concentrate way too much on the bright and shiny, without any care for medium and long term issues, thus is easy to end with junk jewellery project. Another newbie’s mistake I don’t repeat anymore, is taking a just on deliveries payments contract with a government institution, don’t want to get into politics here, but government workers have another work pace —way to chill for me—, so, the option is to set contracts where they delay on the inputs don’t harm me and my team financially.
This the visible part of the iceberg, which is exposed to the final user. And usually it lives as features that come from customer requests to solve a user case applying a business rule.
To ensure this side of quality you basically had to test your software, and in most scenarios you should setup a mixed testing procedure where you do human and automated testing.
How to test:
The basic human testing is to “click around” you product, then you can standardise it with a checklist. I used to have a paper form with a checklist for my employees on the odds, I felt then that signing with a pen over paper next to declaration like: «I have thoroughly checked the items above and I’m confident this website has enough quality to be deliver to customer» put an emotional commitment on the persons doing it, that made them check twice if was necessary.
Today I choose to work with people that put passion and proud on their work, thus I don’t feel the need to be so rhetorical about it, and any digital checklist is enough for me.
Obviously the human testing doesn’t have to be fully manual. You can assist the human with automation tools. Back on the day we used to run Xenu's Link Sleuth, but today I use a dirty mix of local php + sh + FF imacros. Browser sync, is also pretty useful too and comes with zen.
Then comes fully automatable testing, what we all should be doing, but I’m not doing it yet. As I’ve been rather on the prototyping and playing side of Drupal 8 rather, than the let’s build quality code, for the moment.
At the bottom of software testing pyramid we have the Unit Tests which role is to check the smallest pieces of you software (classes) in the wider and extremes scenarios / values, as it’s the most fast (seconds and less) and isolated set of tests, and the main recommendation is those kind of tests should be writed by same developers who made the code to be tested (there are devs who set the unit test to run at they IDE save command).
At the code and classes level you should be doing you unit test as you develop for serious code. Drupal core comes with its own tests and many modules have them too. Which I’ve been avoiding, but it’s unavoidable, so expect a post deeper about it sooner than later.
At the next level of you should start to see a degrade with functional testing, where trough Gherkin language, we can write “human readable”, quoting they own example code:
Feature: Some terse yet descriptive text of what is desired Textual description of the business value of this feature Business rules that govern the scope of the feature Any additional information that will make the feature easier to understand Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action And yet another action Then some testable outcome is achieved And something else we can check happens too Scenario: A different situation …
Which are slower to run (many seconds to few hours) as they test wider and more real world situations. A nice perspective, is to understand them to test “if you hooked up everything right”. Behat test can run with different engines, some engines simulate a simple non js enabled browser, other do js under the hood, and other literally open browsers windows in you computer and click around them —an automated test that can save screenshots on error? Sweeeet—.
But, what to test for?
We reviewed the tools, but let’s get opinionated on their usage; At PM level, I have to said that Behavior-driven development look so sexy, that I’m implementing it for myself. The idea is to translate customer requirements to Gherking AKA ubiquitous language, documents that could be shared by all the project pipeline actors, from customer final user to backend developer and all the people in the middle. So you start most sprints writing one, and end it fulling it.
But aside the successfully execution of customer requirements features, you should also evaluate the how are executed, in at least this five dimensions:
Reliability: Reduce and prevent the probability of failure. This is key for you image and impact on you customer business.
Efficiency: Measure and improve your runtime performance. Evaluate and have a clear picture where will be the next bottleneck on scalability.
Security: Asset and improve your security through not only coding practices and architecture, but also at organisational / SOPs level too.
Maintainability: Maintenances costs and system life expectancy should set you goals here about adaptability, portability and transferability.
Usability: Please bear in mind that usability is not only intuitive and fast learning curve like an iphone, but also speed and precision like a F1 steering wheel:
We can see how «The key behind the Kano Model is for the engineer to discover this “unknown need” and enlighten the consumer» start to emerge, after fullfing the “visible features”. In software there is the concept of structural quality and non functional requirements, which is kind of disputed conceptually, so these are scopes that wikipedia list (scroll down to go into my recomendation on how to use it):
- Audit and control
- Availability (see service level agreement)
- Capacity, current and forecast
- Configuration management
- Dependency on other parties
- Disaster recovery
- Efficiency (resource consumption for given load)
- Effectiveness (resulting performance in relation to effort)
- Emotional factors (like fun or absorbing or has "Wow! Factor")
- Environmental protection
- Extensibility (adding features, and carry-forward of customizations at next major version upgrade)
- Failure management
- Fault tolerance (e.g. Operational System Monitoring, Measuring, and Management)
- Legal and licensing issues or patent-infringement-avoidability
- Network topology
- Open source
- Performance / response time (performance engineering)
- Platform compatibility
- Quality (e.g. faults discovered, faults delivered, fault removal efficacy)
- Recovery / recoverability (e.g. mean time to recovery - MTTR)
- Reliability (e.g. mean time between failures - MTBF, or availability)
- Resource constraints (processor speed, memory, disk space, network bandwidth, etc.)
- Response time
- Safety or Factor of safety
- Scalability (horizontal, vertical)
- Software, tools, standards etc. Compatibility
- Usability by target user community
The list is like dark energy, is huge, is getting bigger every day, and is pushed from inside out. So let me suggest how to use it:
- Start with the vision of you system
- Bring you functional requirements to the table
- Review the list and contemplate each item, perhaps in a more creative context (like with nice music, a whiteboard and even a joint or a beer), but don’t forget whiteboard or pen and paper!
You probably will end up with a lot of ideas in main tree groups:
- Stuff to care now on architecture, to be free on the future to dive into other areas. (ie: good data architecture => will let you add features without rebuilding, no relying on .htaccess or mod_security => will let you switch to nginx or any other web server)
- Stuff to don’t mess up now on the implementation, to be free on the future to get into. (Ie: good html practices => accessibility, keeping keys away from git => security)
- Stuff to schedule to a future iteration, where you can see value but need to “sell” it to stakeholders (boss, client, you CFO alter ego), or you make sense to pack it with coming features.
While you are in that creative mood, is good to come back and bear in mind Kano’s idea of attractive quality.
Adding a quality assurance engineer to you project
I left it to the end mainly because I think quality should be responsibility of the vision holder of the project, but also I never had the chance to work in a project where there is a dedicated QA engineer. But here what you need to know:
They do exist! You can hire a quality assurance expert for you software project, and having one (or more) in you team should give you serious quality muscle; be smart on how to use they’s skills.
You need to have a ratio, don’t expect to have infinite developers and just one QA engineer. I have read experts recommend ratios from 5/1 to 10/1 (general developers / QA engineers), probably this will depend on you project specifics, just bear in mind —as any human resource— they output is limited and much inelastic.
Can you really sell quality? I’m not sure.
On our last Chilean Drupal meeting, Saul did a lighting talk about his experience about Drupal culture in Asia and Oceania, and when he pointed out how they were “doing things right”. A local colleague asked “how they do to convince customers and bosses to”, he responded «You can lead a horse to water, but you can't make it drink.» and made the point that stakeholders needs to learn from their own poor decisions, and eventually they will learn.
He also pointed out that in those lands they made the same mistakes many stakeholders are currently making here in Chile —I want it cheap and fast, don’t care if it breaks in a year— long time ago, learned from it and that why they currently care about structural quality.
I do believe, like in mate choice, it’s not about being to seduce any person any time. It’s about to be able to seduce a matching partner. I already said a little about this on the PM section. The idea is to give potential customers early signals and be consistent on it. Eventually the ones that look for you “quality trades” will gravitate to you, and the ones that are not yet in that maturity level will go elsewhere. And bear in mind that a non matching customer (like a non-matching mate) can have pretty serious consequences on you business. Iif can have this level of honest communication with you customer, I love Dr. Deming answer:
“You don’t have to change (use quality). Survival isn’t mandatory.”
Also If you are in Chile, check out this post, and don’t be fool with the humour that makeup ugly true underneath.
Where I do believe there’s hope to sell quality (long term), it’s trough mouth to mouth and trough “tasting”. Many people value quality once a close one explain their experience or they have access by themselves. That’s why nice car dealers offer you a test drive.
Also don’t forget bring ROI to the discussion, it’s a one dimension / old school view, but al least it will convey you care about it.
If quality is hard to sell, and it’s a matching game, I thought perhaps you —my dear reader— are in the other side of the equation, you are not a provider, but a customer already convinced to buy quality and had come to this post trying to figure out how to sort out your provider choices.
In that case, let me honestly tell you: I feel your pain.
As long timer in tech business, we share this issue to a mayor extend, we need to hire quality personnel, and specially locally, it’s pretty hard and there are many scoundrels out there.
So, there is no silver bullet, you need to seriously do you due diligence on this, and put the effort necessary develop the “buy quality” skills, but you need to start somewhere, so here is a list of suggestions:
- Don’t trust establishment big corporations by default (they are generally one generation behind or more on their tech)
- Learn about you industry / market tendencies (look up to develop countries if you not in one)
- Ask you providers to lift up the hood and guide you trough components (you don’t need to be a mechanic to distinguish between a low end and a high end car engine)
- Try to learn some jargon (this go for all industries, doesn’t matter if you speak it poorly, it will set you apart over the other muggles, few technical terms can give you instant respect as you show effort to comprehend)
May look like a lot of work —as any skill development process—, but it worth it. Even if you switch industry in you career late, learning to procure quality, will still give you a serious advantage over you competition.