Steven Burton

Recruiting test consultants

Wed, 03 Nov 2021 11:11:32 GMT

I’ve been recruiting testers for over a decade there for various companies performing various roles and I feel like I’ve become better at it over time. There are many different aspects of a person and their history that come in to play when interviewing applicants.

Behaviours

The biggest thing that I look for in a tester is the mindset they have and the behaviours they exhibit because of this mindset. An individual can have all the skills in the world but if they don’t have the right mindset they won’t be a great tester. So what makes the right mindset?

Big picture thinking

A great tester always has to have the end goal in mind. What is the reason you are creating something and how does it fit in to the way the users will use your product? Being able to contribute to and make small decisions is useful but only if it contributes to a wider goal.

Focus on quality

I would always want my testers to be thinking about the quality of the end product at any time. Whether they are taking part in refinement sessions, writing some code, deploying some software or anything else they should be thinking about how their actions will lead to a quality product.

Pragmatic

I think it’s important that a tester has values and principles that they look to stick to. However there are times (especially when working for a client) when compromises must be made. It’s important that a tester can understand when these times occur and react to them appropriately. The ability to be pragmatic when required and do the right thing for the situation is very underrated.

Innovative

There are many situations a tester will find themselves in where it’s not clear what the best way to test something is. A tester needs to have the ability to think of different ways of doing things and come up with ways to help a situation that others may not think of.

Value-driven

Ultimately in software development if you are not delivering software then what’s the point? I want a tester to be considering the value in their actions all the time and never just doing something for the sake of it. Every action has to work towards the ultimate goal of delivering quality software.

Skills

I find skills to be the least important thing to look for in a tester. This is because no matter what client or project you end up working on, you may need to use different tools, technologies or test methodologies. So instead of focusing on individual skills, I look for the following:

A variety of tools used -> This indicates an adaptable individual and an ability to use different skills for different situations rather than a tendency to stick to and push one tool they know well
Quick to learn -> I want to see that an individual can quickly learn something new and quickly adapt to any tool
Keeping up to date -> It’s important in any role to ensure you are up to date with the latest in the industry and I look to see how an individual does this through making use of the community and other resources

Personality

In the end it doesn’t matter about the skills or behaviours that an individual has if they are not a good fit for your company. Every company has an culture they want to foster within their teams and the best way to do this is to employ people who buy in to that culture. It is very difficult to convert people to a different culture to one which they enjoy. You can always improve people’s skills and behaviours but you can’t (and maybe shouldn’t) change your employee’s personalities.

Key Takeaway

Skills and experience are useful to build up a tester’s toolkit, but the way a tester behaves to me makes the difference between a good and a great tester. But none of this actually matters if they are not the right fit for the company and vice versa.

Data testing areas of focus

Tue, 12 Oct 2021 13:56:39 GMT

I’ve just recently come off working with a client in the big data space, where we were looking to verify over 9 billions records of data. This was a big undertaking so we had to narrow down the areas of the data that we looked at for verification.

It’s worth saying that I wouldn’t usually recommend verification of all 9 billion records due to the time and complexity involved and I’ll go in to different techniques to ensure you do not have to do this in a future blog. However this client was very concerned about their data and every individual record meant something very specific so they were very insistent on this.

Structure

The first are we concentrated on is the structure of the data. Our data was in JSON-L format, where each line in a file is a fully formed JSON record. However whatever structure or format your data is in, this will always be a big area to look at.

The main way we did this was by verifying the JSON records against schemas. With JSON format this is easier to do so we would verify the record on the way in to the system and on the way out of the system. By doing this, we knew that every single record that we ingested in to the system had the correct structure and that we did not alter or break that structure before the record left the system.

For instance, we may have wanted to verify the following JSON:

{
   "record":{
      "people":[
         {
            "first_name":"jeff",
            "surname":"peters"
         },
         {
            "first_name":"laura",
            "middle_name":"jane",
            "surname":"smith"
         }
      ],
      "action":"CREATE"
   }
}

In order to do this, we would create a JSON schema similar to the below:

{
  "type": "object",
  "properties": {
    "record": {
      "type": "object",
      "properties": {
        "people": {
          "type": "array",
          "items": [
            {
              "type": "object",
              "properties": {
                "first_name": {
                  "type": "string"
                },
                "middle_name": {
                  "type": "string"
                },
                "surname": {
                  "type": "string"
                }
              },
              "required": [
                "first_name",
                "surname"
              ]
            },
            {
              "type": "object",
              "properties": {
                "first_name": {
                  "type": "string"
                },
                "surname": {
                  "type": "string"
                }
              },
              "required": [
                "first_name",
                "surname"
              ]
            }
          ]
        },
        "action": {
          "type": "string"
        }
      },
      "required": [
        "people",
        "action"
      ]
    }
  },
  "required": [
    "record"
  ]
}

These are just example schemas and your situation and data may differ quite a lot to this JSON example. However structure is the most fundamental thing you can assure with your data. You can also assure data structure very early on in the process which is great as the earlier on you can assure it, then you can build up the confidence levels over time. We had excellent automation at all levels of our components to assure they would cope with the data structures they would be presented with. However nothing can prepare the system for the unexpected nature of real, live data which is where this verification was so useful!

Another thing to bear in mind with data structure is what to do if your data does not meet the required structure. In our case, we were streaming in data so we could send the data to a DLQ if it failed and examine the issue when appropriate.

Content

This is the hardest area to validate in the data because you require context about the data itself. Here we are talking about the actual values in the fields in the data. At this point we are confident on the structure of the data, but not necessarily the values within that structure. There are two parts to validating the content.

1. Basic content validation

Firstly, you can validate the contents of specific fields. This can be very useful and following on our example from above, we chose to put some of this within the schema itself. For instance, given the JSON above you can see the following field:

"type":"CREATE"

In our schema, we could do something similar to the below, which would mean we know when the record gets schema validated that the content of this field must be one of the allowed values.

"action":{
    "type":"string",
    "enum":[
        "CREATE",
        "EDIT",
        "DELETE"
    ]
},

There are many different types of validation that can be performed in this way, including:

Length
Special characters
Any form of regex check on the value
Case checks

This is not an exhaustive list by any means! However this kind of basic data validation will only cover individual fields, not the relationships between the fields and the records.

2. Data meaning validation

To really examine the context, you need to understand what your incoming data means and how the data records all relate to each other. In a lot of data systems, they may be designed to be dumb and not really understand the data they ingest, meaning this data validation is not possible. However, in our system we had good communications to the team who owned the source data and so we had a good understanding of the data itself and how records related to each other.

To give an example of what you can do here, let’s take the example we used earlier and consider the action field again, where you can see there are only three valid options:

"enum":[
    "CREATE",
    "EDIT",
    "DELETE"
]

There is a logical order for these records to come in to the system:

CREATE
EDIT (0 to many)
DELETE

Knowing how these records are created is important, because it might be that records can not have a CREATE (for instance for transient or incomplete records) for example, but let’s assume that the above is the only order that makes sense in this case.

The way we resolved this way to perform incoming validation checks, whereby we would check when a record came in what type it was. If it was a CREATE then we would check that there is no record with that id already. If it was an EDIT we would check that there was one record and it was a CREATE and if it was a DELETE we would simply check if there was a record. This allowed us to validate all the scenarios on every single record.

It’s worth noting that these checks were time consuming on the ingestion and so were actually turned off after a period of time in live running when we had built up enough confidence and ingested enough records that the client was happy with this. It’s another area where I would not recommend performing these kinds of checks on every record but in this case it was very explicitly what the client wanted.

Volume

For big data especially volume testing should form a very important aspect of your data testing. My client wanted to ensure that every single record that was ingested was also exported from the system, which is a big ask when we are dealing with billions of records.

Due to time and cost, it’s only not possible (or certainly not cost effective) to check the entire contents of every record. However we didn’t need to. This is because we are building on top of the structure and content tests that we have already done, meaning we are already confident in the data within the actual records, but we do need to ensure that all the records have been ingested, processed and exported.

To do this, we used import and export manifests. This means as we processed the ingested records both in to and out of the system, we were creating lightweight CSV files as a log of all the records that were successfully ingested. Each line in the CSVs would contain a | delimited line (we used a | because some of the record’s had commas in) that had the following information about a record:

ID -> the id field of the record JSON
Timestamp -> the timestamp within the record (rather than processing time)
Type -> Imported record or exported record

This meant the resulting files were very small and allowed us much more manageable amounts of data to compare.

When you have the two sets of manifest files, there are many ways to compare them and it will depend on your context. We were working in an AWS system so it made sense to use an AWS Glue job in the form of a pyspark script to load all the files and compare the data inside. We then produced an output based on the following things:

Are there any IDs that are in the export but not in the import?
Are there any IDs in the export and the import with different timestamps?

We also used this to retrieve statistics about the number of ids we imported and a few other small things but the two things about were the ones that told us about the quality of the volume of data.

Given the answer to the above two questions is no then we are now at a stage where we are very confident that every single record we have been asked to ingest is ingested correctly and is available to export from the system too.

Conclusion

I’ve looked at structure, content and volume here as the main areas to focus on with data validation and this has served me well in previous data projects. The key is that each area of validation builds on the ones before it and they all work together to give a level of confidence in the data. This is in a similar way to how you might create unit tests for an application to give a certain level of confidence and then build on these, rather than repeat them, with component tests and so on up the layers. None of the layers themselves give us the confidence we need for our validation, but when combined we have that high level of confidence we are looking for.

Musings from a remote pairer

Tue, 19 May 2020 08:11:59 GMT

Over the past couple of months I’ve been double a lot of remote pairing for obvious reasons (if you are reading this in the future google “COVID-19”..). I feel like I’ve learnt a lot along the way regarding how to get the most out of remote pairing sessions, both individually and for the pair or team. I wanted to share some of the things to do and things not to do with you.

DO.. remote pair!

I wanted to start off by saying that I’ve found the remote pairing in the main very rewarding and would encourage people to do it. I’ve ended up remote pairing not just on writing code, but peer reviewing design documentation, writing user stories and many other things. It’s a rewarding activity and is a fantastic way to get to know your colleagues better.

DO.. embrace different ways of working

One of the biggest things that I realised early on was that everyone is an individual when it comes to how they code, how they word user stories and how they work in general. Remote pairing gives you the opportunity to see this up close and it should be embraced. There may be many small things that are different to how you work and remote pairing gives you this insight which may help you to understand the person you are pairing with better. It’s very tempting to point out where you believe someone should do something different but I would encourage you not to do this too often. If you genuinely see something they are doing where you think you can help, then by all means talk to them about it, but it’s also important to recognise they might not want to have their ways of working questioned too much. It’s vital to be sensitive to the feelings of the person you are pairing with.

DO.. take turns leading

It can be very tempting to lead the session too much, especially when the subject is something you may have worked on before. A lot more people learn from doing, rather than seeing, so if you are in the position of helping others to learn then allowing them to lead the session will help it sink in easier. This is vital if you are consulting and part of the gig is to help people progress and learn a product that ultimately they will be maintaining in the future. I try to swap the person leading after a set interval as much as possible. Sometimes this feels more disruptive and it’s better to wait for a natural break in the activity, but as long as you are regularly swapping the one in control of the screen, that is the important part.

DONT.. try to take over

This is the one thing I probably still struggle with! When I am pairing with someone I find it hard to not just keep pointing on their screen to direct them where to put the cursor or type something! It can be very demeaning to your partner if you keep doing this because they are not a robot to be controlled, but a real person to bounce ideas off. You have to respect their ability and be sensitive when making suggestions. One of the best things about pairing is having people validate the work as you do it and being able to provide a different viewpoint but when you are the person doing that it’s important to not be too forceful or controlling when putting that viewpoint across!

DONT.. always use a shared IDE

Recently we’ve used shared IDEs on the team via Cloud9 and I’ve used shared IDEs in the part when pairing sat next to someone. They can be very powerful and allow true code collaboration. But they can also lose some of the best bits about pairing as they can split the focus between the pair. When you are pairing through one person’s controls, you have to communicate well with each other and make sure that you talk about the changes you are making, which is a vital part of the learning process. Shared IDEs often don’t encourage that kind of communication and you can end up with two individuals writing code next to each other, rather than together. I do think a shared IDE has its uses, but I think it is best used by people who are experienced already in pairing.

Final thoughts…

I’ve really enjoyed pairing during this lockdown and I feel like I’ve got to know the people I’ve been pairing with more. I’ve always been an advocate of pairing (I even won a flat cap for doing pairing once…) so it’s not hard to convince me to do it. But that isn’t the same for everyone and I’ve done a lot of pairing during recent times with people that haven’t done much of it before and in general they have found it a really useful activity.

I would encourage more people to pair and I hope that we don’t stop doing this when we do eventually end up back in the office.

Why I don’t value test certification

Thu, 21 Nov 2019 08:31:37 GMT

Certification for testers seems to have been a hot topic for a while and many testers seem unsure whether it’s the right thing to do for their career or not.

If you want to work in the USA or India as a software tester then you will almost certainly need an ISTQB qualification. In that case it may be useful for your career but it feels very much like a box ticking exercise at that point.

I’ve recruited software testers for many organisations in different industries and countries and I never value certification as part of the application.

Times, they are a’changing…

The software industry is a fast paced ever changing industry and software testing as a discipline is no exception. You can get some test certification that you never have to retake! Given the pace of change in the industry how can this mean you have learnt knowledge of modern software testing? Given the time and effort put in to creating and arranging the courses many ones I’ve seen are out of date at the time you take them. They simply can’t react fast enough to keep the tests up to date with the industry.

Passing the test

Test certificates will verify that you have a certain amount of knowledge on models and techniques which allow you to pass the tests. They don’t verify that you can apply this knowledge in the right situations or that you understand why these models exist. It’s a similar problem to certification in other areas as you have proved you can pass a test only.

Skills vs behaviours

Test models and techniques are entirely skills based and these are the things certification teaches you. But skills are not a highly valuable asset when I’m looking for new testers. Ultimately technologies, techniques and models can be learned and they change quickly. Behaviours and mindset on the other hand are much harder to learn and change. The behaviours an individual has will determine how well they build relationships, work in a team, lead people, work with clients and so on. This is a much more valuable thing to demonstrate to a potential employer.

I would rather see evidence on an application as to how someone has spent time working on behaviours such as empathy instead of skills learnt.

Forced certification

I’ve heard of and worked with (not worked for, I’m not doing that) companies that force new testers to get ISTQB certification within 6 months of working there if they don’t have it already! Not only does this seem a waste of time and money but as shown above it doesn’t necessarily make them a better tester so no one benefits. Except ISTQB of course.

Should I ever get certified?

Ultimately whether you want to undertake studying for some test certification comes down to your aims. Do you specifically want to work in a country or for a company that requires certification? If you do then it may be worthwhile.

However if you want to become a better tester then reading books about case studies, participating in and learning from the community, and working on your personal skills and behaviours are much better ways of spending your time in my eyes.

If you definitely do want certification, then think about the one that would benefit you most. Is it worthwhile getting something like ISTQB or would you be better learning something like a cloud technology and studying for an AWS certificate or similar?

I’d never get certified to prove you are a great tester, because in my eyes, it doesn’t.

The problem with test coverage

Wed, 04 Sep 2019 07:47:17 GMT

“Test coverage is useless”

Or…

“Test coverage is everything”

Two opposing extremes which I’ve heard recently and many times through my career. As with most things, I think the truth lies somewhere in between.

What is test coverage?

When different people hear test coverage they have there own opinions as to what that means. But it may be entirely different from the measurements that are in place! There are plenty of ways you are measure coverage. You can measure coverage via the requirements – i.e. percentage of acceptance criteria that are covered or feature coverage, but more commonly when it comes to automation, you measure coverage via the the code itself.

Let’s explore some of the common types of code coverage and the issues inherent within them.

Line coverage

You can simply measure the number of lines that are executed in the product code when you run your automated tests (line coverage).

def foo(x, y):
    bar()
    if x == 1 and y > 1:
        bar()

So to get 100% statement coverage you just need one test:

x = 1 and y > 1

What is the issue with line coverage?

If the lines have multiple decisions and multiple possibilities, you do not need to test them to ensure 100% coverage. For example in the above method, do we know what happens when y is less than 2? Maybe there’s a bug where this would still pass but we won’t know because we’ve only made one test. It might actually be that any combination of x and y would produce the same outcome! In this case the coverage statistic might cause us to write less automation than we would of simply because we’ve hit 100%. Can’t beat 100%, right?

Decision Coverage

You could measure the decision branches – i.e. wherever there is a decision made by something like an if statement or a switch statement you ensure that every possible branch of the calculation is covered.

def foo(x, y):
    bar()
    if x == 1 and y > 1:
        bar()
    else:
        baz()

So to cover the above code, you would need two tests:

x = 1 and y > 1
x != 1 and/or y <= 1

What is the issue with decision coverage?

Let’s consider the above code again. Is this scenario it’s a simple if statement, but what if we made the if statement more complicated, like below:

if (x == 1 and y > 1) or z % 2 == 0:

Now we are adding another layer of complexity where we keep the old check but also go in to this branch if “z” is an even number. We would really need to check this with our unit tests as we now have multiple logic possibilities that would cause a TRUE or FALSE result for this if statement. However the above change does not change our coverage percentage. This is because in branch coverage as long as each branch is covered once, then it counts. So our existing two tests will still ensure 100%. Again we are in the situation where we can get top marks while still potentially missing covering some actual logic.

Condition Coverage

This is a type of coverage where you ensure that every condition within your decision statements evaluates to TRUE and FALSE to count. Let’s consider our code again this time with the revised if statement:

def foo(x, y):
    bar()
    if (x == 1 and y > 1) or z % 2 == 0:
        bar()
    else:
        baz()

To get 100% coverage now we need to ensure that both atomic parts of the if statement evaluate to TRUE and FALSE. The atomic parts are the if statement logic broken up by the logical operators of and/or/not. So we have three atomic parts in our statement:

x ==1
y > 1
z % 2 == 0

In order to get 100% coverage we need to have tests that make sure each part evaluates to TRUE and FALSE. However we can cover multiple atomic parts in each test, meaning we can still get 100% coverage with two tests:

x = 1, y = 1, z = 1 (TRUE, FALSE, FALSE)
x = 2, y = 2, z = 2 (FALSE, TRUE, TRUE)

What is the issue with condition coverage?

As you can see we have increased the complexity of the statement but still covered it with two tests. There are many more combinations we could and probably should try. We actually have no guarantee with this type of coverage that the block within the statement has been executed, just that the conditions themselves have been executed.

Common concerns

We’ve seen issues specific to certain types of code coverage, but there are also some common pitfalls with code coverage in general.

Quantity not quality

Test coverage of any form is able to tell you the amount of something, whether that is lines covered, requirements hit, decision branches explored or something else. However it does not tell you the quality of the checks you are performing. You may end up executing every line in your unit tests for example, but missing vital checks anyway.

Let’s say you need to check this method:

def multiply(number_one, number_two):
    return number_one * number_two

Well, that’s simple and we can get code coverage to 100% by doing a test that sends in 1 and 2 and ensures 2 is the response. Awesome! 100%!

But wait… what happens if we pass in null for one or both of the numbers? What happens if we pass in zero? What happens with negative numbers? None of these cases are covered, but going with just test coverage will give us the false sense of security that everything is covered.

Playing the system

Another big issue with enforcing some kind of code coverage metrics the tendency for this to encourage undesirable behaviours in order to satisfy the metric. Unit tests which have no value is the perfect example of this. Let’s say I make a unit test that covers this code:

def foo(x, y):
    if x == 1 and y > 1:
        bar()

Really I’d want to stub the thing that calls bar() and ensure that the method is actually called when I want it to be. But I don’t _need_ to do that in the test to pass the unit test metric.

Consider the following test method:

def test_bar_is_called_given_valid_details():
    foo(1, 2)
    assert(true)

Albeit this is a crude example, but here I couldn’t be bothered creating any kind of stub, so I’ve not bothered. This test would happily pass and will happily hit 100% coverage of the foo() method (well, for line and decision coverage).

This is one example on one method and so it might be unlikely to happen but when you are writing lots of code and you are being forced to hit a measure of 100% or similar coverage this kind of behaviour becomes more and more commonplace when delivery pressures are ramped up.

So don’t bother with code coverage?

Some people would say this, but it isn’t what I would advise. Code coverage can absolutely be a useful metric when used in the right way.

Understand your metric

You need to be aware of what exactly you are measuring with your code coverage. Make a decision as a team what kind of coverage you want to measure and understand what that means and what is and is not included within the coverage.

An indicator, not a target

The bad behaviours and common problems often stem from enforcing a particular amount of coverage on to your code, for example failing the CI build if code coverage is below a specific number. Given the problems that come from this, there is not really any upside from using coverage in this way.

Instead, use it as an indicator that something might be an issue. If one area of code has 80% line coverage and another area of code has 20% then it’s probably worth taking a look at the area with lower coverage. There might be a good reason for this but the coverage metric can give you a handy indicator to look there in the first place.

So…

“Test coverage is useless” -> Rubbish

“Test coverage is everything” -> Rubbish

“Test coverage can be a useful metric when the measurements are well understood and it is used as an indicator of potential risk areas” -> Boring, but true

Unbloating the UI automation

Thu, 11 Jul 2019 08:41:25 GMT

I’m not entirely sure “unbloating” is a word. In fact I’m 95% sure it’s not. Regardless of that, this is about taking those huge, automation packs for the UI and chucking them in the bin as soon as possible.

So, let’s examine the biggest issues with UI automation packs first.

Maintainability

Even with the best CI system set up in the world, the creation of new UI tests can only be pushed so far left. Although the tests themselves can be created as skeletons and even the basic structure and commands, you need to have the components there to finish off the tests and ensure they run as expected. Naturally the UI tests will lag behind the product creation to some extent, which means it’s always a game of catch up. Invariably this means that the UI pack is failing more often than not and failures become commonplace. This leads to not being able to trust the results, whether they pass or fail which means that the pack is ultimately not adding any value.

The ultimate catch all

I think the biggest issue I have with UI automation is the way it is often used as a catch all for covering various risks and issues that should be covered earlier on in the process. The UI pack is often the last to run and the longest pack meaning the feedback loop for any issues is quite large. A lot of scenarios can (and should) be covered further down the stack closer to the code. I’ve seen UI automation used on many occasions as the first place to start adding automation to a product, rather than thinking about a targeted automation strategy and looking to push the automation closer to the code.

Separation of roles

It’s very common to see UI automation separated from the product code in its own repository. While this can have some benefits, the big disadvantage for me is that is makes it seem like a separate task and this does not lend itself to creating the tests as part of a normal feature lifecycle. I also believe that this often leads to developers not getting as involved as possible in the UI test pack.

When automation is part of the product repository and runs when the product builds, it’s easy to encourage people working on the product to create and maintain that automation. If it’s separate and/or in a different language, it’s harder to do this.

My proposal

Remove all the UI automation! Think of that as the crazy goal. Maybe it can’t be achieved, but it can certainly be optimised by reducing the scope and size of UI automation. That’s the aim…now let’s see some ways we can get there…

Push the automation down the stack

This is the biggest one. It’s so common to cover scenarios and functionality at the UI level because it’s not covered elsewhere. Look at the layers of automation you have and examine what your boundaries are at that layer. Have you tested all the enclosed pieces of logic at the unit level? Can you ensure more possible data scenarios and flows through component or integration tests? Are there UI flows that are basically just strings of API calls that could be covered at that layer?

Make a dumb UI

If you are pushing automation down the stack, you can cover most of the functional scenarios and data scenarios. However you will still need to cover the presentation of the data to the user somehow. But rather than a bulky UI automation pack, can you cover some of this in the component testing of the UI? To do this, it helps if you have a simple UI which simply calls endpoints and presents data from those endpoint. Almost all modern languages have excellent automation frameworks allowing you to cover the UI component now. You can write components tests to check the external endpoint connections and contracts and you can write ones to cover how that data is then presented. However if you have a bloated UI with lots of server side logic then it will be very hard to do this.

Introduce continuous testing

An important part of having a good automation strategy is having the automation running as part of a CI system. This way all the automation you introduce at any level will automatically be providing feedback for you. A number of times I have seen CI systems which do not run any automation or ones that some automation but don’t report on it. The result of this is that confidence in the automation is not as high, meaning scenarios get added to the UI automation instead to increase the confidence.

Concentrate on user journeys

A UI pack can easily become bloated by adding defect or edge case scenarios when this logic could be covered elsewhere. One way to ensure you don’t do this is to concentrate on making the scenarios your UI pack covers full user journeys through your system. Take the most used and high risk flows through the system and maybe cover these with a lightweight easy to maintain automation pack. Covering anything else will potentially tip the balance between time and effort to maintain and risks covered the wrong way.

Conclusion

There are absolutely scenarios and products where UI automation makes perfect sense, but it’s hugely overused and misused in my experience.

Removing the UI automation is an aim I always have in any test strategy as this allows faster, higher quality feedback and more time for testing. However this is an aim and it’s not always possible or the right thing to do for the situation.

Pragmatism in Testing

Sat, 18 May 2019 20:32:08 GMT

A really important part of being a tester is the ability to be pragmatic when the situation calls for it. It’s important to have principles and values that you are guided by but there are times where you might need to compromise these to get the job done. I don’t think this should be seen as a bad thing, rather an ability to adapt to the situation in front of you. I’m going to examine some of the most common scenarios from my own experience where you may have to compromise and be pragmatic in your thinking.

One caveat first…

In the scenarios below, you should always push back and attempt to influence the situation when it’s possible to do so, but sometimes this isn’t the case. This is very common when working for a client as a contractor or consultant, where you ultimately can suggest and recommend but do not have the final say on the direction the client takes.

Individual/team skills

Every individual and team has a different skillset and this should always factor in to any decisions you make. When you are looking at tools and techniques to use, these need to be maintainable by the team that will maintain them. Of course people can learn new skills and if, in the company you working for, there is an appetite for this you could go down this route. However a lot of companies will not want to spend time re-training individuals or teams and you may have to accept this scenario.

Rules for rules sake

It’s very important in any team to have ways of working and guidelines for doing things but we should be very careful of becoming slaves to these at all costs. We should treat ways of working as guides, but not strict rules. For instance a team might have a way of working that guides the code review and branching process for all code changes committed to the product. But when there is a very small change is this process absolutely necessary to follow? It’s all about risk and whether the team are comfortable will the level of risk is not following a guideline and understand why an exception has been made.

Delivery pressures

Ultimately every team should be looking to deliver things, whether those things are incremental versions of software, documents of some kind, tools or anything else. At times decisions will have to be made about approaches to take and plenty of factors need to be taken in to account, like the time to do something or the cost that approach will take. Every approach will likely change the thing being delivered and the time it takes to deliver that thing. Risk based testing is very important for decisions like this and lends itself to a pragmatic approach when it comes to delivering as soon as possible which the highest quality possible.

Conclusion

In any walk of live I think it’s important to have values you look to follow and this is especially true in testing. A set of principles and guidelines that you look to when you have to make decisions. However being pragmatic is a vital part of a tester’s skills and understanding when and where to stick to your guns and when to push back is not easy but a skill learned over time.

Testing your contracts (5/5)

Fri, 05 Apr 2019 14:27:41 GMT

In parts 1-4 of this series, I’ve looked at what contract testing is, introduced the pact framework and showed how you can use it to create consumer side and provider side code to ensure a specific contract scenario.

Now I’m going to look at building pact in to your pipeline and tips on how to achieve this in an efficient way that doesn’t block or slow product delivery.

The Pact Broker

Pact broker is something I haven’t mentioned before, but it’s an excellent part of the pact framework and it is helps immensely adding pact to a pipeline.

What is it?

It’s a piece of software that sits between the consumers and providers and acts as a store for the contracts themselves. Let’s examine this be re-visiting the pact process:

Consumer runs a unit test which hits a mock and generates a pact json file
This file is passed to the provider
The provider hits the real endpoint and verifies the response against the details in this file

When you have one consumer and one provider, this process could be managed easily enough with some kind of cloud or network based file storage. But what about when you have lots of providers and consumers? Even if you only have one of each, what do you do during development? How does the provider know which are the latest pacts and which are currently in development? You’d either have a complicated file hierarchy or a confusing naming convention or both!

The pact broker provides a simple interface and API that enable your providers and consumers to share pact files, verify the results and display them to the rest of the business. Much more detail on its purpose can be found at https://docs.pact.io/getting_started/sharing_pacts.

How do I run it?

There are many options for this ranging from getting the code and running locally, running in a container or hitting a version hosted elsewhere. Details can be found here https://github.com/pact-foundation/pact_broker.

The best way I have used before is hosting your own broker instance which can then be accessed by all your build systems.

How does it change the process?

If you were to use a pact broker, then the process at its simplest becomes:

Consumer runs a unit test which hits a mock and generates a pact json file
This file is uploaded to the pact broker
Provider builds and downloads pacts from the pact broker where it is a provider
The provider hits the real endpoints and verifies the responses against the details in these files

Adding pact to your pipeline

Now that we know a bit about the pact broker, we can look at integrating this in to our CI pipeline. It can be done without the pact broker, but the broker is easy to use and adds a lot of visibility to the process. We will look at this in a technology agnostic way so that regardless of the languages and technologies you use for your services and CI tools, hopefully you can take something from the process.

Consumer build

The first bit we will look at is the build pipeline for the consumer. At this point we want to ensure that when the consumer builds, it is generating the pact files and uploading them to the broker.

We have seen in the consumer code explanations how to create the pact generation so it runs when the unit tests run so we will assume that this part is already done. So how do we upload the pacts to the broker? This part is called publishing and handily there is a pact command line tool for this. It’s a ruby package called “pact_broker-client” and installation instructions can be found on the GitHub link above.

We can publish the pacts to our broker using the following command:

pact-broker publish "$pactdirectory" -b="https://example-pact-broker.co.uk" -t="$gitbranch" -a=“$VERSION"

This command uses the pact broker utility we installed earlier and publishes the JSON pact files produced by the build to the given broker. Let’s examine the arguments for the command:

Directory -> This is the folder that the pacts are published to. Every language of pact has a default which you can override when creating the pacts, so if you do that, then you need to provide that folder here too
URL -> Simply the url to your pact broker so this must be accessible to your CI system
Version -> This is used to tell the pact broker that this pact is for this version of the consumer

Tag

This is a really important part of the process, so let’s examine it in more details. Tags enable you to group pacts together and allow the provider to only run verified versions of the pacts. This means that during development of a new pact, you can upload it with various tags and not affect any new builds of the provider as it will not run the pacts with those tags. For the consumer CI, the best method we have found is to use your source control branch name as the tag. This means whatever your branching system, if the pact is under development you will have a way of keeping it separate from the live versions of the pacts.

Provider build

Now that we have our pacts publishing automatically to the broker, we will want to ensure that our provider is downloading the right pacts and verifying them during its own build. Brilliantly we don’t need a new command for this. In part 4, when we looked at the provider code, we saw how the provider pact methods run like integration tests, meaning that simply running the integration tests in your CI will run the pacts.

The only thing you want to ensure here is that when you run the integration tests you are downloading the pacts with the right tag. For the CI system, that tag should be the tag you use when deploying to live (see below). This is because we want to ensure at this stage that the provider changes will not break any of the live consumers. We use “prod” for this.

Deploying to live

Now we have the process in the consumer and the provider builds, we need to ensure that when we deploy to live we can also notify to the broker that a specific version of the pact is now live. This pact then becomes the go to pact for that consumer-provider pair to ensure that when a provider changes it is not breaking the live consumer.

How you fit this in to your specific CI system and scripts will of course vary, but either way you can use the pact broker command line utility again using the following command:

pact-broker create-version-tag -b="$pact_broker_url" -t="$environment" -a=“$consumer_name" -e=“$consumer_version"

This call creates a tag on a specific version of the pact. Let’s examine the arguments supplied to the call again:

URL -> Simply the url to your pact broker so this must be accessible to your CI system
Tag -> This is the tag itself – we use the above command in deploying to all environments and the tag is the environment name itself (live is named “prod”)
Name -> This is used to tell the pact broker that this tag is to be applied to this consumer only
Version -> This is used to tell the pact broker that this tag is to be applied to the pact of this version of the consumer only

Visibility

One of the big advantages that the pact broker gives is visibility of the pacts and the results of the pact verifications. The broker hosts a website which allows you to see a number of different things. The best way to investigate is to get a container up and running and have a play around, but the most important area is the home page which lists all the pacts between different consumer and provider pairs that have been uploaded:

Here you can see at a glance all your contracts, the last time they were published and the last time they were verified. This is excellent for any stakeholders who might want to see this kind of information.

Delving further in to each pact you can see the pact itself (the left icon on each line) which shows a physical representation of the latest JSON pact this consumer/provider pair has uploaded.

Clicking the right icon will show you are history of the pact verification:

This screen show every version of the pact that has been uploaded by the consumer and it shows every time that each one has been verified by a provider version. This screen is very useful for the history of a pact and can also show a development history of the pact as you can see executions against branch and in development versions too.

The broker actually works as a REST API too providing you the ability to view all data via API calls and delete if necessary as well. The website contains an API browser where this data can be visualised.

Development process

A big part of getting buy in from the team for contract testing is ensuring that it does not slow down or get in the way of the development process too much. Therefore the process of creating and testing new pacts is important to refine as you go along with the idea of having maximum confidence in the contracts without slowing down development.
This is the process I have used with the most success:

Create the consumer tests on a branch
Publish to broker, tagged to the branch
Create the provider test on a branch
Run against consumer branch tests – repeat 1-4 until passing
Release consumer
Release provider

This process ensures that new pacts are consumer led, but that the consumer won’t break the provider. An important part throughout this process is the communication between the consumer and the provider, which must be good at all times.

Summary

So, this is the end of the ridiculously large five parter on contract testing. We’ve looked at when contract testing is useful, what Pact is, how to write consumer code, how to write provider code and finally how to integrate it in to your CI system.

I think contract testing is an API world is vital and definitely something to look in to if you aren’t currently. It is not right for every scenario but I would definitely encourage you to have a look.

Testing your contracts (4/5)

Sun, 10 Mar 2019 12:34:37 GMT

If you’ve been ready the parts I’ve written so far, you’ll know we’ve gone through what contract testing is, the pact framework and the consumer side code for a specific scenario.

In this part, we’ll be examining the provider side code of the same scenario. I’d therefore recommend checking out the earlier parts if you haven’t yet, or even for a refresher as this part may not have much context otherwise.

As before I’ll be using Java for this example, but there are many other supported languages and Pact provides excellent documentation and examples: https://docs.pact.io/implementation_guides.

Scenario

Let’s remind ourselves of the scenario we are using for this pact:

Consumer

Our consumer is a simple API that orchestrates data from a number of services and consumes from the Employee Service.

Provider

The provider is a REST service that covers Create/Read/Update/Delete (CRUD) operations for the Employee object.

Pact

The pact for this example is:

Given An employee exists with id of 1**
When** I request to view that employeeThen I am returned success and a single employee object

Provider Code

Now that we have refreshed ourselves on the scenario, let’s take a look at the code. The provider side only has the responsibility of running the tests against itself, so the test code is fairly simple compared to the consumer.

Dependancies

You need to add the pact framework for the provider for your chosen technology. For Java, the following package is needed:

<dependency>
   <groupId>au.com.dius</groupId>
   <artifactId>pact-jvm-provider-spring_2.11</artifactId>
   <version>3.5.23</version>
   <scope>test</scope>
</dependency>

This was the latest version at the time of writing, but I’d use the latest stable one when you come to use it.

Provider test class

Let’s create a simple boilerplate class for running the pacts:

@RunWith(SpringRestPactRunner.class)
@Provider("employee-service")
@PactFolder("pacts")
@SpringBootTest(
classes = EmployeeServiceApplication.class,
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class EmployeeServicePactProviderTestIT {

@TestTarget
public final Target target = new SpringBootHttpTarget();
}

Let’s example the annotations and fields:

We are using SpringRestPactRunner to run this test as the app is a Java Spring Boot app and this works well in this case, other Java runners (like simply PactRunner) are running and what you need will depend on your application under test
Provider – just refers to the provider so we only run pacts for this provider, must match the name used in the pact
PactFolder – tells the test where the pact files are (in this case a local resource folder in the project called “pacts”)
SpringBootTest – part of the spring boot framework and signifies this is an integration test which requires the application to be running when executing
TestTarget – part of the JUnit test runner and tells the test where to hit when running the tests – we give it a SpringBootHttpTarget, which is an addition to the pact framework and tells it to use the running spring boot app as the target

Just having this test class will cause the pact framework to be run as part of our build tests. This means when we build and run tests, pact will look in the given folder and attempt to run those pacts it finds. Currently it will find our pact and if it runs, it will error with the following:

MissingStateMethodException

This error means that the provider has run our pact and the pact is saying it needs a particular state but we haven’t provided a method to create that state.

State Method

So let’s create our method now. All we want to do in this method is use the pact framework to tie in the method to the given state and then set the provider up in that state.

@State("An employee exists with id of 1")
public void anEmployeeExistsWithAnIdOfOne() throws SQLException, IOException {
//mock the service layer
}

There is nothing to it! The reason for this is I haven’t included the mock code because that will be completely different for any application. Although for most applications, mocking a service is extremely simple. The important bit to note is the @State annotation which is the part of the pact framework that ties this method to the given state in the pact from our consumer.

Mocking

Why are we mocking? Well, we are mocking at the layer below the contract because we are not testing this in the contract tests. All we want to test is the contract itself and not the functionality of the provider. Therefore we allow the controller to do it’s work as this is the input point for the consumer and we mock below that as changing this won’t affect the contract. It might affect the returned data values, but we can verify that using unit and integration tests.

Request Validation

There may be occasions when you want to do validation on the incoming request itself – simple data and field checks. If mocking, you have to ensure that this validation is not mocked – this is because it’s a part of the contract. The incoming request structure is a vital part of any consumer contract as it’s the consumer telling you what it will send you.

Summary

So we’ve explored how to create provider code to run your contracts you’ve been given by the consumer and we can now successfully pass our contract tests!

However the approach we have is a bit clunky due to passing files around and not having it in our CI system. In part 5, we’ll explore the pact broker and integrating pact in to a CI system.

Testing your contracts (3/5)

Tue, 19 Feb 2019 20:19:54 GMT

We’ve already looked at contract testing and the PACT framework and in part 3 we’ll be looking at using that framework to create your consumer side contracts.

Language Support

Pact supports many different languages and you can find details and guides for the supported ones at https://docs.pact.io/implementation_guides. I’ve used Pact in Javascript, Java and C# and have chosen to use Java for the examples here as it’s the one I’ve used most recently. The principles are the same, no matter which language you use though.

Scenario

So let’s look at the scenario we will use for this guide:

Consumer

Our consumer here will be a simple API that orchestrates data from a number of services. In this case it will be consuming from the Employee Service.

Provider

The provider will be the Employee Service is a REST service that covers CRUD operations for the Employee object. CRUD is Create/Read/Update/Delete in case it is not a term you have seen before.

Pact

We are going to use the following scenario for our pact:
Given An employee exists with id of 1
When I request to view that employee
Then I am returned success and a single employee object

Consumer Code

So let’s delve in to the code we need for the consumer side.

Dependancies

The first thing we need is to install the pact framework. With Java, this is a matter of adding the following package for the consumer code:

<dependency>
     <groupId>au.com.dius</groupId>
     <artifactId>pact-jvm-consumer-junit_2.11</artifactId>
     <version>3.5.23</version>
     <scope>test</scope>
</dependency>

The version is not too important – I’d generally get the latest stable and stick to that unless new features come up you wish to take advantage of.

Initial Class

So, let’s create the boilerplate code. Firstly we’ll create a class and call in EmployeeApiContractTest – it’s good to have a naming convention up front. We’re then going to create two methods, one will be the method that sets up a mock and the request details and expected response and the other will be the one that hits this mock to generate the JSON pact file.

Whatever language you choose, this is the same pattern you will need – something to set the mock up and something to hit that mock and generate the pact file. This pattern is the supported pattern by the framework and in Java is accomplished by the annotations “@Pact” and “@PactVerification”.

Therefore we end up with the following class:

public class EmployeeApiContractTest {

private RestTemplate restTemplate = new RestTemplate();

private final String PROVIDER_NAME = "employee-service";
private final String CONSUMER_NAME = "public-api";

@Rule
public PactProviderRuleMk2 mockProvider = new PactProviderRuleMk2(PROVIDER_NAME, PactSpecVersion.V3, this);

    @Pact(provider = PROVIDER_NAME, consumer = CONSUMER_NAME)
    public RequestResponsePact shouldGeneratePactWhenRetrievingEmployeeWithIdOfOne(PactDslWithProvider builder) {
    }

    @Test
    @PactVerification(fragment = "shouldGeneratePactWhenRetrievingEmployeeWithIdOfOne")
    public void shouldReturnValidResponseWhenRetrievingEmployeeWithIdOfOne() throws URISyntaxException {
    }
}

Pact Mock Setup

Now we need to set up the mock. To do this, we need to set up the request we are going to send, where we are going to send it and the response that we expect back from the provider. We are also going to set a state for the provider so it knows what data needs to exist when we send it this request.

Each supported language has different ways of setting up this information but pact has unless documentation for each.

For Java, you can use a “domain specific language (DSL)” class called “PactDslJsonBody” to create your request and response.

Request

For this particular request, we have no JSON body we are sending in. We will be using the id of the employee we wish to get in the path so we won’t need to use the DSL to create a JSON body for the request. We’ll set the path we are hitting later on, for now we can go straight on to the response.

Response

For the response, will be expecting a JSON body with the following:

A root employee object, containing
- An integer id
- A string first name
- A string surname

We can use the DSL to create this as follows:

PactDslJsonBody bodyResponse = new PactDslJsonBody()
        .object("employee")
        .integerType("id")
        .stringType("firstName")
        .stringType("surname")
        .closeObject()
        .asBody();

As you can see the DSL describes the JSON as if you were writing it line by line. This is a loose description, which means that if there are extra fields we have not specified, these will not break the pact. This is what we want as we have specific the fields in the body we are using, if there is other information it’s not for us to care about. It may matter to a different consumer, but not us.

Another point here is that we do not verify the data in the fields. That’s because it’s not our responsibility to ensure the provider is working correctly, we just need to ensure we are getting the structure of the response we expect or we can’t extract the required data from the response. Therefore we use the “…type” methods, which just check the type of the field and the name of the field. The data could be anything, it’s not going to affect how we use it.

Mock

Now we have our response object which we are expecting, we need to set up our actual mock. This is where we say which endpoint we will hit and the expectations we have when we do that. This is the information that then forms the generated pact JSON:

return builder.given("An employee exists with id of 1")
        .uponReceiving("a request to retrieve the employee")
        .path("/employee/1")
        .body(bodyRequest)
        .method(RequestMethod.POST.name())
        .willRespondWith()
        .headers(headers)
        .status(200)
        .body(bodyResponse)
        .toPact();

As you can see the PACT framework is written in a very readable language so most lines here are pretty self explanatory. The one I want to highlight is the “given”. This is where we describe the state and it is the link with the provider. As we will see in part 4, the provider needs to know this state and understand what it means. I’ll go deeper in to this in part 4, but it’s important to understand that it signifies any state that the provider needs to be in to satisfy this contract.

Pact Verification

Next we need to write the verification code. This is a piece of code which runs like a test – it will hit the mock with the given response and the pact framework the cause a JSON pact to be generated. The “verification” part of this is that it is effectively verifying the code that hits the mock endpoint. I’ll explain a bit more later on but for now, let’s write the verification code.

@Test
@PactVerification
public void shouldReturnValidResponseWhenRetrievingEmployeeWithIdOfOne() throws URISyntaxException {
Integer employeeId = 1;
EmployeeProvider employeeProvider = new EmployeeProvider(restTemplate);

final ResponseEntity<GetEmployeeResponse> response = employeeProvider
.getEmployeeById(new URI(mockProvider.getUrl()), employeeId);

assertEquals(200, response.getStatusCodeValue());
}

Let’s example some aspects of this fairly simple method. The @Test annotation ensures that this test is run along with the unit tests and the @PactVerification annotation is part of the pact framework which ensures that the Pact method we have already written is invoked when we hit the relevant endpoint and the pact file is generated.

Encapsulated Code

An important part of the method above is the EmployeeProvider class. This class is a piece of code which encapsulates the code that calls the actual endpoint we are writing the contract for. This code can then be called by both the pact and the service code. This is very important because if we change the way we call the endpoint, we would want our pacts to fail. However if we don’t use a piece of shared code for the call, this would not happen unless we change the pact too – which we wouldn’t know to do. For context, this is the code within the EmployeeProvider:

@Component
@AllArgsConstructor
public class EmployeeProvider {

private static final String GET_ENDPOINT = "/employee/";

private final RestTemplate restTemplate;

    public ResponseEntity<GetEmployeeResponse> getEmployeeById(final URI urlPrefix, Integer id) {
        return restTemplate.getForEntity(
                urlPrefix + GET_ENDPOINT + id, GetEmployeeResponse.class
        );
    }
}

Asserting the response

Given the point of this test method is to generate the pact which is verified why are we actually performing any assertions on the result? There are two main reasons for this.

Verifying the mock

The first is that we want to ensure we have coded to hit the mock correctly and the easiest way for this test to ensure that is check we have a successful response. If we have, we can be fairly sure we’ve hit the mock and the pact will have been generated.

Unit testing

The second and biggest reason for performing the assertions is that in doing so, we are performing a unit test on the encapsulated code. We are not looking at these tests in place of unit tests on that code as that is not the responsibility of these tests and we’d still want to write quality unit tests for it. However we want to ensure here that if we change the encapsulated code for any reason that our pact test verification also fails. This will ensure that we investigate and change the pact where needed to match the changes we have made if that’s the chosen route. Without any assertion, we could not be aware of a failure and the pact test could stay as is – meaning we’d have a pact file that doesn’t actually represent the contract anymore!

Running the test

With Java, this is nice and simple because the @Test annotation tells JUnit to run the test as it would any other unit test! For the specific test runner you are using, you just need to treat it like a normal test and it will run as a normal unit test and fail if the assertions fail.

Output

The default for Java is to output the pacts to the target folder in a folder called “pacts”. This will change depending on the language (i.e. C# defaults to the bin folder) and can be configured to be a specific folder if needed. Pact has excellent documentation and examples for every supported language.

So, let’s see the output for this pact! Every field should make sense given the code we have written so far, so we won’t go through it in too much details but a few things to note specifically are:

The matchingRules in the response are the matchers we used earlier telling the provider to match on type and not match the data
The generators are what pact uses when hitting the mock in the consumer – because we don’t provide data for the mocked response, it has to generate some that meet the matching rules
The provider state is clearly noted on its own and is the most important part as the provider can’t verify the pact without knowing and coding for this state

{
    "provider": {
        "name": "employee-service"
    },
    "consumer": {
        "name": "public-api"
    },
    "interactions": [
        {
            "description": "a request to retrieve the employee",
            "request": {
                "method": "GET",
                "path": "/employee/1"
            },
            "response": {
                "status": 200,
                "headers": {
                    "Content-Type": "application/json;charset=UTF-8"
                },
                "body": {
                    "employee": {
                        "firstName": "string",
                        "surname": "string",
                        "id": 100
                    }
                },
                "matchingRules": {
                    "body": {
                        "$.employee.id": {
                            "matchers": [
                                {
                                    "match": "integer"
                                }
                            ],
                            "combine": "AND"
                        },
                        "$.employee.firstName": {
                            "matchers": [
                                {
                                    "match": "type"
                                }
                            ],
                            "combine": "AND"
                        },
                        "$.employee.surname": {
                            "matchers": [
                                {
                                    "match": "type"
                                }
                            ],
                            "combine": "AND"
                        }
                    }
                },
                "generators": {
                    "body": {
                        "$.employee.id": {
                            "type": "RandomInt",
                            "min": 0,
                            "max": 2147483647
                        },
                        "$.employee.firstName": {
                            "type": "RandomString",
                            "size": 20
                        },
                        "$.employee.surname": {
                            "type": "RandomString",
                            "size": 20
                        }
                    }
                }
            },
            "providerStates": [
                {
                    "name": "An employee exists with id of 1"
                }
            ]
        }
    ],
    "metadata": {
        "pactSpecification": {
            "version": "3.0.0"
        },
        "pact-jvm": {
            "version": "3.5.23"
        }
    }
}

Summary

We’ve seen how you can write a simple consumer driven pact file using Java and discussed some of the patterns to look out for when we are writing it.

What do we do with this pact file now? We need to give it to the provider so they can verify it. In part 4, we’ll look at the provider side of this pact and creating the code to perform this verification.

Testing your contracts (2/5)

Mon, 04 Feb 2019 20:53:47 GMT

Pact

In part 1, we looked at what contract testing is and the gap it can cover in an automation strategy. In part 2, we’re going to look at Pact, which is the most widely used contract testing technology and how its framework implements contract testing.

Consumer driven

Pact is referred to as a consumer driven framework, which means the consumer is in control of creating the contract and the contract only contains anything that the consumer actually uses.

Pact is no way replaces any communication between a consumer and a provider, in fact it encourages more communication. Consumer driven does not mean consumer controlled. We’ll examine this more when we see about putting pact in to a CI system, but the main point for now is that the consumer is driving the creation of the pact and leading the discussions as they will benefit the most from the pact being in place. It will give them assurances that the provider they rely on will not break their own functionality when it changes.

The Pact framework

I’ve attempted to sum up the pact process using this simple diagram:

Consumer

On the consumer side, the pact framework works by running a mock server where you can add “interactions”. These interactions are mock endpoints which are the endpoints the consumer would be hitting on the provider. The framework has utilities where you set the mock up so that given a particular state and input then as a consumer you are expecting the following output.
This forms the “contract” and when the pact framework mock is hit, the framework automatically produces a JSON file which represents this contract.

JSON Pact

An example of a simple pact is shown below (I’ve had to blank out any sensitive data to the project I’ve taken this from):

This pact is shown within the Pact Broker, which is a utility I’ll go in to in a later pact but for now I’ve used as it shows the JSON in a nice format.
The first part describes the interaction in human readable form – given a state, upon receiving the below request, then the provider will respond with the below response.
It then describes both the request and the response in simple terms.

Request -> The REST method, the endpoint path to hit on the provider and any headers and body that are in the request are shown
Response -> The details here form the checks the provider will perform when it runs this pact and generally consist of a status, one or more headers and a body, but all of these are optional

Provider

The provider uses the pact framework to perform the following actions at build time:

Download or retrieve pacts when it is the provider
For each pact:
- Run itself
- Hit the endpoint in the request part of the pact with details described in the pact
- Check the actual response matches the checks as described in the pact
- Send result (optional)

The key on the provider side is that the pact is checked by hitting the actual endpoint and not a mock, so if the endpoint code is changed then the pact might fail. I say might as the pact only covers the parts of the endpoint that the consumer is using. If the consumer doesn’t use it, it won’t be covered in the pact. I’ll explain this concept further in later parts.

Testing your contracts (1/5)

Mon, 21 Jan 2019 15:29:13 GMT

Contract.. what now?

I have been doing a lot of work with contract testing recently and wanted to do a blog series on something, so this seems as good a thing as anything else! And it’s more technical and I want to do a mix of technical and non-technical blogs.

So, contract testing… some of you may have done this before and some of you may not but it’s becoming more and more relevant given the rise of micro-services and REST APIs.

What is contract testing?

Simply put, contract testing is about assuring the understood and agreed contract between a provider of an endpoint and the consumer(s) of that endpoint.

That sounds simple, and in general it is, but let’s break it down to make it explicit.

The contract

This is the most important part of any contract test and lends itself very nicely to a given/when/then syntax:

Given the provider is in a specific state

When I send the provider http request X with body/headers/query params Y

Then I expect Z to be returned

The given forms a state that you are telling the provider it needs to be in to accept the request. The when forms the details of the request you will send and the then is the response you are expecting, which could take many forms.

The consumer

This is a service, another API, some UI of some kind or anything else that might call the provider’s endpoint. Contract testing is consumer driven meaning it’s down to the consumer to detail its contract and then talk to the provider about that.

The provider

The simplest part really, its responsibility with a contract test is simply to get the contract and ensure that it satisfies it.

What is contract testing useful for?

To explain this well, I need to use some diagrams to explain the gap contract testing covers. Let’s imagine the below is a rudimentary test approach for a small product made up of a REST API providing data to a UI that simply consumes and displays this data:

The provider here is the API and the consumer is the UI. We are ensuring the individual component’s functionality by their own unit and component tests. We also have some kind of end to end automation pack (let’s ignore the details) which assures the business processes for the product.

The question to ask is:

If a provider change breaks the consumer, when do we find out?

We likely won’t find this out through any exploratory testing of the provider and certainly through the provider automation unless we have an encyclopaedic knowledge of the functionality the consumer uses. And that’s just with one consumer…it’s even more impossible with many consumers.

The answer to this question is we might find out when we run the end to end automation, which would require the provider to be running somewhere after we’ve built and deployed it to wherever that is. However if we haven’t covered that case in the automation, we might find it out when we test the app as a whole, or in UAT testing or…gulp…in live.

But, with contract testing.. we find out at build time for the provider.

This is the big thing that contract testing gives you; it can assure that as a provider any change you make will not cause any issues with any of your consumers before even merging your provider code.

What is contract testing not?

Contract testing is for a very specific scenario like I’ve described above, where you have a provider and one or more consumers. It’s even more useful when the consumers make use of different functionality of the provider. There are a few things that contract testing will not give you however that are common misnomers.

Component testing

Contract tests are not there to assure the functionality of either the consumer(s) or the provider. That should be taken care of by the component itself, not by the contracts.

End to end testing

The contract tests do not assure the end to end process through the consumers and the provider, they simply ensure the exit and entry points of each through a contract as described above.

Conclusion

We’ve examined what contract testing is in part 1 and what it can be used for. We’ve also looked at the component parts of the process and a couple of misnomers about it.

In part 2, I’ll introduce the most widely used tool for contract testing called Pact and examine how the Pact framework works.

Elitism in testing

Mon, 07 Jan 2019 20:14:57 GMT

Since the first half of 2018, I’ve been really bad at blog updates and I want to change this and start blogging much more regularly. So I’m going to write some shorter blogs to get me back in to it and here is the first! I’m still going to write the longer articles, but I’ll be looking to save them for publishing elsewhere.

As always let me know any thoughts in the comments and contact me using the buttons if you wish via the various channels.

Elitism
Something I’ve been thinking about a lot recently is elitism within software development and I have personally experienced elitism from a coding point of view many times in my career. Recently I’ve also started to see a trend of elitism within testing as well.

What is elitism?
Well the Oxford dictionary defines it as:

“The superior attitude or behaviour associated with an elite”

To narrow it down to testing, this would mean an individual or group of individuals who believe they are better at testing or always right when it comes to ways of doing things.

How does this manifest itself?

Non constructive criticism
Whenever you meet someone who questions the methods you are using, it’s important that the criticism is useful and constructive. Someone with an elitist attitude will simply tell you what is wrong with your testing without any suggestions or help for improvements.

No appreciation of learning time
An elitist individual will tend to not appreciate that someone may not know a particular tool or technology already. No matter how experienced you are, there are always new things to discover within testing and new things to learn. This is inevitable and it’s a good thing! However to do this means being a beginner again with that tool or technology.

No appreciation of circumstances
A great tester (and indeed anyone in software development) will have a sense of pragmatism and an ability to compromise. There are many times in your career when you can’t always do things the way you feel they should be done. This might be to do with the company/client you are working for, resource limitations, skills shortages or many other reasons. In those circumstances good testers will do the best they can and always try to improve a situation in every way possible.
An elitist person will make no effort to appreciate that there are always reasons for decisions that happen. They will look at the situation, without appreciating any of the context and tell you what is wrong.

Rather than helping you with getting started an elitist will offer no practical help at all or they will ask you things they are fully aware you don’t know.

Consequences of elitism
An elitist attitude is a frustrating and annoying thing to come across. But more than that there are consequences for people encountering this kind of attitude. The confidence of the individual is often knocked and damaged by elitism. They can increase their feeling of Imposter Syndrome and potentially set their progression in their career back.

How do I ensure I am not elitist?
With the vast majority of companies, compromises have to be made when it comes to software development. There may be a multitude of economic, business or environmental reasons why this is the case. But those compromises are rarely taken likely and without understanding the risks involved.

As an experienced tester looking at any situation and any test strategy you have to recognise this. If someone seems to be an issue or a strange way of operating there is likely a reason that this decision was taken in the first place. You have to be mindful of this when critiquing any ways of working.

So I can’t offer advice?
You can absolutely offer advice and most people would welcome it. However the key is in how you offer that advice.
Always use empathy and understanding for the position the subject is in as there may be very good reasons why it’s the case.

TestBash Brighton 2018

Sat, 17 Mar 2018 14:07:51 GMT

I’m writing this on the train back to Leeds from my second TestBash and the first one since I’ve been blogging so I really wanted to write down my thoughts.

TLDR; Fantastic event!

Firstly congratulations to Ministry of Testing (https://www.ministryoftesting.com/) for hosting an excellent event – very well organised with a great line up off speakers. I can only imagine the effort that goes in to putting this event together and it seemed to be a great success from everyone I met and spoke to. Individually you are all very friendly and welcoming as well so that is appreciated.

Software Testing Clinic (TestBash edition)

On the first night I arrived, there was a Software Testing Clinic held in the evening. For those who haven’t been to these before, they are a great way of connecting with other people in the industry, learning new things and contributing to some great discussions. They are also social occasions where you can meet and befriend lots of good people. I was attending on my own, so was a little nervous given the amount of people that would be attending, but there was no need to be. Lots of games were played, plenty was eaten and drunk and I had lots of great conversations – about testing yes, but also about plenty of other subjects!

Workshops

The next day was the workshop day and there was a selection of workshops to attend which were focused on various different topics. I think the selection was well thought out and involved topics as diverse as “Interviewing for testers”, “Exploratory testing 101” and the two workshops I attended; “Web apps security” and “Communities of practice”.

Talks

Prior to the first workshop there was a talk by Dr Vera Gehlen-Baum on learning within testing. Vera has previously spoken at the UN so it was very cool to be able to hear her speak here! The talk was really informative and it was nice to listen to a different subject than you usually here at test conferences. Vera was asked to step in to this slot at late notice which made her talk even more impressive!

After the workshops, there was a second talk, this time by Anusha Nirmalananthan about having a secret at work and the power of revealing and owning the secret. This was a very powerful, emotional and personal talk and I think the whole audience was impressed and moved by Anusha. For me this was the highlight of the weekend and I’d like to thank Anusha for sharing her story with us.

Web apps security

The morning workshop was held by Dan Billing, who is someone I’ve followed for a while and is one of a number of leaders in the field that are workshopping or presenting at this conference. Dan took us through the ways of identifying security threats and risk in web applications and some techniques we can use to test for the presence of these threats in our own applications. I’ve not got too much experience in this area so I found it very useful and informative.

Communities of practice

The afternoon workshop by presented by Emily Webber, who has very much made communities her own within the industry in the last few years. She has an excellent book out on the subject which I would recommend you to read:

“Building Successful Communities of Practice – Emily Webber”

Communities of practice are something I have tried often at previous companies and my current place with varying success so it was great to get some tips for setting them up and keeping the momentum going. It was also good to hear from lots of other people experiencing the same issues as myself.

Conference Day

The conference day was a single track day with nine talks, breaks in between and a lunch break halfway through. I’m glad it was a single track because it meant that there were no difficult decisions to make about which talks to attend! All the talks were good, but rather than go through all of them individually I wanted to pick out a few of my highlights, although please don’t take this to mean the others weren’t good as they were all excellent talks and I’m always impressed with people who talk at conferences as it’s not an easy thing to do. The talks I’ve highlighted below simply resonated with me due to my experiences or my current position.

Experiences in Modern Testing – Alan Page

This talk was definitely the most head nodding experience for me this weekend given the fact I agreed with almost everything Alan said! I’ve not encountered Alan before, but I’ve now discovered his podcast, twitter and his blog and I’ve found I agree with him on a lot of things! In fact on his twitter page, his top pinned tweet says this:

“If you want to be a better tester and be an expert in testing, do not even bother learning to write test automation”

If you look further down my blog you’ll see very similar thoughts from myself too (i.e. here) so I’ll be following Alan from now on.

So on to his talk…

It was about the evolution of Lisa Crispin’s Agile Tester in to the “Modern Tester”. The core premise of the modern tester is the idea that a tester on lots of teams these days is a Software Engineer that simply specialises in Testing or Quality. I absolutely see this in my job now and I would say this very accurately describes myself. My current team is the most skilled team I’ve ever worked on and every individual is able to pick up most tasks and stories, but everyone also has their own specialism. Alan is an engaging speaker and he articulated his thoughts very well.

To Boldly Go: Taking the Enterprise on a Journey to Structured Exploratory Testing – Aaron Hodder

Aaron is a very experienced tester and has a great blog which I’d recommend you read. Aaron’s talk was very interesting to me and it made me think of previous experiences I’ve had in places that were not as modern in the way they perform their testing and organise their teams.

He took us through an experience he had with a company who would not change the way they worked with regards to testing and showed us the systematic steps he took to add some structure to their testing. I thought it was a very realistic scenario and one that is very common. Aaron’s solutions were a great example of making the best of a situation and making small consistent improvements. The end result wasn’t something Aaron would recommend if you had a blank slate but very few companies have that – especially if you work as a consultancy and the company is your client, as was the case here.

I also managed to meet Aaron and have a chat afterwards and it was a pleasure to do so!

Talk videos

As mentioned all the talks were great and I’d highly encourage you to watch them all and seek out the people who performed them. They are all leaders in the field and great people to learn from. I’m lucky enough to have met most, if not all of them as well and have always enjoyed the experience. The line up can be found on the Ministry of Testing site here:

https://dojo.ministryoftesting.com/events/testbash-brighton-2018

And by signing up to their dojo, you are able to get the videos from this and many other events, so it’s worth a look.

99 second talks

The day ended with about 15 99 second talks on an amazing variety of subjects. Highlights for me were The Artful Tester talking about how to treat people and Ali Hill with a Choose Life monologue!

Summary

I had a great time at Test Bash and will be looking to go to more in the future. I’ll be looking to present at future ones too so hopefully I can maintain the high bar I’ve seen here!

Thanks to all those who presented, everyone who organised the events (especially the volunteers) and to all those I met and talked to. Special mentions to Anja, Ugne, Matt and Martyn – it was great to meet you and Anja and Matt – I hope your weddings are brilliant!

Oh and one final thing…if anyone has a Testing job going in the London area, then you couldn’t do much better than getting in touch with this woman…

#GiveSwannyAJob

Ergo, check your ego

Wed, 14 Mar 2018 15:04:35 GMT

As the saying goes..

“There is no I in TEAM”

This is undeniably true, but there is also no WE in TEAM either. However there is a ME (sort of).

So what does this mean? Does it mean that the team is less about individuals or groups of individuals and actually about thinking what you can do to help the team? Well yes it does actually, but what it more likely means is idioms are pointless.

But there is a serious point hidden in there about the way we deal with our own ambitions and desires while working as part of a team. For me this comes down to Ego. Thinking back to my college psychology course, the Ego effectively represents the selfish part of the individual, which is concerned only with the individuals own desires and needs. It looks to manifest the individual’s base desires (which come from the Id) and isn’t concerned about the social and cultural rules governing us (that’s the Super Ego). This isn’t a psychology blog though so excuse any worryingly inaccurate statements!

Anyway, what’s this got to do with software development? Well, I think there any many times when working in a software team that the Ego seeks to take control and to be the best worker and team member you can be, then you need to try and swallow your pride, check your Ego and do the best for the team.

I’m going to list a few examples from my experience.

Test first

I’ve always been and I’m always going to be a tester first so one practice that I’m a big proponent of is a test first approach. This might be Test Driven Development, Acceptance Test Driven Development, Behaviour Driven Development or some other technique but the key is that the test scenarios are created and defined up front and that in some way they define the requirements of the feature you are implementing. This kind of approach has lots of benefits for the team, for example:

The end feature will nearly always meet the acceptance criteria
The focus is on the behaviour of the feature and not the implementation
The code tends to be easier to maintain and understand due to focus and no loss of scope when refactoring
Free documentation via the tests and the code

However a test first approach does not come easy to many software engineers, especially if you’ve been used to non test first for a long time. Consequently a lot of people will be against the idea and potentially be concerned about trying it. This is the Ego talking. If there are benefits for the team, then as an individual you need to be prepared to try it and potentially adopt it.

Pairing and mobbing

I think it is fair to say that software development has in the past been a practice that can be done by many individuals working on single items, then putting them all together. More and more these days though, teams are working with a model that allows multiple people to work together on the same feature. This kind of collaboration can have short term and long terms benefits to the team, a few of which are:

Multiple people working on a feature means the knowledge of the feature is shared with more of the team
Individuals can learn techniques and practices from other team members
Team members can build relationships quicker
Code reviews can be done during the pairing
“Code Blindness” can cause problems to be missed – more eyes makes this less likely
Allows a tighter focus on the scope and requirements of the feature

There are many different levels and types of pairing (I might do a separate blog on this actually) including developer/tester, developer/developer, shared controls, single controls, remote sharing and so on. All of them have differing benefits but the similarity between them all is that the benefits are generally focused on helping the team.

This kind of working does not come naturally to a lot of people and it is very easy to avoid doing it because it does not sit well with the way you work or your personality. But this is where you need to check your Ego. It’s not about you as an individual but the team. If your team can benefit from adopting these approaches then you should be prepared to support and promote these ways of working.

Doing the ‘dirty’ work

There are many different aspects to developing a feature for a piece of software from the initial creation of the feature/epics/stories through to the deployment of the software in to live and the monitoring of it going forwards. Consequently there are many tasks that a self reliant team will need to complete in order to say a feature is Done. I can’t list every one as that will depend on the team, product and company however a few common ones might be:

Creation and refinement of the stories
Creation of and execution of automated checks
Maintenance of and execution of existing checks
Modifying the existing pipelines
Updating the monitoring
Updating the infrastructure (manually or via code)
And…writing the product code

As an individual you may prefer some of those tasks over others but you have to put this aside to complete the tasks that best enable the team to complete the work they have committed to. You may also find that individuals are better skilled at some of those tasks than others however if that’s the case then you will always have blockers and single points of failure within the team so what is the best way of helping the team? It’s by the weakest person for that task taking it on, pairing on it and learning as they do it. In the long term the team will massively benefit from this approach. As an individual this could be frustrating when you know this isn’t the quickest way to complete a specific task but this is again when you should be checking your Ego and doing what is best for the team.

A focus on completion

When working within any software development team, no feature is adding value to the customer until it’s been Done. Whatever your definition of this is, it must meet it to be delivered to the customer.

With that in mind the focus of the team should always be delivering done or completed features to the business. Anything else will not give the business a return on investment yet. Therefore the team should always have a focus on “working from the right” or looking to do what they can to complete features before they start new ones. As an individual it can be hard to buy in to this mindset because it’s hard to work with others to complete something that is already started. It’s much easier to start something new; but that’s the Ego talking…

Epilogue

I’ve listed above a few common scenarios where it can be easy to do what’s best for the individual instead of the team. I feel I should declare that not all of the above approaches are ones I like to do as an individual either but I do support them due to them generally being better for the team.

I fully appreciate there are occasions when a principle cannot be followed and pragmatism must come in to it. For instance you can’t always have an individual working on a task to upskill themselves. I do believe this is generally best for the team, but if it’s the end of a sprint for example and you need to get something out, then maybe that’s not the best time for it.

You must absolutely be pragmatic when deciding how the team works and when considering what practices and techniques you as an individual will choose to promote within the team.

For me, it always comes down to one question you should always be asking yourself and if you can honestly answer with ’the team’ then it’s likely you are doing the right thing:

“Is this best for the team or for my Ego?”

Automation – The Tester’s Holy Grail (Part 2)

Sun, 21 Jan 2018 10:41:35 GMT

Now that I’ve finally got around to doing part 2, I should probably remind myself of the reason for part 1…

So..

A question that I seem to have been asked a lot recently is:

“How do I get in to automated testing?”

For part 1 I explained a couple of issues I see with the question and the reason people ask it. You can find part 1 here.

Even given any potential pitfalls though automation is a great area to look to get in to so these are my tips for helping anyone start.

Look to solve a real problem

I think a lot of people come unstuck or struggle to focus on learning and using automation because they try to invent something that they want to apply automation too. It’s so much easier if you can find a real life problem to solve. This might be an existing work project where you wish to increase confidence in releases or deliver updates to the customer as quickly as possible. It might be a new personal project that you have started and you want to define the acceptance criteria in the tests before starting the code. There are plenty of things in between too, but if it can be a real life situation it helps a lot.

Pick the tool for the occasion

The important thing here is not to pick the tool because it is the current trend in automation. There are myriad automation tools, languages and frameworks out there so how to pick one to start with? When you have your real life project, think about the problem you are trying to solve.

Do you want to ensure your product is functionally as expected to the user?
Do you wish to ensure it is accessible?
Do you want to ensure it performs under load?

They are just a few examples but when you have the problem you are trying to solve then you can do research on the tools that might help. Pick one that suits the problem and yourself and look to start there.

Remember the principles

I’ve written about the Principles of Automation before and might do again but in a nutshell there are a number of pointers that I think apply in almost all automation situations. When you look to learn to automate checking it’s really important to not learn bad habits at the start. There are plenty of Principles you can research and going in details about them was out of the scope of my idea for this blog but a few headlines from me are:

Automate as far down the stack as possible
Treat automation code as you would production code
Atomicity should be the aim
Run fast, run often
Pipelines…never ignore failing checks
If it doesn’t add value, bin it
Check first, develop later

There are many more but that should give you a few to look up, research and maybe even disagree with!

Focus on the business value

No automation checks are worth doing unless they are adding business value. This value can take many forms though. A few examples might be

Reducing the risk of a release
Decreasing the time to live for new features
Finding issues as early as possible
Certification of your software on many configurations and hardware

There are a few examples for the value that automation checks as a whole can bring but there are many more too!

However you should also consider the business value of each individual check that you create. Is that particular check actually checking anything specific? Can that check be performed closer to code? If an automated check is not adding value then change it to add value or remove it.

Have fun

Most of all learning to automate should be something you want to do and that you enjoy, not something you feel you have to do. There is a lot of satisfaction in writing automated checks and seeing the value they are adding so try to ensure you enjoy that process.

Automation – The Tester’s Holy Grail (Part 1)

Mon, 04 Dec 2017 22:20:31 GMT

A question that I seem to have been asked a lot recently is:
_
“How do I get in to automated testing?”_

I understand why people ask this but to me there are a couple of glaring problems with the question.

There is no automated testing

When people ask the question, what do they mean by the phrase “automated testing”? I believe most people mean the creation and maintenance of code which performs deterministic checks on other code. Which isn’t testing. Instead it’s a specific skill using specific tools which can help to provide feedback in specific situations.

You cannot automate your testing because if you think you can then you are taking all the skill out of testing. I’d argue that if you believe you can automate your existing testing then you probably aren’t testing anyway.

So when someone asks the question above they need to be clear about what automation is and how it fits in with the skill set of a good tester.

Automation is not essential for a tester

Hear me out as I’m aware a lot of people might not agree with this. Automation is absolutely useful…it’s very useful in a number of areas, but it isn’t essential to be able to write automation to be a good tester.

Testing is learning.
Testing is investigating.
Testing is exploring.
Testing is communication.

You can’t automate any of these (there’s a slight caveat around AI with a couple of those but let’s put that aside for now) and you certainly can’t automate the tester’s mindset and way of thinking.

These are the skills that make someone a tester. If you are a good tester then you are a good tester because of who you are, not because of the skills you have. The skills help you apply the mind set you have easily and efficiently.

However…

Having said all that, I do think if someone has the desire to learn some technologies or tools that help them with automation then it’s worthwhile them doing that. Why? Well..

Automation is a useful skill

As a tester you will have a skill set that you can use. Many different skills make up this set and each one will give you something else to use when you are trying to evaluate the quality of a product. Automation is no different in this respect. I’m not going to list the uses and advantages of automation checks themselves as plenty of places do that but as an example the best automation checks provide the team with rapid high quality feedback on new features. Automation can also be very useful when it comes to continuous integration or deployment by increasing confidence in the changes being made.

Technical understanding aids diagnosis

The better that you can read and understand the code that makes up your product, the easiest it will be for you for diagnose and potentially even fix the issues you find. To me this is really the skill that every tester should be looking to acquire. Having the ability to be able to understand and investigate at all levels of the stack will greatly help you out with investigating an issue.

It’s fun!

This is clearly a personal thing, but the ability to use automation tools alongside deployment tools is the main way I get to use my coding skills at the same time as adding value to the team. I enjoy coding and always have and I think a lot of technically minded testers are similar in this respect – automation is a great outlet for this.

What a contradiction

Yeah…it does come across a bit like that. So what’s the point I’m trying to make?

You do not need to learn automation to be a great tester, but if you want to then it will give you extra skills to use in your career.

In part 2, I’ll give my tips for actually starting to learn automation.

Am I a tester?

Thu, 16 Nov 2017 01:00:27 GMT

“My name is Steven and I am a tester“

I was thinking of which subject to do for my first blog post and the above seemed a good way to start, even thought it might sound like a introduction at a self help group session.

It’s taken from a recent presentation I gave around the subject of what being a tester actually means in a modern software development industry. Using myself as a case study, I wanted to explore whether I can still actually call myself a tester.

When I first started in software development, I didn’t choose testing…it sort of chose me. Originally I wanted to be a developer. I know that I’m not the first person to say that and I won’t be the last. The role I took in the end was a Technical Tester role which pretty much involved running through some basic scripts and doing a bit of investigation in to the code and the backend. In all honesty I got pretty bored so I started looking at automation which I loved as I got to use my development skills.

As I progressed through more senior roles, I started leading teams and having an influence on the way the department and the company itself was run. However every time someone asks me what my job is, I would say that I’m a tester. For the last few years my day to day activities have either involved writing code, PO activities such as maintenance of the backlog and customer interaction or scrum master style duties such as running ceremonies or removing blockers for my teams.

So why do I still say I am a tester? To answer this I looked at different areas to see how I test in them.

Testing… Requirements

By participating and pushing 3 Amigos sessions and looking to push BDD, I’m looking to test out a user story and looking for the scenarios that might have been missed. I actively look to get myself involved in cross functionals and system design meetings where again I’m testing the design of the system itself – both from a functional and a non-functional point of view. During all of these events, I’m testing mainly by questioning…questions, questions, questions – a massive part of a tester’s toolkit.

Testing… Code

This is likely to be a more obvious one, but I’m testing both my own code and others’ code every day. Using techniques such as analysing code coverage, TDD, checking cyclomatic complexity and performing static testing I can ensure that I’m making quality checks on the code constantly. The ultimate aim for me when testing code is to provide the best feedback as rapidly as possible.

Testing… Processes

Processes can sometimes be a dirty word, but even the most lean team has processes of some kind – just not processes for the sake of it. When they exist, they must be tested to ensure they are as efficient and practical as possible. Lessons learnt sessions, sprint reviews and team retrospectives are examples of ways of continuously improving these processes and the way the team works and I always try to be an active member of all these sessions, contributing positively where I can.

Testing… People

To me this is all about mentoring and challenging people – if you are line managing, advocating or mentoring in any way then you should be looking to challenge those people to continuously improve. I always look to do this and to give people the confidence they need to venture outside their comfort zone.

Testing… Myself

Lastly I think the biggest thing I probably try and test is myself in similar ways to how I look to test others – by pushing myself to do things I’ve not done before and looking to continuously improve.

Commonality

In all the areas above, I discovered I was using similar skills to examine and hopefully improve the quality:

Analytical thinking -> The ability to see things from a logical point of view
Flexibility -> Accepting that things may change and having an ability to adapt to different situations
Communication -> Knowing how to communicate in the appropriate way for different situations and having the ability to do it effectively
Innovation -> Always trying to see the bigger picture and finding creative ways of looking at and doing things

“My name is Steven and I always will be a tester“