One of the biggest questions when building an enterprise software solution is whether that solution is ready to go into production.

• When is enough testing enough?
• When have all the risks been mitigated?
• When are you ready?

Diffusion is a powerful ingredient software, but it is only one part of a larger solution that can contain clients, publishers, load balancers, adapters, web servers, and any other moving pieces that you need to create a seamless experience for your customers.

To put it another way: Diffusion is the engine; the rest of your solution, is the car. It doesn’t matter how well tested the engine is and how highly it performs if you don’t know that the wheels can handle top speeds without coming off.

The truth is, you can never be 100% sure that your solution is ready. There is not enough time in the universe to test every single possible variation of circumstance and usage that might occur in your system. You also don’t want to spend so much time testing your solution that you miss your moment.

In the end it boils down to risk and confidence. Take the steps you need to reduce your risk and your unknowns until you are confident that your Diffusion solution is ready for the open road.

Your environment

Often weaknesses in a system occur at intersection points. It is important to test not just the components in your solution individually (though this is important), but the whole solution together, just as it would be deployed in production.

Sometimes the smallest change to configuration or the removal of a single component can have an unexpectedly large effect. To hark back to our metaphor: a windscreen may not seem crucial to a car’s job of getting from A to B, but imagine how much being without one changes the aerodynamics when you’re putting that car through its paces (and no driver enjoys flies in his teeth!).

Your usage

Knowing just how your Diffusion solution is going to be used is half the battle. Are all your customers going to hit your site at the same time in response to a big event, then disconnect 10 minutes later? Or are they going to connect at varying times, but stay connected for the long haul?

Being ready to take on the Nürburgring and being ready to race in the World Rally Championships are two different things.

Functional testing

It should go without saying that your solution should work, it should do the task required of it and get your data to your customers.

Functional testing is checking that the engine turns the wheels, the brakes work, and the clutch doesn’t stick.

That’s the easy bit. Now for the fun stuff.

Performance testing

You know your solution does the job, but how fast does it do the job? How well? How efficiently? Are connections or messages backing up?

Your Diffusion servers provide many ways to configure behavior to get the absolute most out of their performance. Buffer sizes, queue sizes, threading, and multiplexing: all of these can be tweaked and tuned until Diffusion’s performance is optimized for your usage.

Load testing

You might have a pretty good idea of the average usage that your solution will see, but what about the extremes? What about when every one of your customers hits your app at the same time? What about those big events that cause the data to flow faster than ever before? How will your solution handle extreme conditions? Will you need to add more Diffusion servers? Do you have to tune the configuration of any components? Can your network and middle-boxes handle the load?

Answering these questions in your pre-production testing will give you the confidence that when your usage spikes suddenly, your app performance won’t be a victim of your popularity.

Every speedometer shows a red line describing a limit that was discovered by testing it until it failed. Know where your red line is.

Soak testing

You’ve looked at performance under average and heavy usage over short amounts of time, but what about the long haul? How does your solution behave if you leave it running for 24 hours? 48 hours?  What resources start to get used up?  How about garbage collection behavior? Do you fall foul of any hidden timeouts in your configuration?

During soak testing is often when resource leaks (for example, file descriptors and sometimes memory) can become apparent.

Penetration testing

Security is important. So much so that it should be baked in to your solution design from the start. The only way of connecting to your production solution should be on those URLs and ports that you explicitly allow. And the data and actions that can be accessed through that connection should be strictly controlled.

Penetration testing comes at this problem from the other end. Instead of thinking about how to prevent insecurity, it is all about how to circumvent security. There are many great companies that spend all their time working in this mindset and are experts at finding the weak points in the security of complex solutions.

Penetration testing is an area where it is often best to bring in a third party to do your testing. I’m a firm believer that this is an area where it’s hard to see your own blindspots.

Planning

You’ve built a solid solution. You’ve tested and tested again. You know your solution’s limits. Are you ready?

Your solution is ready all you need now is a plan:

• When are you going to roll out to production?
• Who do you need to notify in advance?
• What components are you rolling out or updating in production?
• How are you going to roll them out and in what order?
• What is your roll back plan if something unforeseen happens?

These are all questions to consider when creating your road map to production.

You’ve put your Diffusion solution through its paces, planned your route to a successful deployment, and now you’re ready for the open road.

The advice in this blog post is not an exhaustive list of steps to take when getting ready to take your Diffusion solution into production. You might have additional testing requirements based on your solution. Push Technology provides Consulting Services that can work with you to advise on a pre-production testing strategy specific to your requirements.
Image Courtesy