11 May, 2020

The challenge of beta programmes


In a previous post, I looked at the Product Management & Product Marketing model that Redgate Software (a local Cambridge-based company) uses. In it, I identified a component, that didn't appear to be addressed: beta programmes. 
 
A quick diversion into pronunciation
  • In the UK, it is pronounced 'Bee-ta'.
  • In the US, it is pronounced 'Bay-ta'
It's a pronunciation that I found far, far difficult to surmount when I moved to the US and when I moved back. 'Tom-mart-to' and 'Tom-may-to' was easy!

What is the purpose of a beta programme?

The build team (Engineering, Testing, Product Management, Implementation, Technical Support) want the beta programme to technically validate the latest functionality - that it works outside the lab, in real customers hands and that bugs are flushed out and those bugs are validated in subsequent releases. The end goal of a solid product should not be compromised by time constraints.
  • Marketing and Product Marketing want a case study and a quote for the press release - as soon as possible please.
  • Sales, Sales Engineering, the CFO wants a big name brand to do reference case study that ignites the sales pipeline - ASAP as well.
Hmm, plenty of different objectives!


Why should a customer participate in a beta programme?

I'm sure that we have all participated in these CBT (Customer-Based Testing) programmes at some point: buggy software whereby the customer does the testing - seemingly to the shocked surprise of the vendor. This can be immensely damaging to the beta programme, the provider’s brand, the relationship between participant and the provider.

And, for the customer, it's a darned good reason not to participate in a beta programme: too much effort for no advantage (or indeed negative advantage).

For this reason, providers need to ensure that, during the beta, they have the Engineering bandwidth to rapidly fix issues that are identified. This, of course, is difficult to estimate exactly how much resource is required, but essential that this remains top priority. Unfortunately, Engineering’s core capability of building beautiful, well thought-out functionality does not match the issue + investigation + quick response cycle of beta programmes – expertise that is embodied in Support or Professional Services.

Valid reasons for customers to participate in a beta programme are:
  • Access to latest version - see later for my experience with CDK Global
  • High-profile within the vendor's organisation: ie access to Technical resources or insight into the vendor's strategic intentions.
  • Cost-reduction in fees, perhaps for a limited period


Which business functions should participate in Beta Programmes?

There are many good and bad reasons why different parts of the organisation should participate in beta programmes.



Customers
Err, yeah, of course, but some customers have a different propensity towards beta programs.

Customer whales (ie big ticket customers) that that are currently in pre-sales should definitely not be part of a beta. It may destroy the sale altogether: experience with buggy software may leave a nasty impression in the mouth - even if it is exactly the functionality that they are begging for.

Getting a quote out of a whale's marketing department may take months, if at all.

Beta software is usually provided at a discount - possibly a significant discount - a discount which may never be recovered on full release and therefore full price - and too expensive a margin to surrender.


Existing customers are great for betas - they understand what the previous product does, users and the business buyer have a positive relationship with the beta provider. Technically and financially, the provider has been validated and approved. This certainly makes it easy to draw up a shortlist.

However, there may be enough 'baggage' in the account that existing customers may derail the beta at the outset. They may demand so-called 'fixes' (missing pieces of functionality that they perceive in the existing product) or their environment / use may not match the requirements of the beta functionality. They will want compensation for their participation - and this compensation may be more valuable for them then the business benefits of the beta software.

Some of the best beta programmes I have participated in have been with new business divisions of existing customers. Commercially an existing contract is easy to extend. The vendor sees new business and the customer has the assurance of existing proof elsewhere in their organisation.


New Customers
New, NEW customers can also be really good. They genuinely behave like new customers.
They generally have more operational flexibility - they haven't honed their own processes to your product yet - and recognise that they need to create new processes.

Existing customers may say, "Yes, we'll be on your beta - as long as the product is "better" and nothing changes." Well, if the beta product aims to cannibalise the existing product, then the beta product will be "different" - better in many respects, but different in others. These differences may be negative enough to existing customers that they outweigh the new advantages.

The issue may be become more apparent if the existing customer is now off-centre from the target market with the new product. Existing customers may feel that they are being shoved to one side by the relentless progress of the vendor - and this may give the existing customer on the beta programme a sour taste in the mouth.

Another advantage of new customers is that the beta programme doesn't have to tiptoe around Account Management or Sales and get their buy-in.


Testing
Testing needs to valid the software on its way out and also to validate any issue as reported. For this reason, it is often essential that Testing has access to an environment that has beta code on it - let's call it pre-production - they are interested in isolating the real-world issues in a lab environment. Messing around with the production server is not to be recommended as it may create serious degradation to the service.


Engineering
Well, yes, they created the software and they are responsible for delivering it and fixing it. They are critically interested in technical validation in real-world environmental.


Account Management
If an existing customer is involved, the participation of Account Management (or Sales or Professional Services or whatever function manages existing relationships) is undoubtedly required, particularly at the start and end of the beta programme.

This function is compensated with incremental revenue - quite rightly, they don't want to jeopardise their account with the unknown. So, account management need reassurance that their customers' participation in the beta will help their account.

If the beta programme runs in parallel with existing product, then Account Management and Customer Support continue to support the existing in-life product. If the beta programme cannibalises the existing product, then the beta programme manages and supports the customer.


Customer Support
This function is generally wary of beta programmes, as it is often increases the amount of first-line call handling that they are required to do.


Implementation / Professional Services Team
The participation of ProServe is dependent on the amount of customisation required for the beta customers. Development would like 'vanilla' customers with the least customisation. However, this is dependent on the interest of the existing customer base in the beta programme and, with gritted teeth, the beta programme may be obliged to accept a highly customised account into the beta.

The great hope of vendors is that the new version fixes lots of issues in existing accounts - helping Account Management / Sales (unlocking new revenue), Professional Services (less implementation grief) and Customer Support (fewer issues and better relationship). And yes, it should leapfrog the competition and trample all others in the marketplace.


Special Cases - Saturated Markets
Newly all of my product experience has been in small scale-ups. However, I did have one fascinating experience working for CDK Global (see case study). As the Release Director, I was also involved with the Beta Programme - admittedly at arm's length.

CDK Global's product was the first major release in 12 years. It is worth mentioning because CDK Global dominated their market with 60%+ Total Ownership. Their customers (car dealers) operated in a highly competitive market and any new advantage could significantly tip the market. So, getting on CDK Global's beta programme could be a game changer in the short- and mid-term. When a slot on the beta programme was offered to a shortlist of existing customers, each place was eagerly snapped up.

Conclusion

Beta Programmes are extremely important to software creation teams, but managing these diverse expectations can be equally difficult - particularly when the product launch (with all those expectations!) is dependent on it.

No comments: