|
The first testing should occur in-house with
different operating systems and different hardware
configurations. An alpha version may not yet contain
all of the planned features, and may lack documentation.
Use virtual systems or test drives to fully test
software with various operating systems and patches.
Thorough alpha testing will make beta testing
run smoother.
1. Use people within your organization that are
unfamiliar with your software. If you don't have
anyone available in your organization consider
asking family and friends.
2. Before moving an alpha product, to a beta
product consider having an "ignorant" user do
a complete install. Silently watching the tester,
will give you a birdseye view into user habits
and potential problems during the install process.
Beta testing is considered the second phase
of software testing. Beta tests are typically
external tests to identify any performances issues
or bugs prior to an official release. Beta tests
can be open or closed. A closed beta test is used
to control the number of users participating.
An open test is open to anyone who has an interest
in beta testing.
Why Beta Testers are Important
Beta testers are important because it is almost
impossible for developers to test their software
in all of the various conditions that might occur.
Software should never be released without thorough
beta testing. It is impossible to predict or test
software on all kinds of hardware with other applications.
Some developers segment the closed beta into
different release stages so they can maximize
feedback. Historically the majority of feedback
is received from beta testers within the first
week of the beta release.
100 Beta Testers
Week 1 - Alert 1st Group of Testers (34 testers)
of beta availability. Bugs found and fixed.
Week 3 - Alert 1 & 2nd Group of Testers (33 testers)
of beta Bugs found and fixed
Week 5 - Alert 1, 2 & 3 Group of Testers (33 testers)
Solicit Testers
Use all communication channels to solicit
testers. Existing users are the best testers because
they are familiar with software and use specific
software features.
1. Offer Incentives
Lets face it its not easy to convince users to
try buggy software
a. free registered version for feedback
b. discount on registered version for registered
beta testers
c. payment for finding bugs
d. make beta testing a benefit
When recruiting beta testers request an email
address so that you can actively communicate with
them.
2. Layout Beta Test Expectations
Beta testers should have a good understanding
of their role as a beta tester. Educating beta
testers will go a long way to improving the quality
of feedback that you receive. Provide details
on the information they should provide any bug
reports.
3. Know What to Ask
If a user reports a problem, ask the right questions,
so that you can attempt to reproduce the problem.
Refer to Reproducing
Errors for additional details.
5. Dialogue is Important
You should be sure that you have an open channel
of communication with beta testers. Respond to
their queries in a timely fashion (24 hours) and
always thank them!
6. Forum
Create a forum for beta testers to post questions
and concerns. Post file updates to the beta forum.
Create a specific section of the website that
contains beta information.
7. Use Exit Surveys in Beta
Use exit surveys in the beta version. This will
not only give you statistical information about
uninstalls, but will also provide details about
why users uninstalled.
8. Weak Documentation
Be sure to mention any holes in the documentation,
and where the testers can get any additional information
that they might require.
9. Provide Upgrade Details
Be sure to provide details for users that are
using an existing version of the software and
looking to test the beta version. Double check
prior to releasing the beta version that the user's
data will remain intact through the upgrade. Also,
check to make sure they have an adequate key or
code for the beta version so that it does not
expire.
10. Make Beta Versions Backwards Compatible
Can users go back to their old version if they
do not like the new version? In other words will
their registration key work in both the new and
old version? Is there a drop dead date, which
will prevent testers from using the software for
an extended period? Be upfront about these issues
so beta testers know what they are getting into.
11. Don't Rush It
It takes time to conduct a thorough beta test,
don't rush it.
Related Articles:
How
to Recreate Customer Problems
Submit Beta
Versions
Beta Testing Tips
|
Developer
101
How to Conduct Effective Beta Testing