A/B Testing Your Author Website

A/B testing — also called split testing — is the practice of showing two different versions of a page element to different visitors and measuring which version performs better. Version A might have a headline that reads 'Get the Free Prequel Novella.' Version B might read 'Meet the Characters Before the Series Begins — Free.' Half your visitors see Version A. Half see Version B. After enough visitors and enough conversions, you know which headline converts better for your specific audience.

Most authors who think about improving their website conversion rate rely on intuition: this headline feels better, this button color looks more clickable, this form placement seems more prominent. A/B testing replaces intuition with evidence. The results are often counterintuitive — the headline you were sure would win frequently doesn't. The button color that seems minor turns out to matter. What your readers respond to and what you expect them to respond to are related but not identical.

This guide covers what's worth testing on an author website, how to set up a valid test, which tools work without a developer, and how to read results correctly.

What's Worth Testing on an Author Website

Not everything on your author website has enough traffic to test meaningfully, and not everything that can be tested is worth testing. Prioritize elements that combine high traffic volume with direct connection to a conversion goal.

High Value — Test These First

• Email capture headline: the single line of text that is your opt-in offer's primary claim. 'Get the Free Prequel Novella' vs. 'Read the Story That Started Everything — Free' — small wording changes here have outsized conversion impact

• Primary call-to-action button text: 'Sign Up' vs. 'Get My Free Book' vs. 'Download Now' — these are all the same action described differently and they convert at meaningfully different rates

• Opt-in form placement on the Home page: above the book description vs. below it; inline vs. popup

• Book page buy button text and placement: 'Buy Now' vs. 'Choose Your Store' vs. 'Get the Book'; button at top of page vs. after the description

• Reader magnet offer: testing two different reader magnets to see which converts your specific traffic better

Moderate Value — Test When High-Priority Tests Are Complete

• Book page description opening paragraph: the first 50 words that determine whether the reader keeps reading

• Popup timing: exit-intent vs. 45-second time delay vs. 60% scroll depth

• Social proof placement on book pages: reviews above the buy button vs. below vs. alongside

• Home page hero section layout: book cover left + text right vs. full-bleed cover with text overlay

Not Worth Testing — Skip These

• Elements on low-traffic pages — you need sufficient visitor volume to reach statistical significance; pages with fewer than 500 monthly visitors can take six months to produce reliable results

• Multiple elements simultaneously — changing the headline and the button color and the form placement at the same time makes it impossible to know which change caused any difference in results

• Cosmetic changes with no connection to conversion — font size, background patterns, decorative image choices — unless you have specific reason to believe they affect conversion

What Makes a Valid A/B Test

Most informal author website tests produce misleading results because they violate one or more conditions required for statistical validity. Understanding these conditions before running your first test saves you from making changes based on noise rather than signal.

Sufficient Sample Size

A test with 50 visitors in each variation tells you almost nothing reliable. Random variation at small sample sizes easily produces results that look significant but aren't. As a rough minimum for author website tests: 200 conversions total (across both variations) before drawing conclusions. On a page converting at 3%, that requires approximately 6,700 visitors — which at 500 monthly visitors means more than a year. This is the primary practical limitation of A/B testing for author websites with modest traffic.

Lower traffic sites should focus on the highest-traffic pages and be patient — a test that takes three months to reach significance is still worth running. The alternative is making changes based on intuition that may actively hurt conversion.

Statistical Significance

Statistical significance is the measure of confidence that a test result is real rather than random chance. The conventional threshold is 95% confidence — meaning if you ran the same test 100 times, 95 of those runs would show the same winner. Good A/B testing tools calculate this automatically. Do not end a test based on looking at the raw numbers; wait for the tool to indicate the result has reached statistical significance.

One Variable at a Time

Change exactly one element per test. If you change the headline and the button color simultaneously and conversion improves, you do not know which change drove the improvement — or whether they helped together but would have hurt separately. One variable per test, run to significance, then iterate.

Test Duration

Run every test for a minimum of two weeks regardless of traffic, because visitor behavior varies by day of week and time of day. A test run only on weekdays may capture a different reader demographic than a test run across full weeks. Two weeks minimum ensures you capture at least two full weekly visitor cycles.

A/B Testing Tools for Author Websites

WordPress: Thrive Optimize

Thrive Optimize (~$99/year as part of the Thrive Suite) is the most author-accessible A/B testing tool for WordPress. It integrates with Thrive Architect (Thrive's page builder) and allows you to create test variations visually — no code required — and runs statistical analysis automatically. It shows you when a winner has been identified at your specified confidence threshold. For WordPress authors who want to test landing pages, opt-in forms, and book pages, Thrive Optimize is the clearest path to A/B testing without developer involvement.

WordPress: Nelio A/B Testing

Nelio A/B Testing (free and premium versions) integrates with any WordPress theme and page builder and supports testing of pages, posts, headlines, and widget areas. The free version allows one active test at a time, which is sufficient for most author testing needs. Premium adds unlimited concurrent tests and more granular statistics.

All Platforms: Google Optimize Alternative — VWO or Convert

Google Optimize was sunset in 2023. VWO (Visual Website Optimizer) and Convert.com are the most capable replacements, both supporting visual editing of test variants without coding. Both offer free trials and paid plans starting at approximately $200/month — more than most author websites justify. For authors on Squarespace, Wix, or Shopify who want A/B testing, these are the options; the cost makes them appropriate only for authors with significant direct sales revenue.

Free Alternative: Manual Testing with ScribeCount Analytics

For authors without budget for an A/B testing tool, a disciplined before-and-after measurement approach using ScribeCount's Website Traffic provides useful directional data. Change one element, document the date of the change, run for 30 days, and compare conversion rates from the 30 days before and 30 days after. This is not as statistically rigorous as true A/B testing (seasonal variation, traffic mix changes, and other factors can affect results), but it is better than changing things and never measuring the impact.

Reading Your Test Results

When your A/B testing tool reports a winner, look at these metrics before declaring a change permanent:

• Statistical confidence: the tool should report 95%+ confidence — below that, the result is not reliable

• Absolute conversion rate improvement: a 0.5% improvement (from 2.0% to 2.5%) is meaningful; a 0.1% improvement may be noise even at 95% confidence

• Sample size: verify the test reached your minimum threshold before the tool called a winner — some tools have aggressive winning declarations at low sample sizes

• Segment by traffic source: a change that improves conversion for email traffic but hurts conversion for organic search traffic is not a clear winner — it is a change that helps one audience and hurts another

Connecting A/B Tests to ScribeCount Analytics

Configure the conversion event you are testing — email signups, buy button clicks — as a tracked event in ScribeCount's Website Traffic. When your A/B test shows Version B improving email conversion by 28%, you want to see whether that conversion improvement translates to more email subscribers who actually buy books, or simply more subscribers who download the reader magnet and disengage.

ScribeCount's analytics connecting website conversions to downstream sales data is the measurement layer that makes A/B testing results meaningful beyond the immediate conversion metric. A reader magnet offer that converts more downloads but attracts readers who never buy books is less valuable than one that converts fewer downloads but attracts engaged buyers. ScribeCount's data over time shows you the quality of the subscribers different offers attract.

A/B Testing Quick Reference

Intuition builds websites. Testing improves them. The authors who consistently have the highest-converting websites are not the ones with the best design instincts — they are the ones who test, measure, learn, and iterate. Start with one test on your most important conversion element, run it to significance, implement the winner, and run the next test. Each improvement compounds into the next, and over a year of consistent testing, your author website's conversion rate reflects evidence rather than assumption.

-Randall Wood