Did your fan marketing actually work?

A number going up is not the same as it worked

You ran a campaign. Streams went up 40% that week. Did it work?

Almost everyone says yes. Here is why that is not safe.

That week, other things were also true. If it was release week, release weeks rise on their own; that is what a release week is. You might have picked up a playlist. A video might have run further than usual. Someone with a following might have posted. It was a different day of the week from whatever you are comparing against.

Every one of those moves streams. None of them is your campaign. And nothing in any dashboard separates them, because a dashboard reports what happened, and the question is what would have happened otherwise. That is a question about a version of last week that did not occur.

This is structural, not a gap in your particular setup. The trade press has said so plainly: when someone taps an ad and lands on a streaming service, the trail goes cold at the boundary, and the service does not report back which source produced which listen (Music Ally, 2022).

Your metrics measure activity, not impact

Most metrics answer a narrower question than the one people use them for.

What you can see	What it honestly tells you	What people read into it
Email opens	A mail client loaded an image	That they read it, or cared
Link clicks	Someone was curious enough to tap	That they listened. Tapping a link and playing a song are different acts
Total streams this week	The overall number moved	That your campaign moved it
Monthly listeners	A rolling 28-day count changed	Which individual people changed, or when. It is an average, so by the time it moves the behavior behind it is weeks old
Pre-saves	People intended to listen, once, at signup	That they listened on release day
Follower count	Somebody pressed follow at some point	Anything about current listening

None of these are useless. They are real, and the first two are genuinely good at telling you whether your subject line and your artwork are doing their job.

They just cannot answer "did it work", because they all measure activity, and the thing you want is impact.

A holdout tells you what your campaign actually caused

There is a method for this. It is old, it is not complicated, and every serious consumer business uses it.

Leave some people out on purpose.

Before you send anything, take a randomly chosen slice of your fans, say 10% or 20%, and deliberately do not contact them. Afterwards, compare the two groups.

• If the fans you messaged listened noticeably more than the fans you skipped, the campaign caused something, and now you know roughly how much.
• If both groups behaved about the same, it did not. The streams came from somewhere else, and finding that out now is much better than building three more campaigns on the assumption.

That withheld group is a holdout. Its entire job is to be the version of events where you did nothing.

Two rules make or break it.

It has to be random. Not the least interested fans, not the ones whose phone numbers you are missing. If the group you skip differs from the group you contacted in any other way, you are measuring that difference instead of your campaign. Random selection is what makes the two groups otherwise identical, and that is the only reason the comparison means anything.

You have to decide before you send. Choosing the comparison afterwards, once you have seen the numbers, means you will choose, without meaning to, the one that flatters the result. Everybody does this. Deciding up front is the entire fix.

"But I would be leaving streams on the table"

This is the real objection and it deserves a straight answer rather than reassurance.

Yes. If your campaign works, the fans you held back will listen less than the ones you contacted, and you gave up those streams to learn something.

Whether that is worth it depends on how many more campaigns you plan to run. For a genuine one-off, probably not. For anyone running campaigns repeatedly, holding back a tenth of your audience once buys a real answer about which of your approaches does anything, and that answer improves every campaign afterwards. You pay the cost once. The information keeps working.

There is a floor on the cost, too. The holdout only needs to be big enough to see a difference through the noise. A few dozen people tells you very little, because ordinary week-to-week variation will swamp any real effect. A few hundred starts to be informative. If your list is genuinely small, this is a technique to grow into rather than force.

The tools are good at sending and weak at knowing

Worth knowing, because it explains why nobody has offered you this before.

The streaming services give you audience targeting built on behavior. Spotify, for instance, lets artists aim a campaign at previously active listeners, people who used to listen and have not in the last 28 days (Spotify for Artists). Services like Bandsintown offer retargeting on similar logic.

That is useful, and it is the reaching half. What none of it gives you is the measuring half: which individual fans acted afterwards, tied to which message, compared against a group you held back.

So the honest state of things is that the tools available to an independent artist are good at sending and weak at knowing. That is not a small gap. It is the difference between a campaign that teaches you something and one that merely happens, and it is the reason most activation loops stay open for almost everybody.

Try this on Monday

You can run a real holdout on your next campaign, for free, with a spreadsheet.

1. Write your prediction down first. "I think 25% of the people I message will listen in week one." This is what stops you grading your own homework later.

2. Split your contacts randomly. Add a column with =RAND(), sort by it, take the bottom 15%. That is your holdout. Do not look at who is in it and do not adjust it. The randomness is the whole point.

3. Send to the other 85%. Nothing else changes.

4. Wait a full week. Not seventy-two hours. Early on, campaign effects and ordinary release noise are tangled together, and the useful signal appears once the initial spike has passed.

5. Compare what you can actually observe. For most artists that means replies, link taps, ticket or merch purchases, comments. Anything you can tie to a named person. The gap between the two groups is your answer.

6. Write down one line: what you predicted, what happened. Four campaigns of this and you have a record of what genuinely moves your fans, which is worth more than any general advice, this page included.

Doing this every time is where it breaks

The spreadsheet version works, and if you take one thing from this page, take that.

Two things are genuinely hard without help. The first is the piece everyone wants and almost nobody has: knowing which individual fans listened, rather than which ones tapped a link. A tap is a proxy and a loose one. Plenty of people tap and never press play, and plenty listen without ever touching your link.

The second is that this only pays off if you do it every time. One holdout is a data point. Ten holdouts across different kinds of campaign is the thing that actually tells you how to run your fanbase, and doing that consistently, by hand, for years, is the part that quietly stops happening.