You need a loan to open your own restaurant chain. The bank asks for your business plan.

You want to build a new product or launch a new venture. Management asks for the business case and opportunity vs cost.

You want to become a solopreneur. But before you leave your job, you need to know: can I make a sustainable living at this?

All three situations are asking the same question:

Is this idea worth the investment?

A business plan helps you answer this and is an effective way to communicate your vision for something that you want to build. It paints a picture of the problem and proposed solution, the opportunity, and what it will take to make the idea a reality. It’s purpose is to secure resources like money, people, approval, or buy-in.

Most business plans will include:

• Market trends and opportunity

• The problem and proposed product or service

• Competitors and how this product is different

• How the product or service will reach customers

• Resources, partners, and suppliers needed

• Pricing and expected revenue

• Expected costs

A business plan combines both story and data. The story explains why the problem is worth solving, why this idea is different, and why customers might choose it. The data outlines: the size of the opportunity, expected revenue, and expected costs.

No matter how compelling the story is, the investment decision ultimately comes down to these numbers. And each number depends on several assumptions that deserve close examination - whether you’re writing the business plan or evaluating someone else’s.

Share We Dig Data

Let’s dive into each key number and the common assumptions to focus on.

#1 | How big is this opportunity?

The first number in a business plan sizes the potential market for this proposed product or service. This will feed the revenue estimate later, so it needs to be credible.

Base your estimate on credible external data. You may need to combine several sources and make reasonable assumptions, but document where the numbers came from and why you trust them.

Investors don’t expect certainty; they expect a well-supported estimate of the opportunity.

Look for information on:

• Market size, market growth rates, and forces driving that growth

• Market structure, including the number and type of potential customers

• Competitors and current market share, when available

Pressure-test the assumptions:

• Does the market definition match the geography and customer segment(s)?

• Is the data recent enough for a fast-changing industry?

• Are growth estimates based on an established trend?

• Does the analysis include substitute products, not just direct competitors?

• How credible are the sources for the market sizing estimate?

  Subscribe now

#2 | What is the expected revenue?

The potential revenue, how it builds over time, and the underlying assumptions will get much deserved scrutiny from potential investors.

At its simplest, the revenue forecast depends on three things:

• The average price customers will actually pay

• The number of customers you expect to win

• How both change over time

Start with pricing: Use the average amount you realistically expect to collect, accounting for promotions, discounts, pilot pricing, or special terms.

Then estimate how many customers you can realistically win and how quickly. This is where many business plans get overly optimistic. A common shortcut is: “If we capture just 1% of the market…”

Be careful!

Even 1% of a large market can mean winning hundreds of customers, often from established competitors. Your forecast needs to reflect how long that will take and what it will require.

How much will customers pay, and how many customers can you realistically win? And by when?

Build your estimate from the bottom up. How will you win your first 10 customers? Your first 100? What changes as you scale? Use customer interviews, pilot results, comparable businesses, or sales capacity to support those assumptions. Then test a more conservative scenario.

Every assumption should be visible to the reader. Document it and include any sources you used.

Look for evidence that your customer estimates are realistic. This could come from customer interviews, early sales or pilot results, comparable businesses or competitive channels, or the capacity of your sales and marketing channels.

Pressure-test your assumptions:

• Pricing - Will customers pay what you expect, including any discounts and promotions?

• Customer growth - How many customers can you realistically win, and how quickly?

• Sale - How long does it take to go from initial customer interest to a sale?

• Retention - How many customers will stay, and will their spending change over time?

• Evidence - What data or experience supports these assumptions?

#3 | Expected costs

Revenue gets a lot of attention, but costs determine whether the idea can survive.

Costs are easiest to evaluate when you think of them in buckets:

• Startup costs: one-time expenses required to get your business up and running such as legal setup, equipment, initial inventory, or website development.

• Operating expenses: ongoing costs of running the business, such as rent, software, insurance, marketing and administrative labor.

• Cost of goods sold: direct costs of producing the product or delivering the service, such as materials, manufacturing, and labor related to production.

If you are an entrepreneur, do not forget to include your own time.

And unless you have a strong financial background, it is worth talking to an accountant to help build the model or at least review the cost categories. A small investment in advice can prevent a much larger mistake.

Pressure-test your assumptions:

• Which costs rise with sales, and which stay fixed?

• What new costs or investments will be required as the business grows?

• Does the business still work if costs are higher than expected?

Using AI: The point is the model

A business plan is not just a document you use to convince someone else to give you money, people, or support. Before it does any of that, it is a tool for you.

The model helps you work through the ranges and assumptions behind the opportunity. Is the potential audience large enough? Is the pricing strong enough? How quickly could revenue grow? How much will it cost to deliver and scale?

A good model will help you understand what must be true for the business or product to be viable.

Where AI fits and where it doesn’t

Use AI to research industries, competitors, customer pain points, production methods, and potential costs. But don’t let it build the model for you. Creating the assumptions is part of the thinking. Once you have a model, AI is valuable for challenging assumptions, identifying missing risks or costs, and suggesting questions an investor might ask.

Subscribe now

Bottom Line

A business plan is one of the best ways to pressure-test an idea before investing significant time, money, and energy.

Building the supporting model forces you to think through what must be true for the business to succeed - and often improves the idea in the process.

Next up in this series on using data to estimate the future: strategic plans.

And in case you missed earlier installments,

• Introduction: Telling the Future with Data

• The Opinions Behind Forecasts

• Business Plans

• Coming Soon: Strategic Plans

• Coming Soon: Budgets

Share We Dig Data

More reading: Critically evaluate and communicate with data

AI writes emails. It drafts LinkedIn posts. It summarizes meetings. (And yes, it loves an em dash.)

That’s useful. But what I’m much more interested in is what happens when you give AI your data.

Not a polished dashboard or a carefully curated output. Just a spreadsheet.

Can a free AI tool help with a small-to-medium data-cleaning job - the kind of spreadsheet that’s too tedious to fix by hand, but doesn’t justify learning a specialized tool?

That might mean cleaning a few hundred customer contacts before importing them into a CRM, combining product lists from several suppliers, consolidating data from marketing systems, or bringing together transactions from multiple credit cards.

These are everyday data tasks. You should not need to be a programmer or data scientist to tackle them. And they seem like a reasonable place to ask AI for help: the file is manageable, the problems are visible, and much of the work is repetitive.

So I gave the same small contact dataset and the same two prompts to the free versions of ChatGPT, Gemini and Claude. The goal was not to rank the tools permanently. It was to see how each handled this kind of practical cleanup task today, where they made different choices, and how much review their work still required.

A quick note about privacy

Before we get started, a public service announcement:

Don’t upload personal information or confidential company data unless you understand - and are comfortable with - the privacy policies of the AI tool you’re using. If you’re working with company or client data, make sure those policies align with your organization’s requirements.

The main dataset we’ll use in these experiments is fictional. “Synthetic” or anonymized data is perfect for learning. Names, email addresses and company names were AI-generated specifically so we could experiment without risking anyone’s data.

Clean a messy contact list

Most of us have seen a contact spreadsheet like this before. Customer contacts, leads, donor lists, personal contacts housed in Gmail. Regardless of the purpose, they have the same issues.

The same cleanup problems appear in plenty of other small and medium-sized files: product lists from different suppliers, invoices or expenses exported from several systems, shopping or credit-card data from multiple accounts, and customer, membership or donor records collected over time. The columns may differ, but the work is similar: standardize formats, find missing or invalid values, identify likely duplicates and decide which differences matter.

In this contact list, some names are in ALL CAPS. Others are lowercase. A few have no spaces at all. Dates are in several different formats. Some company names are missing. Several email addresses use “at” instead of @. Perhaps you also want to split the full name into first names and surname, and find and resolve duplicate contacts

None of these issues is especially difficult to fix. They’re just tedious. So let’s see how we might leverage AI to clean it for us.

I was testing a fairly specific use case: a one-off file that is large enough to make manual cleanup annoying, but still small enough that a person can review the results. I would approach a recurring process, a very large dataset or a business-critical database differently.

The tools and prompts tested

I used the free versions of ChatGPT, Gemini and Claude. If the free tool offered a choice of models, I used the default offering (for example, Claude Sonnet 5, medium effort).

Each tool was tested twice, using the same prompts:

Prompt 1

Clean this spreadsheet.

Prompt 2

I want to clean this customer contact list, but I do not want to lose information. Please proceed in the following way:

• Standardize names, company names, and dates.

• If you aren’t sure about a date, leave it exactly as it is instead of changing or deleting it.

• Flag likely duplicate contacts, but don’t merge or delete them.

• Flag email addresses that look incorrect, but don’t guess at the correct address.

• Tell me what you changed and anything you think I should review.

I was not trying to engineer the perfect prompt for each model. I wanted to test something closer to how an ordinary user might work: start with a simple request, review the result and then add reasonable instructions.

Note: we selected free versions of the different tools because an average user does not necessarily have access to paid versions that might have more extensive features or capabilities. Even if you do, sometimes you might want to try out a different tool.

First up: ChatGPT

I started with the intentionally vague prompt: clean this spreadsheet.

The results were...underwhelming to poor.

ChatGPT normalized name formatting, converting all lower or all uppercase to title case. But a name like “AlbertHowell” became “Alberthowell,” which isn’t actually correct.

New problems were introduced. Many dates disappeared entirely after ChatGPT failed to correctly understand and standardize their formatting. And while it reported removing exact duplicates, I still had to verify what it considered a duplicate before trusting the output.

Can better prompting fix this?

Often you get better results from AI when you improve your instructions. Better prompts usually include information on what you are trying to accomplish, what shouldn’t change, and how you want the response.

More specific prompting, many of the same problems

So I moved on to the second prompt described earlier - not perfect, but with more explicit instructions on what to find and correct or flag for review.

This time, the AI returned an Excel spreadsheet and described the changes it had made. But when I compared the output to the original file? I found missing dates, incorrect name formatting, and placeholder values inserted where blank cells had been.

For example, look at all those values that say “Nan”- wondering what that’s about? NaN means “not a number,” (humorously, here ChatGPT made it title case as “Nan”) and reflects how ChatGPT is programmatically treating empty cells. You can avoid it by explicitly instructing the AI to leave any missing or empty cells completely blank. Or instruct it not to write ‘NaN’, ‘N/A’, or ‘null’ in the empty cells. But would most people think to include that in their prompt?

In the response, ChatGPT also claimed to have flagged problematic email addresses, but that markup wasn’t actually anywhere in the output file.

The disconnect between the feedback in the chat and what the spreadsheet looked like was one of the more interesting results from the experiment. Better instructions actually improved the conversation with the AI much more than they improved the spreadsheet itself. The model explained its approach more clearly, but it still struggled with several of the same cleanup tasks.

Now for Gemini (free)

Same file, same prompt: Clean this spreadsheet

Gemini approached the task differently. It was slower, but the resulting spreadsheet required noticeably less cleanup. A few examples of how Gemini cleaned the file in the absence of explicit instruction:

• Identified likely malformed email addresses and corrected.
  For example, “someone at company.com” became someone@company.com

• Removed likely duplicates. For example, B Hall and Brianna Hall had similar, but not the same email address (brianna_hall@smith.com vs. brianna.hall@smith.com). Gemini removed the row for B Hall.

• Corrected case and standardized date format. Gemini also caught the impossible date of 2/31/2026 - and deleted it.

• Filled in blank company names based on email domain.

Sounds good, right? Except Gemini made choices without telling me what it did. For example, what if it deleted the wrong record for B Hall vs. Brianna Hall when removing duplicates? How would I know? Filling in company names seems helpful, but is error-prone.

So just as I did in ChatGPT, I ran Prompt 2.

This time, Gemini helpfully gave me a script to reuse for future cleaning (though I didn’t ask for it), but not the actual file - until I asked for it. Alongside the file, it clearly summarized what cleaning tasks were done, and provided a list of items to review including potential duplicates (and how those were determined), incorrect emails and invalid dates. It did not, however, call out that company names had been filled based on email address or flag these changes in the file.

Finally, Claude (Sonnet 5, Medium effort)

Claude was showing off, I think. Just based on my “clean up this spreadsheet” instruction, I received a list of what had been modified, and left alone, along with the file:

But Claude also made assumptions about which duplicate contact I should keep, and while it told me what it kept, I would have preferred it asked me or flagged these for me to decide.

When I gave Claude the more specific prompt, it took even longer to return a result. But I received an extensive review, which turned up more nuance in potential duplicates and grouping the likely duplicates together.

This particular experiment was also the only one that flagged uncertainty with some dates; ChatGPT and Gemini both always assumed that a date like “10/01/2025” was October 1, 2025. However, Claude declined to standardize it, as it could not determine if the date was October 1, or possibly January 10.

This ended up being one of my favorite differences between the tools. ChatGPT and Gemini assumed what an ambiguous date meant. Claude refused to guess.

So what did this experiment teach me?

I went in skeptical about using general-purpose AI tools for data cleaning. Years of working with tools designed for this job are not cast aside easily. Still, AI capabilities have come a long way, and a small, everyday spreadsheet cleaning seemed like a reasonable test.

I assumed the biggest improvement would come from writing a better prompt.

Instead, the larger difference often came from switching tools.

That does not mean one model is universally better than another. In this experiment, the tools made different assumptions, prioritized different kinds of cleanup and varied in how willing they were to act when the data was unclear.

Each tool had a different “personality:”

• ChatGPT explained its approach more clearly than it executed it.

• Gemini produced a cleaner first result but made important choices without telling me.

• Claude was more cautious about ambiguous dates, but it still made decisions about duplicate records that I would have preferred to make myself.

And better prompting had limits.

The more specific prompt helped the tools explain their work, identify some records for review and, in some cases, produce a better file. But it did not guarantee that the instructions would be followed. Adding another paragraph of directions only got me so far.

For this kind of small-to-medium cleanup task, AI may save time. But it can also quietly remove information, insert new values, resolve uncertainty on your behalf or claim to have made changes that do not appear in the file.

How to reduce the risk

Here is how I would approach a similar task next time:

Ask the tool to flag uncertain values instead of fixing them. This is especially important for possible duplicates, ambiguous dates, missing values and malformed email addresses.

Tell it not to delete, merge or fill in records without approval. Do not assume the tool will treat preservation as the default.

Request both a cleaned file and a review list. Ask it to identify every record it changed, every record it left unresolved and every decision that still needs a person.

Compare the file to the explanation. A convincing chat response does not prove that the spreadsheet was cleaned correctly.

Check a sample of ordinary rows, not only the obvious problems. Look for missing dates, changed names, filled blanks and deleted records.

Try another tool before endlessly rewriting the prompt. In this experiment, changing tools sometimes made a bigger difference than adding more instructions.

Use a purpose-built tool when the dataset is large, the stakes are high or this will be a recurring process. Tools like Excel Power Query and OpenRefine are built to apply cleanup rules consistently. A general-purpose AI tool is more likely to interpret the data and make judgment calls.

My takeaway

I am not ready to hand a free AI tool a spreadsheet and accept the returned file as clean.

For a small, one-off job, I might use one to identify problems, suggest cleanup rules or create a first pass, provided I can review the result carefully.

But “working with data” is not one single task. Cleaning, exploring and explaining data require different strengths. AI can be helpful across all three, but that does not mean it should be trusted to handle each of them in the same way.

Forecasts are just opinions with math.

That’s the whole secret. Somewhere behind every “we’re projecting 12% growth“ is a person, or a room full of people, who made a set of educated guesses, ran them through a spreadsheet, and put a number on paper.

The math makes it look official. But underneath the formulas, it’s still an opinion about what the future holds.

When you internalize that, forecasting becomes much less intimidating. Now it’s something you can question, discuss, improve, and defend with confidence.

Too often, people talk about annual targets with a mix of frustration and resignation: “There’s no way we can hit those numbers.” “Management just pulled this out of thin air.”

But others talk about their annual goals differently. Not with blind confidence, but with the calm certainty of people who’ve pressure-tested the plan. That difference isn’t luck. It’s a set of habits you can learn.

By the end of this article, you’ll understand what separates a forecast people trust from a number they quietly roll their eyes at, and why this skill is worth your time.

What exactly is a forecast?

A forecast is an estimate of a future outcome based on analysis of available data, current conditions, assumptions, and domain knowledge.

Here, we’re focusing on financial forecasts - specifically, an annual revenue forecast. But the same principles apply elsewhere, even in your personal life, like planning your retirement savings or estimating how much you will spend on groceries next year.

Annual revenue forecasts tend to carry a lot of organizational weight. Once a company sets its revenue for the coming year, that number is then divided into goals across regions, product lines, and sales teams. It can influence budgets, hiring plans, investments, and bonus payouts.

In other words, it is rarely just a number.

The process matters as much as the number

One talented leader I worked with used to say that setting the next year’s revenue forecast was the most important thing he did. The final number became his division’s goal, tied directly to bonuses and long-term incentives.

But the conversations behind the forecast mattered just as much.

The back-and-forth with his leadership team helped him gather better information, challenge assumptions, and create a sense of shared ownership. Even when someone disagreed with the final target, they understood the thinking behind it, and started planning how to reach it before the year even started.

The forecast was also a negotiation with company leadership (his management). It needed to be ambitious enough to support the company’s goals, but not so unrealistic that the team stopped believing in it.

Missing that forecast would not only affect his credibility; it would damage how senior leadership viewed his entire team.

That is why a strong forecast does more than predict the future. It creates alignment around what the organization believes is possible and what must happen next.

Subscribe now

Where to start: top-down or bottoms-up?

Most organizations lean on one of three approaches.

Top-down

A top-down forecast starts with the big picture.

You might take historical revenue, review past growth rates, consider market expectations, and project the trend forward.

This approach helps anchor the forecast in historical reality. It is useful for answering questions like:

• How quickly have we grown before?

• What would a reasonable growth rate look like?

• How does this compare with the overall market?

Bottoms-up

A bottom-up forecast starts with the individual components that produce revenue. You make assumptions about each one, and build the total from the ground up.

For example:

Revenue = number of clients × average revenue per client

But each part may break down further by region, product line, client size, pricing level, renewal rate, or sales channel.

This approach forces you to name the specific levers you are counting on.

Where will new clients come from? Which products will grow? Will prices increase? How many customers are likely to leave?

Both

Many teams build both a top-down and a bottom-up forecast, and then compare the results.

This isn’t redundant because each serves a different purpose. A top-down forecast keeps you grounded in historical reality and the need to build in growth. A bottoms-up forecast forces you to name the specific levers you’re counting on to achieve the number.

In comparing the results, the gap can reveal overly optimistic assumptions, missing information, or opportunities the historical data does not yet reflect.

You should also decide how accurate the forecast needs to be. In some organizations, reaching 99% of the target is considered close enough. In others, missing by 1% can affect bonuses, hiring plans, or investor confidence.

The consequences of being wrong should shape how carefully you build and monitor the forecast.

Share

Where AI fits, and where it doesn’t

AI can be a useful thought partner in this process. It can help you:

• Surfacing patterns buried in your historical data

• Compare multiple scenarios

• Stress-test assumptions

• Identify missing variables

What AI shouldn’t do is build the forecast for you.

The exercise of thinking it through, wrestling with the assumptions, deciding what you actually believe, and figuring out how you’d achieve the number, is the part that matters. You’re the one who will be held accountable for hitting the number, not the model. “The AI said so” is not a credible answer to “why did we miss the target?”

The same scrutiny applies to other people’s forecasts. Ask where the assumptions came from. If the model uses external data, such as inflation projections, market research, or industry growth estimates, trace them back to a source you trust.

Subscribe now

What data you’ll need

Fair warning: in addition to data, annual revenue forecasts take real time. They require input from multiple people - not just for buy-in, but because each person sees a different piece of the picture. That means calendar time for discussion, plus hours spent gathering data and documenting assumptions so they can survive a conversation.

In terms of data, start with three categories of information:

1. Historical performance

   Review past revenue at a useful level of detail. Look for:

   • Growth or decline over time

   • Seasonal patterns

   • Unusual spikes or dips

   • Differences by region, product, or customer group

   • Patterns that are becoming stronger or weaker

Historical data gives you a starting point, but the fact that something happened last year does not mean it will happen again.

2. Current conditions

   Next, consider what could bend the historical trend up or down. That might include:

   • Market growth or contraction

   • Economic conditions

   • Competitive changes

   • Planned price increases

   • New regulations

   • Product launches

   • Major customer wins or losses

   • Acquisitions or organizational changes

This is where context becomes essential. People close to the work can help determine how much these conditions could impact the forecast.

3. The underlying variables

   For a bottoms-up forecast, identify the variables that drive the total.

   For example:

   Revenue = clients × average revenue per client

   If these groups have a lot of variation within each, you may then consider segmenting them further like in the below.

   

   With multiple variables, you can test the sensitivity of the model to see which is most influential. A 1% change in one variable may barely affect the total. A 1% change in another may dramatically change the result.

   The variables with the greatest impact deserve the most scrutiny.

Make your assumptions visible

Every forecast rests on assumptions. Will last year’s growth rate continue? Will customers accept a price increase? Will the product launch happen on schedule?

Write those assumptions down. Make them visible. An assumption nobody can see is an assumption nobody can challenge, and an unchallenged assumption is often where forecasts go wrong.

Visible assumptions also make the forecast easier to update. When the conditions change, you can revise the relevant assumption instead of rebuilding the entire model or arguing about why the final number moved.

Assumptions worth pressure-testing

Whether you’re building the forecast or reading someone else’s, pay close attention to these assumptions.

• Trends. What’s shifting in the market? What forces might push against that pattern continuing?

• Clients. Is growth expected to come from new clients, existing ones, or both? How realistic are the client acquisition and retention assumptions?

• Pricing and average revenue. Will prices increase? What are the average revenue assumptions per client or per product?

• Products. Which products are launching or being discontinued? How quickly are customers expected to adopt something new?

• Attrition and cancellation. What is the attrition rate? Which customers might leave, reduce spending, or delay renewal? Growth assumptions often receive more attention than the losses working against them.

• Planned changes. Are you counting on a major sponsorship, acquisition/merger, new sales channel, or other one time event?

• Contingency. This is the assumption people most often forget. No forecast survives reality perfectly intact. Build in room for uncertainty: the client who unexpectedly cancels, the launch that slips a quarter, the market shift nobody modeled. This makes your forecast more realistic. A forecast with no room for life to happen isn’t more accurate, just more fragile.

Share

Why forecasting is worth learning

Sometimes you’re handed a forecast. Sometimes you’re the one building it.

Either way, your ability to read, question, and explain it affects whether you can paint a credible picture of what’s possible over the next year.

Get it wrong, and the costs are real.

A fantasy-driven target costs a leader credibility twice over: their team sees through it and quietly concludes they’re out of touch, and their own management watches them miss it and starts to question their judgment.

A technically correct model is not enough either. Perfectly sourced data and a mathematically sound model can still fail if nobody understands the realities behind the number. The real job is making assumptions visible and being ready to talk about them out loud, in a room, with people who will push back.

Learning to work with forecasts pays off in several ways:

1. It turns a vague hope for “a good year” into a concrete plan.

2. It builds alignment around how the organization will get there.

3. It reveals which actions and variables matter most.

4. It creates space to test new approaches and experiment.

5. It helps leaders make better choices about money, people, and time.

Bottom line

Forecasting is also one of the most transferable data literacy skills. When you learn to question data, structure assumptions, weigh evidence, and make a believable case for the future, you move beyond simply reporting numbers.

You become someone people trust to interpret what the numbers mean and help decide what to do next. That difference often determines who gets invited into planning conversations and who gets trusted with greater responsibility.

Next up in the series: how to build a business case that withstands scrutiny and makes a credible case for why your product, project, or business is worth the investment.

Subscribe now

For the last few years, “AI project” has become the catch-all phrase for almost any idea involving data, predictions, or automation. Before that, we called many of those efforts data science projects. The terminology has changed, but the challenge hasn’t.

Someone has an idea and a question. Someone asks whether the data can answer it. Suddenly, the conversation feels highly technical.

It doesn’t have to.

We will cover a lot of ground in this article:

• What a data science/AI project actually is and why it starts with a business question, not a technical one.

• Why most projects disappoint people, and how setting the wrong expectations up front leads to frustration later.

• When to loop in your data science or AI partners and why later than you think is usually the right answer.

• How to build a “Project Brief” that turns a vague idea into something a technical data team can actually evaluate (including access to the template we use).

Let’s start by demystifying data science. At its core, a data science project is simply a structured way of using data to answer questions, uncover patterns, or make predictions that help people make better decisions. Sometimes that involves machine learning. Sometimes it doesn’t. Machine learning is simply one tool within the broader field of data science and one type of AI.

Data science has been described as sitting at the intersection of math, computer science, and domain expertise. I like that definition because it reminds us that technical expertise is only part of the equation.

The other part is you.

Whether you’re a founder, marketer, product manager, operations leader, or program manager, you understand your business in ways no algorithm or outside consultant ever will. You know your customers, your processes, your constraints, and the decisions you’re trying to make. That knowledge is just as important as the technical work that follows.

Preparing for a good technical conversation

I led a data science team responsible for developing new data products and methodologies. We spent a lot of time evaluating whether our data could support new ideas that came from our large product organization and our engaged senior leadership team.

Depending on their business question and the data available, the project may have consumed weeks of multiple data scientists and analysts. We wanted to ensure our data science projects yielded the best return on investment. So, we started asking our business partners to answer key questions before anyone started exploring algorithms, building models, or writing code.

Sometimes the resulting discussions led to exciting new products. Sometimes they revealed that we weren’t asking the right question yet. Sometimes they showed us that the data simply couldn’t support the idea. Those were all successful outcomes.

And those conversations became a framework that I found myself using again and again.

Eventually, we turned that framework into a Data Science Project Brief. Just like advertising agencies use a Creative Brief to gather client needs before kicking off a project, we used that Project Brief over and over with business partners to lay a solid foundation for great collaborations and results.

In this article, I’ll share the key elements of the project brief and we’ll build an example together. By the end, you’ll have a practical way to organize your thinking before your first conversation with a technical partner.

A data science project begins with a business question

Kelvin Goods (a fictional business) launched with one beautifully designed insulated water bottle available in three colors. Over two years, it built a loyal following among commuters, hikers, and parents looking for durable everyday products. With sales growing steadily, leadership is considering its first product line expansion: insulated food containers.

At a planning meeting, the product manager asks:

“How can we use our data to predict whether this new product line will be successful?”

Around the table, everyone immediately sees the problem through a different lens. Marketing is thinking about customer demand. Operations is thinking about inventory. Finance wants to understand the investment. And yes, someone inevitably asks whether AI can help.

At this point, many organizations think the next step is to call a data scientist.

Not yet.

Start with the business question

The company’s original question sounded straightforward:

Can we use our data to predict whether our new insulated food containers will succeed?

It’s a reasonable place to start, but it’s still too broad. What does success mean? What are we trying to predict: Units sold? Revenue? Profit? Demand? Repeat purchases? New customers?

The more clearly you can articulate the questions, the easier it becomes for a technical team to determine whether data can help. For Kelvin Goods, the conversation might evolve into something like this:

Should we introduce insulated food containers next spring, and if so, which customer segments should we target first?

The focus now isn’t simply “predict success.” It’s supporting a specific business decision. That’s something a technical team can work with.

Define the decision

Before you think about data, models, or AI, answer: What decision(s) are you trying to improve?

For Kelvin Goods, this project is about helping people make decisions. Specifically, the answer to the above question will help decisions on:

• Whether the new product line should launch.

• How much inventory should Operations produce.

• Which customer segments should Marketing target.

• Which retail partners should receive the initial rollout.

How much room for error is acceptable?

A data science project that involves predictions doesn’t remove risk; it changes how you manage it. Before moving forward, spend a few minutes thinking about what happens if the answer turns out to be wrong.

For Kelvin Goods, what happens if estimated demand is much higher than expected?

• Inventory shortages?

• Missed sales?

• Frustrated customers?

Now ask the opposite: what happens if anticipated demand is much lower?

• Too much inventory?

• Excess manufacturing costs?

• Discounting?

Understanding the consequences of being wrong helps define how accurate the answer needs to be. Sometimes “good enough” is exactly that. Other times the cost of being wrong means the project requires much greater confidence before anyone acts on the results.

Know the stage of your initiative

The stage of the initiative will also determine the shape of the data science project. For Kelvin Goods, the same business question might look like this as it progresses through stages:

1. Exploratory: Determine whether existing customer and sales data contains enough information to support a decision to move forward.

2. Proof of Concept: Test the technical feasibility of the approach to launch.

3. Pilot: Test accuracy of customer demand estimates during one product launch.

4. Production: Build something that becomes part of everyday operations.

Expectations and the definition of success will change depending on which phase you are in.

Define success for this project

One reason AI and data science projects disappoint people is that expectations are often set too high, too early. If you expect your first model to produce perfect answers that remove all uncertainty, you’re setting yourself up for frustration.

Remember, Kelvin Goods is trying to make a better business decision, not precisely predict the future.

What would success actually look like in different stages for this example?

• Exploratory: determine that the available data contains enough signal to support demand forecasting at all.

• Proof of concept: demonstrate that a model can produce predictions that are directionally useful.

• Pilot: the predictions are in use and improve planning decisions.

• Production: accuracy, reliability, repeatability, and integration into existing business processes.

Not every project ends with a production-ready model. Sometimes success is learning enough to make a better decision about what to do next.

By this point, you’ve defined the business problem. The next question is whether your organization has the information needed to answer it.

Models aren’t magic

If there isn’t a meaningful signal in your underlying data, no algorithm can just figure it out. The model is not a crystal ball and it does not make decisions. People do.

The model or analysis is another source of evidence you use alongside experience, market knowledge, and judgment. When done well, a data science project will reduce uncertainty, enabling a better decision than you could have made before.

For Kelvin Goods, that might mean narrowing a wide array of product line and target market possibilities into something smaller and more manageable. Perhaps the model suggests customer demand is likely to fall within a certain range, giving Operations more confidence in production planning. Perhaps it identifies that customer demand is likely to be strongest in a particular region or age range, helping Marketing prioritize its launch efforts. Those are all successful outcomes because they’ve reduced uncertainty and helped the business make a better-informed decision.

Can your data answer the question?

Here’s where many projects get their first reality check. The conversation shifts from “What do we hope to know?” to “What evidence do we actually have?”

Identify what data you have available, including history, data source, and gaps. Your operations or technical partners who have access to the data will be a great help here. Kelvin Goods likely has plenty of data: historical sales and customer purchasing records, pricing, marketing campaign performance, website traffic and behavior, product reviews.

Ask questions like:

• Do we have enough product launches to learn from?

• Does customer behavior for insulated water bottles translate to food containers?

• Are there important outside factors like weather or seasonality?

• Do we need competitor or market information?

• Are there gaps or inconsistencies in our historical data?

• Who owns this data, and can we actually access it?

Having a lot of data isn’t the same thing as having the right data.

Sometimes this exercise uncovers an important realization: the first project may not be building a model at all. It may be improving data quality, collecting additional information, or establishing better measurement going forward.

Work within your constraints

Even if the data look promising, every project has practical limits. Answer as many of these questions as you can before engaging your technical team, and then ask them too. Each group will have its own constraints and considerations:

• How quickly do we need an answer?

• What’s the budget?

• Are there privacy or regulatory concerns?

• Is this a one-time analysis or an ongoing capability?

• Does this need to integrate into existing systems?

While these questions don’t usually determine whether a project is possible, they do influence what kind of solution makes sense. A quick exploratory project might answer the business question without building anything permanent, while an operational system has a very different set of requirements.

What you know vs. what you think you know

Every project begins with assumptions. Make them visible and recognize that they may or may not be true. The Kelvin Goods team might assume that customers who buy insulated water bottles are also interested in insulated food containers, or that previous product launches are good comparisons.

They need to question those assumptions, and identify the unknowns: What makes one product comparable to another? How much will competitive response influence demand?

This practice of separating assumptions from unknowns helps the team recognize where confidence is well-founded and where it isn’t. It also gives your technical partners a much better understanding of the business decision at hand.

Who owns the decision?

Who decides whether this data science project moves forward? Who decides whether it’s successful? Who ultimately uses the analysis or estimates that come out of the model?

The answers frequently are not the same person or team. Marketing might request the project, but Operations owns the decision. Finance would approve the investment, but Product would be managing the rollout. Knowing who owns which decisions helps everyone understand what success actually looks like and who needs to be involved along the way.

Plan for adoption

Now imagine your technical team delivers exactly what you asked for. Depending on the project, that might be a one-time analysis, a dashboard, or a predictive model that becomes part of an ongoing business process.

Great...but the technical work itself isn’t the goal.

Someone has to receive the results, trust them, and know what to do next. Your Data Science Project Brief should identify who receives the output, how often they’ll use it, what decisions they’ll make from it, and what actions are expected to follow.

For Kelvin Goods, suppose the initial project produces an analysis estimating demand for insulated food containers. Marketing uses those results to recommend launching first to outdoor enthusiasts and parents. Operations uses the same analysis to plan an initial production run.

Six months later, if the company finds the approach useful, that same work might evolve into a forecasting model that’s updated before every major product launch. The important question isn’t whether the output comes from an analysis or a model - it’s who uses it, what decisions it informs, and how it fits into the business.

How to scope a project for success

Smaller is always better. The type of data science project you’re undertaking has a role in defining the scope. I always want to know: how small can we make this project?

Warning: this really irritates “idea people” who want to build a rocket ship. Ask it anyway. How small? One product category? One region? One launch? One year of historical data. One business decision. The more targeted your project, the higher your chance of success.

And if this first data science project does succeed, what earns the next study? Success might earn a larger pilot, or another product category, or automation, or the go-ahead to integrate into an existing process. Thinking about the next step helps you avoid treating the first project like an all-or-nothing decision.

Now bring in your data science partners

Because you have done the work, you’re now ready for a productive, collaborative conversation with your technical partners. Note that I said “have a conversation” not “submit a request.”

You have translated a business idea into a clearly defined question that gives your technical partner something concrete to evaluate.

Note: a good technical partner rarely responds with immediate answers. They respond with better questions, and that is exactly what you want. They might point out a hidden assumption, or suggest a different business question that’s actually easier to answer. Maybe they identify a completely different data source you hadn’t considered. These are signs that the conversation is working.

Together, you will begin answering questions like:

• Is this technically feasible?

• Do we have enough data?

• What additional data is needed?

• What approaches might work?

• What level of accuracy is realistic?

• How much effort will this require?

• What are the biggest risks?

Your project brief isn’t something you hand to a technical team and walk away from. It’s the starting point for a conversation.

Good technical partners will ask questions you hadn’t considered. They’ll challenge assumptions, refine the scope, and sometimes redefine success altogether. That’s not a sign your preparation was incomplete. It’s exactly how good data science projects take shape.

You’re ready

Let’s go back to the question that started this article.

“Can we use our data to predict whether this new product line will be successful?”

Notice that we spent all of our time talking about the business. We didn’t talk about AI or machine learning or data science. Successful data science projects begin when the business problem at hand is well articulated and the full project team understands it.

Throughout this article, you’ve been building your own Data Science Project Brief. If you’d like our template, which brings those conversations together into one practical tool you can use before your next meeting with a technical partner, you can download it here.

Subscribe now

People often talk about the future with frustration and resignation:

“There’s no way we can hit those sales targets.”

“Management just pulled a number out of the air and thinks it’ll magically happen.”

“I know we need to build this, but I can’t get funding.”

Sometimes they’re right to be skeptical. A number handed down without context feels less like a goal and more like a wish.

But imagine talking about the future with confidence instead. Not because you can predict it, but because you’ve thought it through: looked at the evidence, challenged the assumptions, identified the risks, and worked out what would have to be true. And even when the goal is a stretch, you can see a credible path and explain it to others.

That’s what this series is about: learning how to pressure-test the future. Over the next few weeks, we’ll explore the practical tools people use to turn uncertainty into a plan: forecasts, business cases, strategic plans, and budgets. Not with advanced statistics or complicated models, but with better questions, clearer assumptions, and more confidence.

Share

This world exists. You see it in strong managers, leaders, coaches, and founders. Rather than treat future numbers as facts, they treat them as estimates to be tested, challenged, and improved. The result:

• Revenue or sales forecasts that feel like a stretch but don’t ignore reality

• Business cases for new products or services that explain why an idea is worth investment

• Strategic plans that connect long-term goals to everyday priorities

• Budgets that direct limited resources to what matters most

The people who do this well don’t just build a spreadsheet. They build understanding and buy-in by helping investors, teams, and partners see what’s possible.

How do they do this?

The secret is simple. They know that:

No one predicts the future perfectly, but there are practical ways to make better-informed guesses about what’s realistic and what’s just a fantasy with a dollar sign in front of it.

This series is for anyone who’s ever been handed a target and wondered: Where did this number come from? Is it realistic? How am I supposed to make it happen? It’s for anyone who has been asked to build a forecast, business case, or plan and wasn’t sure where to start.

We won’t be talking statistics or advanced modeling. We’ll focus on the practical habits that help you ask better questions, test assumptions, and build confidence when making decisions about the future.

Four reasons these skills are necessary

When done well, building a forecast, business case, strategic plan, or budget improves how you think, plan, decide, and communicate about a future that is uncertain. These tools help you:

1. Set expectations and targets. A forecast gives you a clearer picture of where you want to be and by when. Maybe that target comes from you. Maybe it comes from your boss, your board, your investors, or your family budget. Regardless, forecasting helps turn a vaguely defined future hope into something more concrete.

2. Get buy-in or investment. If you want to build something new, you usually need someone else to believe it is worth the effort, like an investor, an executive, a partner, or your own team. So, you need to lay out what the investment of time, money, or people might produce. A good plan helps people see a possible future clearly enough to support it.

3. Allocate limited resources. Time, money, people, and attention are finite, so every decision is a tradeoff. If you hire for one role, another team may have to wait. If you spend your time on one initiative, something else gets less focus. These tools help you make better - not perfect - choices about where limited resources should go.

4. Become something different. People are not static. Neither are teams, businesses, or organizations. Sometimes forecasting is not just about hitting a number. It is about transformation. A future-oriented plan maps the path from where you are to where you want to be: what needs to change, what milestones matter, and where you might get stuck.

Three key phases with future planning

The real value of these activities is not just the final number. It is also the thinking, the conversations, and the choices you make.

1. Gather your data and build a solid framework. A useful forecast starts with curiosity: What do we already know? What data, research, or benchmarks could help?

Then comes honesty: Where are we making educated guesses? What are we assuming will happen? What would need to be true for this plan to work?

Finally break the problem into its parts. Which variables will influence the future outcome?

This phase is about making your thinking visible so you, and others, can test which parts are solid and which parts need further scrutiny.

2. Pressure-test the plan and assumptions. A good forecast shouldn’t be a sacred document. It should be questioned, challenged and revised. Sales may know what’s shifting with customers; Operations may know where capacity is required, what resources are available; Finance may spot a cost you missed.

Defending your assumptions helps you see where your plan is strong, where it’s shaky, and how it might need to change. Plus, people are more likely to support a future they helped shape.

3. Monitor progress that matters. Once you have a solid plan, decide which metrics will tell you whether reality matches your expectations. What does normal look like? What does good look like? What’s an early warning sign that says “pivot now”?

The right numbers turn your forecasted plan from a document you create once into a tool that helps you make better daily decisions as the future unfolds.

What we’ll explore

Over the next several weeks, we’ll apply these ideas to four common ways people use numbers to think about the future:

• Forecasting revenue, demand and growth

• Creating strategic plans

• Building a business case for new ideas or investments

• Budgeting and decisions on where to focus limited resources

For each one, we’ll cover where to start, what data you need, which assumptions deserve the most scrutiny, how to evaluate a number someone else hands you, and when AI can help or hurt.

Forecasting is not one skill. It’s a collection of practical habits that help you think more clearly, ask better questions, and make stronger decisions before the future arrives.

A dashboard is a decision-support tool.

If you don’t know what decision it supports before you build it, you’re designing a report, not a dashboard.

Most dashboard failures aren’t technology problems, data problems, or budget problems. They’re process failures. Organizations often skip critical design work and jump straight into building.

This is the same pattern we see whenever organizations reach for a tool before they understand the need. We’ve written about this before with platforms, AI, and automation. Dashboards are no different.

Start with the decision

When you’re designing and building a dashboard, before you even think about metrics, visualization or software platform, ask:

What decision will this dashboard help someone make?

The decision comes first. Everything else is downstream of that.

Once you’re clear on the decision, other questions become much easier to answer:

• What information is actually needed?

• Who needs it?

• How often do they need it?

• What would cause them to act?

Reverse that sequence and you risk building a dashboard that looks impressive but doesn’t help anyone do their job.

Need to think about it another way? Try asking one of these questions instead.

Three ways to ask the same question

If “decision” feels too abstract, try asking one of these questions instead:

• What problem am I trying to catch earlier, and what would earlier look like?

• What behavior am I trying to change, and what would prove it’s working?

• What threshold would tell me to change course, and what would I do next?

These questions force clarity about decisions, ownership, and action. They aren’t always easy to answer. But when they get skipped, the dashboard rarely solves the problem later.

Why dashboards get built backwards

Dashboards fail due to several common pitfalls.

Failure pattern #1: The Excel migration

The dashboard request arrives as an upgrade: “Can we get this out of spreadsheets and into a real tool?”

It sounds reasonable - central location, perhaps some automation and version control.

But often the spreadsheet is simply recreated in a browser without first asking what decisions it supports or whether it’s working at all.

Spreadsheets have a tendency to accumulate uses over time, accommodating as many people as possible. It might have a little performance reporting, a little forecasting, a little exception tracking, a little historical archive. The decisions it was meant to support get lost over time.

The dashboard faithfully recreates all of it. It’s more polished, but just as hard to act on

The tool changed. The confusion didn’t.

Failure pattern #2: Built for someone else’s questions

Dashboards can fail because the people designing them and the people using them aren’t solving the same problem.

Leadership dashboards often fall into this trap. They’re designed around assumptions about what leaders should want to see, rather than the questions leaders actually ask. Sometimes the dashboard is technically for the executive, but the real users are the people expected to interpret the numbers and turn them into action.

The same thing happens in reverse. A dashboard gets built by people who don’t understand the decisions the end users need to make. The result is threshold alerts nobody uses, complex visualizations that answer the wrong questions, and metrics that aren’t central to the team’s work.

• What question is this audience trying to answer?

• How often do they need those answers?

• If the numbers change, what would they do differently?

Failure pattern #3: Built assuming data and processes can keep up

This may be the most optimistic failure mode. Someone knew exactly what good, bad, and concerning looked like. They had thresholds, a vision.

What they didn’t have was reliable access to the data needed to support it. A dashboard signals confidence in the underlying metrics. But if the data underneath can’t be sustainably supported over time, that dashboard won’t be credible, and therefore, won’t be used.

This tends to happen when manual work is needed to prepare and load data into a dashboard. Or, there is no clear accountability for maintaining the dashboard on an agreed-upon cadence.

The pattern behind the patterns

These aren’t the only ways dashboards fail. But they share a common thread: the work of defining the need either didn’t happen or didn’t happen thoroughly enough.

The tool drives the design. Available data replaces needed data. Someone copies a template from another business without asking whether the same questions need answering.

By now, this probably doesn’t sound like a dashboard article anymore. That’s because it isn’t really about dashboards. It’s about how organizations approach tools.

The dashboard brief

What most organizations lack is a clear blueprint for the dashboard they want to build.

Before any conversation with an analyst, consultant, or software vendor, there are a handful of questions that should already have answers.

• What decision is this supporting?

• What action should it trigger?

• How will we know it worked?

Those questions sound simple, but they’re often surprisingly difficult. As a result, they’re also the work that gets skipped when teams are eager to start building.

We’ve turned those questions into a one-page Dashboard Decision Brief, which helps teams clarify the need before they rush into building.

The dashboard brief isn’t paperwork. It’s where you decide what the dashboard is supposed to do. It’s the thinking that determines whether a dashboard will ever become useful.

It won’t guarantee a perfect dashboard, but it will dramatically reduce the odds of building one that nobody uses, trusts, or acts on.

AI and automation: the same argument, higher stakes

Replace “dashboard” with “AI” or “automation” and the questions barely change.

What process is this improving? Who owns the outcome? How will we know it helped?

Organizations struggling to realize value from AI aren’t struggling simply because the technology is immature. They’re struggling because they reached for the tool before they understood the need.

While these investments are larger and the expectations are bigger, the fix is exactly the same. Understand the need first. Then choose the tool.

Before you build the next dashboard

The best dashboard you’ll ever build is the one designed around well-defined decisions. The second-best dashboard is the one you revisit after realizing the first one never had a clear brief.

Both are better than the dashboard sitting unused on someone’s bookmark bar.

Download the Dashboard Decision Brief and bring it into your next dashboard conversation. See what happens when the first question in the room isn’t “what data do you want to include?” but “What decision are we trying to make?”

Subscribe now

From the Stanley Cup to the NBA playoffs to the World Cup, we’ve had weeks of high-intensity sports. Professional sports teams run on mountains of data: player statistics, advanced analytics, predictive models. But during the game, they rely on just a handful of numbers on the scoreboard. Those tell them where they stand, how much time remains, and what they need to win.

If I had to pick one data practice that can fundamentally change how a team operates, it would be building these for your work.

Most managers think they already have a scoreboard. They point to annual goals, weekly dashboards, or whatever performance framework is currently in favor. But those numbers rarely guide how the team directs its time and energy each day.

Imagine a basketball team playing without knowing the score or how much time was left. Or imagine their scoreboard showing 45 different statistics. Lots of numbers, very little focus. Now compare that to how an elite team plays when the board shows a close score and a clock counting down.

Are you creating the same intensity and focus for your team? Probably not. That’s not a criticism - there are not a lot of practical roadmaps for this.

Why teams struggle

Building an effective scoreboard is simple, but not always easy.

Teams rarely suffer from a lack of metrics. The shortage is clarity. People don’t understand how measures are defined or calculated. They can’t see how their day-to-day work connects to the numbers they are being asked to move. Sometimes the data isn’t reliable enough to trust. Sometimes the metrics just aren’t part of the team’s rhythm. The numbers aren’t discussed regularly, used in decision-making, or reviewed often enough to shape behavior.

Underneath these issues sits a common root cause: the team never identified and operationalized the few numbers that actually tell them whether they’re winning. And in some cases, they haven’t clearly defined what winning looks like.

What happens when you get this right

A good scoreboard clears out workplace noise and removes the low-level anxiety that comes from never quite knowing if you’re succeeding. Why? Because you can see where you stand and what you need to win the game.

When done well, it creates momentum, protects against burnout, and helps people make good decisions when you are not in the room.

Fewer, more targeted metrics also create more freedom. This means that you might have a different scoreboard for different individuals or groups on the team. When people understand the score, they can experiment and make decisions on their own because the score itself becomes the feedback loop. What did we try? Did it work? What do we do next? How much time do we have left?

Your scoreboard will evolve. You probably won’t get it right on the first attempt. The targets may shift as you build momentum. At some point, you may make enough progress that you turn the focus onto a different set of metrics designed to tackle a different set of challenges. This is a sign that the scoreboard is working.

The most important thing is to start. Most of us are being asked to do more with less: businesses navigating restructuring and layoffs, nonprofits facing funding cuts, and individuals juggling increasing demands on their time and energy.

Subscribe now

How to build your scoreboard

Having a scoreboard is not about doing more. This is about focus, and about being just as intentional about what you leave off the scoreboard as what you include.

When building a scoreboard, I design for these 5 elements:

Measures that you can influence. People need to be able to see the connection between what they do each day and the numbers they’re being asked to move. If they nail those numbers, the larger goal takes care of itself. In our sports analogy, these are the points on the board.

Progress that is quickly visible. When someone can quickly assess where they stand right now versus where they need to go, it changes what they prioritize. They put their energy into figuring out how to move that number.

Timing and constraints. Time remaining, available budget, available hours left for the project. These are the equivalent of the shot clock, available timeouts, or number of fouls. These limits influence what winning realistically looks like right now.

Measures that are trustworthy. If a metric can’t be easily and reliably measured, it doesn’t belong on the scoreboard.

The final test: it fuels energy and momentum. You’ll know you’re there when people are proactively looking at the numbers, the right behaviors are following, and you’re starting to see momentum. It won’t be linear, but the progress will be visible.

What it looks like in practice

Here’s what this could look like in practice.

An email marketer wants to grow website traffic. She could track the percentage of people who open their emails and click-through to the website for each campaign. And she will, but that’s not her scoreboard. Her scoreboard asks a more motivating question: how often are we beating our targets, and is that happening more often than before.

Instead of “What was our open rate this month?”, she asks:

“How many campaigns exceeded our targets? Did we do that more times this month than last, and why?”

A local nonprofit works to house lower-income families and help them flourish. They track how many clients they serve, of course, but that’s not their scoreboard. Their scoreboard tracks two numbers that guide their daily operating decisions: the GPAs of kids in their free afterschool program compared to the district average, and dollars saved through their annual tax help program. That second one matters because every dollar saved in taxes goes directly into a family’s bank account.

Instead of “How many families did we serve this year?” they ask:

“Are the kids in our program outperforming their peers, and are we putting more money back in families’ pockets than we did before?”

Scoreboards can look different by person or team. What matters is identifying the handful of numbers that answer two questions: Where are we? And what do we need to win?

A good scoreboard won’t eliminate uncertainty, but it will give people a clearer way to direct their limited time and attention.

Leave a comment

Share

Subscribe now

Imagine you’re looking at the results of a blood test. One result is highlighted:

22%.

Is that good?

You probably have a few questions:

• What does this measure?

• What’s a normal range?

• Should I be concerned?

• What would cause it to change?

• Do I need to do anything about it?

Your doctor doesn’t just tell you the number. They explain what it means, how it compares to what’s typical, and whether it requires action.

The same thing happens in business all the time. Whether the number comes from a fundraising report, a marketing campaign, a financial statement, or a website dashboard, the number itself is only part of the story.

Most of us spend at least part of our careers translating numbers for people who aren’t close to the data behind them. Over time, I’ve realized I rely on the same set of questions almost every time.

Whether I’m explaining a metric to leadership, reviewing results with a team, or helping someone understand a report they didn’t create, these questions fill in the parts of the story a number can’t tell you by itself. And when I’m on the receiving end, they’re the same questions I want answered before I decide what to do next.

Q1: How do we make it safe not to know?

Before you even talk numbers, make it clear that nobody is expected to know all of this already. Remember that the person receiving this information didn’t build the report, and they may not even use the system it came from. They shouldn’t be expected to instantly understand every metric, assumption, and calculation behind it.

If you’re explaining:

“Before we dig in, let me give you some context on where this comes from. Some of this might be new.”

If you’re asking:

“I want to make sure I’m reading this correctly. Can you walk me through it?”

Good conversations happen when people stop pretending they already understand and the people doing the explaining avoid making others feel foolish simply because they have different expertise.

For example, a development director presents fundraising results to board members who don’t work with donor metrics day-to-day. Before discussing donor retention, campaign performance, or year-over-year giving, the director spends a minute defining the metrics and providing context so board members can focus on the discussion instead of wondering whether they’re missing something.

Two good things happen that turn this into a productive conversation. First, board members gain enough context to participate confidently. Second, everyone stays focused on the problem to solve rather than getting stuck on terminology or debating what a metric means. The discussion moves more quickly from “What are we looking at?” to “What should we do about it?”

Q2: What does this number tell me? What’s missing?

Every metric captures part of reality and ignores the rest. It’s useful to know where a number comes from, but you also need to understand what activity or action the number represents (and what it does not).

A website pageview tells us that someone visited a page. But it doesn’t tell us whether they found what they were looking for, stayed to read, or left immediately.

A customer satisfaction score isn’t just a number. It tells us how a group of customers responded to a survey. But it doesn’t tell us how the customers who didn’t respond felt about their experience.

If you’re explaining:

“This score comes from our post-purchase survey. It reflects the opinions of customers who chose to respond, but not those who ignored the survey entirely. It’s useful feedback, but it may not represent every customer’s experience.”

If you’re asking:

“What does this number measure, and what part is not included here?”

Q3: Why do we look at this number?

The easiest way to waste time in a data conversation is to spend twenty minutes discussing a metric that isn’t connected to the decision you’re trying to make.

If you can’t explain why you’re looking at a number, it’s hard to know what to do with it. And a number can be perfectly accurate and still be irrelevant to the question at hand.

If you’re explaining:

“We’re looking at email open rate because we’re trying to understand whether our subject lines are getting attention. This metric tells us whether people are interested enough to open.”

If you’re asking:

If this number went up, down, or stayed the same, what would we actually do differently?

For example, a retail store sees more visitors this month than last month, but fewer visitors are actually making purchases. Foot traffic and purchase conversion rate answer different questions. Until you know what the business is trying to achieve, you can’t know which number matters most.

Q4: What’s “normal?”

A number needs context, or it’s just trivia. Whether something is good, bad, or unremarkable depends entirely on what you’re comparing it to. This is where you need a reference point such as benchmarks, historical performance or industry trends.

If you’re explaining:

“For an email list like ours, a 20–25% open rate is typical. We’re currently at 22%, which puts us in the normal range.”

If you’re asking:

“Compared to what?”

For example, a nonprofit reports that 48% of last year’s donors gave again this year. Is that good? It depends. If donor retention is typically 40%, that’s very encouraging. If it has historically been 60%, it may signal a problem. The number hasn’t changed. The context has.

Q5: What makes the number move?

Knowing why a number moves helps you respond.

Numbers don’t just decide to be different. Something happened. Usually several somethings. Some of those factors are within our control. Others aren’t. The more clearly we understand the difference, the more likely we are to take productive action instead of reacting to noise.

For example, a nonprofit sees a decline in donations during the summer. Before sounding the alarm, the development team looks for possible drivers. Is this a seasonal pattern that happens every year? Did they do less outreach than normal? Did a major grant end? The answers matter because each explanation suggests a different response.

If you’re explaining:

“Donation totals are influenced by the number of outreach campaigns, donor retention, grant funding, seasonal giving patterns, and broader economic conditions. Some of those are within our control, and some aren’t.”

If you’re asking:

“What would have to change for this number to improve?”

Q6: What should we pay attention to?

Not every movement in the data deserves a reaction, and not every metric has the same clock. A weekly change might be meaningful for website traffic during a marketing campaign, but largely irrelevant for annual donor retention or employee turnover.

Want to create anxiety? Watch a metric more closely than it was designed to be watched. In some situations, moderate fluctuations in the numbers are normal. Some changes may be expected at certain points or only become meaningful when they persist over time.

For example, a Customer Support team sees a spike in tickets the week after launching a new product feature. On its own, that increase isn’t necessarily bad news. More customers may be using the feature, asking questions, and learning how it works. The more important question is whether ticket volume returns to normal after the initial adjustment period.

If you’re explaining:

“Support tickets increased after the launch, which was expected. We’ll continue monitoring for a few weeks, but a short-term spike isn’t unusual.”

If you’re asking:

“What amount of change would actually cause us to act?”

Why these questions matter

Every good data conversation touches on these six questions, whether they get said out loud or not.

When the questions are skipped, people fill in the gaps themselves. They make assumptions about what a number means, whether it’s good or bad, what caused it, and what should happen next.

When those questions are answered, people spend less time talking past each other and more time solving the problem in front of them. Decisions get made with a shared understanding of what the number represents, what it doesn’t, and why it matters.

Just as importantly, everyone gets better at these conversations. The person explaining learns to communicate more clearly. The person asking learns what to look for. Over time, both sides become more confident working with data they didn’t collect and metrics they don’t use every day.

Most questionable decisions aren’t caused by a lack of data. They’re caused by confusion about what the data means, what it doesn’t mean, or why it’s being discussed at all.

Ask better questions, and the numbers become a lot more useful - and the conversation does too.

Subscribe now

“Data” is too often a four-letter word that makes otherwise confident people feel stupid.

After launching We Dig Data, I consistently got the same question: “What do you mean by data?” It baffled me. To me, everything is data: numbers on a financial statement, an offhand comment in a customer interview, the fact that I’ve now heard someone mention the same book twice in one week, or that my kid is sleeping fewer hours than usual.

Anything that can be counted, observed, or treated as a signal? That’s data.

For me, data has always been a quiet superpower. It cuts through noise, helps me make decisions with confidence, and brings sanity to situations where politics might drown out facts. So, imagine my surprise when the main response to my carefully crafted elevator pitch was not “Cool! That sounds incredibly helpful. Let’s talk.”, but instead a request to clarify the fundamental noun (“data”)!

My first theory was that it was a vocabulary problem: maybe “information” or “insights” would land better. Wrong. I got the same question. My second theory was that data just felt boring or irrelevant to them. Also wrong. These people genuinely believed that good data could improve their work and their lives.

So as I continued to ask colleagues, clients, and friends about what the word “data” meant to them, I found an iceberg. The word “data” wasn’t triggering confusion; it was triggering shame.

“I know I should use it, but I don’t know where to start — and honestly, I don’t like it.”

“I am suffocating in data already. What more do you want me to do?”

“I’m not a numbers person.”

“I’m not technical.”

These aren’t lazy or checked-out people. These are driven, capable, goal-oriented people who regularly stretch outside their comfort zones. The problem wasn’t motivation, but the word itself.

What we learned is that the workplace data literacy gap isn’t just a skills problem. It is also a language problem that became an identity problem. The word “data” has quietly accumulated decades of baggage that makes capable, intelligent people feel inadequate before they’ve even looked at a single spreadsheet.

The word “data” comes with baggage.

When many people ask “what do you mean by data?,” they’re actually asking something deeper. Welcome to manager-turned-therapist territory. Here are four beliefs standing between many people and their own data, and what to do about it.

The word “data” can make someone feel exhausted before the conversation even begins. For many managers and business owners, the word conjures an already-overflowing inbox of dashboards, metrics, reports, and spreadsheets, none of which seem to connect to the actual problem weighing on them. When this group asks “What do you mean by data?,” they’re not asking for a definition. They’re really asking: “Which piece of this overwhelming pile are you about to make my problem?” Lower their resistance by focusing on the business issue they care about.

Instead of: “Let’s analyze your data”

Try: “To grow profitability so you can expand the business, let’s look at who your best customers actually are, which days of the week generate the most revenue, and where cash might be quietly leaking out.”

Some people hear “data” and worry they are about to be exposed as bad at math. This one surprised me. Many high-performing managers and successful entrepreneurs still carry an old belief that they “are not numbers people.” When they hear “data,” they brace for regression models or technical explanations that make them feel out of their depth. They want to know: “Are you speaking the language of business, or the language of statisticians and engineers?” The antidote is to focus on what to measure rather than how to measure it. People rarely fear evidence; they do fear looking stupid.

Try: “Before making staffing changes, let’s measure where your team’s time is going, identify the bottlenecks, and total how much time and money their activities save the company.”

Others hear “data” and assume you’re talking about technology. In many organizations, “data” historically belonged to IT. So they picture databases, cloud infrastructure, and cybersecurity. This group is often trying to clarify a practical question: “Are you talking about a dashboarding software or business insights I need to manage my team?” Anchor your conversation with the business insights and benefit, not with a discussion of the data. People do not wake up wanting cleaner data. They wake up wanting to grow lead pipelines, shorten project timelines, and retain more customers.

Finally, some people think “data” equals “Big Data”, not their everyday operations. Media coverage of data focuses on AI, machine learning, and privacy. These people look at their 5-person team and don’t see how it applies to them. What they miss is their data hiding in plain sight. The sticky notes on their desk. Which customers quickly go from quote to sale. The projects that always run late. This group asks for clarification because they don’t see how the concept applies to their messy, real-world operations.

Try: “Let’s keep track of which customers ghost you after sending them a quote, which customers responded quickly, and what the quotes’ services and terms. This will help us identify patterns that will help us make adjustments.”

Data as a supporting character.

Notice what all four reactions have in common: the remedy is to focus on an outcome the person already cares about, not the data itself. You probably can’t talk someone out of their reaction to the word “data.” That baggage is theirs to unpack. But you can stop making data the main character. When you focus on the outcome, data takes its proper place: not as the point of the work, but as a tool that helps the work get better.

Nobody lies awake worrying about their “data collection strategy.” But plenty of people lie awake wondering why a good client went quiet, whether their team is actually making progress, or where last quarter’s margin disappeared. Those are the conversations data was meant to support.

Someone’s reaction to the topic of “data” has been shaped by years of their own professional experience, assumptions, frustrations, and even identity. And that is certainly more baggage than I intend to raise and unpack during an elevator pitch.

So, we are practicing something different. We try to talk about what data does, rather than what it is. We talk about turning an existing, abundant asset (data) into a manager’s survival guide. We talk about finding proof instead of relying on guesses. We talk about spotting patterns before problems become expensive. By shifting the conversation from numbers to evidence and decisions, you lower anxiety and connect more directly to what people care about.

Easier said than done, of course. But if this resonates and you’d like to try it, here’s a cheat sheet we’ve started and are adding to as we go:

If nothing else, this prevents eyes glazing over when you mention the word “data” to someone who doesn’t work in data. :-)

What We’re Reading Right Now

In honor of this week’s theme, here are some articles where people are writing about data as an enabler to affect change in the tech industry, to lead better, to invest better, and to work better.

• and talk on about Substack Bestsellers lists and affecting change in Women Rising: Build the Flywheel. The Badge Follows.

• writes about whether the ‘rules of thumb’ for seasonality and stock market returns actually hold up The Calendar’s Seasonality Quiet Edge.

• writes about leadership and habits of winning teams in Progress Is Fuel: Why Perfectionist Leaders Never Win.

• talks about how to fix the process before jumping into a new tool in Process Before Platforms.

At some point during the analysis you stopped trying to understand the data and started rearranging it. Different date range. Another filter. The same report, broken down just one more way, just to see. Just in case.

That’s not a data problem. That’s a you-and-the-data problem. And more data is not going to fix it.

More data analysis feels productive. It looks like diligence. But past a certain point, it’s just procrastination via spreadsheet.

Why does this happen? Sometimes the stakes feel high, so you keep looking for a level of certainty that doesn’t exist. Sometimes you find one number you don’t understand and now you don’t trust any of them. And sometimes you’re hoping the data will make the decision for you, so you don’t have to.

None of this means that you're bad at data or bad at analysis. It means you're human, you have a real decision to make, and you're avoiding it in a way that feels productive.

Signs you’ve moved from analysis to avoidance

You’re running the same data again just to check. You’re reviewing fresh numbers every day. You’re adding filters and dimensions to reports you’ve already looked at multiple times.

But the biggest signal is simpler:

If you can’t answer the question, “What would change my mind right now?” you’re probably no longer gathering information. You’re avoiding a decision. One of the most common ways people avoid a decision is by convincing themselves that one more cut of the data will finally provide certainty.

Digging yourself into a hole with the data

One of the easiest ways to get stuck is to keep breaking the data into smaller and smaller pieces in search of an answer.

Most platforms, designed to work with datasets of all sizes, will happily calculate a conversion rate on 11 purchases and display it with exactly the same confidence as a number backed by 5,000 impressions.

Consider a small brand running ads for eight weeks, which seems long enough to feel like the data should be telling them something. So they start slicing it: age bracket, placement, device, day of week. By the time they’re done, they’re looking at 15 purchases spread across 24 segments. The platform shows them a conversion rate for every single one. These are confident-looking percentages. They have decimal points and everything!

None of those numbers mean a thing. Not because the math is wrong, but because there’s not enough signal there to support the conclusion.

Any tool that lets you filter data lets you drill deeper and deeper into smaller and smaller segments of that data. Sales data can be viewed by product, then by week, then by customer type until you’re looking at four transactions and calling it a trend.

Customer inquiry data can be viewed by source, then by month, then by service line until each cell contains single-digit counts and you’re making channel decisions based on them.

The problem is the same: the platform keeps showing you numbers regardless of whether those numbers are capable of supporting a meaningful conclusion.

The questions that actually help

When you’ve reached the point where more reports aren’t creating more clarity, the answer isn’t “just decide.” Instead, ask yourself these specific questions to get unstuck:

What would change my mind right now? If you can’t answer this clearly, more data won’t help. You’re not looking for information at this point - you’re looking for permission. The data can’t give you that.

Is the data actually capable of answering this question? Sometimes the answer is no. Not because you’ve analyzed it incorrectly, but because the information simply can’t support the conclusion you’re looking for. When that happens, the decision either needs to wait for better evidence or be made using judgment.

Is this decision reversible? Most small business decisions are easier to undo than they feel in the moment. You can cut advertising spending back next week. You can revisit the product feature next quarter. Before you treat a decision like a one-way door, name whether it actually is one. More often than not, it isn’t.

What’s the cost of waiting versus the cost of a wrong call? Delayed decisions have consequences too - not just money, but time, or momentum. Factor in what you give up by continuing to wait, not just what happens if you’re wrong.

Let’s get practical

Write down three things: what you know, what you don’t know, and what decision you’d make right now with what you have.

That exercise often reveals one of two things: either you already have enough information to move forward, or you discover the specific gap that’s keeping you stuck. Both are more useful than adding another filter to a report that’s already told you everything it can.

Sometimes the gap is real. More often, it’s discomfort dressed up as due diligence.

Data reduces uncertainty. It doesn’t eliminate it. At some point, the job shifts from gathering evidence to exercising judgment. And judgment only gets better when you use it.

Everyone should have a power suit. Or an outfit that makes you feel stronger.

Earlier in my career, I was frequently the youngest in the room. Often the only woman. Sometimes the only foreigner. Occasionally all three at once. So I learned to project confidence that I didn’t often feel.

I found my ‘boss look.’ I bought heels that made me taller. I practiced carrying myself like someone who had been in that room a hundred times before. I trained myself to stop clenching my hands under the table when the conversation was heated and to stop jiggling my foot when I was impatient. I even learned to drop my voice at the end of sentences, because when everything sounds like a question, people treat you like you’re asking one.

And all of that helped, but it only takes you so far. Because once you’re in the room, you have to contribute something. And what consistently helped wasn’t appearance, posture or even confidence. It was data.

Not fancy data. Not a PhD in statistics or a team of analysts. Just the discipline of knowing the numbers before I walked in the door - of doing my homework and being ready to use it.

Confidence helps, but here’s where data changes the game.

When you’re new, the data is your map.

Every new job, new industry, new team is a foreign country. The temptation is to spend your first weeks nodding along and trying not to look lost. Hoping someone will show you the way. Don’t.

Instead, go find the data. It’s almost always sitting right there, buried in old decks, reports, and dashboards that nobody has really looked at in months. Industry size and trends. Where your company fits and how it stacks against competitors. Company financials. Client revenue mix. Product usage. Read it like you’re studying for an exam because you are. The exam is every meeting you’ll walk into for the next six months.

When you know the data, you stop being the new person who doesn’t know anything. You become the new person who sees things others have stopped noticing. That’s not a small thing. That’s your first move.

When you’re the most junior or inexperienced in the room, questions are not enough.

Most people will tell you: ask good questions. And yes, thoughtful questions matter. But questions alone mark you as someone who is learning. If you want to be taken seriously, you need to know something.

Every team has a set of metrics they care about. Find out what they are. Understand where those numbers come from, what’s included and what isn’t, how they’ve moved over time, and what actually drives them. Read the reports. Talk to whoever owns the data. Stay current on news alerts for your clients, your industry, and your competitors - and when something significant happens, actually dig in.

Then, when you speak in that meeting, you’re not asking. You’re contributing. You’re the person who noticed the three-month trend before anyone else raised it. You’re the person who connected the external news to the internal number.

That is a completely different kind of presence in a room. And it is available to you right now, regardless of your title.

You don’t need permission to measure your own work.

You can build a track record before anyone asks you to. Most people don’t realize this until years in.

Whatever you’re responsible for - an email newsletter, a client program, a campaign, a process - start measuring it. Track how many people open the email and then click. Note what changed and when. Run a small experiment. Look at the results. Try something different. Do it again.

This creates a feedback loop, and it does two things simultaneously. You get better at your work, and you build evidence of impact that you can use in a performance review, in an interview, and in the meeting where you make a case for more resources or a bigger role.

You will have evidence of meaningful progress. Not vibes. Not effort. Evidence of results. That is gold in most career conversations.

The most powerful thing you can do is tell someone something they didn’t know. And then tell them what to do about it.

This is where data stops being a confidence tool and starts being a leadership tool.

Every time you’re in a room with someone more senior than you, or a client, or a decision-maker — that’s an opportunity. Not to perform. Not to impress. To add something real.

What does the data say? Prospect leads were down again last month. That’s the third month in a row. What does it mean? A competitor just made a major push on a new product. Here’s what that product does. Here’s the feature that matters and here’s the one that doesn’t. What do we do? Here’s what we should prioritize in our development queue and why

That’s leadership. The data makes it credible, and the insight and recommendation make people listen.

Don’t stop at the insight. Bring a recommendation, even if it might be wrong. Tell them what you think they should do with the information. This is how you go from being someone who reports facts to being someone who influences decisions - and you don’t need a title for that. You need preparation.

Share

One more thing - and this one’s for life.

None of this works if you can’t communicate it.

Learn to deliver information clearly to any audience: your boss, your team, a client, a room full of people who know more than you or less than you. Learn to make people care. Learn to answer the question they didn’t ask but needed to: why does this matter, and what should I do right now?

This is a lifelong practice worth the investment you make in it. I am still guided by solid data-driven messages that I heard 20 years ago because the delivery was that powerful.

Data without communication is just a spreadsheet. Data mixed with extraordinary communication is influence and impact. It’s something people remember about you long after the meeting ends.

Bring the data.

As an introvert, I was not naturally the most commanding voice in the room. But I learned that you don’t need to be the biggest voice.

You need to walk in knowing something. You need the homework done, the numbers understood, and the recommendation ready.

Because in life and career, there will be moments of uncertainty, moments that will determine where future paths open or close, moments where you feel exposed, moments where the stakes are real and the room is not on your side. Those moments determine a lot.

The heels make you taller and the suit more confident, but the data makes you formidable.

Leave a comment

Share

We Dig Data helps you leverage data better at work and practice leadership while you do it. If you want to lead with clarity, strengthen your judgment, and amplify your impact, you’re in the right place.

Subscribe now

First you had to figure out which dashboard tool had the right data. Then you found two report views that seemed to cover it - except they showed different numbers and used the same name for both. No explanation. No warning label. Just two “conversion” numbers, silently disagreeing.

So you clicked around trying to get your bearings. Which turned into reading a tooltip. Which turned into a help article and a YouTube video. Which turned into wondering whether you even have the right tool open. Somewhere in there you saw something interesting, but unrelated to your original question. You followed it because you might need it later. Suddenly it’s been an hour. You saw some things. You’re not sure what they mean. You definitely still don’t have your answer.

It’s the analytics version of scrolling social media. Same vague, slightly guilty feeling at the end.

Nobody teaches you how to walk into unfamiliar data and come out the other side with something useful. Most people think the hard part is learning the platform. It isn’t. It’s knowing how to orient yourself quickly and to figure out what matters and what doesn’t. This is that process.

Before I start…

Before I start digging into a new dashboard, I write down the question I’m actually trying to answer. That sounds simple, but it keeps me from wandering into every interesting metric or report the platform puts in front of me.

First, don’t touch anything.

Before I do anything, I take a few minutes to understand how the tool is organized. I look at the menus. Note how the interface is organized. Get a feel for where things live - settings, search, help - not just how to get to the reports. I ask myself basic orienting questions: what’s the difference between these sections? Where do I go if I need a definition? How do I change my settings?

This sounds obvious, but if you skip it, the danger is that you land on a default dashboard or report view and start reacting to the first numbers you see, but those aren’t usually the numbers that matter.

Here’s what that looks like in practice. When I open Google Analytics for We Dig Data’s website, the first thing I see on the “Home” screen is a chart with active users, event count, and real time data from the last 30 minutes. Below that, a map of where users came from. The prominent positioning feels like that information is important, but none of it may be relevant to my actual question. In fact, the metrics I actually need are tucked into the bottom right, half hidden. I’d miss them entirely if I started analyzing the data before orienting myself.

Turns out you can’t always see everything.

Before I can understand the data, I need to understand my permission levels. Am I an admin or a read-only user? Can I see all reports or only some? Can I export, or just view?

This matters because the answers shape what information you can validate, what gaps you may be missing, and how confident you should be that you’re seeing the complete picture.

So before you go further, note what you can and can’t access. If something is greyed out or locked, that tells you there is more to the picture than your current view. Even if your access is limited, the help materials are not. That’s often the fastest way to understand what you’re not seeing.

Recently, I was set up in a marketing tool as an “analyst,” which meant reporting-only access. I could see the left navigation, but I couldn’t click into most of it. That told me there was more to the workflow and reporting structure than my current view. That’s useful information.

Those words don’t mean what you think they mean

Once I know the boundaries of my access, I start scanning everything I can see: report names, metric labels, filters, definitions, tooltips, and any built-in documentation the platform provides. I’m not trying to answer my analysis question(s) yet. I’m trying to understand how this platform thinks about the business.

The terminology might feel familiar, but don’t make assumptions. Words like “users,” “sessions,” “customers,” and “revenue” sound standardized, but they really reflect decisions made when the platform was set up - product decisions, business rules, and platform assumptions. If you assume you know what a metric means just because you recognize the word, it will cost you later. Take the time to clarify those definitions.

I take notes as I go. I flag the things I can’t fully explain yet and come back to them later once I have more context. I save links to reports that seem relevant or confusing. I look at the dashboards - not to analyze the numbers yet - but to see the metrics and time periods and filters the platform offers.

Here’s why this matters. One platform’s “conversion” may include abandoned checkouts. Another may not. Two dashboards can use the exact same word and be measuring completely different things. That’s how you end up with two reports showing different numbers for the same metric.

A client has multiple marketing platforms. Three use the metric “sessions,” but each has a different way of measuring. I rely on help documentation to clarify the meanings, and then I note down definitions to refer to later.

What was the question again?

This is the most important step. I am now ready to begin analysis. But before I dig further, I anchor on the business goal or objective. What are we trying to change? That question helps me focus.

Not “What does the dashboard show?” That’s the wrong starting point for analysis.

The right questions are:

• “What decision are we trying to make?”

• “What outcome are we trying to improve?”

• “What problem are we trying to understand?”

That distinction matters more than it sounds. A dashboard that initially felt overwhelming becomes much more manageable the moment you realize you’re only looking for the pieces that influence one specific outcome. Everything else can wait.

I was recently working with a client whose goal was conversion. Specifically, understanding where people were dropping out of the sales funnel. That single objective immediately clarified which reports mattered, which metrics deserved scrutiny, and which sections of the platform could safely be ignored for now.

Without that anchor, it’s very easy to spend hours exploring data that never changes the decision. Sound familiar?

Subscribe now

Same metric, different tool, completely different number.

If you’re working across multiple tools (and most businesses are), the same metric will often appear in several places. But that doesn’t mean each platform is equally reliable for your specific question.

Each tool sees a different slice of the customer journey. Traffic in your web analytics platform is not the same as traffic in your email tool. Conversion in your e-commerce platform is not the same as conversion in your ad platform. Each one has blind spots, timing differences, and assumptions built into how it measures behavior.

Part of the job is deciding which platform should serve as the source of record for which metric and understanding why.

For that client concerned with conversion, we compared three platforms. Each measured a different part of the customer journey: browsing behavior, marketing engagement, and completed transactions. We ultimately chose one as the source of record for conversion because it had the most reliable count of completed purchases, and then we documented why we made that decision.

That last part matters. If you can’t explain why you trust one number over another, the conversation quickly turns into debating the numbers themselves instead of making decisions.

The report got you 80% there. Now what?

No platform sees the whole picture. There will almost always be questions the default reports can’t fully answer. Sometimes the gap is small. Sometimes it’s glaring. At that point, you usually have two choices: customize the reporting inside the platform or export the data and do the analysis yourself.

Sometimes I end up doing both. In Google Analytics, I might customize a report to provide some detail needed for deeper insight. But then I often need to extract the data into Excel for further analysis or reporting to my colleagues.

A surprising amount of useful analytics work is not advanced modeling. It’s identifying the gap between the business question and what the default dashboard happens to show, and then finding what’s missing.

Future you will thank present you for this.

The final step is the one that makes the work sustainable.

Once I’ve found the answer or built the view that surfaces it, I make the process of getting back there repeatable. That might mean saving a filtered view, bookmarking a customized report view, documenting what a metric means, or writing down how I got to the answer so I (or someone else) can recreate it next time.

This is how the hour compounds. The first session is always the hardest; you are orienting, mapping, and figuring out what to trust. But if you leave something behind for yourself, the next session starts where this one ended. You’re not rebuilding from scratch. You are building on it.

The goal isn’t a sophisticated analytics setup. It’s a simple, reliable path back to the answer - one you can follow yourself, without starting over every time.

Platforms change. The process doesn’t.

It doesn’t matter whether you’ve seen the tool before. What matters is knowing what you’re looking for, understanding the limits of what you’re seeing, and being willing to go get what’s missing.

That’s what this process gives you. Not a shortcut - the hour is still the hour. But at the end of it you’ll know what the data actually covers, which numbers you can trust, and you’ll have a view you can come back to without starting over. That’s not a small thing. Most people never get there because nobody gave them a map.

Now you have one.

Leave a comment

You’ve updated your resume, but there’s the uncomfortable thought: “What if my numbers aren’t impressive enough? Other candidates will have larger budgets, bigger teams, or have grown revenue faster.”

This isn’t a test where bigger is always better. Numbers get attention, but the explanation behind it is what makes a hiring manager believe you can do the job.

My experience as an interviewer

We are not recruiters or HR professionals. However, my co-author and I have 20+ years of experience building teams: brand new teams, tiger teams, cross-functional teams, and transforming existing teams.

Through this work, we have interviewed people for a wide range of professional roles and seniority levels, from interns to executives. We’ve listened to hundreds of people explain their accomplishments.

How people solved problems - what challenges they walked into, what tradeoffs they faced, and how they figured it out - is one of my favorite aspects of interviewing.

So I want to share what we’ve seen makes for a truly epic interview story.

Because while incorporating data into your resume is important, it’s the story behind that data that gets you hired.

Data on the resume vs. the story behind it

Many candidates worry that others have ‘better’ numbers. But as an interviewer, I don’t just look at the numbers.

For instance, a product manager who grew product revenue by 50% or 20% may be as interesting as one who grew it 300%. A marketer with 45% web traffic growth may be more appealing than one with 6000% social media growth. Similarly, a blogger who gained 10K subscribers in 6 months might be less of a fit than someone who built an engaged audience of 4K over 2 years.

Both candidates are impressive in these examples, but the stories behind their results will reveal very different journeys. The context in which they operated and the actions they took to achieve those outcomes are what truly distinguish them.

Setting the stage

If a number is on your resume, there is likely a good story behind it. And every good narrative needs the stage set.

As a potential employer, I want to hear the context around your numbers. Job search coaches call this step the “Situation” or “Context” or “Challenge.”

The most memorable interview answers will have tension. What needed to change? What problem were you facing? Was there risk? Pressure? Uncertainty? Was it:

• A new opportunity to be conquered, such as a new product launch or a new company that’s going to change how the market operates.

• A villain to fight, like a competitor stealing market share or a sudden market disruption.

• Desperate stakes to overcome, like financial or operational troubles or the cancellation of a major client or grant.

This set-up helps the interviewer quickly grasp the scope of the problem and your role. With smaller problems, I look for a story that showcases a variety of your skills. Larger scopes often involve teams, so I look for you to tie the results to your individual contributions.

With a compelling set up complete, you move onto the best part of the story.

Your hero story

The story behind the data is a tale where you are the hero. How do I know that? Because I’ve already seen the end - the data, the results, on your resume.

This should be the biggest part of your story and be focused on what you did. It features your unique contributions, judgement, and problem solving.

• “I suspected the problem was …… but then learned that was incorrect. So I analyzed ……, and found that we needed to do this instead.”

• “We had budget and time to pursue only option 1 or option 2, but not both. So I…..”

• “Overnight, our most lucrative line of business was made obsolete. I worked with ….. to build a plan to …..”

It is the story of how you faced the challenges in front of you and solved the problems you encountered. Many people can 3x their revenue, but how you achieved that is your differentiator.

The data on your resume are the quantified results. They are the ‘happily ever after’ or the resolution of your epic hero’s journey.

When you bring all three of these together, this becomes the story of your impact that your interviewer wants to hear.

Relevant interviewing frameworks

This approach maps to common frameworks (CAR - Circumstance/Action/Result and STAR - Situation/Task/Action/Result) recommended by job search coaches. They can feel mechanical, so use them to structure your hero story, but not as a rigid script. Focus on telling the story of your unique journey and actions.

Final thoughts

When preparing your resume and quantifying your accomplishments, spend the time to craft your hero story. If writing about yourself feels challenging, imagine a loved one or friend is telling this story about you, celebrating your achievements honestly and vividly.

From an interviewer’s perspective, I have thoroughly enjoyed hearing hundreds of these hero stories, which highlight human creativity, judgment, resilience, and smart risk-taking.

While a resume is like a sales brochure and the interview part of a sales process, the goal is never overselling. It’s about helping the interviewer understand your potential fit for the role and the organization. So be proud of your numbers and accomplishments, and confidently share the compelling story behind them.

Leave a comment

Subscribe now

I ask the dumb questions in meetings. I always have.

It used to make me self-conscious. Now I consider it one of the most useful things I do.

Here’s what we’ve learned about why nobody asks, what happens when someone finally does, and the exact words I use when that someone is me.

Nobody asks because it feels risky

Last week’s post on sneaky metrics got a comment that stuck with me. Ankita wrote: “The part that matters most is getting really concrete really fast…who owns the metric, what’s in, what’s out.” She’s right. But to get concrete, someone has to start the conversation. And that’s the part nobody talks about.

So why don’t people ask? It’s not because people don’t care. It’s because asking feels scary.

It feels like admitting you should already know, but don’t. It can sound like you’re challenging the person who owns the number. And when meetings move fast - nobody wants to be the one who slows everything down for a question that might have an obvious answer.

So people stay quiet. They go back to their desks and work from numbers they don’t fully understand, or they quietly calculate things their own way. And that’s how you end up with two people presenting different revenue figures in the same meeting.

You ask the “dumb” question, and then what…?

A few years ago, we were working to automate financial reporting for a subscription business - analyst reports, multi-year contracts, revenue recognized on release. The setup seemed clean. Then someone asked: “What do we do when a report is released late?”

The room got quiet.

It turned out late releases weren’t unusual. And the harder part was that there wasn’t one clean answer waiting to be unlocked.

The question didn’t just surface the answer. It surfaced the fact that nobody had one and an important discussion was needed.

This situation happens way more than people admit. And when it does, the path forward is usually the same.

First, bring in the person closest to the data - not the person with the biggest title. They’re the ones who have been quietly making the call.

Second, run scenarios out loud: “If this report releases on the 3rd instead of the last day of the month, which period does it land in?” Concrete examples cut through abstraction faster than any definition will, and will raise decisions that need to be made. Use the examples to clarify the boundaries. Perhaps 3 days late doesn’t change anything - but 7 days does.

Third, consider any downstream effects from your decision. If you decide late reports count in the next month, what happens to monthly revenue targets? Do they shift? Do commissions change? Does finance need to adjust forecasts?

Finally, write it down before you leave the room and communicate that decision. Even a Slack message counts! The goal isn’t a perfect policy document, but making sure that information doesn’t go back to living in one person’s head.

This is why I embrace asking those “dumb” questions. Because that question - the one that felt disruptive to ask - was the one that surfaced a problem everyone needed to resolve anyway. The report automation project didn’t create it. It just made it impossible to keep absorbing in silence.

What to actually say

The question itself doesn’t have to be brilliant. It just has to be asked. Here’s what I actually say in different situations. These are not scripts, but starting points.

When I don’t understand a number:

“Can you walk me through what’s included in this?”

Simple and non-threatening. It puts the burden on understanding, not on challenging. Nine times out of ten, the person explaining it discovers something in the telling.

When something feels off:

“I want to make sure I’m not missing something - this looks different from what I expected. Can we look at it together?”

“I might be missing something” does a lot of work. It’s honest, it’s humble, and it makes it about understanding rather than accusation. But you’re still asking.

When you suspect two people are measuring differently:

“Before we go further - are we calculating this the same way? I want to make sure we’re comparing apples to apples.”

This one is especially useful when two numbers don’t match and everyone’s being polite about it. Name it directly. It’s almost always the issue.

When you’re presenting and want to get ahead of confusion:

“Quick note on this number before we dive in. Here’s exactly what’s included and what’s not.”

If you define it first, nobody has to ask. This one takes 20 seconds and saves 10 minutes.

In all of these examples, the common thread isn’t the exact phrasing, but the framing. Curiosity, not accusation. You’re not challenging anyone. You’re making sure everyone’s working from the same understanding.

You don’t have to ask it in the room

Not every question needs to be asked in the middle of a big meeting. If you’re earlier in your career, or just not in a position to break the flow, you still have options.

Listen closely. Take notes. Then follow up with someone close to the number. That’s often the analyst who built it, or the person pulling the report each week.

And use the same language:

“Can you walk me through what’s included in this?”
“I want to make sure I’m interpreting this right…”

These conversations are often easier one-on-one. People will go deeper. You’ll get a clearer answer. And if something important comes out of it, you can bring it back to the group:

“Quick follow-up on that number from earlier - here’s how it’s actually defined…”

You’re still creating clarity. You’re just doing it in a way that fits the moment.

Want more? This article by Gustavo Razzetti has great ways to “unblock” conversations with reframing - not fighting.

Share

Asking “dumb” questions is a superpower

When one person asks, it gives everyone else permission.

Someone senior might assume everyone else already knows the answer. But from the junior person who is afraid to look inexperienced to the new team member who doesn’t want to slow things down or even the long-timer who has been pulling that same metric for years - they all need someone to go first.

The whole dynamic shifts when a group sees that clarifying a definition isn’t a challenge, it’s just good practice. Meetings will (eventually) get faster, not slower. Decisions get cleaner. People stop quietly disagreeing in their heads and start actually resolving it out loud. Real conversations can happen.

And the people who were already doing this? They tend to be the ones others describe as “easy to work with” or “really sharp on the data.” It’s not that they know more. It’s that they’re not pretending to know things they don’t.

Go on - ask those “dumb” questions.

I still ask the dumb questions. I probably always will.

Sometimes I even ask when I think I already know the answer, because I’ve seen too many times what happens when nobody does. The number that everyone nodded at turns out to mean three different things. The decision gets made on a metric nobody fully understands. The rework shows up two months later.

Asking isn’t the dumb thing. Staying quiet is.

Share

Subscribe now

Related articles:

Most data mistakes don’t look like mistakes. They look like smart, reasonable decisions—but based on numbers that weren’t fully understood.

Teams often think they’re aligned because they’re using the same words:

“Revenue.” “Users.” “Customers.”

But they’re not.

They’re working from different definitions and no one realizes it until something breaks. Then it shows up as rework, wasted time, and wrong turns.

This is a problem of false alignment. It’s common, it’s frustrating, and it’s almost never talked about directly.

Let’s change that.

Here are 7 common offenders that wreak havoc on well-meaning teams, and what you can do about it.

Share

The Sneaky Seven: Metrics that always need a definition

#7: Leads

“Leads” (or prospects) sounds simple, but can have very different meanings across teams. Marketing might count a whitepaper download. Sales may only count someone who requests a meeting.

Depending on the system used to capture them, leads coming from different channels (sales vs. web vs. social, etc.) aren’t always captured or categorized consistently.

Now layer in intent and timing. Are all leads equal? Probably not. And how long is someone considered a lead if they don’t convert into a sale - days, weeks, months?

#6: Traffic

“Traffic” can mean foot traffic at a store or event, or digital activity online. And within each, there are multiple ways to measure it.

At an industry event, traffic could mean total attendees or only the people who stopped at your booth. On a website, it could be page views, sessions, engaged sessions, users, or unique users. Each of these tells a different story and has a different context.

#5: New Product Revenue

“New product revenue” is often used to measure innovation and growth, but there’s usually a lot of debate about what counts as “new.” Timing might be limited to products launched just this year, or, if a product has a long sales cycle, it might include the first few years of a product’s lifecycle.

The type of product also needs definition. Teams differ in whether they count only new standalone or if major new features for existing products are included as well.

Don’t forget to clarify revenue. Some teams use projections. Others use actual revenue. Others use sales booked.

#4: Users and Usage

“Users” and “usage” are highly dependent on your product and distribution model, especially in businesses that deliver services online like banking, software, streaming, or gaming.

A user may be anyone with a login or only someone that is active. But what counts as ‘active’ also varies and needs definition. Perhaps it is someone who accessed the system in the past 30 days, or who took a specific action, or spent a minimum amount of time.

#3: Customers

Counting customers gets surprisingly complicated, and definitions can vary between teams or product lines within the same organization.

In B2B, the first question is usually whether the number is counting companies, contracts, or individual users. One company might have multiple contracts and thousands of users.

In both B2C and B2B, the buyer and the end user may not be the same person - like a team signing the contract while another uses the product, or even a parent buying a toy for their child. This is also an important distinction.

Then there is timing and services. A customer who purchased last year, but not this year, may not be counted by one team, but is included by another. How free subscribers or limited trial / promotional customers are counted may also be a discrepancy between teams.

#2: Retention

Retention (or renewal rate) is the percentage of customers, or revenue, you keep over a period of time.

This metric gets muddy quickly and needs clarification on whether it’s measuring:

• customer retention or revenue retention

• a specific cohort (”customers who signed up in January”) or the whole customer base

• upsells, new customers, or pricing changes

Depending on what is included, retention can exceed 100% and mask churn rates, or not. For this reason, the team might decide for multiple retention metrics based on their objectives.**

#1: Revenue

“Revenue is revenue”… until it isn’t.

Sales vs. revenue. Recognized vs. booked. Gross vs. net.

Revenue is one of the most misunderstood numbers because a few key distinctions get blurred.

Sales and revenue are often used interchangeably, but they’re not the same. Sales reflect what was sold in a given period. Revenue reflects what the company is allowed to recognize as earned.

When looking at revenue, you need to understand how it is “earned” or “recognized.” Depending on the product or service, revenue might be counted when a product ships, when it’s delivered, or as it’s used. Or it might follow a defined schedule - so a 12-month subscription sold today might have revenue that is split over 12 months.

And what counts as revenue isn’t always obvious either. In marketplace models, like Etsy or AirBnB, $100K in transactions might translate to $5K in revenue from fees or $105K, depending on how it’s defined.

Why are these metrics so “sneaky”?

With so much variability, you’d think that we’d always want to clarify the definitions. But we don’t.

Familiar words create a false sense of clarity.

Revenue. Users. Customers.

They sound obvious. So teams assume alignment without confirming what’s actually being measured.

And even when something feels off, people often don’t ask. They feel like they should already know. Or they don’t want to slow the meeting down. Or it’s uncomfortable to challenge a number someone else owns.

So instead, people stay quiet and move on.

When you combine:

• One word that can mean many different things

• A tendency not to pause and clarify

You get false alignment. Nothing breaks right away. It shows up later as confusion, rework, wasted time, and mediocre to poor decisions.

How do you get clarity?

Start here: ask the question. What is included in this metric? How is it defined? Just ask.

Yes, it might feel uncomfortable. But chances are, someone else is wondering the exact same thing and once you say it out loud, you’ll realize the room wasn’t aligned after all.

And when the conversation gets fuzzy, use this playbook:

1. See it for yourself. Go to the source. If it’s a lead, walk through the website or flow. What actually happens when it is captured?

2. Run a quick scenario. Get clear on yes/no cases. “If I download the e-book, am I a lead? What if I request a demo? What if I chat with the bot?”

3. Clarify what’s included. “Does this lead number include only website activity? Or also social, CRM, and sales team inputs?”

4. Tie it back to the report. “So 108 leads in April means 108 people completed web forms or chats, and this excludes phone calls and sales-sourced leads. Correct?”

Your goal: understand it well enough to explain it to someone else simply.

And if you’re being measured on it? Don’t leave the conversation until you know exactly how your actions move that number.

Thanks for reading. Do you have another “sneaky” metric we should add? We’d love to hear it!

Leave a comment

Share

**If you want to dig into more on retention, this article from Churnzero is one of the more helpful explanations we’ve seen. We have no affiliation, we just appreciate their clear explanation of retention and when to use it.

Bonus Reading: Other sneaky metrics (but for different reasons)

Percentages add another layer of confusion because they depend on multiple underlying definitions, especially when it comes to conversion rates and trends. We’ve covered these in more detail in other articles. You can read more below.

We have a lot of new subscribers here this week. Welcome! We are glad you are here. This is a longer post than normal because we detail how we are extracting our Substack data for analysis, where we use AI and where we don’t (and why).

For those of you not on Substack, while the example is focused on a specific platform, the general ideas and findings are similar to what we see when looking at data coming from other platforms like Google Analytics, Shopify, etc. We are in learning mode, so if you have any suggestions on tools or things that worked well, we’d love to hear them!

- Rachel

Leveraging data to accelerate results. That’s what we help managers and entrepreneurs do - by using a disciplined approach defining objectives, selecting supporting metrics, evaluating data inputs, and making sense of the outputs. As we build our company, we turn that tough-love advice on ourselves, especially when it comes to our Substack newsletter.

So this week, we share how we’re handling Substack performance data. While it’s a workable model specific for our Substack goals, it is also adaptable for other creators’ goals.

Initially, we used Substack’s native dashboards. While they’re helpful, we soon found that they didn’t fully answer the questions we care about. So we built a more hands-on process, focused on understanding what drives success for us.

In this post, we will:

• Explain how we use Substack data and where we ran into limits.

• Argue why you should look at the data itself and why AI alone is not enough.

• Detail the steps we followed to unlock our Substack data and what we learned.

Substack metrics we care about (right now)

We identified early on what we were not trying to do: we weren’t focused on paid subscribers or trying to maximize every possible metric.

We focused on building the right audience for us, and understanding what content was resonating. That’s what supports our larger goal, which is helping managers and entrepreneurs build a data feedback loop into their everyday work to accelerate their success.

Along the way, we refined our core metrics to a targeted set:

• Relevant audience growth: subscribers and followers, across Substack and LinkedIn.

• Content engagement, such as shares, restacks, and comments

• Community building: interactions, connections and participation

But all of these metrics are not easily viewed over time and analyzed in the Substack dashboards.

How we use these metrics

We are leveraging the data in two key ways:

1. Ongoing monitoring. We review our target metrics regularly - usually weekly, some monthly - alongside what we actually did in that time period.

2. Analysis. We also wanted to compare different types of posts, notes and other activities. But we needed the data to support looking for patterns between effort and response.

Getting closer to the data

To achieve this, dashboard summaries or just feeding it into AI for analysis were not enough. We needed access to the underlying data so we could explore it directly and see what was being captured (and what wasn’t).

We wanted to build our own understanding of what is happening rather than rely on the conclusions coming from dashboards and AI.

These were good instincts because the dashboards didn’t tell the whole story, and the AI made assumptions and had incomplete conclusions.

So we set up a process and supporting system to extract, store, and explore our Substack data outside the platform.

Here’s how we did it - the quick version

Here’s the overview of what we did. If you want more detail, we have you covered. There is a step-by-step guide on how to do it yourself at the end of this post.

Extract the data
We use StackContacts (from ) to pull Substack data, including subscribers, posts, and engagement, into a local database. This saves us from building API integrations ourselves and gives us access to more than what’s visible in the Substack dashboards. Once configured, the data syncs from Substack to a database, and can be automated to run regularly.

Store the data
The data is stored in a local DuckDB database. That means we own it and can query it directly. We aren’t dependent on a platform interface, and we can preserve our history.

Query the data
We created a handful of SQL queries to pull the data related to our key Substack metrics, such as audience growth and content engagement. As we learn more about the data, we refine the queries.

Connect the data to Excel
Our queries are embedded in an Excel file that we refresh weekly after syncing the data via StackContacts. We used an ODBC connection to directly tap the database from inside Excel.

Use the data to monitor progress and build a feedback loop
The Excel file gives us a consistent view of our key metrics over time, alongside what we actually did that week. It supports a simple, repeatable, review process with familiar tools and a critical subset of our data. Now that this is set up, it takes us only a minute to refresh the data.

Explore the data using AI
Of course, we’re also experimenting with AI tools to analyze the data. This can be helpful for generating queries and surfacing patterns - but only once we have a solid understanding of the underlying data. In our case, the Claude desktop app is accessing the DuckDB database via Model Context Protocol (MCP). This configuration is part of StackContact’s installation process.

Overall, this setup isn’t particularly complex once broken down into its parts, and it gives us something we didn’t have before: direct access to our own data, and the ability to shape how we measure and interpret it.

Want more details? A deeper dive is at the end of this article.

Why we collect more than we need right now

Even though we focus on a small set of metrics, we’re intentionally collecting more data than we actively use. Our early-stage history is valuable - and we don’t know yet what we might want later. Patterns that don’t matter now might matter at scale. Older behavior might become useful for comparison. Additionally, platform definitions can change; we’ve already seen shifts in how metrics like open rates are reported on Substack.

Keeping more complete data now gives us flexibility later.

Why the data was harder than expected

Once we had the data, we ran into something we often experience with clients: the Substack data itself was not cleanly organized.

• There were many tables, with many fields, but no data dictionary with definitions.

• Some fields looked similar, but meant different things.

• Other fields were empty or inconsistently populated.

• Mapping fields back to actual Substack activity took work, and several iterations.

In other words, the data didn’t immediately tell a clear story “out of the box.” We had to look at the actual data to get familiar with it - what the fields meant, which similar sounding field was the right one, how tables related to each other, and what was actually reliable.

For example, we track subscribers and followers weekly. We have been doing this manually, and wanted to pull richer data from Substack in a more automated way. So we developed a SQL query via the AI and produced a tidy data output - except for one problem. It said our followers went back to 2021, which isn’t possible. The issue? The field used in the query was the date our followers created their Substack profiles, not the date they began following us.

These structural issues and lack of data definitions are important to know when considering how you apply AI to analyzing your data.

Where AI helped and where it didn’t

Part of our setup allows AI tools to query and analyze the data. It would be tempting to leave it to AI to learn the structure and quirks of the data itself, but in practice, this can (and did!) backfire.

The detailed data is where you learn what your dashboard really summarizes and when to question the overly confident responses AI gives you.

The first question we tested with the AI was subscriber count. The resulting total subscribers was correct, but all free subscribers were labeled as “unsubscribed.” That was a fundamental misunderstanding of the data - and if that is wrong, everything built on top of it is wrong.

We had given the AI little context or instruction, and it was easy enough to prompt for correction. But even after iterating and guiding the AI through the database, there were still issues.

But if you operate with caution, the AI does bring real value. It helped:

• Explore the data structure: it reliably identified which tables could be safely ignored for our purposes, and teased out relationships much faster than we could have. It was also significantly faster in working through the many fields looking for relevant data. Even when there were errors, the second review generally picked up the right field.

• Generate queries: You don’t really need to know SQL (though a little bit always helps so you know what your query is doing). Rapid development and editing of queries saved hours. We know our strengths, and SQL syntax is not one of them.

• Quickly surface key insights: Simple quick analysis, such as top performing posts by a variety of measures, tended to be reliable. We also tried out topical analysis, having the AI classify our posts and break down performance by theme. The groupings were very close to how we think about our content, which gave us confidence in the approach. More importantly, it highlighted both areas of strength and gaps where we may want to develop new content.

Going forward, given that we now better understand the detailed data, we will continue to use that knowledge to prompt AI to generate additional queries as well as be an analysis partner. We may also use it to run data quality checks when we update and refresh the data.

Summary: what this gives us

This process gives us a more useful set of numbers and a way to work with our data more deliberately.

We’ve chosen a small set of metrics to guide decisions, while keeping a broader record so we can learn over time. We’ve built a simple process we can repeat, refine, and actually use. And we’ve tested where AI can help - and where it can quietly lead us in the wrong direction if we don’t understand the data ourselves.

This is what it looks like, in practice, to build a data feedback loop: define what matters, get close to the data, and use it to inform what you do next.

If you’re interested in how this is set up in more detail, we’ve outlined the technical setup below.

Step-by-step guide: How to do this yourself

We’ll walk step-by-step through the setup. Note that we’re operating on PCs, not Macs. So while you can do all of this on a Mac, it’s going to look different.

In addition, the install manual for StackContacts is extensive, and we will not repeat all of that detail - but it may help you to see some of the setup screens and understand where things went right and wrong for us.

Step 1: Extract your data

1. Purchase StackContacts.

Obtain ’s StackContacts. Note that there are a number of editions to fit your environment: we have just one publication (Practical Data Foundations) and do not need the Gumroad or Kit integrations, so we bought the Hobbyist edition.

Note, this part of the process took us a little while, so set aside a few hours (and possibly more) to do this.

2. Review the install manual and complete prerequisite installations.

Read the install manual! It’s long. It’s dense. But very helpful. There are prerequisites before you can install StackContacts itself, including Node.js, npm, and the MCPB CLI tool. You need to know how to open a command prompt or PowerShell, and how to run those as an administrator. (Don’t know how? Do a search - AI results and lots of helpful shortcuts will pop up). In addition, you’ll need to install either the Claude desktop app or Cursor AI app (we used Claude).

3. Install StackContacts

The install manual will direct you to the product page for the latest installer. Run the installer, follow the prompts and then launch StackContacts. Be patient - it took a couple of minutes for our initial launch of the software.

If you run into difficulty: in our case, the Visual Studio install was accidentally interrupted at some point, which led to failures later in the install process. We went back and installed Visual Studio again, manually. Worst case? Uninstall, then start over.

4. Setup StackContacts with publication and cookie info

Follow the install guide to set up your publication. First, you’ll need to tell StackContacts how to communicate with your Substack publication. Using that login, a cookie is pulled from a specific Substack page, establishing the connection between StackContacts and your Substack stats.

When you click Login, the app will give you the following window:

The automated process described in the manual and what the app defaults to did not work for us, producing login errors. We had to manually pull the cookie values. It’s described in the manual, but here’s what it looks like in Chrome.

5. Sync your data

Once login was set up, the sync process itself was smooth. We’re pulling every possible metric we can from Substack. Our syncs are taking about 30 minutes, so be aware that if you have a lot of data or multiple sites, this may take a while. Our install is running on one local PC; the app is not kept running all the time so we’re not using the automatic syncing (it will fire only when the app is open and your PC is “awake”), but you can set up a date and time for an automatic sync.

Tip: When you sync, you are pulling all available data. What you had in the database before is overwritten - you’re not just appending data that’s new since last time. This has implications; notably that if you want to preserve history as it was, you’ll need to periodically back up your database. For example, say Substack stops making data on DM threads available. If you allow the database to be overwritten, you will lose that particular historical data.

6. Finish AI integrations

From here, you finalize the install by configuring either Claude or Cursor AI to enable AI-powered insights atop the data. For Claude, this meant creating and enabling the appropriate extensions. This process was smooth, aligning with the instructions in the manual.

A step we recommend: once you have completed the actual install, open Claude and instruct it how to find the database. Our initial prompts (e.g. show me my last 10 subscribers) were not successful because Claude had no idea what we were talking about. StackContacts is a “deferred” tool - meaning it didn’t load automatically. Claude could only find it after being told the name, triggering a tool_search to load it. So use a prompt like: “Use tool_search to find any connected database tools, then connect and show me what’s available.”

Step 2: Store your data

As part of the StackContacts setup, you will also set up DuckDB, an open-source SQL database. When Substack data is synced down to your local computer, this is where it will be piped. The database is not proprietary to StackContacts. This is positive because we can’t get locked out of our data even if we uninstall StackContacts. We own it, we manage it, we back it up. And even if you’re a large organization or scaling up, you can operate nicely for quite a while with DuckDB, whether locally or in the cloud.

StackContacts’ data screen gives you quick access to database information and tools, including backup and restore of your database, a summary of records and tables, and a list of all tables from your publication.

Take note of the Database Location (circled) - you’ll need that if you choose to connect to it from Excel.

Want to see your data up close and personal?

Download a tool to browse it. We are using DBeaver - a free, open-source tool for database management. From here you can browse through the database structure, view the actual data, write and run queries and export data.

Step 3: Query your data

One of the things we learned in testing the AI integration was that the data structure and definitions were sufficiently complex that Claude could not do analysis without more significant familiarity on our part with the data to instruct it well. We knew we wanted the underlying data, but those experiments with Claude made it clear that we needed it.

So the next step was to set up standard queries for metrics we wanted to follow regularly. And just like we wanted to avoid writing any API calls, we also wanted to avoid writing the SQL queries - this author is great at defining the logic, but self-admittedly, terrible at syntax. This was a perfect use of Claude.

We started with a set of 6 queries that addressed our key needs: post and notes performance, top content by a variety of measures (engagement, subscriptions, shares), subscription activity over time and by source and a rolling summary of all engagement. (Interested in the queries? Send us a DM). We saved the queries in DuckDB (where they were originally tested), documented them in a text file, and began to build an Excel file that could be refreshed weekly after syncing the data.

(Wondering if you can do this with Google Sheets? There are ways, but they’re clunky - the big constraint is that your DuckDB database is local. If you move it to the cloud the path is easier).

Some tips:

• StackContacts and DBeaver want exclusive access to the database while they’re running. So if you would like to run both at the same time, you will need to have DBeaver work in read-only mode.

• Have one place where you keep your master copy of your SQL queries. You can modify them in DuckDB, in text, and in Excel - and it’s easy to lose track of which is the master. Just choose one, and remember to always update that documentation.

Because we are still refining our queries, we test them in DBeaver, keep a backup in a master text file and copy them into Excel. You’ll see how that works below.

Step 4: Connect the data to Excel

We have long used Excel to manipulate and analyze data. In order to be able to run queries from inside Excel, we had to do a couple of one-time setup tasks that enable the connection:

1. Download and install the DuckDB ODBC driver from duckdb.org (free)

2. Configure the ODBC connection on my computer.

To set up the ODBC connection, click on the Windows key on your keyboard or click the Windows key + S to search. Type in odbc and select ODBC Data Sources (64-bit)

• Click on the SystemDNS, then click Add.

• Choose the DuckDB Driver, then click Finish.

• Type in a Data Source name and the path to your database

Now you’re ready to make the final link between Excel and your data.

3. Connecting to your database from Excel

Open an Excel spreadsheet. From the Data tab, click Get Data | From other sources | From ODBC and select the StackContacts data source.

When you select StackContacts from this list, you can either click on Advanced and paste in a query. Or click OK to navigate the tables, preview and load the data into a worksheet.

Once you establish the connection, there are a number of different ways to explore the data in Excel. Because we had already written our SQL queries, we went to a blank worksheet, connected to the database (following the steps above), pasted in a query and loaded the data in the tab. We did those steps for 6 metrics-focused queries plus one to remind us when the data had been synced.

You can view, edit and refresh all your queries by clicking Queries & Connections from the Data tab.

This queries and connections view is really useful on its own - by right clicking on any item, you have a number of actions you can perform - making Excel a suitable place to explore the data without something like DBeaver as an exploratory tool.

Step 5: Use the data to monitor progress and build a feedback loop

We use this Excel file to focus our discussions when we meet to develop a content calendar, refine our messaging, and decide what we want to test next - whether that’s new topics, formats, or ways of engaging.

While we sync our data weekly, we don’t review every metric on that cadence. Instead, we use the data when it’s most useful. Sometimes that’s weekly, sometimes monthly or higher when patterns take longer to emerge.

What matters is the data feedback loop:

• We look at what we did

• We look at how it performed

• We decide what to adjust

Then we repeat.

Having the data at this level of detail makes that possible. We can go back and isolate the impact of a specific post or outreach effort, compare across time, and test whether changes in what we write or how we share it actually make a difference.

When volumes are small, we aggregate to monthly or even quarterly views to get a clearer signal. Over time, this helps us move from reacting to individual results to recognizing patterns, and using those patterns to guide what we do next.

Step 6: Explore the data using AI

Earlier we described some pros and cons we experienced working with Claude. How you use AI will depend on your objectives, but the real value shows up once you understand your data well enough to ask better questions.

Here are a few ways we’ve used AI to extend beyond hands-on analysis:

Content strategy analysis

AI can help you step back and look at your content more systematically:

• Topic and theme analysis
  Have the AI classify your posts and notes into topics, then compare performance across those themes. Review and refine the categories - s a starting point, not a finished taxonomy.

• Format and channel performance
  Look at how format (short vs. long, structured vs. narrative) and channel (posts vs. notes) influence engagement. Different types of content often perform differently in each format.

• Timing and mix
  Identify periods of stronger performance and examine what you were publishing at the time - topics, formats, and frequency.

The goal here isn’t just to find “what works,” but to understand why it works and where you might want to experiment next.

Audience and subscriber analysis

AI is also useful for understanding how your audience is behaving over time:

• Engagement patterns
  Identify which subscribers are consistently active vs. going quiet, and track how engagement changes over time.

• Growth and re-engagement signals
  Look for moments where activity spikes. Did a particular post or note correlate with new subscribers? Who might need re-engagement?

Perfect attribution is less important than spotting directional signals you can act on.

Tips for getting the best out of an AI for analyzing data

Tell the AI what you already know about your data. As we saw when we experimented with Claude immediately after setting up the StackContacts integration - Claude will make reasonable assumptions, but we need to know the quirks of our own data. Surface those to the AI early and the analysis will immediately get sharper. That can include things like:

• How key fields are defined (e.g., free vs. paid subscribers)

• Which fields or tables to trust or ignore

• How you define engagement or success

• Any transformations you’re applying (e.g., weekly/monthly aggregation)

• The specific question you’re trying to answer

Ask the AI to check the data structure before analyzing it. Asking the AI to look at the columns and a sample of rows first. A prompt like “before you analyze, show me what’s in this table and confirm what each key column means” saves a lot of back-and-forth.

Be specific about what question you’re trying to answer. “Analyze my notes” is not a useful prompt for an AI. Questions such as “Which note topics get the most reactions, and does a short vs. long format change that?” provides a clear target.

Correct the AI when something doesn’t look right. For example, an initial topic analysis of notes was off because the AI was mixing original notes with replies and restacks. Flagging that immediately led to a much more accurate analysis.

We were learning while we were building, and we continue to refine our queries and our analysis. We’ve tried to capture our steps - and our missteps - here. If you have tips and ideas on how to do it differently, or if something isn’t clear, we would love to hear from you. Please comment or send us a message. We reply to them all.

Leave a comment

When a workflow feels clunky, it’s tempting to start looking for new tools - a slick new platform, a better tool, AI, the software equivalent of a fresh start.

But the real fix usually starts somewhere else: understanding what’s actually happening beneath the surface. If you skip that step, you’re likely to treat the symptom, not the cause.

It’s like replacing all your light fixtures when the problem is faulty wiring. Flashy fix, same old flaw.

Before you overhaul anything, pause. Assess what’s actually happening. Understand what you already have and what it really needs.

Step 1: Get clear on the goal

“The process is inefficient!”
“This platform is ancient - it’s time for an upgrade!”

Valid frustrations. But not goals.

A good goal gives you direction. It’s something clear - a specific, actionable outcome - that you can work toward and use as a gauge for whether a change is having an impact.

For example:

“We need to shorten the time between content submission and publishing.”
“We want to eliminate manual data errors.”
“With AI’s capabilities, let’s rethink how we deliver client value on this part of our service.”

Before making any changes, ask: What are we really trying to accomplish?
Speed? Accuracy? Better value? Less frustration?

If no one can give you a clear answer, your job might be to help shape that goal. You can’t fix a process - or choose a tool - if you don’t know what problem you’re solving.

Step 2: Map the people and the process

People are great at describing their pain points. The hard part is figuring out what is actually causing them. That’s where mapping helps.

Why mapping matters:

• It reveals disconnects and redundancies.

• It surfaces silent pain points.

• It gives you clarity - and credibility.

In siloed organizations, one team may offer a detailed view of a single step, but not how that step fits into the broader picture. But without the full context, you can end up solving the wrong problem entirely.

Tips for documenting workflows

If you’re lucky, your teams already have documentation. If not, or if it’s outdated, roll up your sleeves!

Try this:

• Shadow people as they perform common tasks.

• Make it clear you’re observing, not evaluating.

• Ask: “Who does this task? What happens next? Who touches what system, and when?”

• Diagram hand-offs. Make it visual. Review with the people who do the work.

Accuracy matters! These diagrams are not just for clarity. They become the foundation for tool evaluations or change management later on.

(Pro tip: Workflow diagrams are a great way to bring objectivity when operating in a low-trust or “finger-pointing” environment.)

Step 3: Analyze and quantify

Now that you have goals and workflows, dig into what you’ve learned.

Ask:

• Where does time pool up?

• Are there steps we can skip or streamline?

• Where do delays or mistakes happen most?

• Do certain approvals always slow things down?

Look for patterns:

• Bottlenecks

• Points of friction

• Risky or error-prone hand-offs

• Bright spots that could scale or add significant value

Focus on moments in the workflow where a small change could have a big impact. These are your biggest potential opportunities for automation.

Step 4: Pinpoint needs, priorities, and risks

With real data in hand, start identifying:

• What needs to change?

• What’s a process issue vs. a tool issue?

• What are the benefits and risks of each path?

With this information, you can evaluate your real options.

Example:
A research team pushed for a new data platform to fix persistent errors. But when they mapped the workflow, the real issue turned out to be inconsistent formatting in client-submitted spreadsheets. Research analysts and data operations were using different templates.

The solution? Align the templates. The errors disappeared. No new system required.

Takeaway: Sometimes the fix is smaller, cheaper, and right under your nose.

The real cost-benefit calculation

Automation projects require money, time, and people. But most teams only ask one question:

“Is the benefit worth the cost to build?”

That’s too simple.

When you evaluate automation, you need the full picture:

a) Build cost (what everyone sees)

• Development time

• Tools / platforms

• Internal or external resources

b) Implementation cost (what gets missed)

• Training your team or clients

• Rolling out new workflows

• Supporting adoption across the organization

c) Opportunity cost (what no one talks about)

• What are your people not doing while this is happening?

  For example, a sales team learning a new forecasting automation tools isn’t meeting prospects. That’s pipeline you’re trading for process.

Don’t get us wrong - automation can absolutely drive value. But only when you weigh it against everything it displaces.

Too often, teams underestimate effort, overestimate speed to value, and ignore the hidden costs of change.

A better question to ask.

Instead of:

“Should we automate this?”

Ask:

“Is this the highest-value investment, right now?”

Step 5: Get ready to recommend and present

By now, you have done something powerful: not only have you identified a problem, but you have built shared understanding around it.

When you present:

• Don’t go for a dramatic reveal. Show how you arrived at your recommendations.

• Share your findings to test understanding.

• Walk through assumptions and risks.

• Invite feedback. Clarify tradeoffs. Be honest about what is still uncertain.

Frame your presentation as a shared discovery, not a pitch.

What makes you invaluable

You don’t need to be an AI or systems expert to lead this kind of change. Curiosity, persistence, and thoughtful observation go a long way.

The hard part is taking the time to understand what’s really happening. It’s the step many teams skip, yet makes the difference between a failed automation effort and a lasting one.

This kind of attention doesn’t just improve processes. It builds lasting value, clarity, and alignment. And that’s what makes your work truly invaluable.

Leave a comment

More Reading: The “AI at Work” Series

• The Human Factor

• Frame the Problem

• Why Data Inputs Matter

• When to Trust, Adapt, or Toss AI Outputs

• Translating AI Results into Decision and Action

• Using AI in a Lean Marketing Machine

There are moments in almost every career where you realize you’re a little out of your league. You’re expected to have an answer, but you’re not entirely sure what’s actually going on.

Maybe it’s your first role and you want to contribute something meaningful. Or you’re stepping into a new team or business and trying to get your bearings. Or you’re running your own operation and every decision feels high stakes.

Data should help in these moments, providing clarity and direction.

But for many people, it does the opposite. The dashboards look polished and the reports look definitive. Yet there’s a worry: What am I looking at? What does this number represent? What if I’m wrong?

So we rely on analysts. We lean on tools. We increasingly turn to AI.

All of these help, but they don’t solve the core problem. If you’ve only seen the final numbers - and not what they’re built from - it’s hard to feel confident acting on them.

What we’ve found is simple: confidence doesn’t come from better tools or delegating. It comes from spending a little time with the data itself looking at real rows, real customers, real events.

In other words, you have to touch the data.

This feels counterintuitive. With all the technology available, it seems like we should be able to skip this step. But you can’t outsource that familiarity. A polished dashboard or AI output can look convincing, but if you haven’t seen what it’s built on, it’s hard to fully trust or interpret it.

We wanted to understand why this step matters so much. So we looked at how people actually learn. It turns out there are three well-established principles that explain what’s happening.

Share

Why touching the data works: three learning principles

1. People learn through concrete examples, not abstractions

Data is abstract. Summaries and dashboards make it even more so. But learning science shows that people understand concepts better when they start with concrete examples and then move to abstraction - not the other way around.

A marketing professor we know teaches a well-known behavioral economics concept about how people overvalue what they own. For years she explained it through a lecture. Students were skeptical — it just didn’t ring true. So she started running a simple experiment: she gives half the class a Kit Kat, asks them how much they’d sell it for, and asks the other half how much they’d pay. Almost every time, the Kit Kat owners demand more than buyers are willing to pay. The concept sticks because students experience it firsthand.

We are wired to learn this way. We remember what we can see, experience, and interact with.

Looking at the data itself works the same way. When you’re looking at actual customers, transactions, and events, you see the building blocks before it gets rolled up into a summary. Once you’ve seen that, the dashboards start making a lot more sense.

2. Better judgment starts with understanding what’s underneath

Research on expertise shows that people who understand how something is constructed make far better judgments than those who only see the final output. Experts organize knowledge around core concepts and relationships; novices rely on what’s visible on the surface. It’s like the physics student who memorizes formulas versus the one who understands the underlying forces. One will have context that sharpens his or her answer significantly.

The same applies to data. Dashboards summarize. AI spots patterns. Neither tells you about the data below it and that gap matters more than most people realize.

Take a simple example: “Average revenue per customer is $40.” That number could reflect two very different scenarios:

• 1,000 customers all paying between $35–$45 (stable, predictable)

• 1,000 customers ranging from $4–$75 (highly variable)

The dashboard shows the same number either way. Seeing the data itself gives a foundation to better understand what that information means and what it does not.

3. Learning and “transfer” (a.k.a. apply what you’ve learned elsewhere)

Research shows that people apply what they learn to new situations more effectively when they understand the underlying principles and structure, not just the outputs.

This means that when you’ve spent time with the data itself, you understand how it is organized, detailed, and start to recognize familiar patterns. You can then apply this when similar data shows up in different reports and contexts.

Instead of starting from scratch each time, you can get oriented faster because you recognize the building blocks. You have seen before how this kind of data works.

Share

What about analysts and AI?

Analysts, dashboards, and AI tools are genuinely valuable. We use them. But they can’t replace your own grounding in the data.

Analysts bring expertise, but not your day-to-day context. They don’t know what feels off in the business right now. And their analysis is always shaped by their own experience and framing.

AI has a different limitation: it can produce excellent analysis, and it can also miss context or make incorrect assumptions. Sometimes it’s hard to tell which is happening.

If you’re not confident with the data yourself, you’re exposed to both of these limitations. You’re relying on someone else’s interpretation without a reliable way to evaluate it.

Even a small amount of time with the underlying data changes that. It gives you enough familiarity to work with analysts and AI tools more effectively, rather than simply deferring to them.

Why this matters

In a world full of dashboards, reports, and AI-generated answers, it’s easy to skip the step of actually looking at the data. Instead choose to actually understand the data. That small step in looking at real rows, real customers, real activity is what builds skill and confidence.

When you’ve seen the data, you build your understanding to judge when an analysis is overreaching, when something looks off, or when a result doesn’t quite line up with reality.

The three learning principles above explain something we’ve observed repeatedly: people who spend even a little time with the data develop a fundamentally different relationship with it. They are not intimidated or afraid. They are more confident and end up using the data well.

Getting started: building confidence with data

Here’s a practical sequence to work through with your team.

1. Choose one report tied to a real question you care about like revenue, leads, customers, usage, donations, support tickets.

2. Export the data. Look for “Export” or “Download.” A CSV file type is usually the easiest place to start.

3. Open it in Excel or Google Sheets. If it opens as a CSV, save a working copy as an Excel file.

4. Set a 15- or 30-minute timer and get curious. Keep a notepad nearby for observations and questions. The goal is familiarity, not a full analysis.

   • Start by scanning the columns. What does each field appear to represent? What looks complete or incomplete? Are there blanks, duplicates, or labels that seem inconsistent?

   • Sort one column at a time - by highest and lowest values, alphabetically, by date. This is the fastest way to spot ranges, outliers, and patterns that disappear in a summary.

   • Filter the data to ask simple questions. Look at one segment at a time: one month, one region, one product type. Move from “What is this report saying?” to “What is actually happening here?”

   • Do one basic summary. Add up a column, calculate an average, count rows in a category. Even simple calculations connect the row-level detail back to the bigger picture.

   • Build one pivot table. If you haven’t used one before, a quick YouTube tutorial will get you there in minutes. Put one category in the rows, one metric in the values, and see what shifts — product line by revenue, region by customers, month by support tickets.

5. Review your notes. What did you notice? What surprised you? What would you now ask an analyst, colleague, or AI tool?

Leave a comment

Share

Subscribe now

Some references

1. Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Harvard University Press.

2. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152.

3. Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school (Expanded ed.). National Academies Press. https://doi.org/10.17226/59.

More Reading

Most people think of data problems as typos, duplicates, and broken formats. You’ve probably run into it yourself: you open a dataset expecting to run an analysis, and quickly realize you can’t sort it, filter it, or trust the totals.

But not all issues show up that clearly. Ask five people in your organization what “active customer” means, and you’ll get five answers. Maybe six. And every one of them will be right based on the data they’re looking at.

Someone will say it’s anyone who made a purchase in the last two years. Someone else will say it’s anyone who logged in this month. Marketing will count qualified leads. Finance will only count currently paying subscribers. And the person who built the dashboard three years ago? They’re not quite sure anymore, because the logic changed and it was never documented.

Now you have two different problems: data that doesn’t work when you try to use it, and data that seems clear but means different things to different people.

Both show up all the time, and both can quietly derail your analysis. And as more teams rely on AI tools to summarize, transform, and act on data, these issues don’t go away - they get harder to spot and easier to scale.

So what makes data “clean”?

Clean data is data that is consistently understood and reliably usable when it matters.

It doesn’t have to be perfect or exhaustive. It does have to be trustworthy for the job at hand and interpreted the same way by the people using it.

Two things tend to go wrong:

• Meaning problems: what does this actually represent?

• Quality problems: is this usable and consistent?

Both matter. Most teams only focus on one.

The data problems you can actually see

These are the kinds of “messy” data you can spot right away - what most people think of when they hear “dirty data.” You open a customer list expecting to run a quick analysis. Instead, you get this:

At a glance, it looks fine. But look closer:

• Date formats don’t match

• Status values don’t match

• Revenue is stored inconsistently and is undefined

• One record has no customer ID

• One customer appears twice

Teams focus on this because it’s visible, and because it will actively break your analysis if you don’t fix it first.You can’t reliably:

• Sort or group by date: 03/04/2024, 2024-04-03, and April 3rd won’t sort or compare without standardizing first.

• Filter categories: Active, active, and ACTIVE look the same to you. They aren’t the same to a database.

• Sum revenue: mix currency symbols with plain numbers and you’ll get errors or wrong totals.

• Join to your customer database: a missing ID isn’t just blank. It’s a record you can’t link.

• Trust your totals: Customer 1042 appears twice with different data. Which is right?

Even simple questions like “how many active customers do we have?” can give you the wrong answer because the data doesn’t line up.

This kind of mess is frustrating, but it’s also fixable. Most of it comes down to standardizing formats and making sure fields are structured consistently.

The usual suspects: messy data in the wild

Once you know what to look for, these patterns show up everywhere:.

Even if you clean all of this up - standardize formats, fix duplicates, fill in missing fields - you can still end up with the wrong answer if a more basic question hasn’t been answered: what does “active customer” actually mean?

When the same data means different things

This is easier to miss: the data looks clear, but people are using it differently.

We dug into this last week, but you’ve seen it:

• Revenue → gross or net? recognized or booked?

• New customer → signed up or paid?

• Traffic → sessions, visitors, or page views?

• Active user → depends who you ask

None of these are unreasonable. They’re just not defined consistently. So two people look at the same report, both are “right,” and still walk away with different answers.

Quick rule of thumb: if you don’t know the definition, you don’t know what you’re looking at.

Share

Clean vs. messy: five questions to ask yourself

You don’t need to audit every field in every report to catch most data quality issues. But you do need to slow down and ask these questions before acting on the data.

1. Do you know what this field is measuring - and would your colleagues agree? If you had to explain this metric out loud, could you? If the answer is “sort of,” that’s your sign to pause and clarify before you go further.

2. Are there values that look the same but are formatted differently? Scan the unique values in a column. Variations in capitalization, spacing, abbreviations, or symbols can affect filtering and totals. If they should be the same, standardize them.

3. Does the total hold up if you check it a different way? If your report says you have 500 active customers, try counting a different way. Pull raw data, check another system, or ask someone else to run it. If the answers don’t match, take a closer look.

4. Do you know where the data came from and how fresh it is? “I got it from the dashboard” is not the full picture. What is the source? How often is it updated? Who owns it?

5. Are you clear who is included in a category or type? If you filter by “active customers,” could you explain exactly who qualifies? If not, the data isn’t ready to drive a decision.

Subscribe now

Clean data in practice

Clean data isn’t about getting everything perfect. It comes down to two things: shared meaning and data you can actually use. Everyone understands the data in the same way, and when you use it, it supports the analysis or decision you’re trying to make.

Formatting and consistency issues make your data hard or impossible to analyze. You can’t sort it, filter it, join it, or trust the totals. Definition issues are quieter, but just as important. They lead to inconsistent assumptions, miscommunication across teams, and decisions that look right but aren’t grounded in the same reality.

The good news is that both are fixable.

On the technical side, establish standards (consistent date, currency, and number formats), clean up duplicates, and structure fields so data is entered reliably (dropdowns, required fields, validation rules). You can also ensure each field contains a single piece of information, choose a clear system of record, and standardize data before it’s used in reporting.

On the definition side, get clear on what key metrics actually mean and document those choices so they hold over time. In practice, that means defining metrics in plain language, pressure-testing them across teams, and making sure everyone is using the same definition when decisions are being made.

Your data doesn’t have to be perfect, and you don’t have to solve everything at once. But it does require a habit: taking the time to make sure you and your team agree on what you’re looking at - and that the data will support the analysis or decision you’re about to make. As AI becomes part of more workflows, these issues don’t just affect reports - they shape the outputs you rely on.

Leave a comment

Related articles from We Dig Data:

We see numbers all the time at work. We track progress, allocate resources, consume research, and pitch ideas. But how well do we actually understand what we’re looking at?

If you’re making decisions or supporting the people who do, you are accountable for the number and for understanding what is behind it.

Today we’re talking about how to critically shine a light on data you use to make decisions at work. We’ll cover:

• Why to revisit what you think you know about your data

• How to determine data’s strengths and limits by looking at its source

• Uncovering hidden assumptions

• Balancing the decision against the risk, ambiguity, and bias in the data

This isn’t about becoming a data analyst. It’s about strengthening the skills to size up data and then use it to confidently choose a path forward.

Share

Get to know your data

Most misunderstandings begin here. We too often believe that we are clear on familiar metrics…until someone asks us to explain them. That’s when we realize the gaps.

When we don’t fully understand what the data represents, we draw incorrect conclusions. Everyday sources of confusion include:

Familiar terms that are used loosely. Customers and users. Sales and revenue. These sound interchangeable, but aren’t, and the data can show up differently. For example, let’s say you sell a $1200 annual subscription. Sales will show up the month the sale closes, but the business recognizes revenue evenly over the duration of the subscription.

One label, many definitions. Metrics like “traffic” sound intuitive, but have many meanings. “Traffic” can mean page views, sessions, visitors, or unique visitors.

Missing context. A percentage or rate change sounds clear until you ask: from what to what? Statements like: “Lead conversion rates have grown 10%!” have little meaning until clarified into: “Last year, the internal sales team converted 10 out of every 100 website leads into a sale. This year, they are converting 11 out of every 100.”

Vague or inconsistent classifications. A report documents “adverse reactions” for a new drug. But what does that mean? A mild rash or hospitalization? Definitions and classifications vary by industry or even by department in the same organization.

Even if a term seems obvious, confirm that what you think the data represents is accurate. Clarify and define, don’t assume.

The source shapes your interpretation

Where the data came from reveals its strengths and its limits. This context will determine how much trust you put into a number’s accuracy and reliability.

For EXTERNAL data sources, ask:

1. Who published the data? Established, credible sources - like census data or a reputable research firm - are generally more reliable, especially when the organization specializes in data and has a verifiable track record. In contrast, vendors or lobbyists may provide research, but they also have incentives that could bias how results are framed or interpreted.

2. Where do they get their data? How did they collect it? Does the data adequately represent the population being measured? What is the sample size? Strong data sources explain how they collected their data and acknowledge limitations.

3. Is the methodology available? How information is collected, defined, and calculated determines whether it is actually relevant to your situation. Credible sources will usually explain how they collect and analyze their data and what limitations apply. If someone dodges your questions, be cautious.

For INTERNAL data, ask:

1. What is the data source? Some systems cover only part of the business when you are trying to understand the whole. Others may have known quality issues when you need greater accuracy.

2. Is it a recurring report or a one-off request? Ad-hoc reports require more scrutiny. Regular reports have usually been stress-tested and refined over time.

3. Who built the report? Different departments bring different lenses. Finance may be more conservative than Sales. Marketing may define ‘customers’ differently than Operations. Each team has different data access, expertise, and perspectives. These factors influence what gets measured and how it gets presented.

No data is perfect, but solid data can withstand scrutiny.

Uncover assumptions behind the data

Every number is built on choices. Some choices are clear (like a time frame), but many are invisible unless you ask. Let’s start with a few that cause the most trouble:

The sample population. Don’t assume a data point represents the full picture. If a survey on fried chicken sandwiches is conducted with 1,000 college students, it doesn’t represent national preferences - it represents college student preferences. So always ask: Who’s actually included in this data? And who’s missing?

Projections and estimates. These are built on assumptions, and you may not agree with the approach nor the level of risk represented in those choices. For example, a revenue forecast will include assumptions like: average sales price, expected growth or decline, revenue from new clients vs. existing clients or new products vs. existing products. Don’t hesitate to have these discussions. They are usually an invaluable step for refining estimates and projections.

Check the math. You don’t need to audit everything, but a quick spot check often catches errors before you get too far. A 2025 Canva survey reported that 57% of respondents said they make spreadsheet errors that impact their work. With numbers like that, it’s not worth just assuming the math is right.

Data skeptics remember to probe on these areas because they’ve learned (often the hard way) how easy it is to act on a false assumption.

“Good enough” is a real answer

You won’t always have perfect information. The question is whether it’s good enough for the decision you are making.

Match the scrutiny to the stakes. Low-stakes decisions, like an A/B test tweak, may not require bulletproof data. High-impact decisions, like a large investment or launching in a new country, warrant more scrutiny.

Watch for false precision. Specific numbers can signal more certainty than actually exists. For example: “40.2% of the population will go on vacation this summer, up from 39.8% last summer.” But if these are based on two consumer surveys of 100 people each, that extra decimal is indicating a level of certainty that does not exist.

Treat surprises as signals. If something looks off - up, down, or just unexpected - pause. Surprises are your cue to investigate. It might be a real shift. It might be a data issue. Either way, it’s worth understanding.

Ranges can be good enough too. If you have a small client base, it could be good enough to use a small range when analyzing annual client survey data. “Approximately 3%-5% of clients plan to increase their marketing budgets, which is about the same as last year.” Being directionally right is sometimes enough, but you need to know that going in.

Spot the bias

Information is produced and interpreted by people. Everyone has a lens, and sometimes an agenda. This doesn’t make their data wrong, but it does mean you should consider that bias in how you apply that information.

Motivation matters. Incentives shape how data is framed. That doesn’t make it wrong, but it does affect how it’s presented. A pharmaceutical company wants positive results for a new drug or product. A salesperson’s forecasts are influenced by how they are compensated. From students to policymakers, people interpret data through their lens. Know the agenda, and factor it in.

Over-generalizing. When data is missing or hard to find, it’s tempting to apply one piece of information to perceived similar situations. But what is true for one group, market, or channel may not apply more broadly. Instagram users do not necessarily represent all social media users. New York City restaurant trends don’t predict what’s hot in Texas, and the top-selling toys in the U.S. won’t match those in France.

History doesn’t always predict the future. Past patterns can break. A competitor has a breakthrough moment. There’s a technology shift, a viral moment, or a market trend that quietly hits a tipping point. ‘It’s always been this way” is an assumption, not a fact.

Why data skepticism matters

Data is never just numbers. It’s a set of choices about what to measure, how to frame it, and who it represents. Your job isn’t to audit every figure. It’s to stay curious enough to ask the right questions, and confident enough to push back when something doesn’t add up.

Leave a comment

Share

Subscribe now

Additional Reading

Here are two excellent resources from fellow Substack writers if you want to go further on assessing external resources:

• , author of Card Catalog, recently wrote a guide to evaluating information sources in the AI age.

• , author of Slow AI, wrote about how to spot a fabricated source and created a game, Dead Reference, to test your skills.

More related articles from :