Is This Data Worth It? How to Vet a New Feed Before You Leap
New data sources can be shiny, exciting, and yet, totally wrong for your problem. This is a practical method to quickly evaluate new data feeds. No SQL required.
👋 We Dig Data helps non-technical professionals build practical data knowledge. We cover what your boss wishes you knew about data.
Why a New Data Feed?
New data feeds can sound like magic, bringing better products, more insights, or stronger operational results. And when they work, they really deliver.
For example, grocers use third-party product databases to enrich online shopping with nutrition information and photos. Libraries use metadata feeds (data describing other data) to organize digital and physical collections. Sales teams pull in company and contact databases to improve lead quality. Marketing tracks social media to monitor brand sentiment.
The idea is simple: integrate a new data feed and unlock better, faster decisions. And with so many providers out there - government sources, niche vendors, even companies selling their own internal data - it’s tempting to think that more data = better results.
But here’s the truth: just because a data feed is available doesn’t mean it’s useful. And not all data feeds are created equal - there are wide discrepancies between providers in terms of quality, completeness, and consistency.
We’ve seen teams spend months on sources that looked promising, but didn’t deliver. And we’ve made those mistakes ourselves.
That’s why we use this quick 3-step process when beginning to look at any new data feed. It’s a simple, no-jargon method to quickly sort out whether a new data source is worth your organization’s time before you kick off a deeper evaluation effort.
Step #1: Make it Real. Where Does the Data Come From?
The dataset is a record of something happening in the real world. What event or behavior does it capture?
Some examples of data by source:
Credit card data: Your monthly statement lists store name, a date, and total amount per transaction - but not what specific items were bought.
Retail receipts: These include the store name and location, a list of items you purchased, the prices, and the time and date of the transaction.
Product databases: Online product pages or physical products often include specs like size, materials, price or country of manufacture.
Spend 15 minutes sketching what this data could include. Grounding in the real world helps you spot both possibilities and limits.
Step #2: Ask for a Sample (Even Just One Row)
Before you commit serious time or resources on a full data sample, start with a sneak peek. Ask for just a few rows of data. Even dummy (fake) records will do the trick.
Here’s why: if the data isn’t a fit, this small step will save you (and your team) hours of analysis and back-and-forth. A full dataset sample often requires technical support, legal agreements, or extra lift from procurement. You don’t need all that - yet. So start light.
Using the example above, you quickly get a feel for what’s included, and what’s missing.
You can see where someone shopped (Target) and how much they spent (76.43).
You might be able to track their shopping habits over time using the user_id.
You can’t see what items they bought.
You don’t get names, contact information, or personal demographics.
Now, compare your sample to what you imagined in Step 1.
Ask yourself:
Are any fields (columns) missing that you assumed would be there? (e.g., store address or store ID) Jot those down as follow-up questions.
Are there fields you didn’t expect? Ask how they were created or collected.
Are any column headings unclear? For example, the zip code field could be the shopper’s billing information or the store’s location.
Pro tip: If it’s an established vendor, you can also request a data dictionary, which explains each field, what it means, and how it’s formatted.
You now have a much clearer picture of what’s in the data, what’s missing, and what still needs clarification.
Instead of making assumptions, you’re working with real examples and real questions to bring back to the data provider.
Step #3: Check for Data Fit
You’ve visualized the data. You’ve reviewed a sample. Now it’s time for the big question:
Can this data actually help solve the problem I care about?
To figure that out, take a few minutes to weigh the strengths and limitations of the dataset in the context of your specific use case.
Start with two questions:
What does this data make possible?
What’s missing, unclear, or a distraction?
Here’s what we’d note down regarding the credit card data sample above:
This doesn’t need to take a lot of time - just jot down what stands out.
Then ask:
Do the strengths help move the needle on my goal?
Are the limitations critical or can I work around them?
You’re not looking for perfect data. You’re looking for a good enough fit. And this quick gut check can bring surprising clarity.
If you can’t clearly see how this data will help solve even a meaningful part of your problem, that’s your cue to pause - or move on entirely.
Because here’s the reality: integrating a new data feed only gets trickier from here. If the early signs aren’t strong - if the value isn’t obvious, or the fit feels shaky - it’s not going to get easier down the line.
This probably isn’t the right dataset or the right provider for you. And that’s okay.
We’ve seen too many teams pour months of effort and budget into the idea of a dataset, hoping it will pay off, only to realize too late it was never going to deliver what they really needed.
But if the strengths line up and the gaps are manageable? That’s your green light to keep going. (We’ll walk you through those next steps in Part II.)
Let’s Recap
You don’t need to be a data expert to make smart decisions about data.
Before you invest time, energy, or budget, start with these three quick steps:
Step 1: Visualize where the data comes from.
Think about the real-world event being captured.
Step 2: Get a sample.
Even a few rows can reveal a lot - don’t skip this step.
Step 3: Check for fit.
Look at what the data can and can’t do for your specific use case, and then make a clear call on whether it’s worth investing time and resources into deeper evaluation and negotiation.
That’s it. You’re already ahead of the curve.
Coming Up
You have decided to move forward and do a deeper evaluation. Now what? In Part II, we’ll walk you through the follow-up questions to ask your data provider to pressure-test the details, validate the fit, and surface any red flags early.
Still no code. Still no jargon. Still no wasted time.
You’ve got this.
Was this helpful?
Click the ❤️ below. This helps other readers find this content and lets us know what resonates.
Also might want to look at meta-data, such as when data was updated (on your product data example). In one of my startups we measured how many products on shelf, differed from the retailers online description and found about half differed in some way.