Less is More: Accelerating AI with Advanced Data Strategies

Last mont, I had the chance to attend a talk at the Barcelona Supercomputing Center (BSC), where NVIDIA researchers Nadine Change and José M. Álvarez presented their work on data-centric strategies for accelerating AI.

The session, title “Less is More: Accelerating AI with Advanced Data Strategies”, touched on something that feels increasingly urgent in today’s AI landscape: the recognition that bigger datasets and larges models aren’t always the optimal path forward. Instead, the researchers argued, we can achieve more with less, if we learn to be smarter about how data is curated, filtered, and fed into training pipelines.

Setting the Stage

Over the past few years, the dominant trend has been simple: collect more data, train bigger models, and scale up compute. This has worked remarkably well in pushing forward the capabilities of large language models, diffusion models, and beyond.

But this approach also comes with growing downsides:

• Soaring training costs in GPU hours and electricity.
• Environmental concerns from massive energy consumption.
• Data redundancy, where models are repeatedly exposed to uninformative or duplicated samples.
• Accessibility gaps as only a handful of organizations can afford to train frontier-scale systems.

The BSC talk addressed these issues head-on asking: what if smarter data management could give us faster, cheaper, and even better AI?

Core Idea

Instead of treating datasets as a static resource, the NVIDIA researchers framed them as dynamic, adaptable components of the training loop. The central thesis was clear

We don’t always need more data. We need better, smarter data.

THis shift from a model-centric to a data-centric perspective has been growing in the ML community, but the talk provided concrete strategies to operate it at scale.

Techniques Discussed

Smart Data Sampling

Not every example in a dataset contributes equally to learning. The researchers showed methods for prioritizing high-value samples, those that correct errors, challenge the model, or diversify its understanding.

• This reduces the volume of training data without reducing effectiveness.
• Sampling can adapt dynamically: once a model learns a pattern well, redundant examples can be downsampled.

Adaptive Curriculum Design

Borrowing from human learning, curriculum strategies introduce training samples in a progressive structured order.

• Early training focuses on a simpler or foundational examples.
• More difficult or rare examples are introduced later, once the model has the capacity to benefit from them.

This accelerates convergence and helps avoid wasted compute on data the model isn’t yet ready to learn from.

Model-Aware Data Filtering

Instead of blindly trusting dataset quality, the filtering process actively considers the current state of the model.

• Examples the model already performs well on can be deprioritized.
• Outliers or overly noisy samples can be removed before they destabilize training.
• Architectures themselves matter: filtering strategies that benefit one model may not transfer to another.

Redundancy reduction

In massive datasets, near-duplicate samples are common. Removing redundancy not only saves training cycles but can also reduce overfitting. Techniques like clustering or embedding-based similarity detection help identify such overlaps.

Technical Benefits

From a systems perspective, the gains are substantial:

• faster training: By reducing low-value samples, HPU hours are used more effectively.
• Improved generalization: Diverse, carefully curated data promotes models that perform better in real-world settings.
• Lower infrastructure costs: This matters not just for companies but also for research centers like BSC, where shared resources are precious.
• Sustainability: Cutting unnecessary computation lowers carbo footprints, making AI greener.

Why it Matters for the Future of AI

The ley takeaway for me was that efficiency is becoming as important as capability. The age of brute-force scaling is giving way to a more balanced perspective, where data intelligence plays crucial role.

This shifts could help:

• Democratize AI research: Smaller labs could achieve competitive results without billion-dollar budgets.
• Improve reproducibility: Curated datasets make it clearer why models behave the way they do.
• Align incentives: Sustainable AI practices align not only with research goals but also with societal concerns about resource usage.

Reflections

What struck me most was the practicality of the ideas. These weren’t distant speculations about future AI, but concrete strategies that can be applied today in training pipelines.

It reminded me of a broader trend in AI research: moving beyond raw horsepower and starting to engineer efficiency, robustness, and sustainability into our systems. As models continue to scale , these principles will likely separate the organizations that can innovate responsibly from those stuck in an arms race of size.

Closing Thoughts

The BSC-NVIDIA talk was a timely reminder that more isn’t always better. By putting data back at the center of the conversation , we can unlock pathways to AI that is not only more powerful but also more efficient, sustainable, and inclusive.

For anyone working in machine learning, whether in academia, startups, or large-scale industry, these strategies are worth exploring. Smarter data could be the hidden accelerator we’ve been looking for.

What do you thin about the “less is more” philosophy in AI? Have you tried techniques like adaptive sampling or curriculum learning in your own work? Do you believe data-centric training could become standard practice, or will brute-force scaling remain dominant?

I’d love to hear your thoughts, drop a comment , share your experiences, or let’s start a conversation about building a more efficient AI future.