Data science
« Back to Glossary Index
Data science is the practice of extracting insight and building predictive capability from data, combining statistics, programming and domain knowledge. In a business setting its output is usually a model that supports or automates a decision, rather than a report describing what happened.
The discipline has matured considerably. A decade ago the constraint was building a model that worked. Today models are comparatively easy to build and the constraint is everything around them — data readiness, deployment, monitoring and whether anyone changes what they do as a result.
What Is Data Science?
Data science applies scientific method to business data: forming a hypothesis, testing it against evidence, and quantifying how confident the result allows anyone to be. That last element distinguishes it from analysis generally — a data science output carries an estimate of its own uncertainty.
The work spans three areas that all have to be present. Statistical and modeling capability to choose appropriate methods and interpret results honestly. Engineering capability to access, shape and process data at realistic scale. Domain knowledge to know which questions matter, which features are plausible, and when a result is too good to be true — usually indicating leakage rather than insight.
Teams weak in the third area produce technically sound models that solve the wrong problem, which is the most expensive failure mode because nothing looks wrong until deployment.
Data Science vs. Analytics vs. Machine Learning
Analytics examines what happened and why, working with observed data to explain and describe.
Data science is broader and includes estimating what has not been observed — predictions, probabilities, causal effects — with uncertainty attached.
Machine learning is a set of techniques data science uses, where a model learns patterns from data rather than being programmed with rules. Not all data science uses machine learning; a well-designed experiment or a causal analysis may use neither.
The relationship to AI is one of overlap rather than hierarchy. Machine learning is the main engine of contemporary AI, and data science is the practice that applies it to business problems. In practice the terms are used loosely, and the distinction matters mainly when scoping what a team is actually being asked to deliver.
The Data Science Process
1. Frame the problem. Translate a business question into something answerable with data, and establish what decision the answer will inform. Projects that skip this produce interesting findings nobody uses.
2. Understand and prepare the data. Consistently the largest share of effort — cleaning, joining, handling missing values and constructing features. Teams new to the field routinely underestimate this by a wide margin.
3. Explore. Examine distributions and relationships before modeling. This is where data problems surface, and where domain knowledge catches implausible patterns.
4. Model. Try approaches, tune, and validate on data the model has not seen. Simple models that are well understood frequently outperform complex ones in production because they fail predictably.
5. Evaluate against the business question. Accuracy alone is rarely the right measure. A fraud model at 99% accuracy that misses most fraud is useless, because the base rate makes accuracy meaningless.
6. Deploy and monitor. Integrate into a workflow, track performance, retrain when it drifts. Where most of the remaining effort actually sits.
Skills and Team Structure
The expectation that one person combines statistics, engineering, domain expertise and communication describes very few real people. Effective teams distribute these.
Data scientists frame problems and build models. Data engineers build the pipelines that make data reliably available — frequently the binding constraint on how much a data science team can deliver. Machine learning engineers take models into production and keep them running. Analytics translators or embedded domain experts connect the work to business reality.
On structure, fully centralized teams build depth but drift from business problems; fully embedded ones stay relevant but duplicate effort and lose technical standards. A hub-and-spoke arrangement — central standards, tooling and career path, with scientists embedded in business areas — is where most organizations of scale settle. Getting this right is as consequential as any hiring decision, which is why our data science and machine learning engagements address operating model alongside delivery.
From Notebook to Production
The gap between a working model and a production system is where most value is lost. A model in a notebook produces no business outcome whatever its accuracy.
Production demands things exploratory work does not: the same feature calculations at training and inference time, so the model sees consistent inputs; latency appropriate to the decision, which differs enormously between a real-time fraud check and an overnight batch; monitoring of input distributions and prediction quality, because silent degradation is the norm rather than the exception; a retraining path with someone accountable for triggering it; and versioning of both model and training data so a past decision can be explained.
These are engineering problems rather than modeling problems, and teams composed entirely of modelers tend to be unprepared for them. Budgeting the majority of a project’s effort for this phase is realistic rather than pessimistic. Broader AI services capability generally lives here.
Why Data Science Projects Fail
No decision attached. The model predicts accurately and no process exists to act on the prediction.
Data not ready. The project stalls in preparation because the data foundation assumed to exist does not.
Target leakage. A feature encodes the outcome, producing outstanding validation performance and failure in production. Suspiciously good results warrant investigation rather than celebration.
Never deployed. Work concludes with a presentation rather than a system.
No ownership after launch. The team moves on, performance decays unnoticed, and decisions are made on a degraded model for months.
Getting models into production depends on the pipeline layer beneath them — building AI-ready data pipelines covers what that requires.
Frequently Asked Questions
Data science is the practice of extracting insight and building predictive capability from data, combining statistics, programming and domain knowledge. Its business output is usually a model supporting or automating a decision, with uncertainty quantified.
Analytics examines what happened and why using observed data. Data science is broader and includes estimating what has not been observed — predictions, probabilities and causal effects — with an explicit measure of confidence attached.
Machine learning is a set of techniques data science uses, where models learn patterns from data instead of being programmed with rules. Not all data science uses machine learning — a well-designed experiment or causal analysis may use neither.
Frame the business problem, understand and prepare the data, explore distributions and relationships, build and validate models, evaluate against the business question rather than accuracy alone, then deploy and monitor with a retraining path.
Statistical and modeling capability, engineering capability to handle data at scale, and domain knowledge to know which questions matter. Expecting all three in one person is unrealistic — effective teams distribute them across data scientists, data engineers and ML engineers.
Because it ignores base rates. A fraud model can be 99% accurate while missing most fraud, since the overwhelming majority of transactions are legitimate. Evaluation should measure what the business decision actually depends on.
Target leakage happens when a feature encodes the outcome the model is meant to predict, producing excellent validation results and failure in production. Unusually strong performance should prompt investigation rather than confidence.
Most often because no decision was attached to the output, the data foundation was less ready than assumed, target leakage went undetected, the model was never deployed, or nobody owned it after launch and performance decayed unnoticed.
