Hcfyk

Is there a conceptual or procedural difference between fitting a model to data and fitting data to model? An example of the first wording can be seen in https://courses.washington.edu/matlab1/ModelFitting.html, and of the second in https://reference.wolfram.com/applications/eda/FittingDataToLinearModelsByLeast-SquaresTechniques.html.

Pretty much every source or person I've ever interacted with except the Wolfram source you linked refers to the process as fitting a model to data. This makes sense, since the model is the dynamic object and the data is static (a.k.a. fixed and constant).

To put a point on it, I like Larry Wasserman's approach to this. In his telling, a statistical model is a collection of distributions. For example, the collection of all normal distributions:

$$ { text{Normal}(mu, sigma) : mu, sigma in R, sigma > 0 } $$

or the set of all Poisson distributions:

$$ { text{Poisson}(lambda) : lambda in R, lambda > 0 } $$

Fitting a distribution to data is any algorithm that combines a statistical model with a set of data (the data is fixed), and chooses exactly one of the distributions from the model as the one that "best" reflects the data.

The model is the thing that changes (sort of): we are collapsing it from an entire collection of possibilities into a single best choice. The data is just the data; nothing happens to it at all.

In the field of Rasch modelling it is common to fit the data to the model. The model is assumed to be correct and it is the analyst's job to find data which conform to it. The Wikipedia article on Rasch contains more details about the how and the why.

But I agree with others that in general in statistics we fit the model to the data because we can change the model but it is felt to be bad form to select or modify the data.

Typically, the observed data are fixed while the model is mutable (e.g. because parameters are estimated), so it is the model that is made to fit the data, not the other way around. (Usually people mean this case when they say either expression.)

When people say they fit data to a model I find myself trying to figure out what the heck did they do to the data?.

[Now if you're transforming data, that would arguably be 'fitting data to a model', but people almost never say that for this case.]

Usually, we assume our data corresponds to the "real world" and making any modifications means we are moving away from modelling the "real world". For example, one needs to take care removing outliers since even if it makes computation nicer, outliers were still part of our data.

When testing a model or estimating properties of an estimator using bootstrap or other resampling techniques, we may simulate new data using an estimated model and our original data. This makes the assumption that the model is correct, and we are not modifying our original data.