Re: CCL:Re: CCL:optimization with constraints
Hello,
Eric Martin wrote:
>
>My understanding is that fitting a transformed function might not only
>affect the efficiency, it can also affect the answer. The least
>squares best fit of the transformed problem will have different
>parameter values (even after the reverse transorm) than the best fit of
>the untransformed (albeit constrained) problem, since the sum of
>squared residuals of a different function is being minimized. This can
>sometimes be rectified by applying weighted least squares. See:
>"When, Why and How to Use Weighted Least Squares", Robert de
Levie, J.
>Chem. Edu., 63, (1986) p. 10.
This is a very good point, but it refers to a transformed _model_,
not to constraints on the model itself (the latter being a purely
computational issue).
A classic example is
y = A \exp (b*x) (1)
with additive error.
In the old days (before computers), this would be changed to
\log (y) = A' + bx (2)
which is now a linear model (in terms of the parameters A and b).
One can now use "good old linear regression" to solve this problem.
However, as Eric correctly points out, this is only (statistically)
correct if the error is _additive_ in Eqn. (2). Of course, if the
error was additive in (1), this surely will not be the case in (2).
The parameter values resulting from the 2 fits will indeed be
different.
One can (partially) compensate for the transformation by
_weighting_ the residuals in fitting Eqn (2). In general, the
weights for a transformation f(y) are given by
w_i = \frac{1}{f'(y_{i}}^2}
which in the case of (2) becomes
w_i = \frac{1}{y_{i}^2}
(Of course, the above comments also assume normally distributed
errors with equal variances, etc.)
I never cease to be amazed by the fact that regression (and
parameter estimation) is taught so poorly in universities -
it's as if computers had never been invented! There's a _huge_
emphasis on linear models and on normally distributed errors, and
neglect of many important topics (errors in the independent
variables, nonlinear models, non-normal error distributions, .... )
Maybe it's related to the fact that beginning stats courses (where
regression is first taught) are oriented towards the social sciences -
where there is no such thing as a "fundamental law" and one might as
well use a linear model.
Some years ago, I wrote a report discussing the above and other
problems related to nonlinear parameter estimation. If anyone would
like a copy, please send me an E-Mail request.
Best Regards,
W. R. Smith Professor
Dept. of Mathematics and Statistics
and School of Engineering
University of Guelph
FAX: 519-837-0221 Guelph, Ontario
Tel: 519-824-4120, ext. 3038 CANADA N1G 2W1