curve fitting examples

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

curve fitting examples

Rainer M. Engel
Hello everyone,

I retrieve some data, which can have missing data points even in the
beginning or at the end. So what I needed was both; interpolation as
well as extrapolation, to fill these positions.

Typically the deviation over the data is not that huge and a straight
fitting works well under this circumstance. Otherwise on more
fluctuating/arbitrary data I got no satisfying results with any of the
available fitting functions (see example makro below).

Maybe my expectations are wrong about this and I thought that there is a
way to adjusted methods in how close a resulting fitting would be
applied. So I used median/mean as prefilter-methods.

What wonders me is that sometimes a certain method works very well. In
my case Rodbard, Error or a Gaussian. But sometimes, like in the given
example, it makes no sense to use these at all.

Is this typical or is my data strange :)

Regards,
Rainer




// START (makro) #############################################
// Curve Fitting Demo
//
// This macro demonstates how to use the Fit.* functions,
// which were added to the macro language in v1.41k.

// modified to see some larger data sets

x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75,
76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93,
94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136,
137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150,
151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178,
179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192,
193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206,
207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,
221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234,
235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248,
249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276,
277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290,
291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304,
305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318,
319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332,
333, 334, 335, 336, 337, 338, 339);
y = newArray(96, 97, 98, 101, 101, 102, 101, 101, 101, 101, 101, 101,
100, 100, 101, 101, 102, 101, 100, 100, 101, 100, 99, 99, 100, 101, 100,
100, 101, 101, 100, 99, 99, 99, 99, 99, 99, 98, 99, 99, 98, 97, 96, 96,
96, 96, 95, 94, 95, 95, 94, 94, 93, 90, 89, 89, 88, 87, 87, 87, 87, 84,
85, 84, 85, 84, 83, 81, 81, 81, 80, 80, 78, 77, 77, 77, 76, 75, 75, 75,
75, 74, 73, 73, 73, 73, 72, 71, 71, 71, 70, 69, 68, 68, 68, 67, 66, 66,
65, 66, 65, 66, 64, 64, 64, 64, 64, 65, 65, 66, 66, 65, 65, 65, 65, 66,
66, 66, 67, 68, 68, 68, 68, 69, 69, 70, 69, 69, 69, 70, 70, 71, 70, 72,
72, 72, 71, 71, 72, 73, 73, 73, 73, 75, 75, 75, 74, 75, 75, 76, 76, 77,
76, 76, 77, 78, 77, 77, 77, 78, 79, 80, 79, 79, 80, 81, 80, 80, 81, 81,
81, 80, 81, 81, 82, 82, 82, 81, 82, 83, 83, 83, 82, 83, 83, 83, 82, 84,
84, 84, 84, 84, 83, 83, 84, 84, 83, 84, 84, 84, 83, 83, 83, 83, 85, 85,
84, 83, 84, 84, 84, 83, 83, 84, 84, 85, 84, 83, 83, 84, 83, 83, 83, 83,
84, 83, 83, 84, 84, 84, 84, 84, 83, 84, 84, 84, 83, 83, 83, 84, 84, 83,
83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 85, 84, 83, 83, 84,
84, 84, 84, 85, 85, 84, 83, 84, 84, 83, 84, 83, 84, 85, 84, 84, 84, 84,
84, 85, 84, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 84,
84, 83, 83, 84, 84, 83, 83, 84, 84, 84, 83, 83, 84, 84, 84, 84, 83, 83,
84, 84, 83, 82, 83, 83, 83, 83, 83, 83, 84, 84, 83, 83, 83, 84, 84, 83,
83, 83, 83);

doAllCurveFittings("raw", x, y); //pretty dense x-range

x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118,
119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132,
133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146,
147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160,
161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174,
175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188,
189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202,
203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230,
231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244,
245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339);
y = newArray(100.8147, 100.7783, 100.7480, 100.6839, 100.6290, 100.6147,
100.6335, 100.6794, 100.6647, 100.6252, 100.5897, 100.5647, 100.5397,
100.4647, 100.3897, 100.3647, 100.3647, 100.3147, 100.2897, 100.2897,
100.3147, 100.2897, 100.2147, 100.1647, 100.0897, 100.0147, 99.9147,
99.7897, 99.6647, 99.5897, 99.5397, 99.4147, 99.2647, 99.1147, 98.9397,
98.7147, 98.4397, 98.1897, 97.8897, 97.5897, 97.2647, 96.9647, 96.7147,
96.3897, 96.0730, 95.7147, 95.3397, 94.9480, 94.5897, 94.1897, 93.7147,
93.2397, 92.7897, 92.3397, 91.8897, 91.4147, 90.9647, 90.4397, 89.9647,
89.4397, 88.9647, 88.4647, 87.9147, 87.3647, 86.8064, 86.2397, 85.6647,
85.0814, 84.4397, 83.7647, 83.1647, 82.5897, 81.9897, 81.3897, 80.8147,
80.2397, 79.6647, 79.1147, 78.5147, 77.9397, 77.3147, 76.7397, 76.1897,
75.6397, 75.1147, 74.6147, 74.1147, 73.5647, 73.0397, 72.5647, 72.0647,
71.5397, 71.0147, 70.5397, 70.0147, 69.5397, 69.0397, 68.6397, 68.1647,
67.6897, 67.2397, 66.7897, 66.3897, 66.0897, 65.7897, 65.5397, 65.3147,
65.1397, 64.9897, 64.8647, 64.7397, 64.6897, 64.7147, 64.7147, 64.7897,
64.9147, 65.0397, 65.1397, 65.3647, 65.6147, 65.8647, 66.1647, 66.3897,
66.6147, 66.8397, 67.0647, 67.3147, 67.6147, 67.9147, 68.2647, 68.6147,
68.9147, 69.1647, 69.4397, 69.7147, 69.9647, 70.2397, 70.5147, 70.7647,
71.0897, 71.4147, 71.6897, 71.9897, 72.3147, 72.6147, 72.9147, 73.1897,
73.4647, 73.7147, 73.9147, 74.1647, 74.4397, 74.7147, 74.9897, 75.2397,
75.4647, 75.7397, 76.0647, 76.3397, 76.5397, 76.7897, 77.0647, 77.3397,
77.5397, 77.8147, 78.0647, 78.3147, 78.4897, 78.7397, 78.9897, 79.2397,
79.4647, 79.7147, 79.9397, 80.1897, 80.4647, 80.6897, 80.8397, 81.0147,
81.2147, 81.3647, 81.4897, 81.6147, 81.8397, 82.0147, 82.1647, 82.3147,
82.5147, 82.6397, 82.7647, 82.8647, 82.9647, 83.0647, 83.2147, 83.3147,
83.3647, 83.3647, 83.4147, 83.4897, 83.5147, 83.6147, 83.7397, 83.8397,
83.8147, 83.8147, 83.8147, 83.8397, 83.7897, 83.8147, 83.8397, 83.8397,
83.8897, 83.9147, 83.8647, 83.8397, 83.8397, 83.8397, 83.8397, 83.8397,
83.8397, 83.7897, 83.6897, 83.6147, 83.6397, 83.6397, 83.6397, 83.6147,
83.6647, 83.6397, 83.6397, 83.6647, 83.6147, 83.5647, 83.5647, 83.5397,
83.5397, 83.5897, 83.5647, 83.5647, 83.5897, 83.6147, 83.6647, 83.7147,
83.7147, 83.7147, 83.7397, 83.7647, 83.7897, 83.8397, 83.8897, 83.8897,
83.9397, 83.9897, 84.0147, 84.0397, 84.0647, 84.0897, 84.1647, 84.2147,
84.2647, 84.2897, 84.2897, 84.2897, 84.2897, 84.2897, 84.2397, 84.2397,
84.1897, 84.2147, 84.2147, 84.2397, 84.2397, 84.2897, 84.3397, 84.3897,
84.4397, 84.4397, 84.3897, 84.3647, 84.3397, 84.3147, 84.3397, 84.3647,
84.3897, 84.4147, 84.4647, 84.4897, 84.5397, 84.5397, 84.5397, 84.5397,
84.5147, 84.4897, 84.4397, 84.4147, 84.3647, 84.3647, 84.3897, 84.3647,
84.3647, 84.3397, 84.3147, 84.2647, 84.2147, 84.2147, 84.2147, 84.1897,
84.1647, 84.1147, 84.0647, 84.0397, 84.0397, 83.9897, 83.9647, 83.9397,
83.9147, 83.8647, 83.8397, 83.8397, 83.7897, 83.8147, 83.8147, 83.8147,
83.8147, 83.7647, 83.7147, 83.6647, 83.6397, 83.6647, 83.6647, 83.6647,
83.6252, 83.6369, 83.6794, 83.6960, 83.6814, 83.7004, 83.7224, 83.7480,
83.7783);

doAllCurveFittings("pre-filtered", x, y); //full x-range

function doAllCurveFittings(stackTitle, xpoints, ypoints) {
        // Do a straight line fit
        Fit.doFit("Straight Line", xpoints, ypoints);
        //print("a="+d2s(Fit.p(0),6)+", b="+d2s(Fit.p(1),6));
       
        // Do all possible fits, plot them and add the plots to a stack
        setBatchMode(true);
        for (i=0; i<Fit.nEquations; i++) {
         Fit.doFit(i, xpoints, ypoints);
         Fit.plot();
         if (i==0)
             stack = getImageID;
         else {
             run("Copy");
             close();
             selectImage(stack);
             run("Add Slice");
             run("Paste");
         }
         Fit.getEquation(i, name, formula);
         //print("");
         print("index: "+i+", "+name+ " ["+formula+"]");
         /*print("   R^2="+d2s(Fit.rSquared,3));
         for (j=0; j<Fit.nParams; j++)
             print("   p["+j+"]="+d2s(Fit.p(j),6));*/
        }
        setBatchMode(false);
        run("Select None");
        rename(stackTitle);
}
// END ##################################################


--
Rainer M. Engel
Berlin

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
Reply | Threaded
Open this post in threaded view
|

Re: curve fitting examples

Herbie
Good day Rainer,

if I remember correctly you've discussed a similar topic before...

Please understand that curve fitting or general regression, to be more
precise, needs a "null hypothesis" or "model", i.e. you need to know the
type of function that underlies your data. If you don't have an idea
about the data generating process, you are lost or the results become
arbitrary.

There are very few exceptions from this rule and two of them are
interpolation and smoothing (they are no regression techniques).
Interpolation is well-defined if the data is correctly sampled from a
continuous process. Smoothing and related operations are less
well-defined and they are more an art than hard science.

Please keep this in mind and consider the proper approaches:

Curve fitting provided by ImageJ is based on "general regression" and
requires a "null hypothesis" or "model". The most general "model" is a
polynomial of high degree but I don't recommend to use it without
profound reason.

If your data doesn't fit a mathematically formulated "model", then
"general regression" according to an arbitrary chosen "model"-function
is like a shot in the dark.

I fear that the "general regression"-approach is not what you looking for.

Regards

Herbie

:::::::::::::::::::::::::::::::::::::::::::::
Am 07.05.19 um 11:40 schrieb Rainer M. Engel:

> Hello everyone,
>
> I retrieve some data, which can have missing data points even in the
> beginning or at the end. So what I needed was both; interpolation as
> well as extrapolation, to fill these positions.
>
> Typically the deviation over the data is not that huge and a straight
> fitting works well under this circumstance. Otherwise on more
> fluctuating/arbitrary data I got no satisfying results with any of the
> available fitting functions (see example makro below).
>
> Maybe my expectations are wrong about this and I thought that there is a
> way to adjusted methods in how close a resulting fitting would be
> applied. So I used median/mean as prefilter-methods.
>
> What wonders me is that sometimes a certain method works very well. In
> my case Rodbard, Error or a Gaussian. But sometimes, like in the given
> example, it makes no sense to use these at all.
>
> Is this typical or is my data strange :)
>
> Regards,
> Rainer
>
>
>
>
> // START (makro) #############################################
> // Curve Fitting Demo
> //
> // This macro demonstates how to use the Fit.* functions,
> // which were added to the macro language in v1.41k.
>
> // modified to see some larger data sets
>
> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
> 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75,
> 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93,
> 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
> 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
> 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136,
> 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150,
> 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
> 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178,
> 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192,
> 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206,
> 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,
> 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234,
> 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248,
> 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
> 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276,
> 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290,
> 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304,
> 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318,
> 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332,
> 333, 334, 335, 336, 337, 338, 339);
> y = newArray(96, 97, 98, 101, 101, 102, 101, 101, 101, 101, 101, 101,
> 100, 100, 101, 101, 102, 101, 100, 100, 101, 100, 99, 99, 100, 101, 100,
> 100, 101, 101, 100, 99, 99, 99, 99, 99, 99, 98, 99, 99, 98, 97, 96, 96,
> 96, 96, 95, 94, 95, 95, 94, 94, 93, 90, 89, 89, 88, 87, 87, 87, 87, 84,
> 85, 84, 85, 84, 83, 81, 81, 81, 80, 80, 78, 77, 77, 77, 76, 75, 75, 75,
> 75, 74, 73, 73, 73, 73, 72, 71, 71, 71, 70, 69, 68, 68, 68, 67, 66, 66,
> 65, 66, 65, 66, 64, 64, 64, 64, 64, 65, 65, 66, 66, 65, 65, 65, 65, 66,
> 66, 66, 67, 68, 68, 68, 68, 69, 69, 70, 69, 69, 69, 70, 70, 71, 70, 72,
> 72, 72, 71, 71, 72, 73, 73, 73, 73, 75, 75, 75, 74, 75, 75, 76, 76, 77,
> 76, 76, 77, 78, 77, 77, 77, 78, 79, 80, 79, 79, 80, 81, 80, 80, 81, 81,
> 81, 80, 81, 81, 82, 82, 82, 81, 82, 83, 83, 83, 82, 83, 83, 83, 82, 84,
> 84, 84, 84, 84, 83, 83, 84, 84, 83, 84, 84, 84, 83, 83, 83, 83, 85, 85,
> 84, 83, 84, 84, 84, 83, 83, 84, 84, 85, 84, 83, 83, 84, 83, 83, 83, 83,
> 84, 83, 83, 84, 84, 84, 84, 84, 83, 84, 84, 84, 83, 83, 83, 84, 84, 83,
> 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 85, 84, 83, 83, 84,
> 84, 84, 84, 85, 85, 84, 83, 84, 84, 83, 84, 83, 84, 85, 84, 84, 84, 84,
> 84, 85, 84, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 84,
> 84, 83, 83, 84, 84, 83, 83, 84, 84, 84, 83, 83, 84, 84, 84, 84, 83, 83,
> 84, 84, 83, 82, 83, 83, 83, 83, 83, 83, 84, 84, 83, 83, 83, 84, 84, 83,
> 83, 83, 83);
>
> doAllCurveFittings("raw", x, y); //pretty dense x-range
>
> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
> 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
> 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
> 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
> 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118,
> 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132,
> 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146,
> 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160,
> 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174,
> 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188,
> 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202,
> 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
> 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230,
> 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244,
> 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
> 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
> 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
> 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
> 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
> 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
> 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339);
> y = newArray(100.8147, 100.7783, 100.7480, 100.6839, 100.6290, 100.6147,
> 100.6335, 100.6794, 100.6647, 100.6252, 100.5897, 100.5647, 100.5397,
> 100.4647, 100.3897, 100.3647, 100.3647, 100.3147, 100.2897, 100.2897,
> 100.3147, 100.2897, 100.2147, 100.1647, 100.0897, 100.0147, 99.9147,
> 99.7897, 99.6647, 99.5897, 99.5397, 99.4147, 99.2647, 99.1147, 98.9397,
> 98.7147, 98.4397, 98.1897, 97.8897, 97.5897, 97.2647, 96.9647, 96.7147,
> 96.3897, 96.0730, 95.7147, 95.3397, 94.9480, 94.5897, 94.1897, 93.7147,
> 93.2397, 92.7897, 92.3397, 91.8897, 91.4147, 90.9647, 90.4397, 89.9647,
> 89.4397, 88.9647, 88.4647, 87.9147, 87.3647, 86.8064, 86.2397, 85.6647,
> 85.0814, 84.4397, 83.7647, 83.1647, 82.5897, 81.9897, 81.3897, 80.8147,
> 80.2397, 79.6647, 79.1147, 78.5147, 77.9397, 77.3147, 76.7397, 76.1897,
> 75.6397, 75.1147, 74.6147, 74.1147, 73.5647, 73.0397, 72.5647, 72.0647,
> 71.5397, 71.0147, 70.5397, 70.0147, 69.5397, 69.0397, 68.6397, 68.1647,
> 67.6897, 67.2397, 66.7897, 66.3897, 66.0897, 65.7897, 65.5397, 65.3147,
> 65.1397, 64.9897, 64.8647, 64.7397, 64.6897, 64.7147, 64.7147, 64.7897,
> 64.9147, 65.0397, 65.1397, 65.3647, 65.6147, 65.8647, 66.1647, 66.3897,
> 66.6147, 66.8397, 67.0647, 67.3147, 67.6147, 67.9147, 68.2647, 68.6147,
> 68.9147, 69.1647, 69.4397, 69.7147, 69.9647, 70.2397, 70.5147, 70.7647,
> 71.0897, 71.4147, 71.6897, 71.9897, 72.3147, 72.6147, 72.9147, 73.1897,
> 73.4647, 73.7147, 73.9147, 74.1647, 74.4397, 74.7147, 74.9897, 75.2397,
> 75.4647, 75.7397, 76.0647, 76.3397, 76.5397, 76.7897, 77.0647, 77.3397,
> 77.5397, 77.8147, 78.0647, 78.3147, 78.4897, 78.7397, 78.9897, 79.2397,
> 79.4647, 79.7147, 79.9397, 80.1897, 80.4647, 80.6897, 80.8397, 81.0147,
> 81.2147, 81.3647, 81.4897, 81.6147, 81.8397, 82.0147, 82.1647, 82.3147,
> 82.5147, 82.6397, 82.7647, 82.8647, 82.9647, 83.0647, 83.2147, 83.3147,
> 83.3647, 83.3647, 83.4147, 83.4897, 83.5147, 83.6147, 83.7397, 83.8397,
> 83.8147, 83.8147, 83.8147, 83.8397, 83.7897, 83.8147, 83.8397, 83.8397,
> 83.8897, 83.9147, 83.8647, 83.8397, 83.8397, 83.8397, 83.8397, 83.8397,
> 83.8397, 83.7897, 83.6897, 83.6147, 83.6397, 83.6397, 83.6397, 83.6147,
> 83.6647, 83.6397, 83.6397, 83.6647, 83.6147, 83.5647, 83.5647, 83.5397,
> 83.5397, 83.5897, 83.5647, 83.5647, 83.5897, 83.6147, 83.6647, 83.7147,
> 83.7147, 83.7147, 83.7397, 83.7647, 83.7897, 83.8397, 83.8897, 83.8897,
> 83.9397, 83.9897, 84.0147, 84.0397, 84.0647, 84.0897, 84.1647, 84.2147,
> 84.2647, 84.2897, 84.2897, 84.2897, 84.2897, 84.2897, 84.2397, 84.2397,
> 84.1897, 84.2147, 84.2147, 84.2397, 84.2397, 84.2897, 84.3397, 84.3897,
> 84.4397, 84.4397, 84.3897, 84.3647, 84.3397, 84.3147, 84.3397, 84.3647,
> 84.3897, 84.4147, 84.4647, 84.4897, 84.5397, 84.5397, 84.5397, 84.5397,
> 84.5147, 84.4897, 84.4397, 84.4147, 84.3647, 84.3647, 84.3897, 84.3647,
> 84.3647, 84.3397, 84.3147, 84.2647, 84.2147, 84.2147, 84.2147, 84.1897,
> 84.1647, 84.1147, 84.0647, 84.0397, 84.0397, 83.9897, 83.9647, 83.9397,
> 83.9147, 83.8647, 83.8397, 83.8397, 83.7897, 83.8147, 83.8147, 83.8147,
> 83.8147, 83.7647, 83.7147, 83.6647, 83.6397, 83.6647, 83.6647, 83.6647,
> 83.6252, 83.6369, 83.6794, 83.6960, 83.6814, 83.7004, 83.7224, 83.7480,
> 83.7783);
>
> doAllCurveFittings("pre-filtered", x, y); //full x-range
>
> function doAllCurveFittings(stackTitle, xpoints, ypoints) {
> // Do a straight line fit
> Fit.doFit("Straight Line", xpoints, ypoints);
> //print("a="+d2s(Fit.p(0),6)+", b="+d2s(Fit.p(1),6));
>
> // Do all possible fits, plot them and add the plots to a stack
> setBatchMode(true);
> for (i=0; i<Fit.nEquations; i++) {
> Fit.doFit(i, xpoints, ypoints);
> Fit.plot();
> if (i==0)
>     stack = getImageID;
> else {
>     run("Copy");
>     close();
>     selectImage(stack);
>     run("Add Slice");
>     run("Paste");
> }
> Fit.getEquation(i, name, formula);
> //print("");
> print("index: "+i+", "+name+ " ["+formula+"]");
> /*print("   R^2="+d2s(Fit.rSquared,3));
> for (j=0; j<Fit.nParams; j++)
>     print("   p["+j+"]="+d2s(Fit.p(j),6));*/
> }
> setBatchMode(false);
> run("Select None");
> rename(stackTitle);
> }
> // END ##################################################
>
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
Reply | Threaded
Open this post in threaded view
|

Re: curve fitting examples

Rainer M. Engel
Good day Herbie,

you are right about that. Some 1-2 years ago, but right now I'm refining
my code and I have a better overview now, since the underlying data
retrieval is more robust.

> more an art than hard science

I think I tended to 'misuse' it, even though it worked in 90% of the
cases without the need to set filtering windows. So I wanted to check on
the limits again.

One thought:
The data I get is very stable and missing data positions are marked to
be filled. Right now I'm linear interpolating these, but by this these
positions have the same "weight" like the former analysed ones, which
then tend to be more deviating and hence will be smoothed stronger. This
is one reason why I came back and tested the fitting methods, which are
doing a nice job at least in some regions..

Thank you for your fast reply and thoughts on this..

Regards,
Rainer


Am 07.05.2019 um 12:06 schrieb Herbie:

> Good day Rainer,
>
> if I remember correctly you've discussed a similar topic before...
>
> Please understand that curve fitting or general regression, to be more
> precise, needs a "null hypothesis" or "model", i.e. you need to know the
> type of function that underlies your data. If you don't have an idea
> about the data generating process, you are lost or the results become
> arbitrary.
>
> There are very few exceptions from this rule and two of them are
> interpolation and smoothing (they are no regression techniques).
> Interpolation is well-defined if the data is correctly sampled from a
> continuous process. Smoothing and related operations are less
> well-defined and they are more an art than hard science.
>
> Please keep this in mind and consider the proper approaches:
>
> Curve fitting provided by ImageJ is based on "general regression" and
> requires a "null hypothesis" or "model". The most general "model" is a
> polynomial of high degree but I don't recommend to use it without
> profound reason.
>
> If your data doesn't fit a mathematically formulated "model", then
> "general regression" according to an arbitrary chosen "model"-function
> is like a shot in the dark.
>
> I fear that the "general regression"-approach is not what you looking for.
>
> Regards
>
> Herbie
>
> :::::::::::::::::::::::::::::::::::::::::::::
> Am 07.05.19 um 11:40 schrieb Rainer M. Engel:
>> Hello everyone,
>>
>> I retrieve some data, which can have missing data points even in the
>> beginning or at the end. So what I needed was both; interpolation as
>> well as extrapolation, to fill these positions.
>>
>> Typically the deviation over the data is not that huge and a straight
>> fitting works well under this circumstance. Otherwise on more
>> fluctuating/arbitrary data I got no satisfying results with any of the
>> available fitting functions (see example makro below).
>>
>> Maybe my expectations are wrong about this and I thought that there is a
>> way to adjusted methods in how close a resulting fitting would be
>> applied. So I used median/mean as prefilter-methods.
>>
>> What wonders me is that sometimes a certain method works very well. In
>> my case Rodbard, Error or a Gaussian. But sometimes, like in the given
>> example, it makes no sense to use these at all.
>>
>> Is this typical or is my data strange :)
>>
>> Regards,
>> Rainer
>>
>>
>>
>>
>> // START (makro) #############################################
>> // Curve Fitting Demo
>> //
>> // This macro demonstates how to use the Fit.* functions,
>> // which were added to the macro language in v1.41k.
>>
>> // modified to see some larger data sets
>>
>> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
>> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
>> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
>> 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75,
>> 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93,
>> 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
>> 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
>> 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136,
>> 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150,
>> 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
>> 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178,
>> 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192,
>> 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206,
>> 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,
>> 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234,
>> 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248,
>> 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
>> 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276,
>> 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290,
>> 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304,
>> 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318,
>> 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332,
>> 333, 334, 335, 336, 337, 338, 339);
>> y = newArray(96, 97, 98, 101, 101, 102, 101, 101, 101, 101, 101, 101,
>> 100, 100, 101, 101, 102, 101, 100, 100, 101, 100, 99, 99, 100, 101, 100,
>> 100, 101, 101, 100, 99, 99, 99, 99, 99, 99, 98, 99, 99, 98, 97, 96, 96,
>> 96, 96, 95, 94, 95, 95, 94, 94, 93, 90, 89, 89, 88, 87, 87, 87, 87, 84,
>> 85, 84, 85, 84, 83, 81, 81, 81, 80, 80, 78, 77, 77, 77, 76, 75, 75, 75,
>> 75, 74, 73, 73, 73, 73, 72, 71, 71, 71, 70, 69, 68, 68, 68, 67, 66, 66,
>> 65, 66, 65, 66, 64, 64, 64, 64, 64, 65, 65, 66, 66, 65, 65, 65, 65, 66,
>> 66, 66, 67, 68, 68, 68, 68, 69, 69, 70, 69, 69, 69, 70, 70, 71, 70, 72,
>> 72, 72, 71, 71, 72, 73, 73, 73, 73, 75, 75, 75, 74, 75, 75, 76, 76, 77,
>> 76, 76, 77, 78, 77, 77, 77, 78, 79, 80, 79, 79, 80, 81, 80, 80, 81, 81,
>> 81, 80, 81, 81, 82, 82, 82, 81, 82, 83, 83, 83, 82, 83, 83, 83, 82, 84,
>> 84, 84, 84, 84, 83, 83, 84, 84, 83, 84, 84, 84, 83, 83, 83, 83, 85, 85,
>> 84, 83, 84, 84, 84, 83, 83, 84, 84, 85, 84, 83, 83, 84, 83, 83, 83, 83,
>> 84, 83, 83, 84, 84, 84, 84, 84, 83, 84, 84, 84, 83, 83, 83, 84, 84, 83,
>> 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 85, 84, 83, 83, 84,
>> 84, 84, 84, 85, 85, 84, 83, 84, 84, 83, 84, 83, 84, 85, 84, 84, 84, 84,
>> 84, 85, 84, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 84,
>> 84, 83, 83, 84, 84, 83, 83, 84, 84, 84, 83, 83, 84, 84, 84, 84, 83, 83,
>> 84, 84, 83, 82, 83, 83, 83, 83, 83, 83, 84, 84, 83, 83, 83, 84, 84, 83,
>> 83, 83, 83);
>>
>> doAllCurveFittings("raw", x, y); //pretty dense x-range
>>
>> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
>> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
>> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
>> 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
>> 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
>> 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
>> 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118,
>> 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132,
>> 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146,
>> 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160,
>> 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174,
>> 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188,
>> 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202,
>> 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
>> 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230,
>> 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244,
>> 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
>> 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
>> 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
>> 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
>> 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
>> 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
>> 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339);
>> y = newArray(100.8147, 100.7783, 100.7480, 100.6839, 100.6290, 100.6147,
>> 100.6335, 100.6794, 100.6647, 100.6252, 100.5897, 100.5647, 100.5397,
>> 100.4647, 100.3897, 100.3647, 100.3647, 100.3147, 100.2897, 100.2897,
>> 100.3147, 100.2897, 100.2147, 100.1647, 100.0897, 100.0147, 99.9147,
>> 99.7897, 99.6647, 99.5897, 99.5397, 99.4147, 99.2647, 99.1147, 98.9397,
>> 98.7147, 98.4397, 98.1897, 97.8897, 97.5897, 97.2647, 96.9647, 96.7147,
>> 96.3897, 96.0730, 95.7147, 95.3397, 94.9480, 94.5897, 94.1897, 93.7147,
>> 93.2397, 92.7897, 92.3397, 91.8897, 91.4147, 90.9647, 90.4397, 89.9647,
>> 89.4397, 88.9647, 88.4647, 87.9147, 87.3647, 86.8064, 86.2397, 85.6647,
>> 85.0814, 84.4397, 83.7647, 83.1647, 82.5897, 81.9897, 81.3897, 80.8147,
>> 80.2397, 79.6647, 79.1147, 78.5147, 77.9397, 77.3147, 76.7397, 76.1897,
>> 75.6397, 75.1147, 74.6147, 74.1147, 73.5647, 73.0397, 72.5647, 72.0647,
>> 71.5397, 71.0147, 70.5397, 70.0147, 69.5397, 69.0397, 68.6397, 68.1647,
>> 67.6897, 67.2397, 66.7897, 66.3897, 66.0897, 65.7897, 65.5397, 65.3147,
>> 65.1397, 64.9897, 64.8647, 64.7397, 64.6897, 64.7147, 64.7147, 64.7897,
>> 64.9147, 65.0397, 65.1397, 65.3647, 65.6147, 65.8647, 66.1647, 66.3897,
>> 66.6147, 66.8397, 67.0647, 67.3147, 67.6147, 67.9147, 68.2647, 68.6147,
>> 68.9147, 69.1647, 69.4397, 69.7147, 69.9647, 70.2397, 70.5147, 70.7647,
>> 71.0897, 71.4147, 71.6897, 71.9897, 72.3147, 72.6147, 72.9147, 73.1897,
>> 73.4647, 73.7147, 73.9147, 74.1647, 74.4397, 74.7147, 74.9897, 75.2397,
>> 75.4647, 75.7397, 76.0647, 76.3397, 76.5397, 76.7897, 77.0647, 77.3397,
>> 77.5397, 77.8147, 78.0647, 78.3147, 78.4897, 78.7397, 78.9897, 79.2397,
>> 79.4647, 79.7147, 79.9397, 80.1897, 80.4647, 80.6897, 80.8397, 81.0147,
>> 81.2147, 81.3647, 81.4897, 81.6147, 81.8397, 82.0147, 82.1647, 82.3147,
>> 82.5147, 82.6397, 82.7647, 82.8647, 82.9647, 83.0647, 83.2147, 83.3147,
>> 83.3647, 83.3647, 83.4147, 83.4897, 83.5147, 83.6147, 83.7397, 83.8397,
>> 83.8147, 83.8147, 83.8147, 83.8397, 83.7897, 83.8147, 83.8397, 83.8397,
>> 83.8897, 83.9147, 83.8647, 83.8397, 83.8397, 83.8397, 83.8397, 83.8397,
>> 83.8397, 83.7897, 83.6897, 83.6147, 83.6397, 83.6397, 83.6397, 83.6147,
>> 83.6647, 83.6397, 83.6397, 83.6647, 83.6147, 83.5647, 83.5647, 83.5397,
>> 83.5397, 83.5897, 83.5647, 83.5647, 83.5897, 83.6147, 83.6647, 83.7147,
>> 83.7147, 83.7147, 83.7397, 83.7647, 83.7897, 83.8397, 83.8897, 83.8897,
>> 83.9397, 83.9897, 84.0147, 84.0397, 84.0647, 84.0897, 84.1647, 84.2147,
>> 84.2647, 84.2897, 84.2897, 84.2897, 84.2897, 84.2897, 84.2397, 84.2397,
>> 84.1897, 84.2147, 84.2147, 84.2397, 84.2397, 84.2897, 84.3397, 84.3897,
>> 84.4397, 84.4397, 84.3897, 84.3647, 84.3397, 84.3147, 84.3397, 84.3647,
>> 84.3897, 84.4147, 84.4647, 84.4897, 84.5397, 84.5397, 84.5397, 84.5397,
>> 84.5147, 84.4897, 84.4397, 84.4147, 84.3647, 84.3647, 84.3897, 84.3647,
>> 84.3647, 84.3397, 84.3147, 84.2647, 84.2147, 84.2147, 84.2147, 84.1897,
>> 84.1647, 84.1147, 84.0647, 84.0397, 84.0397, 83.9897, 83.9647, 83.9397,
>> 83.9147, 83.8647, 83.8397, 83.8397, 83.7897, 83.8147, 83.8147, 83.8147,
>> 83.8147, 83.7647, 83.7147, 83.6647, 83.6397, 83.6647, 83.6647, 83.6647,
>> 83.6252, 83.6369, 83.6794, 83.6960, 83.6814, 83.7004, 83.7224, 83.7480,
>> 83.7783);
>>
>> doAllCurveFittings("pre-filtered", x, y); //full x-range
>>
>> function doAllCurveFittings(stackTitle, xpoints, ypoints) {
>>     // Do a straight line fit
>>     Fit.doFit("Straight Line", xpoints, ypoints);
>>     //print("a="+d2s(Fit.p(0),6)+", b="+d2s(Fit.p(1),6));
>>     
>>     // Do all possible fits, plot them and add the plots to a stack
>>     setBatchMode(true);
>>     for (i=0; i<Fit.nEquations; i++) {
>>      Fit.doFit(i, xpoints, ypoints);
>>      Fit.plot();
>>      if (i==0)
>>          stack = getImageID;
>>      else {
>>          run("Copy");
>>          close();
>>          selectImage(stack);
>>          run("Add Slice");
>>          run("Paste");
>>      }
>>      Fit.getEquation(i, name, formula);
>>      //print("");
>>      print("index: "+i+", "+name+ " ["+formula+"]");
>>      /*print("   R^2="+d2s(Fit.rSquared,3));
>>      for (j=0; j<Fit.nParams; j++)
>>          print("   p["+j+"]="+d2s(Fit.p(j),6));*/
>>     }
>>     setBatchMode(false);
>>     run("Select None");
>>     rename(stackTitle);
>> }
>> // END ##################################################
>>
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html
Reply | Threaded
Open this post in threaded view
|

Re: curve fitting examples

Herbie
Again,

what do you mean by:

"it worked in 90% of the cases"

How can you know that the missing data is correctly interpolated. Did
you make tests with all kinds of data and have you cut out parts of them
and after interpolation compared the cut-out data with the reconstructed
ones?
What criterion do you use when comparing both?

I think such a comparison with a suitable criterion is a reasonable way
to show that your method works correctly.

Regards

Herbie

:::::::::::::::::::::::::::::::::::::::::::::
Am 07.05.19 um 12:48 schrieb Rainer M. Engel:

> Good day Herbie,
>
> you are right about that. Some 1-2 years ago, but right now I'm refining
> my code and I have a better overview now, since the underlying data
> retrieval is more robust.
>
>> more an art than hard science
>
> I think I tended to 'misuse' it, even though it worked in 90% of the
> cases without the need to set filtering windows. So I wanted to check on
> the limits again.
>
> One thought:
> The data I get is very stable and missing data positions are marked to
> be filled. Right now I'm linear interpolating these, but by this these
> positions have the same "weight" like the former analysed ones, which
> then tend to be more deviating and hence will be smoothed stronger. This
> is one reason why I came back and tested the fitting methods, which are
> doing a nice job at least in some regions..
>
> Thank you for your fast reply and thoughts on this..
>
> Regards,
> Rainer
>
>
> Am 07.05.2019 um 12:06 schrieb Herbie:
>> Good day Rainer,
>>
>> if I remember correctly you've discussed a similar topic before...
>>
>> Please understand that curve fitting or general regression, to be more
>> precise, needs a "null hypothesis" or "model", i.e. you need to know the
>> type of function that underlies your data. If you don't have an idea
>> about the data generating process, you are lost or the results become
>> arbitrary.
>>
>> There are very few exceptions from this rule and two of them are
>> interpolation and smoothing (they are no regression techniques).
>> Interpolation is well-defined if the data is correctly sampled from a
>> continuous process. Smoothing and related operations are less
>> well-defined and they are more an art than hard science.
>>
>> Please keep this in mind and consider the proper approaches:
>>
>> Curve fitting provided by ImageJ is based on "general regression" and
>> requires a "null hypothesis" or "model". The most general "model" is a
>> polynomial of high degree but I don't recommend to use it without
>> profound reason.
>>
>> If your data doesn't fit a mathematically formulated "model", then
>> "general regression" according to an arbitrary chosen "model"-function
>> is like a shot in the dark.
>>
>> I fear that the "general regression"-approach is not what you looking for.
>>
>> Regards
>>
>> Herbie
>>
>> :::::::::::::::::::::::::::::::::::::::::::::
>> Am 07.05.19 um 11:40 schrieb Rainer M. Engel:
>>> Hello everyone,
>>>
>>> I retrieve some data, which can have missing data points even in the
>>> beginning or at the end. So what I needed was both; interpolation as
>>> well as extrapolation, to fill these positions.
>>>
>>> Typically the deviation over the data is not that huge and a straight
>>> fitting works well under this circumstance. Otherwise on more
>>> fluctuating/arbitrary data I got no satisfying results with any of the
>>> available fitting functions (see example makro below).
>>>
>>> Maybe my expectations are wrong about this and I thought that there is a
>>> way to adjusted methods in how close a resulting fitting would be
>>> applied. So I used median/mean as prefilter-methods.
>>>
>>> What wonders me is that sometimes a certain method works very well. In
>>> my case Rodbard, Error or a Gaussian. But sometimes, like in the given
>>> example, it makes no sense to use these at all.
>>>
>>> Is this typical or is my data strange :)
>>>
>>> Regards,
>>> Rainer
>>>
>>>
>>>
>>>
>>> // START (makro) #############################################
>>> // Curve Fitting Demo
>>> //
>>> // This macro demonstates how to use the Fit.* functions,
>>> // which were added to the macro language in v1.41k.
>>>
>>> // modified to see some larger data sets
>>>
>>> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
>>> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
>>> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
>>> 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75,
>>> 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93,
>>> 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
>>> 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
>>> 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136,
>>> 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150,
>>> 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
>>> 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178,
>>> 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192,
>>> 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206,
>>> 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,
>>> 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234,
>>> 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248,
>>> 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262,
>>> 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276,
>>> 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290,
>>> 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304,
>>> 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318,
>>> 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332,
>>> 333, 334, 335, 336, 337, 338, 339);
>>> y = newArray(96, 97, 98, 101, 101, 102, 101, 101, 101, 101, 101, 101,
>>> 100, 100, 101, 101, 102, 101, 100, 100, 101, 100, 99, 99, 100, 101, 100,
>>> 100, 101, 101, 100, 99, 99, 99, 99, 99, 99, 98, 99, 99, 98, 97, 96, 96,
>>> 96, 96, 95, 94, 95, 95, 94, 94, 93, 90, 89, 89, 88, 87, 87, 87, 87, 84,
>>> 85, 84, 85, 84, 83, 81, 81, 81, 80, 80, 78, 77, 77, 77, 76, 75, 75, 75,
>>> 75, 74, 73, 73, 73, 73, 72, 71, 71, 71, 70, 69, 68, 68, 68, 67, 66, 66,
>>> 65, 66, 65, 66, 64, 64, 64, 64, 64, 65, 65, 66, 66, 65, 65, 65, 65, 66,
>>> 66, 66, 67, 68, 68, 68, 68, 69, 69, 70, 69, 69, 69, 70, 70, 71, 70, 72,
>>> 72, 72, 71, 71, 72, 73, 73, 73, 73, 75, 75, 75, 74, 75, 75, 76, 76, 77,
>>> 76, 76, 77, 78, 77, 77, 77, 78, 79, 80, 79, 79, 80, 81, 80, 80, 81, 81,
>>> 81, 80, 81, 81, 82, 82, 82, 81, 82, 83, 83, 83, 82, 83, 83, 83, 82, 84,
>>> 84, 84, 84, 84, 83, 83, 84, 84, 83, 84, 84, 84, 83, 83, 83, 83, 85, 85,
>>> 84, 83, 84, 84, 84, 83, 83, 84, 84, 85, 84, 83, 83, 84, 83, 83, 83, 83,
>>> 84, 83, 83, 84, 84, 84, 84, 84, 83, 84, 84, 84, 83, 83, 83, 84, 84, 83,
>>> 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 85, 84, 83, 83, 84,
>>> 84, 84, 84, 85, 85, 84, 83, 84, 84, 83, 84, 83, 84, 85, 84, 84, 84, 84,
>>> 84, 85, 84, 83, 83, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 85, 84, 84,
>>> 84, 83, 83, 84, 84, 83, 83, 84, 84, 84, 83, 83, 84, 84, 84, 84, 83, 83,
>>> 84, 84, 83, 82, 83, 83, 83, 83, 83, 83, 84, 84, 83, 83, 83, 84, 84, 83,
>>> 83, 83, 83);
>>>
>>> doAllCurveFittings("raw", x, y); //pretty dense x-range
>>>
>>> x = newArray(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
>>> 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
>>> 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,
>>> 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
>>> 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88,
>>> 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
>>> 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118,
>>> 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132,
>>> 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146,
>>> 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160,
>>> 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174,
>>> 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188,
>>> 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202,
>>> 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216,
>>> 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230,
>>> 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244,
>>> 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258,
>>> 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272,
>>> 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
>>> 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300,
>>> 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314,
>>> 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328,
>>> 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339);
>>> y = newArray(100.8147, 100.7783, 100.7480, 100.6839, 100.6290, 100.6147,
>>> 100.6335, 100.6794, 100.6647, 100.6252, 100.5897, 100.5647, 100.5397,
>>> 100.4647, 100.3897, 100.3647, 100.3647, 100.3147, 100.2897, 100.2897,
>>> 100.3147, 100.2897, 100.2147, 100.1647, 100.0897, 100.0147, 99.9147,
>>> 99.7897, 99.6647, 99.5897, 99.5397, 99.4147, 99.2647, 99.1147, 98.9397,
>>> 98.7147, 98.4397, 98.1897, 97.8897, 97.5897, 97.2647, 96.9647, 96.7147,
>>> 96.3897, 96.0730, 95.7147, 95.3397, 94.9480, 94.5897, 94.1897, 93.7147,
>>> 93.2397, 92.7897, 92.3397, 91.8897, 91.4147, 90.9647, 90.4397, 89.9647,
>>> 89.4397, 88.9647, 88.4647, 87.9147, 87.3647, 86.8064, 86.2397, 85.6647,
>>> 85.0814, 84.4397, 83.7647, 83.1647, 82.5897, 81.9897, 81.3897, 80.8147,
>>> 80.2397, 79.6647, 79.1147, 78.5147, 77.9397, 77.3147, 76.7397, 76.1897,
>>> 75.6397, 75.1147, 74.6147, 74.1147, 73.5647, 73.0397, 72.5647, 72.0647,
>>> 71.5397, 71.0147, 70.5397, 70.0147, 69.5397, 69.0397, 68.6397, 68.1647,
>>> 67.6897, 67.2397, 66.7897, 66.3897, 66.0897, 65.7897, 65.5397, 65.3147,
>>> 65.1397, 64.9897, 64.8647, 64.7397, 64.6897, 64.7147, 64.7147, 64.7897,
>>> 64.9147, 65.0397, 65.1397, 65.3647, 65.6147, 65.8647, 66.1647, 66.3897,
>>> 66.6147, 66.8397, 67.0647, 67.3147, 67.6147, 67.9147, 68.2647, 68.6147,
>>> 68.9147, 69.1647, 69.4397, 69.7147, 69.9647, 70.2397, 70.5147, 70.7647,
>>> 71.0897, 71.4147, 71.6897, 71.9897, 72.3147, 72.6147, 72.9147, 73.1897,
>>> 73.4647, 73.7147, 73.9147, 74.1647, 74.4397, 74.7147, 74.9897, 75.2397,
>>> 75.4647, 75.7397, 76.0647, 76.3397, 76.5397, 76.7897, 77.0647, 77.3397,
>>> 77.5397, 77.8147, 78.0647, 78.3147, 78.4897, 78.7397, 78.9897, 79.2397,
>>> 79.4647, 79.7147, 79.9397, 80.1897, 80.4647, 80.6897, 80.8397, 81.0147,
>>> 81.2147, 81.3647, 81.4897, 81.6147, 81.8397, 82.0147, 82.1647, 82.3147,
>>> 82.5147, 82.6397, 82.7647, 82.8647, 82.9647, 83.0647, 83.2147, 83.3147,
>>> 83.3647, 83.3647, 83.4147, 83.4897, 83.5147, 83.6147, 83.7397, 83.8397,
>>> 83.8147, 83.8147, 83.8147, 83.8397, 83.7897, 83.8147, 83.8397, 83.8397,
>>> 83.8897, 83.9147, 83.8647, 83.8397, 83.8397, 83.8397, 83.8397, 83.8397,
>>> 83.8397, 83.7897, 83.6897, 83.6147, 83.6397, 83.6397, 83.6397, 83.6147,
>>> 83.6647, 83.6397, 83.6397, 83.6647, 83.6147, 83.5647, 83.5647, 83.5397,
>>> 83.5397, 83.5897, 83.5647, 83.5647, 83.5897, 83.6147, 83.6647, 83.7147,
>>> 83.7147, 83.7147, 83.7397, 83.7647, 83.7897, 83.8397, 83.8897, 83.8897,
>>> 83.9397, 83.9897, 84.0147, 84.0397, 84.0647, 84.0897, 84.1647, 84.2147,
>>> 84.2647, 84.2897, 84.2897, 84.2897, 84.2897, 84.2897, 84.2397, 84.2397,
>>> 84.1897, 84.2147, 84.2147, 84.2397, 84.2397, 84.2897, 84.3397, 84.3897,
>>> 84.4397, 84.4397, 84.3897, 84.3647, 84.3397, 84.3147, 84.3397, 84.3647,
>>> 84.3897, 84.4147, 84.4647, 84.4897, 84.5397, 84.5397, 84.5397, 84.5397,
>>> 84.5147, 84.4897, 84.4397, 84.4147, 84.3647, 84.3647, 84.3897, 84.3647,
>>> 84.3647, 84.3397, 84.3147, 84.2647, 84.2147, 84.2147, 84.2147, 84.1897,
>>> 84.1647, 84.1147, 84.0647, 84.0397, 84.0397, 83.9897, 83.9647, 83.9397,
>>> 83.9147, 83.8647, 83.8397, 83.8397, 83.7897, 83.8147, 83.8147, 83.8147,
>>> 83.8147, 83.7647, 83.7147, 83.6647, 83.6397, 83.6647, 83.6647, 83.6647,
>>> 83.6252, 83.6369, 83.6794, 83.6960, 83.6814, 83.7004, 83.7224, 83.7480,
>>> 83.7783);
>>>
>>> doAllCurveFittings("pre-filtered", x, y); //full x-range
>>>
>>> function doAllCurveFittings(stackTitle, xpoints, ypoints) {
>>>      // Do a straight line fit
>>>      Fit.doFit("Straight Line", xpoints, ypoints);
>>>      //print("a="+d2s(Fit.p(0),6)+", b="+d2s(Fit.p(1),6));
>>>      
>>>      // Do all possible fits, plot them and add the plots to a stack
>>>      setBatchMode(true);
>>>      for (i=0; i<Fit.nEquations; i++) {
>>>       Fit.doFit(i, xpoints, ypoints);
>>>       Fit.plot();
>>>       if (i==0)
>>>           stack = getImageID;
>>>       else {
>>>           run("Copy");
>>>           close();
>>>           selectImage(stack);
>>>           run("Add Slice");
>>>           run("Paste");
>>>       }
>>>       Fit.getEquation(i, name, formula);
>>>       //print("");
>>>       print("index: "+i+", "+name+ " ["+formula+"]");
>>>       /*print("   R^2="+d2s(Fit.rSquared,3));
>>>       for (j=0; j<Fit.nParams; j++)
>>>           print("   p["+j+"]="+d2s(Fit.p(j),6));*/
>>>      }
>>>      setBatchMode(false);
>>>      run("Select None");
>>>      rename(stackTitle);
>>> }
>>> // END ##################################################
>>>
>>>
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html