Nonlinear Regression in MATLAB
Вставка
- Опубліковано 21 сер 2015
- A three parameter (a,b,c) model y = a + b/x + c ln(x) is fit to a set of data with the MATLAB APMonitor toolbox. This tutorial walks through the process of installing the solver, setting up the objective (normalized sum of squared errors), adjusting the parameter values to minimize the SSE, and plotting the results. Download source code from apmonitor.com/che263/uploads/M...
- Наука та технологія
Hi,
the problem that my experimental values are divided into 8 .txt files and they are voluminous so I can't import it in excel to have the data.csv
Do you have a solution ?
Thank you very much for all tutorials videos , I want to ask you about Apmonitor, when for example students completed successfully their online courses, will you give them a certificate?
The only way to get credit is to register for the course through BYU.
Thanks - great quality videos. Please keep making more of them.
Thank you for this great video. Easy to follow, with good explanations.
Dear John I interest with this program, let me know in the case when I have 6 independents parameter. I want to construct a non linear regression. Can I use this formula and adding x1, x2...x5?. Thx
It is no problem to add additional variables - it should work with up to 100,000+ variables.
Hello John! Thank you very much for this great video! I was wondering if it is possible to replace the equation with the parameters a,b,c by an ODE with the same parameters. Just a simple one like: y = a*xm + b*xm^2 +c*xm_dot.
I would be very greatful if You could help me with that. Thank you very much!
Ilja Sagradov, you can use a derivative value with $xm instead of xm_dot. You could also use xm_dot if you crate it as a new parameter and insert a new xm_dot column in your data file if you want to provide the derivatives.
Great video, helping me to find my parameters for the nonlinear curve, I have 2 variables which I have input and the code as per yours. It is taking a lot of time to complete the run. Can you tell me how I can speed up my work?
How many data points are you solving? Sometimes it helps to install a local server instead of using the public server: apmonitor.com/wiki/index.php/Main/APMonitorServer Also, Python gekko is likely faster: apmonitor.com/che263/index.php/Main/PythonDataRegression
Thanks for this tutorial. But I am getting an error message regarding fgets:
Invalid file identifier. Use fopen to generate a valid file identifier.
Error in apm_load (line 21)
aline = fgets(fid);
could you please suggest what to do !
Thanks in advance :)
+Somnath Paul it appears that you are missing a file. Please download the source code for this example from: apmonitor.com/che263/uploads/Main/MATLAB_nonlinear_regression.zip. You can see additional course content at: apmonitor.com/che263/index.php/Main/MatlabDataRegression
What differences would be necessary to do, if instead of the equation y = a + b/x + c ln(x) I would have a mfile with several input vectors and one output vector to do the exact same minimization? In other words, how can i put this equation in another file and call that file in model.apm to have the same result?
+Alexandre Lopes if you have an M-file then I'd recommend that you use fmincon or a solver package like Tomlab. You can also dynamically define the APM file if you have multiple cases or change inputs to that model with apm_meas. There are a couple examples here: apmonitor.com/do/index.php/Main/ParameterStatistics
Professor can you suggest me good article to calculate by hand(manually without matlab) coefficients - a , b and c(slope) related with your example - 3PL nonelinear?
You can find an analytic solution to this problem if you substitute two new variables x1 = 1/xm and x2 = ln(xm) and then perform linear regression on y = a + b * x1 + c * x2. With P = diag[a, b, c] and X = diag[1, x1, x2], the equation is Y = X * P and the solution is P = (X^T * X)^-1 X^T * y. The key is that you use multivariate linear regression (see a good textbook such as Applied Multivariate Statistical Analysis by Johnson and Wichern) on the transformed problem to make it linear.
Good Video. I was wondering what can we do if we only have the data but do not know the function/model, for example, the data is so complex that not sure the function. Is there any method to find the function?
Yes, try regression from machine learning: apmonitor.com/pds/
It seems to not work as good when you have a function with a singularity, do you have any idea on how to handle that?
y = a * ((sin(bx)/(bx))^2
Edit: It worked rather okay, when I slightly shifted the function and added a restriction a > 10
y = a * ((sin(bx)/(bx+0.000001))^2
Great observation. Setting a bound or rearranging the equation to avoid divide-by-zero is often effective.
i dont knwo why, but when i want to display the solutions i get an error; at z = y.x; it says Reference to non-existent field 'x'. How can i fix it? Thx
There is likely a problem with the solution. You'll need to print the solver output and resolve the error.
Excellent video and very helpful! Can this approach be implemented on more than one plot at a time? For example: solving for the coefficients of a similar mathematical formula which is attempting to predict 10 curves instead of one like above?
There is no limit. Please see apmonitor.com/do/index.php/Main/ModelIdentification for multiple time series models. You can add more equations and variables to this problem as well.
@@apm Thanks for the quick response! I will investigate, thanks again.
I have narrowed down my issue to the following: this is the error Im getting: "Error using fmincon (line 619)
Supplied objective function must return a scalar value." So its because I am trying to use this approach for more than one [xm, ym] data set, i.e. [xm,ym;xm,ym;....;.....]. I hope that fmincon can accommodate this?
@@trotion fmincon always needs a single value that returns from the objective function. Instead of returning a vector, try summing up all the objective contributions into one value. Here is a Gekko (Python) example of multivariate regression: stackoverflow.com/questions/57726954/what-is-the-correct-way-handle-with-multidimensional-array-in-gekko-nonlinear-re You can also do the same thing with APM MATLAB or run Gekko in MATLAB: github.com/BYU-PRISM/GEKKO/tree/master/gekko/utilities/MatlabWrapper
Hello, Thank you for your video. For my research, I am trying to come up with a calibration equation. I used this tool to develop four of my model. others are polynomial equations. One of the four model is y= a+b*x^m+c*x^n. The reason for using this tool is to get non-integer value for the parameter m and n. I think my equation is not non-linear. My question is can I use this tool to get parameters value for my models. Thank you very much.
Yes, this tool will work for that application.
@@apm Thank you very much. I was confused because I thought that this tool is only for non-linear regression.
Error : Reference to non-existent field 'x'.
How do I fix it?
Make sure you get the latest apm package from apmonitor.com/wiki/index.php/Main/MATLAB Also, you should print the solver output to make sure a solution is correctly generated.
Thank you for this tutorials they are very useful ,but I am getting an error message :
Warning: Name is nonexistent or not a directory: apm.
> In path at 110
In addpath at 87
??? Undefined function or method 'apm' for input arguments of type 'char'.
I have downloaded the apm file and did everything like in the video , please what should i do , to get it right (i don't have Wolfram Mathematica) , ihave only matlab , and thank you very much again.
+brown Sara please download the apm folder from apmonitor.com or at github.com/APMonitor/apm_matlab. Place the apm folder in your directory so that Matlab can add the functions to make this work.
Hello Sir
I would like to use model y=a+b*cos(x), It says reference to non-existent x. Can you please help me with this?
At 5:22 you can use y=a+b*cos(xm). Otherwise, you need to define a new parameter x.
How i can solve this Eqation Na= Nd(Sa/Sd)^(-k).?
I know only Value Of Na nd Sa and try to fit the line by getting the value of Nd,Sd,& k.
+sandip vassani You can download the sample files for the example problem in this video from: apmonitor.com/che263/uploads/Main/MATLAB_nonlinear_regression.zip. You can download, unzip, and then modify the files for your application. If you run into any trouble, please send the files to the APMonitor discussion group for help: apmonitor.com/wiki/index.php/Main/UsersGroup (at apmonitor@googlegroups.com).
how can write error function
Hello John, wonderful video!
Only a question: I have to fit a lot of data (almost 12000) with the double exponential decay model
[y=A*exp(-a*x)+B*exp(-b*x)+C],
a,b>0
but the response is
No feasible solution
Solver : APOPT (v1.0)
Solution time : 76.6786999999968 sec
Objective : 11278.6596207652
Unsuccessful with error code 0
I know the order of magnitude that the 5 parameters should assume, can I write down the range in which to vary the 5 parameters?
Thanks a lot
At 4:30 in the video, you can add constraints on the parameters such as: a = 0. Sometimes better guess values or constraints can help the solver converge. You can also try changing the solver to the IPOPT solver with apm_option(s,a,'nlc.solver',3). Let me know if you need additional support with your application - you can send the files to the APMonitor user's group by subscribing and posting to apmonitor@googlegroups.com.
Thanks for answer. I have added the constraints to the parameters and the result of the objective value was about 0.07 and the measured data was quite close to the predicted data. Thanks a lot.
Hi, thank you.It seems it will solve my problem. I am trying to repeat the example and I got the error.
" Error using urlread. The value of 'url' is invalid. Expected input to be one of these types". Any recommendation to resolve this?
Thank you
Please check that your MATLAB version is 2014+ and that you are
connected to the Internet. If either of those is a problem, I
recommend that you try Python Gekko that doesn't require an internet
connection in local mode. There is also a way to call Gekko from
Matlab: apmonitor.com/che263/index.php/Main/MatlabCallsPython
Can I use Marquardt Method to fit this curve?
Yes, Levenberg-Marquardt method is another option. Unfortunately, I don't have this method in the code.
Professor I have a problem, I am trying to change the model and data from the APM file and CSV file respectively, but matlab always gives me the same result, suggesting that it does not consider those files
Make sure the run directory is set correctly. You may also want to try Python Gekko: apmonitor.com/me575/index.php/Main/NonlinearRegression
i can't seem to make an apm file.. once i make a new text document, it only saves as a text document and not an apm file
Here is help on changing the extension: ua-cam.com/video/YlN9ax7yC0A/v-deo.html
the software looks really good but the description is so vague!!! I have some experience with Matlab and couldn't get what you are trying to say sir so I guess I ll keep trying with examples
Thanks for the feedback. There are additional examples at apmonitor.com/do
Hello John,
A stupid question is that why we should include s = byu.apmonitor.com ?
It solves the problem on the remote server and then returns the solution to your MATLAB console. There is an option to download and install the APMonitor software (apmonitor.com/wiki/index.php/Main/APMonitorServer) but most users will probably just want to use the web service (available from github.com/APMonitor/apm_matlab).
In my case, I only have two unknowns, what should I do toward the existing file to solve my problem?
+Muflih Arisa Adnan you can modify the model file as show at 4:27 to include a different number of parameters. You can calculate those parameters by turning status=1 (on) in your Matlab script as shown at 9:55. Good luck!
I have tried to reduce the number of parameters from 3 to 2 parameters (I removed c parameter). Then I deactivate these two scripts:
apm_info(s,a,'FV','c');
apm_option(s,a,'c.status',1);
But, there is an error message:
Reference to non-existent field 'x'.
Error in regression (line 33)
z = y.x;
what should I do to solve this error?
It looks like there is another error in your script. There is probably a message before the one that you showed. It should point to the place in your model where there is an error.
Hello, I am curious how I can add regression statistics to this demo file? I ultimate want to run a for loop and run multiple model files for the same data set and compare a model statistic, like the root mean square error to determine the best fit.
So the RMSE calculation is easy but I was wondering if apmonitor has functions to do this? my RMSE code is :
%results check
tau = z.t(1) + z.k(1).*z.xm.^(z.n(1)); %(this is the model im fitting so just change this to whatever you need to)
%RMSE Calcualtion
difsq = zeros(1,length(z.ym));
count = length(z.ym);
for i = 1:1:length(z.ym)
difsq(i) = (tau(i)-z.ym(i)).^2;
end
error = sqrt(sum(difsq(:))/count);
disp(['RMSE = ' num2str(error)])
APMonitor doesn't return the R^2 value. However, there are functions in MATLAB for this: www.mathworks.com/help/stats/coefficient-of-determination-r-squared.html Here is some additional analysis that you may be able to adapt from Python: apmonitor.com/che263/index.php/Main/PythonRegressionStatistics
@@apm Hello, how does your first example (in the comment link) translate to the example from this video. In other words. how and where would I put in code to obtain a R^2 value for the measured and predicted values for the final plot in this video? So far this has been very helpful. But I am hitting a wall at this point.
@@patricksinko4375
I tried using this but get a error
Undefined operator '-' for input arguments of
type 'struct'.
Note: I am working off this video exactly, with just a change in the model. From there I am trying to get an R^2 from the measured and predicted values.
@@biomilitus8139 You can calculate the R^2 from this code:
Bbar = mean(B);
SStot = sum((B - Bbar).^2);
SSreg = sum((A - Bbar).^2);
SSres = sum((B - A).^2);
R2 = 1 - SSres/SStot
Where B is your vector of measurements and A is your vector of predictions.
Hello. My model runs to 100 iterations and it it displays: "Maximum iterations" and also "Creating file: infeasibilities.txt
Use command apm_get(server,app,'infeasibilities.txt') to retrieve file
@error: Solution Not Found" How can I solve it. Thank you very much.
You can try a different solver apmonitor.com/wiki/index.php/Main/OptionApmSolver or else increase the maximum number of iterations: apmonitor.com/wiki/index.php/Main/OptionApmMaxIter as the command:
apm_options(s,a,'apm.solver',3)
I tried to fit my (500 data points) data to a cos squared function . My model equation is --> y =a + b*((cos((c*xm)+d))^2)
Maybe switch over to Python Gekko. There are examples 3 and 4 here: apmonitor.com/do/index.php/Main/DynamicEstimation or a similar problem here: apmonitor.com/me575/index.php/Main/NonlinearRegression
APMonitor.com thank you so much for your helpful amazing video :) keep on ;)
I am facing a problem. Can anyone help me plz. i have a dataset of about 500 data points in my csv file but my xm and ym is showing only one value instead of a array of 500
Try running the source code at apmonitor.com/che263/uploads/Main/MATLAB_nonlinear_regression.zip (after you extract the archive) and then replace the data.csv values with your data. Also, you'll want to open the data.csv file with a text editor to make sure the numbers are formatted correctly as a CSV (Comma Separated Value) file.
Dear Prof.
I'm trying to make my code with this equation : y=p*(ke/(kd-ke))*exp(-ke*xm)-exp(-kd*xm))
But I'm getting an error message:
Creating file: infeasibilities.txt
Use command apm_get(server,app,'infeasibilities.txt') to retrieve file
@error: Solution Not Found
Solution
a = 1
b = 1
c = 1
The solver didn't find a solution. Here are suggestions on StackOverflow. You may want to consider switching to gekko: www.google.com/search?q=infeasible+stackoverflow+gekko
I tried and received this error:
"Error: include CSV data in 81.240.25.145_data_regression.csv
when CSV_read is ON
STOPPING. . ."
Anyone who can help me to fix it, please?
+Sergio Valdivia, it looks like you are missing the data file. Could you try to start with these files? apmonitor.com/che263/uploads/Main/MATLAB_nonlinear_regression.zip Don't forget to extract the archive after you download and run the script.
Thank you!
Dear Professor how can i display Output U?
function fval = conductor_output_voltage(u, p, b, ok, I, xs, ys, xi, yi)
%This function calculates output-voltage of each conductors
fval = u * p * exp(1i*cos(b))*I./2*pi*sqrt((ys.-yi.)^2-(xs.-xi.)^2)
end
%Define parameters:
% u = magnetic permeability
% p = Gain factor
% b = Sensing directional angle
% r,ok = Position and angular distance between conductor and sensor
% xs,ys = Coordinate of sensor
% xi,yi = Coordinate of conductor to be measured
% I Current
u = 4*pi*10^(-7);
p = 6.824*10000;
b = 13.61;
ok = 0.01;
% Define the positions of each sensor.
xs = [27.69; 27.15; 15.24; -13.16];
ys = [-14.78; 9.55; 24.07; 24.71];
% Define the positions and values of each conductor.
xi = [1.242; -6.6884; -2.346; 5.0564];
yi = [+9.9012; +6.0868;+1.5672; +1.9709];
I = [4.6689-4.9448i; -10.363-2.2886i; 1.6691+5.6988i;4.025-1.5346i];
Are you trying to calculate u or is it a constant? You can print a value with disp(u). You'll need to use regression if you need to calculate u: apmonitor.com/che263/index.php/Main/MatlabDataRegression
Dear Prof.
I want to find 3 Parameters(xi, yi,I) of a non linear least square function. But i dont know how to use "Isqcuervefit" because there are three initival of them.
Hier is mein Code:
clear all; close all; clc
%Define parameters:
% My = magnetic permeability[H/m]
% Rho = Gain factor
% phi = pahse delay [°]
% Theta Sensig directional angle
% r,phi = Position and angular distance between conductor and sensor
% xs,ys = Coordinate of sensor
% xi,yi = Coordinate of conductor to be measured
% I Current [A]
My = 4*pi*10^(-7);
Rho = 6.824*10000;
phi = 13.61;
Theta = 0.01;
RI = 21; % in[mm]
% Define the magetic fields in [µT] of sensors
B = [0.17; 0.47; 35.30; 35.32; 6.66; 6.59; 36.28; 35.43; 13.36; 15.04; 42.17; 44.05];
% calculate the output voltage of sensors
V = (My*Rho*exp(j*phi*pi/180))*B;
% Define the measured voltage of sensors
Vm = [0.0152 + 0.0034i; 0.0392 + 0.0095i; 2.9421 + 0.7123i; 2.9437 + 0.7127i
0.5751 + 0.1344i; 0.6 + 0.1330i; 3.0238 + 0.7321i; 2.9529 + 0.7149i
1.1235 + 0.2696i; 1.2535 + 0.3035i; 3.7147 + 0.8509i; 3.6713 + 0.8889i];
% Define the positions of each sensor
xs = [-21; 21; 0; 0; -14.5; -15; 16; 15.3; -8.800; -7.900; 10.3; 9.4];
ys = [0; 0; 21; -21; 14.5; 13.8; -12.9; 13.9; 18.3; -18.8; -17.9; 18];
x0 = [30,30,10];
% Contraints of function NNLS
x = lsqcurvefit(@(xi,yi,I)NNLS_fuct(My,Rho,phi,Theta,RI,V,Vm,xi,yi,xs,ys),x0,V,Vm);
function NNLS = NNLS_fuct(xi,yi,I,My,Rho,phi,Theta,RI,V,Vm,xs,ys)
% calculate the distance between the sensor and conductor
dis_array = sqrt((ys-yi).^2-(xs-xi).^2);
% calculate cos angular distance between conductor and sensor
cos_delta_a = (xs.*cos(Theta)-ys.*sin(Theta)).*(xs-xi)+(ys.*cos(Theta)-xs.*sin(Theta)).*(ys-yi);
cos_delta_b = (xs-xi).^2+(ys-yi).^2;
a = My*Rho*exp(j*phi*pi/180)*I;
% calculate the Output-voltage
V = a.*cos_delta_a.*cos_delta_b./dis_array;
Sum_V = sum(V);
end
Not enough input arguments.
Error in Noninvasive_current_sensor>NNLS_fuct (line 46)
dis_array = sqrt((ys-yi).^2-(xs-xi).^2);
Error in Noninvasive_current_sensor>@(xi,yi,I)NNLS_fuct(My,Rho,phi,Theta,RI,V,Vm,xi,yi,xs,ys)
Error in lsqcurvefit (line 213)
initVals.F = feval(funfcn_x_xdata{3},xCurrent,XDATA,varargin{:});
Error in Noninvasive_current_sensor (line 41)
x = lsqcurvefit(@(xi,yi,I)NNLS_fuct(My,Rho,phi,Theta,RI,V,Vm,xi,yi,xs,ys),x0,V,Vm);
Caused by:
Failure in initial objective function evaluation. LSQCURVEFIT cannot continue.
Thx. help!!
I haven't used that package. Does this help? www.mathworks.com/help/optim/ug/lsqcurvefit.html
please explain me! why can't read data.csv file.
i get z.xm and z.ym are 1.
You need to download the data.csv file and extract the zipped archive.
I do it but i get both z.xm and z.ym are 1.
Please try this archive: apmonitor.com/che263/uploads/Main/MATLAB_nonlinear_regression.zip Make sure you unzip the archive before you run it. I just tried it with MATLAB 2017a and it works. What version of MATLAB are your running?
i tried to use another model but it didn't work.
y={(415/b)*(c+6.464*(1-c)*(1/cos{0.33*asin[-13.5*xm*(xm^2-0.33)]}-1)*[(sqrt((1+a^2)/3)*sin{0.33*asin[-13.5*xm*(xm^2-0.33)]}+a*(xm+0.33*cos{0.33*asin[-13.5*xm*(xm^2-0.33)]}}^(-1/0.25)
thank you to tell me what i have to change .
could you post your problem files to the APMonitor Google group?
Thank you very much professor john . I will joint the group and hope to find your kind help
very wonderful video . many thank to you professor please prof. can I used different equation in my model kind regard
Yes, you can modify the equations in the model file. See the part of the video starting at 4:28 for information on modifying the model.
Many thanks for your response , please professor kindly ,I make some change in the model equation and reply the code but I don't see any change in the results value of the coefficient or other parameter .what is that mean kind regard
Please post your code to the APMonitor Discussion group. You'll need to subscribe first at groups.google.com/forum/#!forum/apmonitor
can matlab identify nonlinear processes?? I want two methods
A solver such as fmincon can also perform nonlinear regression. I'm planning to make a video tutorial on this next week so check back soon.
I just added an fmincon tutorial to apmonitor.com/che263/index.php/Main/MatlabDataRegression
how about if i have imaginary number?
The solver fmincon can use imaginary numbers. See example 3: apmonitor.com/che263/index.php/Main/MatlabDataRegression
Dear Prof,
I get this error message when I run the script(even when I run the example script shown in the video). Could you please help me with this problem? Thank you.
Error using urlread
The value of 'url' is invalid. Expected input to be
one of these types:
char
Instead its type was java.net.URL.
Error in urlreadwrite>parseInputs (line 144)
p.parse(args{:})
Error in urlreadwrite (line 15)
inputs = parseInputs(fcn,varargin);
Error in urlread (line 47)
[s,status] =
urlreadwrite(mfilename,catchErrors,url,varargin{:});
Error in urlread_apm (line 51)
output = urlread(url);
Error in apm (line 31)
response = urlread_apm(url);
Error in regression (line 12)
apm(s,a,'clear all');
You need to be connected to the Internet to use the public server to solve it. If you don't have an Internet connection, you can use Python Gekko through MATLAB with remote=False apmonitor.com/che263/index.php/Main/MatlabCallsPython or else get a local server: apmonitor.com/wiki/index.php/Main/APMonitorServer
Also, you'll need at least MATLAB 2014 to run this. If you have an older version, it may not work.
Thank you for your tutorial and reply. May I ask how to define the objective function? And wondering why data input is not required in this simulation?
@@jessielim569 here is information on the objectives. The default is to use the l1-norm error: apmonitor.com/do/index.php/Main/EstimatorObjective The measurements are defined in the data.csv file. The measurement is the column labeled ym.
You atrted to talk in chinese at 8:27 😔
I'm so lost...
Sorry for the confusion. This additional material may help: apmonitor.com/pds/notebooks/06_regression.html
Obviously this is a LINEAR problem to be easily solved e.g. by least squares normal equations and a Cholesky decomposition of a 3x3 system.
+Matthias Ehrhardt, you are correct that it is linear in the parameters. The purpose of this tutorial is to demonstrate the platform with a simple example. Nonlinear, differential, and mixed integer variables are also allowed and are shown here: apmonitor.com/do
so the title is a bit misleading
It is still a nonlinear regression problem because I didn't do a parameter transform. It can be simplified to a linear regression but the actual method that I demonstrated is nonlinear.
at least in german universities this is classified as a linear problem. Linear does not necessarily mean that the approximant is a straight line; here it is a linear dependence of y on (a,b,c).
I agree that the problem can be simplified to a linear problem by reformulating it as y = a + b*w + c*z with w=1/x and z=ln(x). In the tutorial, the nonlinear transforms for w and z are performed with the solver, not as a precursor step. The data is only provided as y and x, not as y, w, and z as would typically be the case when performing linear regression. I think we both understand that this problem is linear in the parameter space. The purpose of the tutorial is to demonstrate how to solve a parameter estimation problem with a very simple correlation. You are welcome to try nonlinear relationships like y = a * sqrt(x) + b and they should also work in this environment. One other thing to consider is that the objective function at 5:27 is normalized by the measured value, making it different than a standard least squares problem. Other objective function forms such as the 2-norm (square root of squared errors) or 1-norm (minimize absolute value) are also possible. I appreciate your concern over this example problem, especially when a simple linear regression is possible. In many problems there are constraints, nonlinear correlations, or custom objective terms that don't fit a standard linear least squares regression. The significance of this platform is that large scale parameter estimation is possible even though this simple example problem doesn't fully exercise the functionality.